Thursday’s downtime: network bug; more downtime to follow

The rumour mills (serious and highly humorous – the latter particularly on Twitter) churned mightily yesterday, Thursday April 26th, after SL log-ins were blocked. Users attempting to log-in to SL were treated to something of a dire message, rarely seen in recent years, prompting me to tweet at the time:

The outage certainly brought back memories for many, particularly of the old Black Wednesdays, when the Main grid would be down for anything up to eight hours (the rule of thumb being six) for weekly maintenance.

Yesterday’s outage brought with it memories of the “good old days”

A Grid Status update posted on April 27th later revealed that the cause of the fault to be a core network upgrade going awry, impacting both the Main grid and the SL website at secondlife.com. The Grid Status update also warns of further maintenance work that will take place on Monday April 30th as a result of the issue. The update reads in full:

[UPDATED] Scheduled Maintenance Monday, April 30, 2012

[UPDATED 2:30pm PDT, 27 April 2012] On April 26, 2012, the Second Life engineering team was performing an upgrade to the core network. The upgrade triggered a bug that led to downtime and instability throughout the day. The maintenance described in this post is scheduled for Monday, April 30 at 6am PT in order to correct the bug and may result in additional service interruption. We apologize for the service disruption and appreciate everyone’s patience.

[POSTED 1:00pm PDT, 27 April 2012] We will be performing scheduled maintenance Monday April 30, 2012 beginning at 6:00am until approximately 9:00am. Please save all builds and refrain from rezzing no copy objects or making inworld L$ transactions. Some residents may experience delays logging inworld. Please follow this blog for any updated information.

ANS launches: “I have some good news and some bad news…”

The good new is… Automated Notification of Sales (ANS) rolled out yesterday for Direct Delivery. ANS allows information on sales to be forwarded to an external URL, allowing merchants to not only track sales, but to perform a range of different analyses on their sales and customers, in order to provide things like more focused support, identify product trends and so on.

Originally, it had been hoped that ANS would form part-and-parcel of the overall Direct Delivery roll-out, and many merchants were disappointed that this was not the case, with ANS being delayed for reasons unknown.

Yesterday’s announcement that ANS is now available came with a highly-informative user-written wiki article on how to make use of ANS. This is extremely well-presented and spells-out exactly how ANS data can be received and used.

However, it’s not all good news, sadly, as Darrius Gothly reports in the forum thread making the announcement:

 Sadly the ANS for Direct Delivery has a Severe bug and IS NOT SAFE TO USE YET!! The ANS Transaction being sent via the Marketplace service is duplicating the Item ID# (the numeric part of the Product’s listing page) into the Location field. The Location Field is supposed to have the Order Line Item ID number instead, showing which line item in an Order correlates to the ANS Transaction. As long as that field contains the wrong data, you CANNOT track an ANS transaction back to the specific line item in an Order.

Darrius has raised a JIRA on the matter, and merchants who use ANS / have an interest in using ANS are urged to log-in and WATCH the JIRA.

Marketplace: LL updates further, but communications hardly “regular”

Linden Lab has issued a further brief update about the on-going Marketplace issues, to whit:

Today we updated Marketplace to address two of the top three outstanding issues:

  • WEB-4580: purchases are now delivered to recipients with the inventory name (which does not allow unicode characters). This will prevent future orders from getting stuck in the Being Delivered state due to this issue. In addition, all orders affected by this problem have been pushed through.
  • WEB-4587:  updates have been made to support updating store search results, which we will process over the next week; we continue to work on the issue related to mismatched data on listings. We do know that this issue has existed since September 2010 (during the migration from Xstreet to the Second Life Marketplace).

We continue to work on the other Marketplace JIRAs and will provide additional updates as soon as possible.

Menwhile, Rodvik has stepped in to defend how matters have been handled in terms of communications, stating on Twitter:

While it is true that the Commerce Team clearly engaged with individuals experiencing problems through the medium of e-mail exchanged, it is nevertheless also true that feedback on this matter in the broader sense has been severely lacking from the Lab, with little or nothing being posted to either the main forum thread on the WEB-4587 issues or the JIRA itself. This left many merchants both frustrated and feeling as if they’d been abandoned, while those who had received some feedback from the Lab via e-mail tried to pass on the information to a wider audience in lieu of LL doing so.

It is good that progress is being made – but equally, it would be nice if LL would do more to keep users openly informed. As those responding to Rodvik’s tweets note:

Related Links

Linden Lab obtains right to sub-licence Havok engine

Linden Lab has recently acquired the right to sub-licence the Havok physics engine technology used within their Viewer. This has resulted in the Lab issuing new guidelines to third-party Viewer developers wishing to incorporate advanced Viewer capabilities developed using the Havok technology within their offerings.

The guidelines read in part:

The technology is provided in the form of an autobuild package ‘llphysicsextensions’ containing header files and the required library. This does not directly expose the Havok APIs, but a set of higher level interfaces specific to the viewer. Sources for the wrapper itself will not be open source. The llphysicsextensions package includes all features that use Havok (currently convex decomposition and features related to navigation mesh for pathfinding).

This move is already a subject of debate among TPV developers and the OpenSim community, because the sub-licence associated with the guidelines appears to place clear restrictions on TPV developers, notably in clause (b) of the Conditions to Grant, which reads:

(b) Sublicensee must require the Third Party Viewer to connect only to servers owned or operated by the Company; [i.e. Linden Lab]

So if a TPV developer wishes to work on both Second Life and OpenSim, they’ll have to look at options very carefully, as Maria Korolov points out in Hypergrid Business.

Within Second Life, there is concern as to what this may mean for some TPVs – specifically those utilising GPL rather than LGPL. Such Viewers appear to be effectively excluded from applying for a sub-licence. While this will not prevent such Viewers from accessing Second Life, it does mean that they’ll be excluded from using code that implements the Havok capabilities. The requirement for TPVs wishing to obtain a sub-licence being required to be publicly listed on the Third-Party Viewer Directory may also have a negative impact in some quarters.

The flip side to this, however, is that it means Havok physics will effectively be in the Viewer itself, which could pave the way to many new enhancements and capabilities within Second Life. As such, it is far to say that the move to sub-licence the Havok engine is less about LL attempting to restrict Viewer development per se (the apparent attempt to push out V1-based Viewers not withstanding), but rather to provide a means by which they can integrated what is effectively a closed-source, licenced product (Havok) into what is essentially an open-source project (the Viewer) without breaking the terms of their agreement with Havok.

The program itself is not available as yet, and discussions within the community are ongoing, with TPV developers aiming to seek further clarification from Linden Lab on possible impacts on their work – again, specifically where OpenSim support is concerned.

Related Links

Scanning for the SL scammers

SL has tended to have its share of scams over the years, running from the misuse of Account Debiting scripts (wherein an unknown object sends you a request asking to be allowed to take money from your account), through to quite involved and complicated data-scraping efforts as most clearly exemplified by the infamous RedZone affair of 2010/2011.

Recently, we’ve had two attempts at what amounts to phishing for SL user’s log-in credentials (and possibly other information). These attempts are focused on trying to take advantage of the Second Life name and the widespread popularity of the Phoenix and Firestorm TPVs.

SL Log-in Scam

This problem first appeared in March which people began receiving seemingly genuine information directing them to what appears to be the SL web log-in page, with a request to log-in to SL The site was actually a false page, geared solely towards gain people’s user name and password.

Lindal Kidd was one of the first to report this issue, alongside of covering the Account Debiting scam. Shopping Cart Disco also covered the issue, with an excellent piece on what to look for, complete with screen shots. More recently, the Phoenix / Firestorm team themselves blogged about the problem.

Phoenix / Firestorm Survey Scam

Over the Easter weekend, a new scam appeared using the lure of a L$ reward to tempt people. It comprises a message the can be received either in-world or relayed to e-mail (if you are offline), encouraging you to visit a website and “confirm your details” in return for a L$1000 reward. The following is a typical example of such a message (as I received today, relayed via e-mail):

The object ‘Second Life’ has sent you a message from Second Life: Happy Holidays Everyone! Get 1000L just for signing up here and confirming your email –http://bit.ly/????

Second Life is owned by FirestormRelease Resident

(Note that I have redacted a part of the URL short link to avoid any accidents with people copy/pasting it out of curiosity.)

This is a particularly insidious scam because it is using the names of SL’s two most widely used TPVs in order to gain a veneer of authenticity – notice the name of the avatar responsible for sending the message. This has prompted the Phoenix / Firestorm team to issue a cautionary Message of the Day warning, seen when logging in to either of their Viewers:

Multiple accounts are being used to circulate such messages – “FirestormRelease Resident”, the attribution for the message I received being just one. Indeed, when I contacted Jessica Lyon about this account name and location she replied, “I just got a bunch of those accounts shut down, however, if more show up please send me the SLurls to the objects and account names.”

How to Deal with a Scam Message

The important thing here is that if you are in receipt of such a message / e-mail either asking you to log-in to the SL website or which gives the impression it is associated with a valid group or organisation within SL (such as the Phoenix / Firestorm team as seen here), do not click on any link it contains or provide any information to the website you’re taken to if you do.

In respect of the SL log-in page, you can always test the validity of the page you are displaying prior to logging-in simply by looking at the URL. The genuine SL website log-in page will always commence with: https://id.secondlife.com/, regardless of whether you are trying to log into your Dashboard or the Marketplace or your web Profile.

SL log-in page: the real McCoy (click to enlarge)

If the URL for the page contains any other information than this, regardless of how “real” the rest of the page may look, then the URL is bogus; do not follow it. An example of such a bogus URL which was circulated last month commenced: “http://marketplacesi.altavista…..”.

False prophet – note the (made up) URL (click to enlarge)

Where messages appearing to come from established in-world groups or organisations are concerned, check the message carefully and if you have any doubts at all, contact a representative of the group / organisation to verify whether the message is genuine or not.

If you have followed any such link and supplied information to a website / possibly had something download from the website, then you should:

  • At the very least, change your account password immediately
  • If you believe the account has already been tampered with, contact Linden Lab and inform them of the situation. They may lock the account while they investigate. Note that you’ll have to supply RL information in order for them to release it back to you
  • Raise an Abuse Report if you have sufficient information on the perpetrator. Contrary to popular myth, LL do take Abuse Reports seriously and will investigate
  • Run an anti-virus / malware sweep of your computer.

In the case of scam messages relating to Phoenix / Firestorm, you may wish to inform Jessica or a member of the team, so they can continue to work with LL to get bogus accounts shut down en masse.

Linden Lab are working pro-actively on matters as well – not long after I’d informed Jessica about “FirestormRelease Resident”, a representative from the Lab was on-hand investigating the location being used.

Related Links

Marketplace listing issues: LL provide feedback

With listing errors still apparent on the SL Marketplace, Linden Lab have provided some feedback relating to their investigations. An update to their post relating to Marketplace JIRA currently under investigation reads:

[UPDATE: April 5, 2012]
We’ve identified two issues contributing to WEB-4587.

The first issue concerns mismatched data appearing on Marketplace listings. This impacts a very limited number of Merchants and has been occurring since the original migration from Xstreet to the Second Life Marketplace in September 2010. We continue to actively work on a resolution. 

The second issue concerns incorrect listings appearing in search results. We have identified the issue and are actively working on a fix. 

The information provided by Merchants on the forum and directly to the Marketplace development team helped greatly in identifying the root cause of these issues. Thanks to those who provided information.

We will continue to provide updates as we work through the issues.

There is no indication as to haw far along any resolution / fix for either issue might be – and probably wisely so. Event tentative dates might be taken literally and cause additional strain in LL / merchant relationships if missed.

That one of the errors dates back to the original migration from XStreet to the current  SL Marketplace back in 2010 will come as no real surprise to merchants who have been impacted by the issue; it is a view that was aired as a part of ongoing discussions into the matter. It might event be fair to say – as the update hints – that it was such speculation and  investigations by merchants themselves that may have helped Linden Lab confirm this particular root cause for the problems.

Even without potential dates for a fix or fixes to be rolled-out, the feedback from the Lab is to be welcomed. Hopefully, now issues have been identified and the line for updates has been re-established, the Commerce Team will take further steps to ensure the merchant community as a whole is kept informed of the situation through the Commerce forum.

Related Links