Lab blogs on recent billing / transaction failures

On Monday, June 26th, 2017 many Second Life users encountered issues  when trying to purchase Linden Dollars or purchase goods via the SL Marketplace following a period of scheduled maintenance and updates to the billing system. These incidents were escalated to the Lab, resulting in a further period of unscheduled maintenance, during which time payment processing was suspended, which itself had some knock-on impact.

Some of these issues were still being felt on Wednesday, June 28th, 2017. However, Xiola Linden took time out to blog about what has been happening, in which she provides the following update on the situation:

After some intensive troubleshooting that – despite our best efforts – took longer than we’d have liked, the base cause of these failed transactions was resolved as of 3:51PM PDT on the same day. However, we still have the task of refunding the thousands of failed transactions to the impacted users ahead of us. A large number of Lindens across several departments have been coming in early and staying late between Monday and today to fulfil that responsibility to our users. Your patience is greatly appreciated, and we hope to complete the work as quickly as possible.

We’ll post an update to this blog as soon as we have more information to pass along. Our support agents on the phone lines and live chat do not have any additional information to provide at this time. Support cases which have already been submitted will be answered once this situation has been resolved.

Xiola goes on to note that as a result of the issues “Shrek ears” are being worn by Lab staff  – a reference to April Linden’s May 10th blog post about a bit of a SNAFU with Place Pages, in which April explains:

We encourage people to take risks and push the limits of what we think is possible with technology and virtual worlds. It helps keep us flexible and innovative. However… sometimes things don’t work out the way they were planned, and things break. What we do for penance is what makes us unique.

Around the offices (and in-world!) we have sets of overly sized green ears. If a Linden breaks the grid, they may optionally, if they choose to, wear the Shrek Ears as a way of owning their mistake.

If we see a fellow Linden wearing the Shrek Ears, we all know they’ve fessed up, and they’re owning their mistake. Rather than tease them, we try to be supportive. They’re having a bad day as it is, and it’s a sign that someone could use a little bit of niceness in their life.

At the end of the day, the Linden takes off the Shrek Ears, and we move on. It’s now in the past, and it’s time to learn from our mistakes and focus on the future … If you see a Linden wearing them, please know that’s their way of saying sorry, and they’re really having a bad day.

Posts of this nature, explaining what went wrong and why (and, where possible steps being taken to try to ensure there are no  – or at least few – repeats) are an important part of reassuring users and shining a light on just how complex a beast SL can be when things do go wrong.

In that vein, I’m still hoping we’ll get one of April’s informative blog posts on concerning the June 14th log-in issues (although I appreciate she and the Ops team likely have their hands full right now!).  but in the meantime, the “Shrek ears” notes remind us that those who work at the Lab are no different from the rest of us – something we can at times lose sight of in our own frustrations when things go wrong.