Marketplace: it’s not only the tempo, Rod, it’s time for practical steps

The SL Marketplace has appeared in these pages a few times these past couple of weeks, and not in a good way. First was my coverage of the latest errors relating to listing enhancements, to be followed by the news that LL’s Commerce Team had apparently seen the error and taken steps to correct it and refund people. Only as it turned out, they’d only fixed half the problem, which resulted in people again being incorrectly billed.

Marketplace: a dispiriting place?

Some Good News

On the 8th October the Commerce Team issued the following update:

Merchants,

There has been quite a bit of discussion on Product Listing Enhancements and stuck orders over the past few days on this Forum. Here is an update on these issues.

Product Listing Enhancements:
Last week, we deployed a fix for Product Listing Enhancements to allow them to start billing again. We refunded all PLEs for the prior two months.

After we refunded merchants, billing for Product Listing Enhancements started again. Not all of the billed enhancements correctly updated their renewal date, so we stopped billing. We have updated the PLEs that had an incorrect date and will be releasing a fix this week before we start billing again.

We are aware that there are some Merchants who have Product Listing Enhancements stuck in the “Charging, cannot edit” state. We are continuing to look into this and investigate what we can do to get those PLEs unstuck.

Stuck Orders:
Late Friday night through early Saturday morning, many orders got stuck in the being_delivered state. This morning we were able to force those orders to a completed state (allowing payments to Merchants to complete) and are working to prevent this from happening again.

If you are seeing anything different than [sic] the behavior described above (or have an additional problem), please contact support or file a JIRA. Please include order numbers or listing information as needed.

The Commerce Team

On the whole, this is welcome news, regardless as to how the problems originated.

Please use the page numbers below left to continue reading this article

Marketplace issues: not so much eroding trust as completely undermining it

Well, it seems news over the correction in one aspect of the ongoing SL Marketplace listing enhancements debacle (itself merely one part of the overall Marketplace debacle) was premature.

No sooner had the Commerce Team announced they were refunding people for the mess-up over payments, that automatic debiting for enhancements resumed, with the same level of confusion as to what is actually going on, and people unable to determine exactly what they have or have not been charged for. How this came to pass is unclear, although I do tend to agree with Darrius Gothly’s assessment of the situation, vis:

When your staff went through and refunded everyone, you should have AT THAT TIME tested to be sure your code modifications would not immediately undo everything just done. But did you? Nope. As a result it went through and lickety-split re-billed everyone .. not only for what they’d just been refunded but additional charges too. Pardon me but .. WTH?!? By dint of your lack of attention you have just completely undone everything your staff did .. by hand .. at great expense to your employer. You have WASTED a very large amount of money. Wasted because you could not or did not want to bother testing your changes. 

It is perhaps bad enough that people have seen refunds enter their accounts only to evaporate once more. But it would also appear that people are again getting charged for enhancements they cannot cancel due to WEB-2974 (an issue now some two years old, and resolution of which was “on hold” as of July 2012).

This state of play is, frankly, ridiculous. While mistakes can and do happen, what has been going on within the Marketplace and on the part of the Commerce Team long ago reached a point of farce. Even the simplest of tasks appears to be beyond their capabilities (or the capabilities of the software they manage). Remember the change to the sales notification e-mail address I mentioned as being rolled-out on September 26th as a part of my last general SLMP update? Guess what was rolled back just 48 hours later, only to be rolled-out once more on October 4th?

One has to question a) the level of competence within those responsible for managing and coding SLMP; and b) the overall condition of the Marketplace code itself, as it seems utterly incomprehensible that even the most basic issues within the system appear to be beyond LL’s grasp to fix.

In his comment on the matter of listing enhancements, Darrius concludes:

Communication from your team to us is a major issue. I’ve no doubt why this is the case. Most people have a very difficult time going to others with the need to say “We’re sorry, we screwed up.” With the number of times you must begin a blog post in that manner, it’s no wonder you don’t post very much at all. So here’s an idea … stop being lazy, stop short-cutting things and rushing changes into production, stop screwing up .. and STOP having to begin every post with an apology.

While I agree with his point of view, I’d go a step further.

It doesn’t matter as to whether or not these issues are only affecting a “small number” of merchants (as the Commerce Team have repeatedly stated); it also doesn’t matter as to whether LL regard L$ as “real money” or “tokens”.

What matters is that the company actively encourages people to get involved in their platform’s commerce engine, and to invest time and money in it – and they promote the Marketplace as a major means for people to do so. People have taken LL at their word, and for many of those affected by all the Marketplace screw-ups over the years, it very much is the case that real money is involved, and real stress and real upset.

As such, it is time for someone within Linden Lab to recognise this, take responsibility and step forward with a sincere apology for the manner in which the entire litany of mistakes, errors and mishaps going back as far as at least 2010 has been handled. They then need to go on to ensure issues are managed in such a way that people are kept properly informed on progress, and that issues are not exacerbated by what appears to be either flaws in internal processes – or carelessness.

Simply saying people are busy “crunching numbers” doesn’t really cut it any more.

As it is, a decent projection as to when LL will “have a fix” for Marketplace problems, would appear to be, “Around the 12th of Never”.

Related Posts

Run silent, run deep: the SL Marketplace and eroding merchant trust

Update October 4th: Linden Lab have issued a forum post on this matter, please see LL updates on listing enhancements.

Linden Lab depend on land tier (server space, call it what you will) for  80%(ish) of their revenue. This places them in an awkward position vis-à-vis providing any form of tier easement, even if they wanted to (as I’ve commented before).

The remaining 20% comes from the likes of Premium membership, and more particularly, the SL Marketplace (SLMP). In the case of the latter, the revenue doesn’t only come from the 5% commission on goods sold through the Marketplace, a lot of it comes via the listing enhancements merchants are encouraged to pay for. These theoretically boost sales by placing items in places such as the SLMP home page, or on the Checkout pages, and are paid for on a 30, 15, or 7-day rolling subscription basis, costing merchants between approximately $5.00 and $12.00 USD a month per item, with some merchants paying over $200 USD per month for enhancements.

Listing enhancements – can amount to a pretty penny in outlay per item

Earlier this year, the system went haywire, failing to take due subscriptions for around a two-week period (JIRA WEB-4638). It was finally resolved by Linden Lab taking a single, large payment from merchants’ accounts. While people had no issue in paying for services rendered, the problems here were that a) next to no forewarning was given that accounts were about to be so debited, leaving many merchants with a sudden and unexplained drop in their account balances; and b) many were billed in excess of the two weeks subscriptions actually owed (with some reporting being billed for up to four weeks); while c) the billing information received made it hard for merchants to actually determine which of their listing enhancements had been billed, or even if the right enhancements had been billed.

This understandably led to some confusion within the merchant’s forum, and not a little upset, particularly as some merchants had also been faced with an inability to cancel some of their listing enhancements due to an ongoing issue with many items remaining stuck in a “locked” mode, preventing them from being edited – a situation itself which at the time was some 18 months old (and is now some two years old, and still awaiting resolution – see WEB-2974).

While merchants were refunded for any overcharging on their account as a result of the billing issue, the manner in which the situation was handled by LL resulted in something of a drop in trust where the Marketplace is concerned, and a number of merchants publicly indicated they would be ceasing in their use of enhanced listings.

At the end of July, the problem started again, and was raised as a topic for discussion in mid-August, as well as having a new JIRA (WEB-4927) raised against it. Neither the discussion thread nor the JIRA drew comment from the Commerce Team. Instead, a single payment was taken from all “at fault” accounts, again without any forewarning, and again with Merchants facing issues over what, precisely, they have been charged for, and whether those listings they have made payment against are actually active.

Failed subscriptions for August – courtesy Ry0ta Exonar

Again, the problem here is not so much that things Went Wrong and broke again – that’s pretty much taken to be the standard operating condition for the Marketplace nowadays – but how the Commerce Team managed the issue. Almost nothing was said on the matter (again), with the only communications forthcoming from the Commerce Team being a terse instruction not to re-open WEB-4638 after Ry0ta Exonar attempted to do so (hence WEB-4927), and a brief Marketplace dashboard message posted on September 28th, which simply said:

We are aware of some issues with Product Listing Enhancements. Keep an eye on the Grid Status page for more details.

With neither the dashboard message or Status page message actually stating what was about to be done.

So is it little wonder that merchants are again looking at listing enhancements with a jaundiced eye? Several have re-stated the fact that they will no longer participate in the process and others have stated they are terminating – either automatically or manually – their subscriptions. Given that LL are seeking to increase their non-tier related revenues, one would think that ensuring the one service which does so is run with a level of professionalism and communication that would not undermine customers’ faith in the service, or their willingness to place money into it.

Currently, and added to the rest of the ongoing litany of issues and problems with the Marketplace, this doesn’t appear to be the case.

Related Links

Of blogs and sundry thoughts

The Second Life Blog was once a place where the Lindens talked casually with you about policy, their projects, recent news, the future of SL, etc.  Residents regularly told us that they loved having access to such broad insight into the company and frequent communication with the full range of Lindens.  And Lindens loved the ongoing dialog with residents.

Sounds like something I might have said here – or you might have read from Tateru or a dozen other SL commentators. A harkening-back to the “good ol’ days”.

But it’s not. It’s actually from – wait for it – a Linden, who went on to say:

Over time however, as more Lindens came to participate, the blog got a bit manic. Some of you complained that reports of temporary performance issues would eclipse larger conversations related to long-term plans and features while others believed that tutorials and opinion pieces were distracting them from the hard news of inworld issues they needed to know about in order to run their businesses.

In other words, we outgrew our single channel blog […] We knew it was important to get back to using the blog as a key means of constructive two-way conversation with the community.

“We knew is was important to get back to using the blog as a key means of constructive … conversation…” How times have changed, hmmm?

These quotes come from the Linden Lab blog archive on WordPress. Written by Blue Linden (sadly gone in the re-organisation of June 2010), they demonstrate how much attitudes have changed within LL over the last three years.

The LL blog archive on WordPress

Continue reading “Of blogs and sundry thoughts”

Grid Maintenance: all silent on the Linden front

On Friday, I drafted a piece on the lack of direct feedback from Linden Lab vis-a-vis the recent rounds of maintenance carried out on the 8th, 9th and 10th May. I held off publishing  because I opted to wait until Close of Business in California to see if anything would be forthcoming. Then I got involved in other things, and completely pushed the post out of my head.

In the meantime, Tateru has, with her usual incisiveness asked the question I was planning to ask, but far more succinctly and directly. It’s a good read, and I suggest that if you haven’t you go read it – if only to save yourselves from hearing the same old, same old from me.

Elsewhere – well, Nalates Urriah’s blog to be precise – it may appear that the poor state of communications isn’t restricted to the amount of information flowing out of the company to the users, but many extend to internal communications within the Lab. In her piece, and in reference to the maintenance periods, she reports that Andrew Linden commented:

I think part of that is an operating system upgrade on some hosts, not network level maintenance, but I’m not sure. We’re definitely working on migrating to later versions of Debian, but there will be a few upgrades along the way before we arrive at Debian/Squeeze.

Andrew, together with many of the other Linden staff involved in User Group meetings usually try to be as informative as possible. Following the outage of the 26th April, for example Oskar attempted to provide some explanation via the forums, as Jack Abraham pointed out on this blog, and Daniel Voyager later reported via his.

So when you do see normally forthcoming staff using terms like “I think,” and “I’m not sure,” when describing a situation that impacts the entire SL service, it’s a little disconcerting, and might be seen as suggesting internal communications at LL may be lacking. After all, Andrew is a part of a group of LL staff who do face users, and who are likely to get asked the hard questions when something major is going on, so you’d perhaps expect the company to ensure they are furnished with sufficient information to be able to field such questions with a measure of clarity.

There was a time when we would see blog reports related to upcoming service improvements, and posts on outage postmortems. But no more. Instead, we’re left hanging or having to resort to scrabbling around the forums in the hope that someone has had the foresight to say something. And while it is great that news is frequently passed out at User Group meetings, not everyone attends these, so the message tends to go more unheard than heard.

Simply put: there is no substitute for clear, open communications – and however you look at it, the SL blogs are the best place to communicate when it comes to major announcements and service issues. In these cases, all other channels should point back to the blog, not used in place of the blog. As Tateru points out, the Grid still isn’t the most stable of environments following the recent work; we have no idea as to what the general status of the grid is or what is going on – and that leaves us guessing.

Going on Andrew’s comments, we may not be the only ones. Either way, it would be nice if someone at Linden Lab stepped up to the keyboard and took a little time to let their users and customers know exactly what is / has been going on, and how things stand.

Dear Rod Humble…

Dear Rod Humble,

It is now some ten months since I last wrote to your company regarding its apparent inability to keep customers informed as to issues and problems impacting the service it provides. And while it may appear presumptuous of me to do so, I feel compelled to now write to you directly.

In that missive, I made mention of the fact that once upon a time, whenever there were problems, or when maintenance – planned or otherwise – was about to commence, Linden Lab would push out an in-world notice. As I said at the time:

It was informative; it was helpful; it was reassuring to know you guys were out there, keeping an eye on things and letting us know what was going on. It gave us a nice warm fuzzy feeling inside. In short, it was communicative.

And then one day it stopped, leaving us with no option but to find out about Things Going Wrong or that planned maintenance had started by experiencing it the hard way: through teleports failing or transactions going astray or No Copy items poofing into the ether, never to be seen again.

I’m not alone in feeling the in-world notifications need to be re-instated, and I drew your attention to this fact at the time of my initial letter:

At the same time we have seen what amounts to something of an erosion in the use of the Grid Status page. Once, matters pertaining to the grid were displayed directly on the Viewer splash screen, up in the top right corner with other useful information. For some reason never really clarified, they were removed. I commented on this to you directly about this on a couple of occasions, specifically with regards to the “new” log-in splash screen introduced around the time of SLCC2011. While your initial response was non-committal…

…You did seem to respond more positively when I raised the issue later in 2011:

Yet here we sit, almost a year on from my original letter and our exchanges, and nothing has changed. This fact was brought sharply into focus by the outage which occurred on the 26th April 2012, and the terse explanation that was eventually given for it happening.

The matter appears to have been the result of “unscheduled maintenance” – although the subsequent explanation released on the 27th suggests that the risk of it causing problems may have been anticipated. To me, the use of the term “Triggered a bug” – rather than say, “Resulted in a bug” – suggests there was a known issue / risk here, even before the maintenance commenced. Was this perhaps the cause for the maintenance in the first place? But I digress into speculation. Whether or not the potential for issues arising from the work was anticipated ahead of time, the fact remains that even as unscheduled maintenance, there was an opportunity to inform users of what was about to happen ahead of time.

Indeed, can you not see how much better it would have been if there had been an in-world broadcast that the work was about to commence? While such a broadcast would not have prevented the subsequent outage, it would have given fair warning to those already in-world and encouraged them to proceed with care, rather than people suddenly and unceremoniously booted out of SL and bewildered as to why. Of course, you did provide the log-in warning for people attempting to log back in to SL – but really, that was pretty much akin to saying to someone, “This may hurt,” after you’ve suddenly kicked them in the shin.

It is hard to fathom why Linden Lab appears determined not to re-implement such warnings – and I can only take it as a determination on your part, given that a) it’s almost a year since the idea was mooted both personally with you and with the likes of Viale Linden and despite the positive feedback, nothing further has happened; and – more particularly and relevantly – b) it appears that the JIRA (VWR-20081) from Marianne McCann remains unassigned. Further, given the lack of feedback following Oz’s comment from May 2011, it would appear that “support and ops” simply weren’t interested enough in the idea to warrant any such feedback; which in itself could be seen to speak volumes.

There can’t be any technical issues as to why in-world notices cannot be re-implemented; after all, they are used to give warning during the weekly server roll-outs. This being the case, one can only assume that in-world notices are not used is down to a complete lack of interest / concern on the Lab’s part, and the same holds true for providing links to the Grid Status page on the Viewer’s splash screen.

I know that from our direct exchanges that you feel you have, as CEO, been far more communicative than your predecessors at Linden Lab – and I’m not about to deny the fact that you have. But reaching out on Twitter or Plurk  – as welcome as it is – is no substitute for ensuring your company is fully and properly engaged in the process of communicating with its users through the channels that are most likely to reach the majority of said users.

To be more succinct: when it comes to keeping people informed of matters of import that may impact the Second Life platform, the most appropriate place for Linden Lab to communicate with its users is through the platform – Viewer and website. Everything else should be seen as a secondary or back-up means of getting the word out.

So again, why oh why do you, as a company, refuse to accept this?

No-one expects Linden Lab to handle everything perfectly; there will always been times when the unpredictable and/or the unexpected happens. There will be times with the best will in the world, the sky falls in or SL simply blows a raspberry at everyone and disappears up its own left nostril. We don’t expect you to be superhuman in your efforts to communicate.

But we do ask that the company communicates, and does so through the channels most likely to reach the majority of your users. When it comes to notifying us of the need for grid maintenance – whether it is scheduled or not – or in informing us of issues that may impact our use of the platform, then the channel most likely to reach the majority of your users is in-world notifications. As such, it’s hard not to interpret the ongoing refusal to make any attempt to do so as anything less than a cavalier disregard as to how users might be affected by either the need for immediate maintenance or by known issues.

And frankly, we deserve better than that.

Yours sincerely,

Inara Pey.