It’s been a busy time at Amazon’s AWS Re:Invent conference, which closes in Las Vegas USA on December 1st. At the start of the event, Amazon announced the launch of their VR / AR development / publishing platform Sumerian (see: Sumerian: Amazon’s VR / AR app building platform for more).
Meanwhile, on November 28th, and potential of more interest to Second Life and Sansar users, the event saw Tara Hernandez, Senior Director of Systems and Build Engineering at Linden Lab, give a presentation covering Sansar and touching on plans for Second Life, entitled How Linden Lab Built a Virtual World on the AWS Cloud.
Most of the video delves into the intricacies of building a complex platform like Sansar and how Amazon’s products have empowered the Lab. As such, it does come across as quite a dry listen; however, within it there are some useful areas of focus which are worth noting.
For example, the early part of Tara’s presentation touches on some core truths about Second Life. Such as the fact it is a platform now 14+ years old, which started as an environment engineered almost down to the bare metal, taking advantage of what were, at the time, deep-seated optimisations in graphics and networking capabilities.
Over time, these have not only been layered upon almost organically over the years, but have also become – in Tara’s own words – “kinda ugly” in terms of trying to maintain and enhance. This monolithic, deeply rooted approach to the core elements of the platform is – along with the user-driven expectation than the user-generated content within the platform will not break as a result of changes to the platform – one of the major reasons why “updating” Second Life isn’t simply a matter of JFDI, as might be thought.
Aspects such as compliance – another issue which is perhaps a lot more complicated than many might appreciate, given the complexities involved in running services like Second Life and Sansar, where the ability to cash out money adds a lot of additional regulatory overheads (visible and invisible from a user’s perspective) over platforms which only allow users to pay-in.
The video also reveals the depth of the relationship between Linden Lab and Amazon, which in the case of Second Life, stretches back to 2008, and which has encompassed the Lab’s other product, Blocksworld. In particular, it touches on Linden Lab using (and sometimes breaking!) Amazon’s more recent offerings, such as their ECS services, as a beta customer. This is something that Amazon has itself highlighted, featuring Linden Lab and Sansar in one of their own ECS use-case studies (see my article “Project Sansar”: an Amazon ECS case study, from January 2016).
ECS in fact drives almost all of the Sansar back-end, from the Atlas through to the store. In particular, the way in which the ECS application layer is used to present the Sansar Atlas, and manage the entire management of the experiences offered by the Atlas and their instancing, utilising Amazon containers (see 27:40-30:58).
What’s interesting here is not only the way in which Amazon’s services are being used, but in understanding what is going on from the moment a Sansar user clicks the Visit button in the Atlas, and the lessons the Lab are learning even now, as people use Sansar.
This latter point is itself of interest, as it helps to explain why Linden Lab opened Sansar up to wider audience in what seemed to many of us familiar with virtual space – myself included – to be a premature move. Simply put, they needed more of a flow of people moving through experiences to better judge how experiences can be more efficiently / effectively managed within the Amazon environment – spinning them up / down, instancing, optimising server use, etc.
In terms of Second Life, perhaps the most interesting part of the video can be found at 32:14-34:36, with a look at the recently announced attempts to move all of the Second Life service – including (eventually) the simulators, if possible – the cloud. Officially announced as a project in August 2017, but has been discussed at various in-world meetings such as the TPV Developer meetings.
In particular, the presentation touches on one of the major reasons for attempting the move: costs. Right now, Second Life is dependent upon hardware the Lab has to source and operate through a data centre. Updating this hardware, and the underpinning infrastructure – network, fibre, rack space, etc., – requires continuous and high levels of expenditure (even allowing for re-purposing / write-down of old equipment).
There are also limits, as touched upon in the earlier part of the video, on what can be done within specific areas of Second Life support and maintenance. For example, Tara specifically mentions the core database services (which have been subject to numerous issues over the last year plus). While recovery times for these services has been halved – from three hours to 45 minutes – it is still a considerable outage period from the users’ perspective, and one difficult to bring down further.
Thus, an attempt to move Second Life to AWS could resolve a lot of issues for the Lab, and potentially allow them to leverage lessons learned with Sansar together with the capabilities of newer services – like ProxySQL – to further update and improve SL. It might also allow the Lab to move their database operations away from MySQL to more robust products, again following Sansar’s lead.
The shift of a platform from being data centre centric to cloud based is obviously non-trivial, and involves considerable challenges, some of which are outlined by Tara (above). However, from the comments she makes, she is anticipating possibly a dramatic level of progress over the next year. If so, it could be an interesting twelve months.
With thanks to Dassni – The Mesh Cloud for the Twitter pointer to the video.