2020 SL project updates week #14: TPVD summary

The Muse – The Library, February 2020 – blog post

The following notes are taken from the TPV Developer meeting held on Friday, April 3rd, 2020. These meetings are generally held every other week, unless otherwise noted in any given summary. The embedded video is provided to Pantera – my thanks to her for recording and providing it. Time stamps are included with the notes will only the video at the point(s) where a specific topic is discussed.

This was a short meeting – less than 20 minutes.

SL Viewer News

[0:13-2:43]

There were no viewer updates in week #14, leaving the official viewer pipelines as follows:

  • Current Release version  version 6.3.8.538264, dated March 12, promoted March 18th. Formerly the Premium RC viewer – No change.
  • Release channel cohorts:
    • Camera Presets RC viewer, version 6.3.9.538729 March 25.
    • Love Me Render RC viewer, version 6.3.9.538760, March 25.
    • EEP RC viewer updated to version 6.4.0.538823, March 20.
    • Zirbenz Maintenance RC viewer, version 6.3.9.538719, issued March 19.
  • Project viewers:
    • Copy / Paste viewer, version 6.3.5.533365, December 9, 2019.
    • Project Muscadine (Animesh follow-on) project viewer, version 6.4.0.532999, November 22, 2019.
    • Legacy Profiles viewer, version 6.3.2.530836, September 17, 2019. Covers the re-integration of Viewer Profiles.
    • 360 Snapshot project viewer, version 6.2.4.529111, July 16, 2019.

General Viewer Notes

  • EEP is now extremely close to release. The hope is to have the final RC version available for users in week #15 (commencing Monday, April 6th).
    • Providing no major issues are encountered with that version, and allowing for it gaining sufficient user hours as an RC cohort, it will then be promoted to de facto release status.
    • This viewer still has one of the lowest crash rates for the official viewer, and which is described as being “dramatically lower” than the current viewer release.
  • The Love Me Render (LMR) and Camera Presets RC viewers are both getting close to a point where they could be released at some point after EEP.
  • Tools update (Visual Studio 2017 and a more recent version of Xcode): the first full viewer build is ready to be issued, so an RC could be appearing in week #15. If so, it may be fast-tracked to release status behind EEP and ahead of other RCs.
  • There is still work to be done on the Copy / Paste and Legacy Profiles viewers to get them up to RC status.
  • Work is also continuing on the mesh uploader viewer, a version of which had yet to be made available to users as a compiled viewer.

Server / Simulator News

[3:44-6:42]

  • The server team believe they have fixes for the issue off-line inventory losses from objects (see: BUG-227179 “All offline inventory offers from scripted objects are STILL lost”).
  • These fixes should be going to a simulator RC release in week #15, and no viewer-side updates are required for either of the fixes (UDP and HTTP).
  • TPVs have been asked to confirm the HTTP fix works, and if so, to switch to that mechanism (if they have not already done so), rather than continuing to rely on UDP messaging for off-line inventory offers, so that path can be deprecated.
  • Details on where the fixes can be tested will be made available to TPVs through the Open-Source Dev mailing list.
  • Apologies have been offered for the time it took LL to fix and fix the underlying causes.

Lab Gab 20 summary: Second Life cloud uplift & more

via Linden Lab

The 20th edition of Lab Gab live streamed on Friday, April 3rd, featuring Oz Linden, the Lab’s Vice President of Engineering and a member of the company’s management team, and April Linden, the Lab’s Systems Engineering Manager. They were appearing to primarily discuss the work in transitioning Second Life to commercial cloud environments. Ekim Linden had also been scheduled to appear, but was unable to do so.

The official video of the segment is available via You Tube, and is embedded at the end of this article. The following is a summary of the key topics discussed and responses to questions asked. Note that the first half of the video is related to the cloud uplift, and the second half to broader engineering-related questions.

April Linden has some 20 years of experience in systems engineering, and is genuinely passionate about Second Life. She first became involved in the platform in 2006 as a resident (and is still extremely active as a resident). She joined the Lab in 2013. She worked within the systems engineering team, and was promoted to her current position of Systems Engineering Manager, Operations, some 18 months ago. For her, the great attraction of the platform has been, and remains, the empowerment it gives people to express themselves positively.

Oz Linden joined the company in 2010 specifically to take on the role of managing the open-source aspects of the Second Life viewer and managing the relationship with third-party viewers, a role that fully engaged him during the first two years of his time at the Lab. His role then started expanding to encompass more and more of the engineering side of Second Life, leading to his currently senior position within the company.

Both are genuinely passionate and enthusiastic about Second Life and its users.

The bunny and the wizard who bring us Second Life: April Linden (Systems Engineering Manager, Operations) and Oz Linden (Vice President, Second Life Engineering)

The Cloud Uplift

What is It?

[5:40-9:45]

  • Cloud Uplift is the term Linden Lab use for transitioning all of Second Life’s server-based operations and services from their own proprietary systems and services housed within a single co-location data centre in Tucson, Arizona, to  commercial cloud services provided by Amazon Web Services (AWS) and Google.
  • The process of moving individual services to the cloud is called “lift and shift” – take each element of software, making the required adjustments so it can run within a cloud computing environment, then relocate it to cloud infrastructure and hardware in a manner that allows it to keep running and avoids disruptions that may impact users, and continues to run exactly as it did prior to the transfer.
  • The current plan is to have all of this work – up to an including moving all of the SL region simulators – to cloud services by the end of 2020.
  • Numerous services have been transitioned to date.
    • The Lab generally prefers not to discussion which specific services have been moved, to prevent users seeing the move as a placebo reason for issues they may be encountering, thus biasing their bug reports.
    • However, one service that is known to have moved is the inventory (asset) database, so that all users’ inventories are obtained via the cloud, and not from a dedicated asset cluster within the Lab’s co-lo facility.
  • With the services that have moved, the Lab has seen noticeable improvements in performance, partially as a result of cloud services using more recently / more powerful hardware configurations than the Lab can run without making a major new capital expenditure in equipment (which the uplift is intended to avoid).
  • A practical advantage of cloud operations is the ability for LL to scale services to meet demand.  The recent increase in users logging-in to SL, for example, placed a strain on the services that feed the CDNs that in turn deliver the majority of asset data to users (mesh data, textures, sounds, gestures, clothing, etc.). These services we then able to dynamically scale to an increased number of nodes to handle the load, something LL would not have been able to do without first sourcing, installing ans configuring the required hardware.
Oz and April with Strawberry Linden (c)

What Improvements Might Users See from the Uplift?

[9:48-14:42]

  • Between now and the end of 2020, no appreciable different should be observable to users.
  • The move is initially being made to a single AWS centre, so things like ping times to regions (once they are moved) shouldn’t change.
  • In terms of reducing simulator-side lag, the answer is unclear, as simulators have yet to be tested – this is due to start with simulators internal to the Lab Soon™. This will enable the Lab to begin to get real numbers in terms of simulator performance.
    • It is believed that simply moving simulators to the more recent, more powerful hardware used by cloud services should on its own result in a modest improvement in simulator performance.
    • That said, the outcome of performance adjustments in distributed environments is “really, really hard to predict”.
  • Longer-term, as the Lab is able to start exploiting the advantages of being in the cloud, there is confidence performance will improved in various areas.
    • For example, if simulators can be distributed in accordance with the geographical locations of their primary audiences (e.g. simulators that tend to get the majority of their audience from South America being located in South America), then this could reduce network time in connecting to them for those audiences, and so help boost performance as seen by those users.
    • While this is a longer-term goal for the cloud migration (it’s not going to be there from “day 1”), it is a part of the motivation to make the transition.

How will the Lab Handle Costs?

[14:45-18:40]

Sidebar note: cloud services typically bill based on demand and usage. This has given rise in some quarters to concerns / beliefs that LL could find themselves facing unexpected large bills for hosting.

  • Two answers: the first is nothing is ever certain.
  • The second is, the Lab, with April and Ekim in particular leading the effort, put a lot of work into modelling their likely operations and costs when using cloud services and infrastructure.
    • This work involved a lot of assumptions on how LL anticipated their costs would look based on how the planned to operate SL in a the cloud.
    • This model was then put to both AWS and to an independent, outside consultancy with expertise in advising clients on the use of cloud-base service provisioning, both of who gave positive feedback on the approach the Lab would be taking and the likely costs involved.
  • Further, the fact that SL isn’t a service that dynamically expands under use. All of its services are operating 24/7, so the costs can be readily calculated and pretty much consistent, therefore, the dynamic surges that can lead to high service bills don’t actually apply.
  • While there are some back-end services that can leverage dynamic hardware use in times of heavy load, these are in the minority (all of SL’s back-end services account for only 15% of its server fleet), so again, dynamic increases in hardware use for those services that can leverage it, are not going to be massively excessive.
  • As such, and allowing for answer (1), the Lab isn’t overly concerned about costs spiralling.

Will There Be Cost Saving that Can Be Passed to Users?

[18:41-19:54]

  • Unfortunately, the engineering teams are not responsible for determining fees charged to users.
  • More practically, it is not going to be possible to make any informed judgements on costs to users until the Lab has had the opportunity to see how actual operating costs compare with their predicted costs model.
  • Further, it is not anticipated that any cost savings will be made in the first 1-2 years of cloud uplift, so any decisions on if and where to reduce costs to users won’t be made for a a while to come, and those involved in making such decisions are not in the engineering teams.

Second Norway and Sailor’s Cove East – status update

Second Norway, March 2020

I recently reported (with updates) on the situation with Second Norway and Sailors Cove East (SCE), both of which were facing possible closure due to physical world issues, including the current SARS-CoV-2 pandemic – see: Second Norway & Sailor’s Cove East: rumours & statements.

On Friday, April 3rd, Ey Ren, founder of Second Norway contacted me to request that people wishing to keep up-to-date with developments on both estates refer to his Bad Elf Blog, and I’m only too happy to point those concerned about the situation to that blog.

In particular, Ey has posted three updates, all dated April third, and summarised below:

24 SCE regions to Transfer Ownership

A transfer ticket for 24 of the 45 Sailor’s Cove East regions has been submitted today. Original co-founder of Sailor’s Cove, Patrick Leavitt, has stepped up to ensure that over half of the estate continues to exist.

See Ey’s full blog post on this topic, which includes a list of the affected regions.

Ey is Still Working to Secure a Future for Second Norway and the Rest of SCE

As per the notice presented by Mialinn Telling in her profile (again, see: Second Norway & Sailor’s Cove East: rumours & statements), Ey is seeking to secure a viable future for the estates and the regions within them. In particular he notes:

  • Outside of the SCE regions that will be transferred to Patrick mentioned above, there should be no significant changes to either estate before May 2020. In particular:
    • No regions should be taken off-line in April 2020.
    • Rental payments are suspended until such time as the future of the regions within each estate is determined and / or regions are transferred to new ownership (at which point rental agreements will need to be entered into with any new owners).
  • Ey is actively engaged in seeking new ownership to secure the future of as many regions as possible, and investigating the means to finance those regions which cannot be transferred to new owners. However, due to his personal situation, it is possible that some regions that cannot be transferred to new ownership could eventually be removed from the grid.
  • He also wishes to extend thanks to Linden Lab for all they have done in expediting the unlocking of his account and in providing leeway for him to seek alternative arrangements to try to save as much as possible of both of these estates.

For full details on all of the above points please refer to Ey’s posts All Good Things Must Come to an End and No Tier Payments Until Changes are Made.

 

Again, if you are a tenant of Second Norway or SCE, or wish to be kept appraised of the situation directly be Ey, please make sure you bookmark or subscribe to his blog.

2020 Content Creation User Group week #14 summary

Garrigua, February 2020 – blog post

The following notes were taken from my audio recording and chat log of the Content Creation User Group (CCUG) meeting held on Thursday, April 2nd 2020 at 13:00 SLT. These meetings are chaired by Vir Linden, and agenda notes, meeting SLurl, etc, are are available on the Content Creation User Group wiki page.

A large part of the meeting concerned options for what might be done when handling complex avatars that fall outside of what is currently being done through ARCTan, including esoteric discussions on when things like impostering should occur in the download / rendering cycle, etc. Discussions also touched on the sale of Sansar (see elsewhere in this blog) and SL’s uptick in user numbers as a result of the current SARS-Cov-2 pandemic.

Environment Enhancement Project

Project Summary

A set of environmental enhancements (e.g. the sky, sun, moon, clouds, and water settings) to be set region or parcel level, with support for up to 7 days per cycle and sky environments set by altitude. It uses a new set of inventory assets (Sky, Water, Day), and includes the ability to use custom Sun, Moon and cloud textures. The assets can be stored in inventory and traded through the Marketplace / exchanged with others, and can additionally be used in experiences.

Resources

Current Status

  • Is caught on a couple rendering bugs related to Linden Water and how the water / things under water are rendered by EEP.
  • The plan is still to have EEP promoted before any other viewer project is promoted to release status.

ARCTan

Project Summary

An attempt to re-evaluate object and avatar rendering costs to make them more reflective of the actual impact of rendering both. The overall aim is to try to correct some inherent negative incentives for creating optimised content (e.g. with regards to generating LOD models with mesh), and to update the calculations to reflect current resource constraints, rather than basing them on outdated constraints (e.g. graphics systems, network capabilities, etc).

As of January 2020 ARCTan has effectively been split:

  • Immediate viewer-side changes, primarily focused on revising the Avatar Rendering Cost (ARC) calculations and providing additional viewer UI so that people can better visibility and control to seeing complexity. This work can essentially be broken down as:
    • Collect data.
    • Update ARC function.
    • Design and provide tool within the viewer UI (i.e. not a pop-up) that presents ARC information in a usable manner and lets users make decisions about rendering / performance.
  • Work on providing in-world object rendering costs (LOD models, etc.) which might affect Land Impact will be handled as a later tranche of project work, after the avatar work.
  • The belief is that “good” avatar ARC values can likely be used as a computational base for these rendering calculations.

Current Status

  • Internal testing is awaiting a Bake Service update related to the issue Vir identified that was causing issues in gathering data.
  • In the interim, Vir has been looking at the tools available for manipulating viewer performance (e.g. imposters, the Jelly Dolls tools, blocking, etc.). He’s specifically been looking at “peculiarities” in how the various options work and raising internal questions on possibly re-examining aspects of how they work.
  • One point with imposters / Jelly Dolls is that while the settings may be used – and as was raised as a concern prior to that project being deployed – is that rendering data for all attachments on an impostered or jelly dolled avatar is still downloaded to the viewer, which is not optimal.
    • Removing attachment data could improve performance, but would also make jelly dolled avatars in particular look even more rudimentary.
  • A bug with the  Jelly Doll code means setting an avatar to never render causes it to load more slowly than just lowering the complexity threshold so it doesn’t render. This is viewed as a known bug.
  • There have been suggestions for trying to limit access to regions (particularly events) based on avatar complexity.
    • Right now, this would be difficult, as the simulator does not have authoritative information on avatar complexity – it’s calculated in the viewer, which in turn is based on data the simulator doesn’t even load.
    • This means there would have to be a significant refactoring of code before the simulator could be more proactive around avatar complexity. Given the cloud uplift work, this is not something the Lab wishes to tackle at this point in time.

General Discussion

  • Arbitrary skeletons: The question was raised on SL allowing entirely custom / arbitrary skeletons.
    • This again would be a complex project, one that was rejected during the Bento project due to the risk of considerable scope creep.
    • There is already a volume of available humanoid mesh avatars, each operating with their own (mutually incompatible) ecosystems of clothing and accessories that can already cause confusion for users. Adding completely arbitrary skeleton rigs to this could make things even more complicated and confusing.
  • The major reason there is little work being put into developing new LSL capabilities is because the majority of the LSL development resources are deeply involved in – wait for it – cloud uplift work.

Next Meeting

Due to the Lab’s monthly Al Hands meeting, the next CCUG meeting will take place on Thursday, April 16th, 2020

Sea Brook’s haven in Second Life

Sea Brook, April 2020 – click any image for full size

A full region with the 10K bonus LI, Sea Brook is a remarkable setting that offers a stunning location that forms a rich, eye-catching, highly-photogenic haven of a destination that offers a tour de force of what can be achieved with vision and considered execution in region design in Second Life.

The work of Muira (Angelique Vanness) on behalf of Rahnn Parker (Rahnn) and Carrie Parker (Cari2017), the region is a tour de force demonstration of Muira’s remarkable eye for region design, something I first noted in 2019 after visiting Season’s Cove (now closed, but see The magic of Season’s Cove in Second Life). As with that design, this is one that again feels far bigger than its 256m on a side size. In this instance, the sense of size and space is made all that more remarkable by the fact that much of the centre of the region is given open to open water.

Sea Brook, April 2020

The water takes the form of an extensive lake fed by falls that drop from a massive up-thrust of rock that rises to the north-east of the region in great granite or basalt blocks, topped by high fir trees. A broad, paved footpath winds its way around the lake’s shoreline in a loop, connecting three small terraces that thrust their own out into the clear blue waters. One of these terraces  forms the regions landing point, whilst all three present impressive views over the lake. At one end, this footpath connects to an imposing lodge that whilst grand in size, utterly fits with its surroundings. To the other end the path gives way to a rocky path – one of two in fact – that switch-backs up to the top of the high plateau.

Between the lake and the waters beyond the edge of the region, the land is entirely-low-lying with the exception to the huge plateau. Theses lowlands are rich is detail and  – if I might use the term again – present an expansive setting. Rich in tall Scots pines, they are marked by gravel tracks that run around the outside of the paved path around the lake, the woodlands between pavement and gravel cut through with winding trails that allow visitors to wander and discover all that lies under the shade of the trees: ponds, little camp sites, a children’s playground, picnic spots – the list is extensive without – the setting ever feeling crowded.

Sea Brook, April 2020

The paths also provide links to other locations within the region. These include a west side beach, tucked between two headlands. One of these is home to the ruins of an ancient church that now offers a cosy retreat. A second, intact chapel forms a book-end to the ruins, sitting on a low hills on the other headland, resting atop a low hill that allows it to look north across the beach towards the ruins of its companion.

East and south, behind the great lodge – which appears to be open to the public and itself offers an impressive place to explore – the land opens a little as at sits between rocky highlands and a growth of mangroves that surround one of the smaller islands sitting just off the coast. This little island is home to an old gazebo that offers a place to dance. Across the narrow channel separating the gazebo from the lodge, sits a little fenced meadow, a place where visitors can rez a horse to ride around the region – something that is well worth doing.

Sea Brook, April 2020

Atop the plateau there is yet more to discover, the switched paths leading up to it connected one to the other by gravel trails that wind across the plateau, separating the woodlands to offer obvious paths for people (and horses) to follow and which take visitors past table-top games, and along an arched path to another dance area that offers an elven theme.

As with the lands below, the plateau is also cris-crossed by wooded paths that reveal more secrets among the trees, and which I’m not going to spoil by mentioning here. However, I will say that look carefully enough and you will find a zipline that runs down to the little finger-like island rising from the middle of the lake and where bumper boats can be rezzed by those looking for a little fun.

Sea Brook, April 2020

Nor is this all; below the plateau, and nestled in the roots of the cliffs, are wooden doors awaiting discovery. They lead to a network of tunnels and chambers that run through the rocks from on side to the other. With paved floors and faced stone walls, these tunnels and the halls and rooms that open off of them make for an intriguing point of exploration on their own; one looks like a former wine cellar, others present more intimate spaces.

A truly stunning design, Sea Brook is absolute perfectly set within the encircling region surround of high mountain peaks that – with the right windlight – give it tremendous depth, this is not a setting to be missed. It has a huge amount to discover (I’ve only scratched the surface here), and is finished with a matching sound scape.

Sea Brook, April 2020

SLurl Details

Mapping Second Life’s mainland railways

via Linden Lab

The Second Life Railroad network is one of the major features of Second Life mainland – notably Heterocera – with lines also to be found on south and east Sansar and on Bellisseria, with number smaller (and private) lines also to be found across Second Life (such as the system in Second Norway).

Much as anyone can operate a car on the roads in Second Life, any Resident may use this public facility for any purpose consistent with the Second Life Community Standards/TOS (and at times the Lab has provided automated rail services). However, finding your way around the network can be a tad confusing, so enter 由里子 (Rydia Lacombe).

Not only is Rydia the creator of the sci-fi themed Aoshima, a homestead region Caitlyn and I enjoyed visiting in February (see: Beaming in to Aoshima in Second Life), she is a SLRR enthusiast, and something of a cartographer. She recently sent me her most recent map charting the major Linden-supplied train routes across and around Heterocera, Sansar and Bellisseria – and it is an impressive piece.

Rydia is a keen SL rail-roader, something she noted to me as we chatted about the map.

It’s what keeps me exploring! Mostly it stems from the time the WARR started building in front of the Burns freebie warehouse. I made my first SLRR railway maps in 2013 / 2014. I don’t have a formal means of distributing the maps, except through the various groups. The Virtual Railway Consortium [VRC] distributed my previous maps, but they don’t appear to be quite as active now.

– Rydia talking about her SLRR map work

WARR is the West Atoll Railroad, an electric railway line founded by Hilto Meridoc in 2010 and operating in southern Heterocera Mainland. The line opened in stages, first from Neumoegen to Electra, then east to Hera and west to Elpenor, before eventually closing in 2013, although I understand a part of it reopened in 2019.

Click the map above to go to the full-size version which you can examine on-line or right-click and download.

Despite its age and need of upgrade or overhaul, the SLRR has remained popular over the years and can offer a unique way of seeing Second Life mainland. Such are the number of lines and routes in Heterocera alone, that having a map makes a lot of sense, and the version Rydia has produced is an extensive, impressive,  polished, professional, and informative piece of work.

As with a physical world railway map, the various lines are colour-coded, and the map includes all the major routes associated with the SLRR, form the SLRR Main Line through to the likes of the old WARR line,  the the Okemo, Nakiska, and Southern Railway (ONSR), the Great Second Life Railway (GSLR), as well as the smaller and the more metropolitan routes such as the Northern Branch or the East River City Metro, the Bay City Trolleys and the current routes available in Bellisseria. Stops for the likes of airports are provided, together with crossover stations.

Rydia’s 2014 map of the Second Norway system

Unsurprisingly, Rydia’s work has been positively received in the past, with her approach to maps being adopted in a number of mainland regions.

As well as the 2013/2014 maps and this new iteration that would add grace to any SLRR station, Rydia  has also produced maps for some private region transport network, such as the Second Norway system (also perhaps in need of a little TLC, depending on what eventually happens to that estate).

There is a wealth of information available in the SL wiki about the SLRR – although I cannot vouch for how up to date it is – starting with the official page, and also covering the likes of the VRC, and from these, it’s possible to find out more about various lines and routes. Links from these pages also point to more technical discussions of the SLRR and Second Life vehicles. There are also various private estate lines (as with Second Norway), but these are currently outside the scope of the current map.

Despite some of the issues that can be encountered on the SLRR, if you’ve not tried it before, it’s certain worth exploring – even if only be reading about it initially. For my part, it’s something I’ve never actually blogged about per se in these pages, although I’ve ridden various trains and tracks.

Hmmm… so perhaps it might be time for an occasional series in these pages, something perhaps called From the Footplate or similar.

My thanks to Rydia for contacting me and for our chat.