Category: Events

Meet the Lindens is a series of conversations / Q&A sessions with staff from Linden Lab, held as a part of the SL Birthday celebrations in-world. They provide opportunities for Second Life users to get to know something about the staff at the Lab: who they are, what they do, what drew them to Second Life and the company, what they find interesting / inspirational about the platform, and so on.

Tuesday, June 20th saw Landon Linden sit down with Saffia Widdershins, and this article hopefully presents some “selected highlights” of the chat, complete with audio extracts from my recording of the event. The official video of the event is embedded at the end of this article.

About Landon Linden

Landon Linden joined Linden Lab in August 2008, and is currently VP of Operations and Platform Engineering, based in the Lab’s Virginia offices. He has led the transition of live operations and the production platform to support the company’s new products. With a BSc in chemistry. he worked as a research chemist before moving into the IT sector. Since then, he has worked in telecommunications, launching numerous products.

For SL users, he’s possibly most recognised as the man responsible for re-opening the technology blogs the Lab publishes after major issues / outages occur. These had dried up after FJ Linden departed the Lab in 2011, and Landon revived them in 2014. April Linden has since taken over core responsibility for these posts since then.

Landon loves building large-scale systems, and says his passion for virtual worlds is fuelled by his interests in sociology and economics. As he notes, the nature of the work his teams undertake  – running the services, architecting them, improving them, migrating them where appropriate, etc., – is such that most of it goes sight unseen by users, unless there is a problem.

The Discussion

the initial part of the discussion looks at Landon’s background, his interest in sociology and economics – he notes that by working with the Linden Dollar and the Lab’s transactional services he’s learned a lot about economics – and touches on the Lab’s own studies with users.

In this latter point, Landon makes it clear that the Lab does not conduct direct social experiments on users, but obvious does monitor the use of services and capabilities such as the user on-boarding process, games like PaleoQuest, etc., to see how they are being used, where points of weakness lie which might be improved, what kind of metrics are being generated, and so on.

In terms of general SL trends, he makes the point of noting that – and contrary to claims otherwise – the Lab has seen a “considerable strengthening” of the Second Life economy over the last six months, probably sponsored in part by the arrival of Bento, which the Lab is obviously pleased to see.

This moves into a broader chat about the evolution of things like mesh and breedables, and how that helped grow Second Life, the way in which the Lab cannot always anticipate how new features will be used – but do try adapt to how users take them on and start using them.

Using Amazon  Services

One of Landon’s responsibilities has been to oversee and drive the evolution and enhancement of these supporting services and the infrastructure which supports them and Second Life. Most recently, this has included moving various services in to Amazon cloud.

The Lab has been a long-time user of Amazon services, and this current work not only involves moving services to Amazon, but also moving them to a container model, making them easier to test and deploy, whilst leveraging the flexibility offered by cloud-based services. These include reducing the complexity of having to manage a dedicated data centre environment to run the services, the complexities of having to manage capacity, plan ahead for growth and the purchase, delivery, installation and testing of new hardware, etc., in order to meet specific demands (as the cloud provider can “simply” turn on additional servers and facilities as they are required, and add them to the current billing.

Right now, the intention is not to reduce costs per se  in making the move – Landon rather describes the Lab and trying to break even – but is rather geared to leveraging AWS (and ECS?) and thus doing more, infrastructure-wise with the money the Lab has coming in.

Lab Working Environment

While he is based in Virginia, Landon spends a good deal of time at the Lab’s head office in San Francisco, and notes that while the Lab operates a number of office – Virginia, Boston, Seattle, San Francisco – a lot of people actually work from home, and the Lab has a relaxed approach to office-based work requirements – if it is possible to work from home and be more productive in doing so, there is no problem with doing this.

From his personal perspective, Landon views the Lab as the best place he has ever worked, describing his colleagues as “an amazing group of intelligent, passionate people”. Like others at the Lab have said, it is also a place where he tends to learn something every day, whether about technology, how SL is being used by the residents or about people.

This topic touches on the Lab’s history, going back to the late 1990s and attempts to build a VR / haptics system (aka “The Rig“).

General Q&A

What is being done to improve platform stability and performance?

Landon: We’re always working on these problems. One of the things that is frustrating for residents  – and it’s frustrating for me too – is that lag and crashing out seems to be like a perennial problem. And it is, but the reason it’s a problem is that it’s never, ever just one thing. It’s a near-infinite number of issues and problems, and we’re always working on trying to smooth those things out and reduce them, but it’s always ongoing work. And we’re always trying to balance being able to do new features versus performance improvements and stabilisation work, and I think we strike a pretty good balance there …

… This is going to come dangerously close to sounding like I’m blaming the residents for some of this stuff – and I’m not. But I think … it’s a very creative and expressive place, Second Life, and we really like people to be able to express themselves  in whatever way possible  – and within the confines of the law, at least! But that also means that the complexity of whatever it is that you’re doing, whether it’s in your region or in your parcel or on your avatar, can impact the people around you.  And so we’re trying to strike this balance of how can you express yourself without negatively impacting the people around you. And I think [Jelly Dolls] were a pretty good solution. And it also had the added benefit of feedback to the users, “Hey! Your avatar looks great, but maybe you should tone it down a bit.

Why can we have an unlimited inventory but only 60 groups?

Landon: Inventory is relatively cheap, you’re talking about a very small amount of storage when you have something sitting in inventory, and probably more importantly in the context of this question is the inventory doesn’t necessarily have to interact with other pieces of inventory. So you can pretty much just add anything in your inventory without bound, and UI problems notwithstanding, it doesn’t really have any negative impact on your experience and it certainly doesn’t impact anyone else.

When you’re talking about groups, you have this exponential impact on performance with the number of people who you’re adding into the group [particularly all the Group data which needs to follow you around SL so you can receive group notices, remain part of a group chat, etc.] … I think that’s the kind-of short and long of it. [Groups] have an impact on you and the people around you.

What is the number one cause of lag, and will improved server hardware improve SL?

Landon:  We’re always beefing up the hardware we’re using, and I can tell you the hardware is not a big factor at all in terms of lag. And this is going to be a really unsatisfying answer, but I can tell you that in my experience the single greatest contributor to lag is the network between you and wherever the server is.  So if you are physically far away from the server, you’re going to have a much more laggy experience. Most of our equipment – I dare say all of our equipment is in North America, and the west coast of North America at that. So if you’re in South America, you’re going to have more lag than some that’s sitting in Seattle, Washington. Likewise people who are in Europe and Africa are going to have a more laggy experience than people in North America.

… This is where I’m really going to get into trouble, because I don’t want to come out here and make a bunch of promises, because the things that I’m talking about are going to take probably years to do. But one of the things I absolutely have in the back of my mind is that once we get Second Life fully functioning on cloud services there is the possibility – and I will stress “possibility” – but there is the possibility we can co-locate regions more easily in other parts of the world, in south America or in Europe or in East Asia or Australia. And that would make the experiences for the people who are in those regions a lot better. The flip side to that is, if I’m moving the simulation closer to you and further away from somebody else, you’re making the lag worse for someone else.

… We did some analysis several years ago, regarding this. And what we saw was not a lot of geographic affinity for regions. One of the amazing things about Second Life is that people from all over the world come together and talk and get to know one another and chat and experience Second life together, and there’s not a lot of geographic affinity. There are a few notable exceptions to that, and I think language is one of those things; I think one of the exceptions is people who speak Portuguese, and then tend to almost exclusively come from Brazil. So we can say that if you have a region that caters to, or is attracted to Portuguese speakers, we would probably want to co-locate that region in Brazil.

This is just really stuff that we’re thinking about, there’s no hard plan to do any of this; I think we’ve got a lot of work to do before we can even considering doing something like that, but I’ve absolutely got that in the back of my mind.

Would LL ever consider adding any of the reliable language translation tools back into the viewer?

Landon: For what it’s worth, I’ve actually looked into some of that. I mean … there’s just some amazing tools that are becoming available now using AI machine learning, and I’m really interested in doing some things along those lines. That said, no promises, no commitments; I don’t control the product direction, so I’m looking at it just out of more-or-less professional curiosity and not something I’m actually planning on implementing.

But I think, to try to answer your question as best I can, I think it’s getting easier and easier to put translation and text-to-speech and speech-to-text services into your products, and I would hope that we get back to doing some of that – but no promises and no commitments, and I don’t control it anyway … I don’t make that call.

 

 

Meet the Lindens is a series of conversations / Q&A sessions with staff from Linden Lab, held as a part of the SL Birthday celebrations in-world. They provide opportunities for Second Life users to get to know something about the staff at the Lab: who they are, what they do, what drew them to Second Life and the company, what they find interesting / inspirational about the platform, and so on.

Monday, June 19th saw Dee and Patch Linden sit down with Saffia Widdershins, and this article hopefully presents some “selected highlights” of the chat, complete with audio extracts from my recording of the event. The official video of the event is available at the end of this article.

About Dee and Patch

Dee Linden is the Land Operation Supervisor for the Lab, and her introductions often includes the phrase, “older than the terrain itself”, reflecting her experience from the physical world realty market. She discovered Second Life in 2003, and quickly decided she wanted to be a part of the Lab’s and of Second Life’s growth, taking to dropping note cards on various Lindens, including Char and Philip, encouraging them to consider hiring her.

This happened in around 2005, when she was recruited as a liaison, prior to joining the concierge team, where she was responsible for training Patch. When he moved to set-up the land team, she lobbied him to join the team, where she has a particular interest in supporting non-profits and groups seeking land for events.

Patch Linden started as a Second Life resident, first joining the platform in 2004, and has been a male fashion designer, mentor, and community lead. His efforts with the latter brought him to the attention of the Lab, and in 2007, it was suggested he consider applying to work for the company.

Initially working as a support agent, he worked his way up through the concierge team, eventually becoming the team’s manager. He later moved to the role of Operations Support Manager for a year prior to pivoting away from support entirely and joining the Product group at the Lab, the group responsible for defining the features, etc., found within Second Life. Here he developed the land operations team, which includes the Land Department of Public Works (LDPW) and the Moles. He’s now the Senior Director of Product Operations, a role in which he is also responsible for the Lab’s support organisation.

Recalling the Early Days

The first part of the interview focuses on Patch and Dee’s time before Linden Lab, on-boarding with the Lab: Patch through his men’s fashion business mentoring new starters, Dee through working with hosting events, mentoring new starters, etc. In discussing their backgrounds and joining the Lab, Dee brought up the very early days  of Second Life, mentioning two things in particular those of us with long memories are likely recall.

The first of these was the avatar rating system. Back then, the old-style profile floater (still used by some TPVs) had a tab in it called Ratings, which had five categories in it: appearance, behaviour, building skills, and two others I now cannot remember (age does that…).

These rating categories were open to others to vote on  – so as Dee said, became a kind of “popularity contest” among users (and one that tended to be gamed – “you rate me and I’ll rate you!”).  Ratings could only be awarded once per avatar per category, and cost L$25 to award, the money  going to Linden Lab.

Also recalled was the original means for the Lab to raise revenue – the “prim tax“. This started when Second Life entered its “closed” Beta phase in late 2002 and ran through the “open” beta which started in April 2003. Initially, this saw a fee of L$10 charge for rezzing a prim, with a flat weekly fee of L$3 (I think) to keep an object in-world. Over time, this system evolved to calculating taxes based on each prim’s volume and also its altitude, and with a sliding scale of fees for objects using prim lighting (starting at L$5).

These changes gave rise to the infamous tax revolt during the latter half of 2003, when users who were working to build infrastructure in SL for other to enjoy – communities, roads, event centres felt they were being unfairly penalised against. This revolt in turn brought about the shift in the economic model of Second Life from the “prim tax” to land tax (tier), with the release of version 1.2 of the service, in December 2003 – although this also wasn’t without its own controversy.

Patch also discusses the teleport hubs (“telehubs”) and their impact on community and immersion in Second Life. Again, for those who don’t remember, in the early days of SL there was no direct teleporting. Everything was mainland-based, and teleporting was via teleport hubs, with a free payable to LL. These hubs naturally became the focal points for people to build around, so they’d get traffic / an audience. Thus, the hubs in turn encouraged people to hop around the mainland and then go and explore what lay around each hub and get involved in activities they discovered.

He went on to say the Lab may be looking at the surviving telehubs and doing something with them. Whether this means re-establishing the network in some manner than encourages people to use it, or whether it is a case of “restoring” them as museum pieces is something he and Dee refused to offers further hints about, other than in the most general terms.

Selected Q&A Items

Most Interesting / Challenging Projects To Work On

Patch: There’s PaleoQuest, learning island and social island, I think rank right up there with it; you know, coming from the whole mentor, “I like to help people, I walk in the residents’ shoes every day” perspective that I put on all of our projects, I’d have to say that one’s very important to me.

Probably the Portal Park would be my third favourite … as far as most challenging, I would say Linden Realms. You know, it’s the grandfather of the Experience Keys system [aka “Experiences” today] … it’s basically what experience tools were built upon / along side of or in parallel with, were developed for. We basically set out to say, if we were to try to give people a way to create games or put a gaming engine creation tool within Second Life, what would we do? And out of that was born both experience tools, the tool set and Linden Realms at the same time as demonstrator for it.

The interesting thing about that is, that over all of the years it’s been running it is still one of the most popular experiences, with a very high number of unique visitors daily. And it keeps us on our toes! Again, with ageing content and such, more maintenance tends to go into it to keep it running over time; so we want to make sure it doesn’t break down, and with that, we also have to have the agility to be able to respond to any changes to the tool set that it runs on, as that may occur. So keeping up with all of that over the years has probably been the primary challenge, because just short of rewriting the whole thing or just scrapping the whole thing, we’ve found some pretty interesting ways to keep it going.

Dee: Linden Realms will always hold a very special place in my heart. It was one of the first really big one that I worked on; Linden Homes as well, the originals. PaleoQuest is still my favourite to play, and Horizons, I think, was probably the most challenging of the more recent ones … Unlike Linden Homes, with Horizons parcels you can deed your parcel into your group, and we had to make the mailbox [configuration controllers] and the houses work for the group, where we didn’t have to worry about that with Linden Homes … there are now a few businesses in Horizons, but it is predominantly residential.

Continue reading “SL14B Meet the Lindens: Patch and Dee” →