SL13B talks: Oz and Landon inside Second Life

Oz Linden (centre, left) and Landon Linden (centre, right), flanked by Saffia and Elrik
Oz Linden (centre, left) and Landon Linden (centre, right), flanked by Saffia and Elrik

Meet the Lindens is a series of conversations / Q&A session with staff from Linden Lab, held as a part of the SL Birthday celebrations in-world. These present opportunities for Second Life users to get to know something about the staff at the Lab: who they are, what they do, what drew them to Second Life and the company, what they do, what they find interesting / inspirational about the platform, and so on.

Wednesday, June 22nd saw Oz and Landon Linden sit down with Elrik Merlin and Saffia Widdershins, this article hopefully presents some “selected highlights” of the chat, complete with audio extracts from my own recording of the event. Note that these are not necessarily presented in the order items were discussed during the session; to maintain a sense of flow, I have grouped some items together. However, for those who would like to hear things chronologically, the video the session  is embedded at the end of this article.

About Oz and Brandon

Oz Linden
Oz Linden

Oz Linden is perhaps best known for his work with the viewer and open-source communities in Second Life. He joined the company in 2010, and is perhaps one of Second life’s most unabashed and proud promoters.

Starting in support of the Lab / open-source community relations, Oz moved upwards and forwards in the company, managing the relationship between the Lab and the third-party viewer community, and thence on to Director of Engineering, and lobbying for the post of Technical Director for Second Life  when the Lab commenced re-aligning itself to manage two large-scale core products: Second Life and Project Sansar.

He has a strong background in open-source development and in web technologies, including voice applications, communications protocols and defining industry standards definition. He notes of working for the Lab:

Working on Second Life comes with some odd benefits… you get to pick your own avatar name, and it turns out that’s what everyone at work calls you by. So I became Oz Linden. Four years later, I’m the Director responsible for Second Life core product engineering, and having more fun than a barrel of virtual monkeys.

Landon Linden is the Lab’s VP Operations and Platform Engineering, a post he has held since December 2013. Originally a research chemist with a long-term involvement in MUDDs and MMOs, he decided that there were probably saner pastures in which to work than research chemistry (he relates with a smile), and so hopped over into IT, working in consultancy prior to telecommunications before joining Linden Lab in August 2008 as a Lead Systems Engineer.

Landon Linden
Landon Linden

Since that time, he’s been literally at the heart of Linden lab and Second Life, initially leading the engineering team that designs and implements the network, infrastructure, and low-level systems on which Second Life runs as well as managing the team responsible for creating all the Lab’s internal applications:  support tools, service administration apps, and continuous integration systems including test automation.

In October 2011 he became Director of Systems Engineering and Operations, responsible for technical operations and platform engineering (data centre, network, system infrastructure, build systems, internal tools, and application security) as well integrating some of the Lab’s third-parties service (Amazon AWS, CDN providers). From here he moved on to Senior Director Platform Engineering and Operations, overseeing the team which creates the platform for all Linden Lab products (e.g. platforms, payments, virtual currency, data warehouse) and ensuring  production services run as smoothly as possible. With his move to VP Operations and Platform Engineering, Landon now also oversees the  foundational infrastructure and services being developed for Project Sansar.

It’s interesting you have this cross-over between chemistry and virtual worlds. do you see any kind of common ground, apart from the madness?!

Landon: I think about this a lot, and I really wish I could come up with a really interesting answer, but I don’t. In terms of what I was doing in chemistry and what we do in Linden Lab; I don’t see a lot of base overlap. One of the things that I used every day in chemistry and what I tend to use in my job today is the underlying methods can be similar, particularly with regards to statistics.

… One of the things that fascinates me about virtual worlds is that it is a human-created space but it’s also part of the machine; it’s in the computer. And so we have lots of information, just like social networks, about what is happening, how people are behaving. So, one of the things that has always fascinated me is sociology, psychology and economics. What frustrated me about those disciplines back in my hard science days, back working as a chemist, it was very difficult, and it remains very difficult, to do hard science research on that. And in virtual worlds, there’s this kind of perfect collision of the kind-of fuzzier side of science and the things you can directly measure.

Economics in particular is something that I’ve been desperately interested in, and I think most economists would just drool to see what is happening, directly measure what’s happening, in the Second Life economy.

Oz: I think those disciplines are not as far apart as you think they are; at least based on my experience. I have a whole bunch of former chemists that I’ve worked with in programming over the years, and I think the thought processes are very similar.

Landon:  I think it’s worth pointing out though, that I think that virtual worlds and Second Life in particular can be tremendous tools in teaching people about chemistry. In fact I have a story – I will spare you the gory details – but I had a professor in college who just put it quite plainly that everything you needed to know about organic chemistry is summed-up in two kinds of principles.

Texas A&M interactive learning environment demonstrating how Second Life can be used for teaching chemistry
Texas A&M interactive learning environment demonstrating how Second Life can be used for teaching chemistry

One is electronic effects, and that essentially means that negative or opposite charges are attracted to each other, or like charges repel each other. And stereoelectronic effect, which essentially means you can’t fit a square peg through a round hole. And if you can visualise what is happening in a chemical reaction, it will help you understand whether or not something’s going to work, or at what rate it will work. So virtual worlds can be a really powerful educational tool for chemists, to help them understand that, “Oh these things actually have physical size, and they willing fit together if they want.” And we’ve seen some of that.

Oz: More broadly, Second Life is really a whole bunch of overlapping, interconnected experiments that are all happening at the same time.  Both from the development of Second Life, right up through what all of the different resident communities are using it for. And, of course the ongoing experiment of keeping it working…

You know, we often make the analogy that we’re rebuilding the railroad from a moving train.

Saffia: That sounds like one of those comedy films where they’re running round to the back, picking up the tracks and running around to the front and putting them down in front while the train keeps moving throughout.

Oz: Except we’re doing things like changing the gauge of the track!

I hear stories from the much older hands than I about the early days when they could actually take the grid down on, what was it? Wednesdays, or something. And they would take the whole grid down for part of a day and then they could change things and bring it back up. We don’t get that luxury any more.

The famous "Black Wednesday" Second Life website notice which indicated all user-facing services (grid and web) were down for weekly maintenance
The famous “Black Wednesday” Second Life website notice which indicated all user-facing services (grid and web) were down for weekly maintenance

Landon: Yep, we don’t do that (laughing).

Oz: (laughing) I really envy that. I really do, but it’s just not the world that we live in.

What were your first impressions on joining Linden Lab?

Landon: My first impressions were, I think, awe, I think the word comes to mind. I’d never before worked with such a talented group of people, and that remains true today. But I think that was also something that took some getting used to – not in any bad way. but yeah, I think it is just an amazing product, amazing community and an amazing group of people to work with, and I feel privileged every day to be able to do it.

How much of Linden Lab’s daily activities are carried-out in world?

Oz: We do a fair number of our routine meetings, things like status updates and just sort-of general chatting about routine matters, in-world. I mean, I have probably 15 in-world meetings, not even counting the ones that exist for meeting with residents.

I actually don’t find it very useful for the ones where you’re trying to solve a problem. If you’re really trying to work together with a group of people to solve a problem, come up with a solution for something, or to resolve a dispute about something, then I find that using a video conference application is quite a bit superior. Not all Lindens agree with me on that, but that’s my personal take. Because we have not, in Second Life, got the emotional bandwidth of understanding people’s expressions and so forth. You can’t use their body language because of course it’s artificial, and it’s not what the person behind the avatar is doing.

So we do both, and that’s sort-of how I pick which one I want to use, for any given thing.

As well as chairing user-facing in-world meeting, such as the Third-Party Viewer Developer meeting (above) Oz Linden attends over 10 Lab staff meetings in-world a week
As well as chairing user-facing in-world meeting, such as the Third-Party Viewer Developer meeting (above) Oz Linden attends over 10 Lab staff meetings in-world a week

Landon: I use both. Most of my meetings I do in-world, but for some I use video conferencing apps as well. I’m a little bit more random about it than probably Oz is, he’s clearly given this a lot of thought in terms of which medium to use.

The only additional thing that I would add is that we work on a distributed team; we don’t all work in the same office or the same location, many of us don’t work in offices at all. And adding the virtual world component, the Second Life component, to our meetings makes it a little more “water coolery” or social, I don’t know how to put it; but you come in wearing a different avatar, and it becomes a point of conversation, it’s like a team building sort of thing.

Oz: Absolutely. One of the women who works for Landon  who’s been here longer than I have, certainly, I don’t know how she compares to  you,  has actually made a resolution, a few years ago, that she would come up with a new outfit every day. And so a part of any meeting that you have with her is, “what’s she got on today?” And she comes up with the most amazing stuff! It’s really great! And that’s part of the whole thing, you know? I’m boring! I hardly change my avatar at all; I have maybe a dozen outfits!

If we had facial tracking software and could use it with our avatars to provide facial expressions, would that make in-world meeting more like real world meetings?

Oz: Ah, probably. And there were experiments, before my time, called puppeteering, where you could make your avatar follow real-world movements. but there are some technical hurdles with how the avatar is controlled that make that pretty difficult to do at scale. A gathering like this, where we have a lot of people in a small space, if everybody were trying to do that, it would really be a big problem. Even for modern internet bandwidth and latency.

Project Bento

This brings me to the subject of Project Bento, which is a big thing you’re working on. That has the possibility of making avatar faces more expressive, doesn’t it?

Oz: Oh yes, absolutely. Some of the stuff I’ve been seeing people do as part of our testing, and the community’s testing, more importantly, is really quite impressive.

Perhaps we could explain what Project Bento is for anyone in the audience who hasn’t heard of it.

Oz: Well, if you haven’t, it’s a very big thing and something we’re really excited about. So, what we’ve done is added to the avatar skeleton. The avatar skeleton is just what it sounds like: It’s a set of bone-like constructs that govern the shape of your avatar and how it moves. So I’ve got a couple of bones in each arm and a couple of bones in the neck; one that moves my head around and one that moves my legs.

The basic SL avatar and avatar skeleton (courtesy of Machinimatrix)

And what we’ve done is added a whole bunch more detail to that. So there are now all the bones of the hand, so there are five fingers, each with a couple of joints; and we’ve added bones for a tail; we’ve added bones for a second set of limbs, so that you can do quadrupeds easily; we’ve added bones for wings; and a whole bunch of bones that rigged meshes can be rigged to for facial bones. So there’s an extra set of bones for the eyes, I think it’s six for the eyebrows, several for the mouth. I don’t even keep track of what the final counts were; but all of that can be manipulated using animations and cause the mesh of your avatar to move. So mesh avatars are going to have tremendously more expressiveness when this comes out.

But we are talking about animations here, we’re not talking about your avatar movements being related to things you’re doing in real life.

Oz: Well, not in real-time. The way a lot of animations are developed is by motion capture; so you film somebody making a movement and then you translate that into the same motions for the underlying skeleton. And you get a lot of great dance moves, that’s the way a lot of the coolest dance moves are created.

But doing that where you can pre-process it and figure it all out and break it down and then optimise it, so that you’re not capturing a thousand data points for each movement is one thing. Doing it in real-time, where you have to track what’s happening in real-time, and then get that expressed to everyone in Second Life, given the Second Life architecture, is pretty challenging. We might get to that point some day, but it’s not on our near-term roadmap.

Is the collaborative nature of Project Bento, working with residents a fundamental new departure for the Lab?

Oz: I don’t know that it’s a complete departure. It certainly was a critical element of this project; we could not have done this project, and had we attempted it, we would not have done nearly as good a job as we have done by collaborating with a very talented group of residents.

Early on, we recruited a bunch of content creators who had well established reputations for excellence, and some of our own in-house content creators, the Moles. And actually, they ended up changing the priorities of the project very significantly. As I said before, we weren’t planning on doing ten fingers when we started the project. We thought, well that’s an awful lot of motion, that’s an awful lot of new bones for just hands. We weren’t planning on doing anything like the number of facial bones that we ended up with.

We were looking at wings and tails and maybe an extra set of limbs, and maybe a few other things here and there. And when we got together with the content creators, they said, “Faces! Hands! Gotta have them!” So we took that and ran with it, and that’s what got us to where we are today.

Aki Shichiroji demonstrates a wearable wyvern utilising Bento bones for animation. Inset: the model under development at a Bento meeting.
Aki Shichiroji demonstrates a wearable wyvern utilising Bento bones for animation. Inset: the model under development at a Bento meeting.

And that is something we’re trying to do, trying to put more and more emphasis on, is  collaborating with residents, as much as we can figure out how to do. On what our priorities should be and on how things should behave.

It’s customers who tell you what matters; so we’ve got to communicate with them to get that to happen.

Will we be able to keep our “old world” avatars versus these new ones? Will we lose the system clothes, since they don’t work on mesh? – Vampire Hunter

Oz: The Bento avatars are an extension of the standard avatar, so they are a superset. And there’s nothing about that new technology layer on top of the existing avatar that alters the existing avatar. So you should be just fine to keep using your existing avatars as long as you want to, as far as we know. Certainly nothing about Bento limits that in any way.

And anybody who knows me knows that I’m a real stick-in-the-mud, and it’ll probably be ages before I get around to adopting a Bento extension to my avatar. I have a couple of ideas, but I’ll probably be slow about implementing them.

System clothes at this point don’t work on meshes. You can’t put a system-based shirt or tattoo layer over a mesh body. but if you’re continuing to use the system avatar base, all that will continue to work exactly the same way it does now.

Working at the Lab: Collaboration, Complexity, Challenges

It’s an interesting challenge, isn’t it, the balance between internal and external, so to speak,  because we as users don’t know what you might have in your back pockets. So you could imagine things that we couldn’t imagine, because you understand the platform and what it can do, but once those things are there, we can point them in the right direction.

Oz: That’s not actually the hardest part, though. and that’s not actually the thing that provides the strongest counter-pressure to deciding whether or not we’re going to engage publicly about something. The biggest problem is that once we starting talking about doing something publicly, people immediately leap to a bunch of conclusions about what it’s going to be and how far it’s going to go and how it’s going to behave.

And the truth is, most often with any really important change, we don’t know what it is we’re going to be able to accomplish until we get really quite far into it.

So, early in the Bento project, we started talking to this closed group under NDA, and one of the things we did very early on was make an extended skeleton that just had ten tentacle arms with ten joints each, and did a bunch of performance testing of what happens if you start waving all these tentacles around wildly. What’s the effect? Do all the viewers fall over? Does the network get crushed? Does the simulator get confused?

And it’s entirely possible we could have gotten part way into this project and then discovered that, yeah, we can add a pair of wings and a tail, but that’s pretty much it, if we do more than that, we’re going to knock everybody down. And if we’d started talking early about having ten fingers and thirty facial bones, or whatever it is, and then it turned out we couldn’t deliver, the end result of that project would have felt very different to the community than it has ended up doing. So, often we’re starting project that we don’t really know how they’re going to come out and how they’re going to scale.

And of course, Landon’s team is responsible for keeping things going once we’ve put them out in the world. My team develops stuff; his team then tests it and tries to deploy it and keep it operating 24/7. And that’s a whole different ball game.

And presumably making sure it doesn’t break half-a-dozen other things by mistake?

Landon: Oz is correct, it certainly does fall on us to make sure that things are going to work correctly. But I would describe our relationship as being much more collaborative than that; I mean we work really closely together to make sure that we’re shipping something that’s going to work for everyone.

Oz: But, early in the development, you don’t always know. We all take our best guess, and we try to figure out ways to simulate and test it.

But one of the real fundamental issues we have is that there is not way to create a test bed that even remotely approximates the size and complexity of the real production Second Life grid, with all of its residents. We just can’t do it. There isn’t enough computing power; even if we built another whole grid, we’d have to simulate everybody’s viewers. and even then, we could simulate all the wacky things people do with it!

I literally haven’t had a week go by in six years of Second Life, literally not one week, where I didn’t learn something new about either what our product was built to do or how some group of users has found a way to do something we never intended!

Landon: Oz and I joke about that all the time, but it’s completely true. I would say every day, not every week! But you know, it’s this complex system, and we have  tens of thousands, hundreds of thousands of people in there, using it every day, and you cannot imagine all of the things people will do with it. And that’s just one of the truly awe-inspiring things about Second Life…. It’s just this amazing, complex system, and I learn something every single day I come to work; whether it’s about the system itself, or about life!

Oz: One of the most common reactions I find myself having I response to a description from users about what they are doing, or some problem they have or some new thing they want done, or something that they do every day, is to say, “you’re trying to do what? That works?!”

I mean, it’s just amazing! And despite the fact, as I say, I’ve gone six years with that happening constantly – not even occasionally, constantly – I still am astonished at the fact that it keeps happening! I keep being surprised! I keep waiting for a week to show up where it doesn’t happen. Hasn’t happened yet! Not even close!

One of the things that must have been very true of was the introduction of mesh. I imagine your original concept was that mesh was for furniture and vehicles, and may not, “oh! we can make hair with it!”

Oz: Mesh was just coming in about the time I joined the company, and I didn’t get to see the beginning of the project, when people were developing their “limited” conception of what it was going to be. So I can’t say. Certainly, everything we’ve developed while I’ve been here has ended up being used in ways we didn’t anticipate. Which is great; like I said earlier, it’s this giant ongoing experiment, and we just get to keep doing it.

As well as introducing new things, you have to deal with the challenge of dealing with old things. Which is the more challenging?

Landon: No comment! (laughing).

Oz: Well, we put a lot of emphasis on not breaking old things; at the same time we have to constantly try to find ways to simplify the system enough that we can make its behaviour predictable and reliable.

So, one of the things we’ve been doing a lot over the last couple of years is saying there’s more than one way to do something, some of those ways are better than others, we’re going to give people lots of warning, the third-party viewers especially, that this particular set of the old ways is going to stop working.

Invisiprims is maybe an example, and we got some of the functionality we needed that invisiprims were providing, done in other ways that provide good qualities; and that, you know, we still haven’t solved the water in boats problem. And that’s unfortunate, it’s regrettable, but there are pretty good reasons why that was a good trade-off to make. And I’m not really qualified to explain that in depth, because I’m not a graphics guy, despite the fact that I’m working on Second Life.

But, we’re working on a transition like that with inventory now. We’ve been on a big programme – been going for a year-and-a-half now – to make  inventory more reliable, so people don’t lose things. And we’ve made a huge amount of progress on that.

But one of the things that we still have to do, and that we’re in the process of doing now, is saying, we’re doing a version of the viewer that no longer uses some of the old interfaces and provides a model of how to do everything you need to be able to everything with inventory, using newer interfaces. And some time soon we well say, “OK, as of this point in time, the old ways will stop working.” And yes, there are some older versions that will just not be able to do inventory any more. It won’t work, because they’ll get errors when they try to do  some of their inventory operations.

And that’s sort-of the process of moving forward. If we maintain 100% backwards compatibility, we would pretty soon grind to a complete halt.

And presumably, the deeper you go, the more likely you are to encounter things that you just can’t do anything about?

Oz: There are deeply fundamental things that there’s just no way to touch any one. You can build on top of them, but you can’t go back and change them. Because there are certain assumptions built into all the code about how something works, and therefore if you touch it, you run the risk of breaking almost everything.

What are the greatest challenges and rewards of your jobs?

Landon: I think we talked about this a little bit, but I think my answer is the same. It’s this amazing system that is just full of wonder, I learn something all the time; that is the thing I love the most about it. It is also the thing that is most challenging! (laughs).

It’s very difficult to predict what’s going to happen on any given day, and I know that Oz can appreciate this as well. That you find new and unexpected things crop up, and it very difficult to plan. I mean, one of the things I would love to be able to do is to commit to having something done on a particular day, but that make it kind-of difficult due to the complexity of what we’ve done and the unexpected things that crop up. Oz and I talk about that a bunch as well.

There are certain things, like security issues might come up, and you have to drop whatever you’re doing and take care of that immediately, because that is the most critical thing. But we do so at the expense of all these other wonderful things that we want to be working on and delivering. So that’s probably the most challenging thing about what we do.

Oz: We often find ourselves three levels deep in emergencies. I’m not exaggerating. You’re working on problem A, problem B, that’s much more urgent suddenly comes up, and you have to start working on problem B. Then you have to drop that to work on problem C. And then you have to remember, “OK, what was I doing before B?” And it’s true.

And I have to say, Landon’s team, they’re the people who keep the grid and everything behind it – and you can’t even imagine how much there is behind it – going all the time. And one of the things that’s happened in the last couple of years that wasn’t happening when I got here, is that Landon’s team has started putting out regular reports about just what went on when some big event happened. Landon started the trend with a fairly famous blog post about that at one point.

It’s just astonishing, the level of complexity they deal with every day, and how well they cope with  spectacular, multiple simultaneous failures. when you’ve got a system really big, when you’ve got as many different components as we have, it’s no longer a question of whether anything is going to fail on any given day. Something is going to fail; you just don’t know what it is. And on some really bad days, it’s going to be more than one thing. And if those things interact, they you’re really in deep sneaker. And his team are just amazing at how they do it.

The other thing that is just fascinating is I’ve had the privilege of hiring some people, and bringing them in and acquaint them with Second Life, which is a process we call drinking from the firs hose. But I’ve hired some pretty amazingly talented people, and I get the same reaction from all of them. Most of them are people who were familiar with Second Life before they came; we try to do that; but they come and they begin to learn how it all really works, how it is all put together, and the reaction is uniform: “this is amazing! This is just amazing!”

We’re doing things nobody else is doing, and we’re doing things on a scale that almost nobody else is doing. And it’s just astonishing to just come in and see it all, and see that it keeps working every day. And you know, it’s creative and it’s fun. and you get to constantly be exposed to the crazy things that people are doing.

My favourite example, which I learned early on, is how many other places can you work where it’s a perfectly normal thing to go into a meeting and say, “Hey, can anybody here see my pants?!”  It actually happens all the time!

So it’s an endless creative thing, and an amazingly stable organisation. We just keep moving along all of the time, and the very best part of my job is that I’ve got just a tremendous best part of my job  is that I’ve got such a tremendous team of people who have such dedication to making this happen. Their level of passion is just incredible.

It feels like the people who stayed with Second Life after others went of to Sansar are the people who feel passionate about it.

Oz: That’s exactly true, and there shouldn’t be anything surprising about that; it was a self-selection process. I mean, some people wanted to go and work on the new thing – and more power to them – we all hope that they succeed brilliantly  and embarrass us with how much better it is.

But I think in many ways, it’s just going to end up being very different. I know it’s going to have some ways in which it is going to be just amazingly better, but it’s not going to have initially, as everybody has said publicly, the level of diversity and scope that Second Life has, because you simply can’t build anything as big as Second Life has been built, in whatever it is – fifteen years, thirteen publicly, and a significant time before that. You just can’t turn round and do that in a couple of years.

But I was lobbying to get the job I’ve got now about the time we were really beginning to ramp-up the commitment to Sansar, and begin really taking people off of Second Life to go do it. And I literally went around to everybody who was doing things, and what I asked them was, if what you want is to keep working on Second Life, then I want you on my team. And that’s how we got the people who are primarily responsible within Landon’s organisation for maintaining Second Life and driving Second Life. That’s who those people were; and the people who are doing the development, that’s who those people are. They are the ones who wanted to be involved in Second Life, continuing to make it better.

And we kind-of see it as a challenge. Sansar is out there trying to do better everything that we do – and more power to them, I hope they succeed, let me be clear about that – but I see it as my mission to keep raising the bar they have to get over.

Where do you see Second Life going? What do you see as the future?

Oz: Oh, gee. I’m not sure I can even predict that. Kind-of the way we do things, is we get together, the whole team, a couple of times a year; we’ve been doing this a couple of years now. We get everybody together  in one place out of our regular environment, and we get in a few conference rooms, and we throw out ideas of what needs to be done or we describe problems that need to be solved. And Bento came out of one of those, a year and a bit ago, where an Engineer [Vir Linden] stood up and said, “Here’s the skeleton, here’s what it can and can’t do, and here’s what we can do about it while maintaining backwards compatibility.” And that was the beginning of the project. Six months before that? No-one could have predicted we were going to do anything even remotely like that.

So I don’t think I could any more predict what we’re going to be doing a year from now, than I could predict what it was that some group of residents is doing right mow that’s going to present me with a big problem next week! Neither of those is predictable.

Landon: So just to add to that, I would agree, and I think my answer is pretty unsatisfying in that the thing I love about Second Life is that you can’t predict where it’s going to go. [But] there are a couple things I probably would say about it.

I’m expecting Bento to be a huge step for content in Second Life, and we were talking about the economics of it a little bit ago, going back to mesh. Mesh was a very fundamental addition to what you could do with content creation in Second Life. and it was adopted very quickly and it became huge and did a lot to invigorate Second Life economy. That is the thing that’s important; that’s the thing that keep Second Life new and interesting and relevant, thirteen  years into its run.

I’m expecting Bento to be similar in its impact on Second Life. I mean I certainly cannot predict in any way what people are going to be able to do with it; I just know it’s going to be an important part of what Second Life looks like  and how people use Second Life in the years to come.

Do you think Oculus Rift is significant in Second Life’s future?

Oz: It’s a pretty amazing experience. We’ll be coming out with a new version of the Rift viewer shortly; one or two things need to get fixed, and we’re putting on a full court press to try to get those done … But I don’t know if it’s going to have a big impact, I really don’t.

Obviously, it’s an extra filter; it’s pretty cool, especially for just wandering around. but it requires that you have a headset, so in terms of the scale of its impact, I think it will be pretty limited for  a while until the costs on those things begin to come down. And we’d like to find ways to support the cheaper ways to get some of those effects. But if what you want is the real immersiveness of being inside, you really do need the high-resolution wraparound screen, good optics and quite high frame rate.

Using Oculus Rift on the (no longer maintained) CtrlAltStudio viewer - courtesy of Dave Rowe
Using Oculus Rift on the (no longer maintained) CtrlAltStudio viewer (image courtesy of Dave Rowe)

Landon: The one thing I can say about the Oculus is I think it is very cool, I’ve gotten into Second Life with one of the DK2’s, I thought it was the coolest thing, I’ve seen people go and do it, I think it’s really cool; one of the problems we have with the new wave of VR tech today is that there is not very much content, and Second Life has at least the surface ability to deliver a lot of content for VR.

The thing I think [that] is really limited is – and I’m one of the people who get really motion sick in general – [with] the technology behind Second Life, it’s going to be extraordinarily difficult if not impossible to deliver the frame rates  necessary to have a comfortable experience in VR. I mean, there is a reason why we’re doing Sansar.

I think it will be interesting – this is just my prediction – to use Oculus in Second Life, but it’s difficult for me to see that it will get to the point where it can be a comfortable experience for a main mass market using Oculus.

At this point, the discussion moved to solving VR motions sickness, and a more general discussion of personal experiences of VR, which can be heard in the video below, prior to the session drawing to a close. This article will move to the more general questions asked by the audience.

Audience Questions

What does inventory cache do when you clear it? – Angie Mornington

Oz: To start with, it helps to explain what the inventory cache is. Your inventory is a bunch of rows in a database. And each of those rows has a name associated with it, and a pointer to something in the underlying asset system. So, my coat, your hair, all these things. The object itself is in the asset system, and might have many pointers to it from people’s inventories, but your inventory is how you organise it.

That’s a lot of data. People have inventories with 100,000 or 200,000 items in them. And if every time you logged-in, you had to read all of that, it would very noticeably slow down how long it took you to log-in; it can take quite a while.

So you’re inventory cache is literally just a copy of all that data, that’s kept locally on your computer. And when you’ve got it, your computer can, when you start Second Life, just look up the real inventory and say, “how close is this to what I’ve got stored locally?” and if there are differences, it can just get the differences. And that means you can access that information very much more quickly; you can get what your outfit should be …

So, it used to be true, I am told, with some regularity that the local copy would get badly out of sync with what was in the real inventory, the one on the servers. And the only way to make things behave correctly and actually be there correctly, was to delete your local copy and get a completely fresh copy from the server. And that’s what clearing your cache means.

It means, delete my local copy of everything I know about my inventory, and fetch it all again, next time I log-in. And that slows you down, it’s only actually of any value if the information in your local inventory cache in your local copy was bad. And these days, that’s actually really unusual. It has become almost folklore that you can solve almost any problem just by deleting your inventory cache; it’s really not true, most of the time. It should be, at very, very best, a last resort these days, because we’ve done a lot to make it more reliable.

So, that’s what you’re doing, you’re forgetting everything you know, and you have to load it all again.

Has the Lab looked at cubemap reflections for Second Life? It’s already possible to simulate the effect with projected lights. – Paulo Gibbs   

Oz: Well actually I saw an excellent YouTube video on the simulating of reflections for bouncing lights just recently. It was great.

But real reflections, mirror-style reflections, which is what cubemap reflections gets you? We have looked at it, there was an open-source prototype limited implementation a while ago. And as I said a few minutes ago, I am not a graphics guy; I’m a network protocols guy. What I know about graphics I could write on a few sheets of paper. So when that came around I took it to the graphics guys – and we have good ones – and I said, “What about this?” And they said, “Don’t go there!”

All the examples we were being presented with were a single mirror on a perfectly flat surface in a relatively uncrowded area where there wasn’t a lot of tricky lighting going on, and there weren’t a lot of avatars there…

Basically, what you have to do, to implement a reflection, is you have to pretend that you’re re-rendering the scene from a different perspective; and then you take a slice from that perspective and you project it onto a surface.

So imagine, we’ll say this wall behind me is a mirror. The way to simulate what should appear in that mirror, is to put another camera behind that wall, treat the wall as clear, and then re-render this entire scene. And then take the piece of that render that would be visible through that hole in the wall, and render it on the wall.

So you need to re-render the entire scene for each reflective surface. Now picture what happens if you’ve got two of them. Now you’ve got to do it twice, and you’ve got to account for the infinite mirrors problem, two mirrors facing each other. How many times do you render that, to make that come out right? And then think about, what do you do if the surfaces aren’t flat? Now you have to render the entire scene and then distort it, based on the shape of the surface that you’re mapping that image onto. The complexity goes up exponentially really fast, in a bunch of different dimensions, and basically the graphics guys said, if what you want to do is kill the performance of every viewer in the world, do this.

How about something like an Undo button in Second Life for fixing accidental in-world deletions or correcting mistakes when changing settings? – Freya

Oz: In order to do Undo – and it’s a classic example, you can read about architectures for doing Undo, it’s a very common example in a lot of design patent books in computer science – you have to keep track of everything you did. So, picture everything that might be happening around you in Second Life and  imagine how much information you have to store, in order to be able to reverse it. Oh, and by the way, you also have to have a way to reverse any given action.

It’s pretty easy if what you’ve got is text being typed in an editor window; all you’ve got to do is remember the sequence of keystrokes [and] play them backwards. But if what you’re talking about is the creation and linking of a bunch of prims and the texturing of them, and the distorting of them; it just keeps getting more and more complicated.

So, it’s not impossible, and within some limits it’s a pretty achievable thing, but  how large a community are we serving, by making that change, as opposed to adding new bones in skeletons and enabling the creation of really radically cool non-human avatars. We just have to pick our targets.


3 thoughts on “SL13B talks: Oz and Landon inside Second Life

  1. Reblogged this on thomas mcgreevy and commented:
    Wednesday, June 22nd saw Oz and Landon Linden sit down with Elrik Merlin and Saffia Widdershins, this article hopefully presents some “selected highlights” of the chat, complete with audio extracts from my own recording of the event.


Comments are closed.