HTTP and Group Services updates

There are a number of projects underway at the moment to improve various aspects of Second Life performance. Some of these have been reported on as a part of the Shining Project, others are being dealt with elsewhere are reported on through the likes of the SL Scripting User Group and the fortnightly TPV/Developer meetings.

The following is by way of a brief update on the ongoing HTTP Library and Group Management projects with information taken from the most recent TPV/Developer meeting (recording link).

HTTP Library

The focus of this aspect of the Shining Project is to improve the underpinning HTTP messaging that is crucial to simulator / simulator and simulator / viewer communications, and it is under the management of Monty Linden.

Discussion on progress with the project commences at 36:36 into the recording.

The project code (textures only) is with the Linden Lab QA team and is expected to be in the 3.4.1 viewer once it has been released by QA. In the meantime, the HTTP project viewer was updated at the end of July. Many people are noticing improvements in viewer performance that go beyond initial texture loading, although there have been reports of other aspects of the viewer which use HTTP apparently being “slower” to use. This latter issue is most likely a false impression, with Monty commenting at the August 24th meeting that, “Most parts shouldn’t be affected. It’s competitive, when you’re doing both texture downloading and some of that work … but other things aren’t being cheated if you’re not downloading textures at the time.”

An issue has been noted in older Macbook Pro systems (late 2007 into 2008 dual-core systems, although the span of the problem isn’t clear) using nVidia drivers, wherein the expected speed-up with cached data which can be seen on other systems isn’t occurring. Monty is still investigating this. Overall, however, feedback on this project has been positive.

Group Management Functions

Large group loading: a familiar problem

Baker Linden has been working to resolve this problem, and his plan is also to go the HTTP route, which will require changes on both the server and the viewer sides of the equation. His comments on progress start at 42:53 into the TPV/Developer meeting recording.

The server-side code for an initial implementation of the solution has been passed to LL’s QA and is expected to be rolled to selected regions on the Beta (Aditi) grid soon.

In terms of the viewer, the plan is to develop a Project Viewer, which will be made available in the near future for people to use with the Aditi test regions. How soon this viewer is likely to appear is open to question – the code will initially need to be passed by LL’s QA (who may have received it on the 24th August) prior to the viewer being built. Once in the project viewer repository, the code will also be available for TPVs to produce test viewers of their own.

How long the testing period will last is also open to question and dependent upon feedback / issues arising. However, the plan will be to follow the usual pattern for roll-outs in that once the code has been tested on Aditi and necessary updates made, it will be rolled to a main grid RC for more more involved testing. This is important, as there is a significant different in the number and sizes of groups operating on the two grids. For example, the largest group on Aditi numbers some 40,000 members; on the main grid the largest group is about 112K, and there are many more groups with between 40K and 112K members.

One thing that has been made clear is that there will be no attempt at backward compatibility with V1-based viewers on the Lab’s part; the new code will be aimed solely at the V3 code base. However, V1-based viewers will still be able to use the UDP protocols for group management, although the LL servers will limit UDP access to groups with 10K members or fewer, so V1-viewers will have some more code backporting on their hands.

There will also initially be some issues around the new HTTP protocol. For example, in the first implementation, the data will be uncompressed. This means that a 40K member group is around 5Mb in size, which can take up to a few minutes to download, depending on someone’s connection speed, so some frustrations are liable to continue. While data compression will eventually be used, this is not planned for the initial implementation.

The discussion involved providing an option to routinely clear-down group lists based on people’s last log-in date, or who have not logged in for a (group owner specified) number of days. However, LL are not going to implement such a feature on the grounds that it could lead to mistakes being made, and people being accidentally removed from a group.

Time Scale and Implementation

As mentioned above, there is no definitive time scale for this work to be completed. Testing is liable to take several weeks at the very least, so it is unlikely the new group management capabilities will be rolled-out on a widespread basis for at least another month, or possibly longer.

However, and like the upcoming new avatar bake service, once the server code is available on the grid, the switch-over will be transparent. If a viewer has the code to use the new group management HTTP service, it will do so, if it has not been updated, it will continue to use the UDP service (with the aforementioned 10K “cap”) until such time as that capability is “retired” from the grid.

Lab seeks to improve how TPV support issues are addressed

C & TM Linden Research

As mentioned in the TPV/Developer meeting of the 24th August, Oz Linden has been taking steps to try an improve how issues are addressed by the company’s support teams when dealing providing support to users who are using a TPV as their viewer of choice.

That TPVs are collectively more popular than the official SL viewer is not that surprising. However, a lot of people still turn to Linden Lab for help when they encounter issues. As a result of this, LL have come in for criticism as to how they handle users who report that they are using TPVs, and it is this that has prompted Oz to try to improve how matters can be handled and addressed.

Identifying the Problem

The first part of dealing with any problem is correctly diagnosing whether it is in fact viewer-related or server-related. This isn’t as easy as it sounds because there are many parts of SL where the problem could reside either within the viewer or on the server-side of things (inventory issues being a good example) – hence why LL often get the call when things go wrong.

Because of this complexity, and in order to help improve the initial viewer issue / server issue diagnosis, Oz is working with LL’s support teams to put together a better set of heuristics for use in support staff training and guidance in identifying where a particular problem may reside. To help with this work, he has asked the TPVs supply lists of issues they have encountered which they know are not viewer issues, and how to recognise them. These lists can then be added to the information supplied to LL support staff to both speed the initial diagnosis of a problem and reduce the chances of a problem being mis-diagnosed from the outset.

It’s a Viewer Problem – But Can it be Reproduced on the LL Viewer?

When it comes to trying to resolve what appears to be a viewer issue, LL support staff will ask a) whether the user is using the official LL viewer; and b) if they have tried to reproduce the issue using the official LL viewer. These questions are often taken to mean LL’s support staff “do not want to help” with the problem if it appears to be TPV related.

However, this is not the case; the question is a perfectly valid part of trying diagnose a problem because:

  • If the problem can be reproduced using the official viewer, there is a chance support staff may be able to provide SL-viewer based assistance to resolve the issue
  • If the problem cannot be reproduced on the official viewer, then it at least helps point to the problem potentially being related to the TPV itself.

Obviously, if the problem does appear to be viewer-related but only manifests in a TPV, LL’s support personnel are unlikely to be able to give detailed help (simply because it is unfair to expect LL’s support personnel to be intimately versed in how to resolve issues occurring with all of the TPVs used to access SL). As such, they are going to pass the matter back to the user. When this happens, it can lead to frustrations and a feeling that LL “aren’t interested” in solving the problem.

To avoid this in the future, Oz is working with TPVs to ensure LL’s support staff are better placed to provide onward guidance rather than leaving users feeling they “don’t want to help”. This is being done by each TPV listed in the TPV Directory being asked to:

  • Add the details of any in-world support group(s) they operate to their Directory listing if they haven’t already done so
  • Use a new field in the Directory to give details of any additional locations where help on a specific TPV might be obtained (e.g. a website, a support forum, etc.)

Thus, should an issue appear to be related to a specific viewer which LL staff cannot help resolve, they will at least be able to point the user concerned in one or more directions where they can receive more focused assistance in order to resolve the problem.

Asking People to Complete the Survey

During the discussion, Oz reiterated that every support issue dealt with by LL staff should trigger a follow-up e-mail to the user concerned. While this might not happen until up to four days after the event itself, the e-mail does include a customer satisfaction survey. This is important for two reasons:

  • All survey responses are reviewed by a Linden Lab staffer; they are not farmed out to a third-party survey company or ignored or handled by an automated process
  • They are seen as a primary mechanism for determining how well support is identifying and dealing with issues to the satisfaction of LL’s users.

As such, Oz emphasised the importance for feedback to be given, particularly where there is strong evidence to show that support have failed to provide the correct assistance. While completing the survey may not help in resolving the issue itself, it may help pin-point errors within the support process, particularly if a number of surveys are received highlighting the same fault.

The current process by which support issues – particularly those with TPV problems reported to LL – are handled doesn’t always run smoothly, and there are times when issues do get mis-directed. However, Oz’s response to concerns raised during recent TPV developer meetings demonstrates that steps are being taken to address them. It has been suggested that LL post a blog entry on the initiatives explained here (particularly on the need for TPV users to understand why LL do ask about reproducing issues encountered using the official viewer). In lieu of that happening, I hope this piece will serve as an informational.

Curiosity: the trek begins with a song

NASA’s Trekkin’
Across Gale Crater’s plains,

On rover Curiosity, run by JPL.
NASA’s Trekkin’
Across Gale Crater’s plains

Boldly seeking science, what wonders will it tell?

(To the tune of “Star Trekkin'”)

Singing to us from Mars

The recent broadcast from Mars of a recorded message by NASA Administrator Charles Bolden (Monday 27th August) was followed on Tuesday the 28th by a more ambitious broadcast, designed to be an inspirational fanfare from Mars to encourage young people to get involved in science, technology and engineering.

Written by, Reach for the Stars is the first-ever song written on Earth and transmitted from the surface of another planet. Opening with the lines “Why do they say the sky’s the limit / When I see the footprints on the Moon?”. The song was originally written in February 2012, but following a meeting with NASA representatives, undertook to rework elements of the piece. “I don’t think it’s the right thing to do by sending a computer beat to Mars, so I wanted to put an orchestra together to show human collaboration,” he explained when talking about the song’s evolution. “That robot is going to Mars, but a piece of humanity, of art, should go with it as well.”

As well as an orchestral arrangement for the music, the song features children singing the verse. The entertainer, well-known for his advocacy of science and technology education through his Foundation, said the debut of the song, transmitted from Gale Crater and played during a NASA special event, is a message of inspiration. “Today is about inspiring young people to lead a life without limits placed on their potential,” he said. “And to pursue collaboration between humanity and technology.”

Stereo View

Also on the 28th (Sol 22) Curiosity departed Bradbury Landing on its longest drive to date. Travelling some 16 metres (52 feet) eastwards, the rover stopped at another of the scour marks created by the descent engines as they blasted the Martian “topsoil” away to expose the rock below.

The rover was due to spend around a full Sol at the spot, using the Mastcam to collect a further set of images of the mission’s ultimate driving destination, the lower slopes of “Mount Sharp”. These images, once received on Earth, will be combined with images already gathered from the last location the rover occupied, some 10 metres (33 feet) away, to produce 3D images of the mound, which should help planners determine potential driving routes up the slopes as well as furnishing more information about the surface features and mesas themselves.

On the road: Curiosity’s rear HazCam captures the tracks left by the rover as it moves away from the centre of Bradbury Landing.

After this, on Sol 24 (August 30th), it is expected the rover will be commanded to start its traverse to Glenelg, an intersection of three terrain types some 400 metres (1300ft) from Bradbury Landing. “We are on our way, though Glenelg is still many weeks away,” said Curiosity Project Scientist John Grotzinger, following the initial move to the second scour mark.

One of the reasons a relatively “close” destination is “weeks” away is that mission personnel hope to find a suitable area in which the rover’s robot arm can be further tested, particularly the scoop system which will be used to gather soil samples for analysis. Should a suitable area be found, it is likely the rover will pause there for at least a week while the robot are is further calibrated and the scoop system tested.

There is also a possibility that the drill system and the MAHLI (Mars Hand Lens Imager, the camera mounted on the robot arm’s turret) may be tested en route to Glenelg, should a suitable opportunity arise, although testing of both may not take place until the rover reaches Glenelg.

A wide-angle image of Curiosity’s destination: the lower slopes of “Mount Sharp”, captured by the Mastcam on August 23rd during the focusing calibration exercise

Curiosity reports in this blog

BURN2 seeking builders and scripters

The BURN2 team are seeking builders and scripters to help with this year’s BURN2 temple build. A call, put out on the BURN2 blog, reads thus:

In keeping with the Burning Man principles of gifting and participation, we’re looking for volunteers to help build this year’s Temple!  More importantly, in keeping with the principle of communal effort, this year’s Temple build will be a group effort, each individual on the build team contributing to the whole: an awesome Burn2 Temple!

If you are a builder extraordinaire, a sculpting god/goddess, or a scripting whiz, we need you!  If you’d like to join the Temple build team, please contact Shendara Destiny, either via in world IM, or by e-mail at

Help us create something truly beautiful that will enrich the entire Burn2 community!  Burn on!

The BURN2 temple, 2011

If you are a builder / scripter and enjoy participating in communal projects, why not give Shendara a call?

Related Links

Firestorm pathfinding, Flickr and more

Monday 27th saw Firestorm released. Unfortunately, this included a visual bug being inadvertently introduced into the release which made moving items such as doors and wheels appear to be “broken”. While this was only a visual impact rather than a code breakage, the decision was taken to withdraw 4.2.1 and replace it with 4.2.2 once the problem had been fixed.

As a result, and in case the release of version 4.2.2 included additional updates necessary as a result of fixing the issue, I opted to hold-off on my review of 4.2.1, and wait until I’d been able to look at 4.2.2 before Pressing a review.

So here it is, a look at Firestorm 4.2.2, featuring some of the key changes and updates which include an initial implementation of pathfinding. Alongside this, the release sees includes Katharine Berry’s snapshots-to-Flickr option, temporary uploads from the snapshot floater, new toolbar buttons and more.

A Note on OpenSim

This release does not include any fork between Second Life and OpenSim. That will be coming in a future release, which, as Jessica reports in her blog post on this release, might be a while in coming as the team have a lot of work on their collective plate.


The windows installer is some 33.7Mb in size – so par for the course with Firestorm. If you’ve previously installed version then a clean install is not required. However, if you’re upgrading from or earlier, a clean install is required / recommended.

Pathfinding Tools

As mentioned above, and with the exception of navmesh visualisation, all the main pathfinding tools are present in this release, complete with the expected Firestorm finesse when it comes to Rebake Region.

The Linksets and Characters floaters can be accessed using both the context and the pie menu when right-clicking on an object or character. The Build and Object Profile panels also have their pathfinding information panels added.

The Firestorm team have implemented the Rebake Region functionality somewhat differently to Linden Lab. Rather than incorporating a button displayed at the bottom of the viewer window when a rebake is required, the team have combined the rebake function with the pathfinding icon displayed in the Menu Bar / Navigation Bar (if displayed). Thus, when the icon is displayed (either with or without the initial warning pop-up, as shown in the image below), clicking on the icon will display a dialogue allowing a rebake to be initiated.

Region rebaking in Firestorm: The Menu Bar icon is used to trigger a rebake, rather than a button displayed within the viewer window. The default warning of the need for a rebake (top) may also be displayed, depending upon whether you have left the option enabled or not after its first appearance.

Combined with disabling the initial pop-up (by checking Do not show this again),  this option makes the need for rebakes less intrusive when using Firestorm.

New Buttons

There are three new toolbar buttons in the 4.2.0 release: Asset Blacklist and Sound Explorer, both of which toggle open / close the Asset Blacklist floater or the Sound Explorer floater respectively (each otherwise accessible via the World menu), and Ground Sit – which is pretty self-explanatory.

Snapshots: Flickr, Temp Upload and More

Flickr option on Snapshots floater

Flickr is a popular medium for SL photographers, so having an option to save pictures directly to it is likely to be a benefit to many. With this release, Firestorm obtains Katharine Berry’s code to enable snapshots to be uploaded directly from the viewer to a Flickr account.

In order to work, this functionality requires Firestorm is authorised to access a Flickr account. Therefore, the first time the Flickr tab on the snapshot floater is clicked, a pop-up is displayed, both explaining the need for authentication and what will happen. Clicking on NO on the pop-up will stop the process, and you can use another option on your snapshot floater for saving the image.

Clicking on YES will take you to the Flickr authorisation page, which will outline the possible risks of connecting Firestorm to Flickr (a standard alert page, common when using inter-application authorisation). Read the warning carefully, and if happy, confirm you wish to proceed (refusing cancels the link and denies Firestorm the ability to upload to Flickr).

Confirming that you’re happy to proceed will display a code number on the Flickr web-page. Type this into the authorisation pop-up displayed in Firestorm to complete the authorisation process. Once done, you’ll be able to upload pictures to your Flickr account without further hindrance.

This release of the snapshot floater also includes an option to temporarily upload a snapshot to your inventory. Temporary snapshots are saved to your Photo Album, where they will be available for personal use (e.g. non-transferrable, etc) until your next re-log. Finally for the snapshot floater, all settings changes are saved between sessions.

Use the page numbers below left to continue reading

Curiosity: speaking from Mars

Monday August 27th saw NASA host another news briefing on Curiosity’s progress, which included some amazing new images and well as updates on SAM and recent manoeuvrings with the rover.

The briefing started with a message “from Mars”, in the form of a recording of a greeting by NASA Administrator and former astronaut, Charles Bowden. Carried by Curiosity to Mars and then transmitted back to Earth, the message, lasting just over a minute, represents the first broadcast of a human voice from another planet. While primarily of PR value, the broadcast demonstrated the capabilities available with the rover working with (in particular) the Mars Reconnaissance Orbiter (MRO) when transferring large amounts of data to Earth; the speech represented some 4 megabits of voice data, which was transmitted alongside other information. As it is, in just three weeks, Curiosity has returned more data to Earth than the entire Pathfinder and MER rover missions combined.

SAM takes a Sniff

SAM, the Sample Analysis at Mars, is a suite of instruments designed to analyse organics and gases from both atmospheric and solid samples. On Saturday August 25th SAM was allowed to take its first sample of the Martian air, albeit as an engineering calibration exercise rather than a science experiment. After a slight “blip” in the proceedings when traces of calibration gas which should have been evacuated from the system beforehand were measured rather than Martian air, the tests successfully confirmed SAM’s atmospheric sampling capabilities are ready for action. In the future SAM will not only be able to sniff the air (and “taste” samples of Martian rock and soil delivered to it by the robot arm), it should even be able to tell us what Mars smells like (which, given it is thought Gale Crater has levels of sulphur dioxide present, is liable to be, “Ewww! Rotten eggs!”).

Mastcam Demonstrates its Power

The two Mastcam cameras (M100 (telephoto) and M34 (wide-angle)) have almost completed their characterisation check-outs and the Malin Space Science Systems team is getting ready to turn them over to the MSL science team for the start of science operations. Key to this work has been ensuring the cameras are properly focused, a process that has required using the both camera lenses to take high-resolution images across a range of distances. This has resulted in some amazing mosaics of Gale Crater being produced.

In the above image, Mastcam reveals Gale Crater: the grit-like surface on which the rover sits extends outwards to the South-west for about 125m (406ft), and is followed by a slight dip in the land that extends a further 105m (341ft) from the rover to the lip of a small crater about 20m (65ft) across. Beyond the rim of the crater can be seen what is described as a low-lying “moat” surrounding the flanks of “Mount Sharp”, the middle of which is about 3.7km (2.3 miles) from the rover. Beyond this is a field of sand dunes some 5.5km (3.4 miles) distant, with the base slopes of the mound just beyond them. The regions the science team are particularly interested in extend from about 6.6km (4 miles) from the rover out to about 10.7km (6.7 miles).

The image above shows a zoomed-in view of the region of interest, with the dune field in the immediate foreground. The inset image is of a rock roughly the size of Curiosity (the black dot in the inset image), pictured to hep give a sense of scale to the mesas, and show what the rover might look like if it could be pictured from Bradbury Landing once it starts exploring “Mount Sharp”.

While the area of interest is only 10km (6 miles) from Bradbury Landing, it would take Curiosity 100 days to reach it, were it to drive at full speed – which is obviously not what is going to happen, as there is much to study along the way.

On August 27th, NASA released a panoramic movie of Gale Crater and “Mount Sharp” put together using images from the Mastcam captured on the 8th and 18th August (prior to the focus calibrations being completed). The images have been white-balanced to match sunlight levels on earth, with 640×360 and 1280×720 Quick Time version of the movie available on the JPL website (note the 1280 movie is over 242Mb in size and may take time to download and stream).

Continue reading “Curiosity: speaking from Mars”