Materials processing reaches beta – overview and more

After recent delays as the code passed through QA, the Materials Processing viewer unexpectedly reached beta release on Friday, May 31st with the release of version 3.6.0.275764, accompanied by a blog announcement from the Lab.

What is Materials Processing?

In case you’ve missed the news to date (if you already are, feel free to skip to the info on the viewer :)).

Materials processing brings normal and specular maps to object surfaces in Second Life (prims, sculpts and mesh – but not avatar skin / clothing layers). The capability has been developed as a collaborative effort between TPV developers (notably from the Exodus, Catznip and Firestorm teams), content creators and Linden Lab.

Note: The following two sections are not intended to be a technical discussion on computer graphics mapping or a detailed analysis of normal & specular maps. It is intended purely as a broad, non-technical of the latter.

Normal Maps in a Nutshell

Normal maps (sometimes referred to as bump maps, although they are more rightly the most common form of bump map) are a means of faking high levels of detail on an otherwise bland surface by means of simulating the bumps and dips that create the detail. Normal maps can be created in several ways.

For example, when working with 3D models, a common method is to make two models of the same object: one a very complex, highly detailed model with a high polygon count, the other a much lower polygon count model with significantly less detail. An overlay process is then used to generate a normal map of the detailed model’s surface features which can be applied to the less complex model, giving it the same appearance as the highly detailed model, but for just a fraction of the polygon count, reducing the amount of intensive processing required to render it.

Using a normal map to enhance the detail on a low-polygon model. The image on the left shows a model of some 4 million triangles. The centre image shows a model with just 500 triangles. The image on the right shows the 500-triangle model with a normal map taken from the model on the left applied to it (credit: Wikipedia)

Another common way to produce a normal map is to generate it directly from a texture file. Most modern 2D and 3D graphics programs provide the means to do this, either directly or through the use of a plug-in (such as the nVidia normal map filter for Photoshop or the GIMP normal map plugin). When combined with diffuse maps, the normal map creates the impression of surface detail far greater than can be achieved through the use of the texture alone.

Normal map from a texture: left – the original texture (diffuse map) and its normal map shown as a split view; right – the material resultant from applying both maps to surfaces inside a game (credit: Valve Corporation)

Specular Maps in a Nutshell

In the real world, every highlight we see in an object is actually the reflection of a light source. Surfaces and surface details reflect light differently to one another, depending on a range of factors (material, lighting source point(s), etc.). Specular maps provide a means of simulating this by allowing individual pixels in an object to have different levels of brightness applied to them, giving the illusion of different levels of light being reflected by different points on the object.

When life gives you lemons: a mesh lemon with (l) a normal map applied, and (r) a normal and a specular map together. Note how light is apparently being reflected across the surface of the latter (credit: Mind Test Studios)

Like normal maps, specular maps can be produced in a number of ways, both within 3D graphics modelling programs and in tools like PhotoShop. As shown above, they can be combined with normal maps and textures to add detail and realism to 3D models and flat surfaces.

Second Life itself already includes a very dynamic example of how materials can be combined to create in-world effects in the form of Linden Water. This is created using an animated normal map to create the wave-like effect for the water, while an animated specular map adds the highlights and reflections. The result is a very realistic simulation of moving water able to catch and reflect sunlight.

Some key points about materials processing:

  • Effects will currently only be visible when using the Materials Processing beta viewer
  • In order to be visible in your world view, whether you are creating objects using materials processing OR whether you simply want to see them in-world, you must have the Advanced Lighting Model (previously “Lighting and Shadows” and often referred to as “deferred rendering”) option in Preferences > Graphics enabled
    • Note that this does not mean you also have to enable shadows themselves – you can leave the drop-down set to None and still see materials in use in-world, thus minimising any impact on your frame rates
    • The Lab has done a lot of work on the rendering pipeline to improve performance; as such, if you’ve not run with the Advanced Lighting Model / deferred rendering enabled on your computer due to previous performance issues you might find you can now enable Advanced Lighting Model in your viewer without experiencing a dramatic loss of frame rates
  • When creating objects which use materials processing, note that they will automatically move to the “mesh” accounting system for land impact, which can affect  the impact value assigned objects and models
  • Materials capabilities do not work when under Linden water.

Continue reading “Materials processing reaches beta – overview and more”

SL projects update 22 (2): SSB/A issues, materials, server issues

Server Deployments – week 22

The server channel deployments were delayed 24 hours this week due to Monday May 27th being Memorial Day in the USA.  This being the case:

  • On Wednesday 29th May, the Main channel received the server maintenance project previously on Magnum. This includes bug fixes, comprising two for crash modes and one for BUG-2424 (Overriding “Sitting on Ground” animation while sitting on the ground makes “stand up” button disappear). This deployment also included the LSL support to create and parse JSON-formatted strings, which also included the bug fixes for this capability deployed to Magnum in week 21 (see my SL projects update report from week 21). Release notes
  • On Thursday 30th May, the three Release Candidate (RC) channels received the interest list improvement project deployed to LeTigre in week 21. The core change in this update should reduce scene loading time when entering a new region (again, please refer to my week 21 report for background information). Release notes (BlueSteel, but applicable to all three RCs).

Server-side Baking / Appearance

As noted in these pages, the Lab formally announced the forthcoming arrival of SSB/A on May 29th. This has prompted questions of “when?” Again, as I’ve previously reported, the Lab is proceeding cautiously towards a server-side deployment, even though they are encouraging people to swap to a version of their preferred viewer which is SSB/A-enabled sooner rather than later.

Currently, the two regions for TPV testing have been enabled with the new service and TPVs are putting the new capability through its places – and this has already revealed a reason for the Lab’s understandable reluctance to give out firm dates, as a potentially major issue has been identified.

SUN-74, raised on May 29th, shows that if you are wearing a MOD skin, hairbase or eyes and you enter an SSB/A-enabled region using a non-SSB/A enabled viewer, an alert will appear on your screen which, on clearing, is followed by an innocuous-looking prompt.

The alert and prompt which are displayed when using a non-SSB/A enabled viewer when entering an SSB/A-enabled region
The alert and prompt which are displayed when using a non-SSB/A enabled viewer when entering an SSB/A-enabled region (image courtesy of Whirly Fizzle)

Clicking YES in reply to the prompt can result in the currently worn skin / eyes / hairbase to become irreparably corrupted, with a skin turning  a mixture of black / invisible and eyes turning white. Rebaking will not fix the issue. Relogging to an SSB/A-enabled viewer seems to result in the avatar rendering as a cloud, and / or ending up with a default skin and ruthed. Replacing the affected items (skin and/or eyes and/or hairbase, depending on which has / have been corrupted) with others from you inventory will fix the issue, but re-wearing the corrupted item(s) results in the avatar once more appearing corrupted (and again ruthed, if running an SSB/A-enabled viewer).

Whirly Fizzle demonstrates the result of the SUN-74 issue
Whirly Fizzle demonstrates one aspect of the SUN-74 issue – on a non-SSB/A viewer, her MOD skin has turned black / invisible and her MOD eyes have turned white as a result of entering an SSB/A-enabled region and responding with YES to the given prompt.

Continue reading “SL projects update 22 (2): SSB/A issues, materials, server issues”

Lab formally announces Server-side baking / appearance

Regulars to this corner of the SL blogsphere know I’ve been covering Project Shining – the various projects the Lab is currently undertaking to improve Second Life on the technical front in order to give us all a (hopefully) better experience.

Part of this work includes Project Sunshine, which is more colloquially know as server-side baking (SSB) or server-side appearance (SSA) or server-side baking/appearance (SSB/A) – the choice is yours, depending on personal preference, and which I’ve covered throughout numerous reports in this blog. The primary aim of project Sunshine is to resolve the issue of avatar bake fail – those situation wherein your avatar (or other avatars) fail to render correctly to either yourself or to others around you.

Today, the Lab itself moved to formally announced the forthcoming arrival of SSB/A with a special blog post of their own on the matter, which includes a short video explaining matters:

As the post indicates, SSB/A is being deployed in three parts:

  • A viewer update  – which is available now for the majority of commonly used SL viewers
  • The deployment of server-side changes, which should be commencing shortly
  • A further viewer-side update once the server deployments are completed.

The server-side deployment will take a while to complete, as the new service will require a degree of testing. As such, it is expected that a number of regions on the main grid will be enabled for SSB/A (if they have not been already), and these will be used to measure performance over a period of time prior to a decision being made on “throwing the switch” to enable the entire grid is SSB/A enabled (the test regions may even be scaled-up over time, depending upon how the initial testing goes.

Server-side baking: find out what it is and why you'll need to update your viewer if
Server-side baking / appearance: must viewers should (or will shortly) support SSB/A – make sure you update to a current release of your preferred viewer to avoid seeing grey avatars as the server-side of the new capability is deployed in the coming weeks.

As you won’t be able to tell which regions are using the new SSB/A service and which are using the existing avatar baking service, it is important that you make sure you are using a viewer which supports both capabilities – otherwise you might find yourself encountering grey avatars in increasing numbers. This means updating to a viewer which has the SSB/A code; at the time of writing, these are:

Doubtless, Catznip (R8 with SSB/A has been in development for a while), Dolphin and Exodus will have SSB/A-capable viewers out shortly as well.

Those wishing to obtain a further overview on SSB/A and also on the most recent updates out of LL on the server-side deployment plans are welcome to refer to the following reports from this blog:

SL project updates 22 (1): server releases, SL viewer update

Server Deployments – week 22

The server channel deployments are delayed 24 hours this week due to Monday May 27th being Memorial Day in the USA. As usual, the latest updates, feedback and comments can be found on the deployment discussion thread. Anyone encountering a specific bug is asked to file a JIRA.

Second Life Server (Main) Channel

On Wednesday 29th May, the Main channel should receive the server maintenance project previously on Magnum. This includes bug fixes, comprising two for crash modes and one for BUG-2424 (Overriding “Sitting on Ground” animation while sitting on the ground makes “stand up” button disappear). This deployment also includes the LSL support to create and parse JSON-formatted strings, which also includes the bug fixes for this capability deployed to Magnum in week 21 (see my SL projects update report from week 21). Release notes.

Release Candidate Channels

On Thursday 30th May, the three Release Candidate (RC) channels should receive the interest list improvement project deployed to LeTigre in week 21. The core change in this update should reduce scene loading time when entering a new region (again, please refer to my week 21 report for background information). Release notes (BlueSteel, but applicable to all three RCs).

SL Viewer

The SL release viewer updated to 3.5.3.276452 on May 28th (release notes), leapfrogging it ahead of the last of the beta channel viewers. This release may include updates which may lessen the issue of “missing prims” which people have been encountering over the last couple of months.

The anticipated move of the Materials Processing project viewer to a beta release status in week 21 did not happen. The current project status is unclear, but hopefully the move to beta should occur sooner rather than later.

Other Projects

As a result of the long weekend in the USA (Memorial Day) it is currently a light week for news on the projects LL are working on for SL. Expect this to be rectified as the week progresses!

SL projects update week 21 (3) more server, viewer and interest list bits

Server Deployments

As noted in part 2 of this report, due to issues with the JSON deployment made to all three Release Candidate channels in week 20, there was no Main channel deployment in week 21.

Maestro Linden likes to keep fit while chairing the Server Beta meeting
Maestro Linden likes to keep fit while chairing the Server Beta meeting
  • On Wednesday May 22nd, the three Release Candidate (RC channels should each receive the following updates:
  • Magnum received an update to the server maintenance project deployed to all three RC channels in week 20 which includes fixes for bugs within the LSL support to create and parse JSON-formatted strings. Release notes.
  • BlueSteel received a further update to the experience tools project, and there should be no visible changes with this update. Release notes
  • LeTigre received an update related to the simulator’s interest list subsystem which reduces scene loading time when entering a new region. Release notes.

As mentioned above, the Magnum updates included a fix for an issue within the LSL JSON capabilities deployed to the three RC channels in week 20. Currently, the fix appears to have resolved the issues, so it is possible the JSON capabilities will reach more of the grid once more in week 22.

“json is a javascript way to describe data and pass data between scripts or services. In that regard one of the biggest benefits of json in LSL is for improving the ability of LSL to interact with 3rd party APIs,” Kelly Linden explained at the Server Beta meeting on Thursday May 23rd after questions were asked as to the purpose of the capabilities. “Because it is simple and relatively ‘complete’ even non-javascript services use it on the internet to exchange data …  I’ve been interested in services like parse.com for example which let you store and retrieve data very easily … in json …  https://parse.com/docs/rest this is what would work with LSL.”

Kelly also noted the new capability might be used within LSL, but with a small caveat, saying, “If you are only working within LSL there may be some specific cases where it is beneficial, but other string to list functionality will probably be fine.”

The package deployed to Magnum also included a fix for the bug introduced into the RC channels in week 20 which affects control event triggering in attachment’s child prims after changing regions. However, the fix as deployed is described as “interim”, and corrects the problem (which is described in full here) by reverting the fix for  SVC-8227 (ApplyImpulse now works only in the root prim). Referring to the situation at the Server Beta meeting on Thursday May 23rd, Maestro Linden said, “we hope to get a ‘real’ fix in for that sometime in the future.”

SL Viewer

Further to part one of this report, Oz Linden has clarified the function of the new “Willing to update to release candidate” option in project / beta viewers. As I’ve previously mentioned, when a viewer is believed to be of release quality, it will be put into a release candidate, which will be released to a chosen number of users (the number determined by Linden Lab). By leaving this new option checked, users are indicating that they are willing to receive any such release candidate updates if they are selected by LL to receive them. Unchecking the option means that a user will not be included in the count for any release candidate update, and so will not receive any updates until such time as the viewer reaches release status.

The new "Willing to Update" option
The new “Willing to Update” option

Even so, leaving the box checked does not mean a user will automatically receive release candidate updates – as noted above, LL will determine the total number of users who will receive any given release candidate updates. These will be chosen at random from those who are using the project / beta viewer, and once this number has been reached, no further users will receive the update regardless as to whether the option is enabled or not. If necessary, the selection process can be additionally targeted at specific operating systems, but the Lab currently don’t have plans to use this capability.

While the new release process is not dependent upon Materials Processing project viewer reaching a beta release status, it still appears unlikely that the new processed will be deployed until after Materials has done so.  Once the new release process has been deployed, Oz indicates that it is likely that a number of viewer candidates will appear – such as a bug fix candidate a Snowstorm candidate and possibly others as well, although the exact timing and spacing of the releases is unclear.

Interest List News

Andrew Linden
Andrew Linden

Also attending the Server Beta meeting, Andrew Linden provided a further update on the interest list updates deployed to LeTigre. These amount to a number of fixes and updates to the code.

The main thing in that RC is some minor tweaks to help the scene load a little faster on login and teleport. The effects are small, especially in the case where you have a full cache for that region so I’m guessing that no one has really noticed the scene loading any faster.” Andrew explained, “There was [also] one minor bug I fixed for people with really low-bandwidth settings… the updates were not properly getting re-sorted when the camera moved around, so the scene would continue to stream in based on where you were standing when you arrived but most people with >500kbps bandwidth shouldn’t notice that problem — the scene usually loads fast enough now. ”

He went on to reiterate that the LeTigre deployment also includes a partial fix for the “Meeroo update” problem of objects not updating correctly after being outside of the camera’s field-of-view. Again, as mentioned in part 2 of this report, the fix works with affected objects which are up to 10 metres away.  However, he believes he now has a more complete fix for the problem, but has yet to test it. He also believes that the issue causing the “Meeroo update” problem may also be responsible for BUG-2644 (pathfinding characters not updating behind the camera) is the same problem as the Meeroo animation, and is hopeful his intended fix will correct that as well.

Finally, the update fixes a minor bug where the green avatar dots on the mini map would not update correctly for avatars behind the camera.

The downside to the LeTigre update is that it did introduce a crash mode, which Andrew described as “rare… only about 6 per day,” and which is currently being investigated.

Server-side Baking / Appearance

As noted in week 20, it has been hoped that the server-side of the SSB/A code would be enabled on two test regions on the Main grid.  These regions are Intended specifically for TPVs to carry out functional tests on the viewer code away from the distractions of broader issues which interfered with testing on Aditi. As such, they should not be considered a sign that deployment of the server-side code had commenced. It had been hoped that the two regions would be enabled this week, but at the time of writing, this is not yet the case. This doesn’t necessarily mean the project is delayed, however.

In terms of overall deployment, matters are unlikely to have changed since week 20, and the Lab will still in part be looking at this initial “TPV test” period as an opportunity to gain further additional metrics on the system and to look for anything untoward occurring prior to committing to possible dates.

SL projects update week 21 (2): server releases, Interest list, group ban list

Server Deployments

As always, please refer to the release forum  thread on the weekly deployments for the latest updates and discussions.

Due to issues with the JSON deployment made to all three Release Candidate channels in week 20, there has been no Main channel deployment in week 21.

On Wednesday May 22nd, the three Release Candidate (RC channels should each receive the following updates:

  • Magnum should receive an update to the server maintenance project deployed to all three RC channels in week 20 which includes fixes for bugs within the LSL support to create and parse JSON-formatted strings, which I reported on in my week 20 report.  This week’s update fixes some bugs related to the changes. Release notes.
  • BlueSteel should receive a further update to the experience tools project, and there should be no visible changes with this update. Release notes
  • LeTigre should receive an update related to the simulator’s interest list subsystem which reduces scene loading time when entering a new region. Release notes.

Interest List Bits

Andrew Linden
Andrew Linden (image captured by Opensource Obscure)

As noted in week 18, Andrew Linden has been working on fixing a bug which he specifically mentions in terms of Meeroos, but which can affect other animals as well, which he described as:

If you turn your camera away from a crowd of Meeroos, wait several seconds, then turn back around… the Meeroos will be updated, but not quite in the right order. So sometimes you’ll see a head move to the new position, then a fraction of a second later the rest of the body.  So I have a theoretical fix that doesn’t crash the simulator (anymore)

Providing an update at the Simulator User Group meeting on the 21st May,, he said, “I do have a little news about the Meeroo animation bug… I wasn’t able to fix it after all… but I did reduce it or eliminate it for meeroos that are nearby (closer than 10m).” He also noted an issue with the Meeroos’ animations which he believes to be “Mostly by slow scripts, low bandwidth connection, or general lag,” which results in the Meeroos walk animation appearing to be broken. He believes the fix in his new project will enable nearby Meeroos to update correctly when being viewed, and he’ll be revisiting the problem once the initial fix has been deployed, although he’d be interested in hearing back on how well the partial fix works, once the fix has gone out.

Baker Linden: Group Ban List and Other Work

Baker Linden: getting closer to working on group bans
Baker Linden: getting closer to working on group bans

Baker Linden reports that he is making “really great progress” on fixing leading and trailing spaces in display names. He’s currently working on some unit tests and dealing with a couple of minor issues, but he hopes that overall it will be ready for QA later in the week. He did admit that, “I’m unsure how useful it’ll be — anyone that wants to game the system will just append a bunch of other characters that appear before letters… But at least whitespace will be stripped.”

Once this has happened, he’ll be finishing-off the fixes for name searches using the People floater and the correct removal (unmuting) of muted avatars and objects from the viewer’s mute list. As soon as these two issues have been dealt with, Baker will be pushing forward with the new group ban list capability as requested in JIRA SVC-8127.