Lag in OpenSim and Second Life, and how to deal with it.

Lag is a common problem in almost anything computer-related, yet to most people, it needed Second Life to make them aware of how annoying it really is. This post tries to address the issue and offer possible solutions.

What is lag?

Lag is the perceived delay between the input and the output of a computer program. This means that, at its core, it is a human-machine interface problem. It stems from the fact that I expect a simulated environment to respond as quickly as a natural environment would, and, if it doesn’t, that the simulated environment feels “broken”. I use this in a very broad term, since lag is not just limited to 3D-environments, but appears on every kind and level of human-machine interaction. Still, for the scope of this entry, we will limit our focus on OpenSim-specific topics. In that regard, we have to distinguish three different kinds of lag:

1. Viewer-side lag.

The most common and well-known kind of lag is the so-called “viewer-side lag”, meaning that the viewer, which is the software you use to access a Second Life or OpenSim grid, does not respond in a timely fashion to provide a seamless “fluid” experience. Whenever your avatar moves or turns, and the screen stops for a short time, you’ll experience it.

The first question here is: When do we experience something as laggy? A good tool to answer that is the statistics bar (ctrl+shift+1 on a PC, cmd+shift+1 on a Mac), which will, among other things, at its very top display the current frame rate. Since every computer simulation is actually a very rapid succession of still frames, which, if watched uninterrupted, provide the impression of a continuous movement of events, the rate at which these frames will be rendered (i.e. displayed on the screen) is crucial to whether we perceive lag or not. A rate of 24 frames per second (fps) will typically be sufficient to make the frames indistinguishable from one another to the human eye, and give the impression of a continuous flow of events. Thus, most videos and television broadcasts will use a standard framerate of 24 or 25 fps. Below that, a frame rate between 12 to 24 fps will be visibly “laggy”, but usually not enough to break immersion to a critical level, whereas frame rates below 10-12 fps will become increasingly unpleasant and unacceptable.

The frame rate you will be able to achieve will depend on several factors. Most importantly, it will depend heavily on the graphics hardware you have in use on your computer. Since the viewer is a very graphics intensive application, it will require at least a decent to good graphics card. Plenty of RAM (talking about 4 to 8 GB here) would be useful as well, while computing power (CPU) is of a lesser importance. Tateru Nino (whose blog is highly recommended) provides a very useful guide to a decent computer setup for the viewer software.

Still, even on a high-end PC, you are likely to encounter lag. This is due to the fact that SL and OpenSim are, at their core, user-created environments, while most computer games, by comparison, are created by experts with knowledge of and an eye to how efficient their environment will perform on even low-end computers. The key here is: Do not try to display too much at once. If you’re in-world and  in an environment you don’t control yourself, one of the few things you can do is lower your graphics setting (especially the draw distance) so only objects in close proximity are being displayed and in a lower quality.

Mainly, though, you will need to learn how to make your own environment (your avatar, and the things you display in-world) less lag-inducing. This starts with the simple amount of things rezzed, because every additional object will be one more for the viewer to render. So try to keep your objects to a minimum. Likewise, the amount of textures displayed will influence lag to a great deal. At any given moment, you’re surrounded with literally hundreds of textures anywhere you go, and if you think of how your web browser would behave, if it would access a website with 100 pictures on it (all between 512×512 and 1024×1024 pixels in size), you might understand why the viewer can be overloaded with that. Sadly, most creations you’ll encounter aren’t built very efficiently, using too large and too many textures for the effects they’re trying to achieve, which will make it hard to optimize this effect. If you’re creating something yourself, try not to use high-resolution textures (anything bigger than 512×512 pixels) and try to re-use the same texture, or parts of it, on as many surfaces as possible. Likewise, if you’re using a creation of someone else, try to understand how it will impact your viewer. (At least compare the framerates you get before and after you rezz the item.)

In addition to the resolution of textures, the kind of texture used will impact your viewer differently. It’s a big difference if your texture uses transparency or not, since with transparent textures, the viewer will not only have to render the surface your texture is on, but also everything that’s behind, and try to determine what, and to which extent, is visible. So in this regard, try to keep the use of transparency to a minimum. (You will have noticed that, for example, in almost every computer game, hair is mostly a non-transparent, rigid part of the avatar mesh itself, whereas in SL/OpenSim, it is a partially flexible object using lots of transparent textures, thus creating a huge performance problem.)

Finally, the complexity of an object will have a big effect on viewer performance. Sadly, it’s not sufficient to just count prims to determine this, which is why Second Life introduced a new system of calculating the impact of an object. Also, the Imprudence Viewer has a highly useful feature that will display the vertex count of an object upon inspecting it, giving you a much more detailed information about its complexity. To give you a general impression, the avatar mesh (your shape, without any attachments such as hair, prim clothes, etc.) has about 4000 vertices, which is a medium complexity for an avatar mesh. In comparison: a single cube prim has 54 vertices, a torus prim (the most complex basic primitive) has between 625 to 1250 vertices (when hollow), and a sculpted prim has about 1000 vertices. On the other hand, average avatar hair has around 50.000 to 100.000 vertices and meshes are limited to a maximum of 65536 vertices, which goes to show just how (unnecessarily) complex the environment in SL and OpenSim can be, and how important efficient creation really is. The statistics window (see above) also shows you the amount of triangles in your current frame (KTris Drawn per Frame), and it seems a sensible amount would be around 1000 (which is 1 million triangles!), whereas anything above 2000 KTris (2 million triangles) will seriously impact performance.

The bottom line here is: Both users and creators need to learn to be much more efficient in how they shape the environment and themselves, as most environments and avatars in both OpenSim and Second Life are much too complex in and of themselves.

2. Server-side lag.

In addition to all the things that can impact viewer performance, the server itself can also be lagged down, mostly due to different reasons. While the viewer displays the world to you by producing the graphics on your screen, the server’s job is to send the data to your viewer fast enough and to update any changes that occur in-world. Every time an avatar or object changes position, every chat message that gets transmitted, every time properties of an object get changed (through editing, for example), every script that’s running, will impact the servers performance.

Again, the most important factor here is the hardware. While you have no influence on that in Second Life, OpenSim offers you the ability to run your server on a variety of setups. The most important resource is, once again, RAM, and you should at least reserve 1 GB for your OpenSim installation. While computing power isn’t a high priority, it would be great to have at least one core just dedicated to OpenSim itself. And just if you’re the only (or one of a handful) of avatars on that simulator.

Because contrary to popular belief, the one thing that impacts server performance more than anything is not the amount of regions, or prims on these regions, but the amount of avatars. And logically so, because prims and regions are just laying dormant unless someone visits, and the simulator will have to send data packets to them while also keeping track of their avatars movements, and if there’s more than one avatar, not only will the data need to be sent to several recipients, but also the data about each other (avatar appearance, position, facing, chat, etc.) will need to be sent between each user. Thus, it is wise to calculate at least 250 MB of RAM for every avatar you expect to be simultaneously on your simulator.

Still, while there’s not much you can do to reduce the amount of avatars, there are some things you can do to reduce the impact they have on your simulator. Physics are one of the major sources of lag. Every time an avatar moves, the simulator needs to calculate its collisions with the ground it’s walking on, and the objects it might “bump” into, so every object that’s not crucial to collisions (like plants, animals, waves, light effects, etc.) should be set to phantom, which will reduce the weight on the server as these objects will simply be ignored by the physics engine. Again, most professional 3D-games run smooth because they come with a very crude mesh, which will not calculate physics on the shape of objects, but on a very rough boundary around them.

A second source for server-side lag is the scripting engine. While scripts are essential to create an immersive environment, too many scripts will considerably slow down the server. Consequently, you should not use scripts unless absolutely neccessary, and even then you should learn to use them efficiently, as there’s many different ways to achieve a certain result. Luckily, many things that required a scripted solution in the past are now part of the viewer software, such as radars (which display information about the other avatars around you) and animation overriders (which allow you to customize the way your avatar moves). On the other hand, certain tasks which can be handled by the viewer are sometimes handled through scripts, either out of convenience or in a misguided attempt at providing some sort of DRM. Objects rezzers allow you to conveniently move and position very large builds, but once you position them, make sure to remove the scripts again, otherwise you will have a script running in each and every part of the build you just rezzed. Likewise, almost any attachment comes with resizer scripts now, which allow you to fit it to your avatar easily, but will still leave you with several scripts (or, when badly scripted, with one script in every single prim of the attachment) running in your attachments at all times if you don’t remove them. Sadly, many times we’re not even aware of the scripts that are in effect both in the environment, and on the avatars themselves, but there is at least a script function that is able to return the total script count of an avatar or object, which can be combined with an automated reminder to people to lower their script count somewhat.

3. Network lag.

Finally, all data between the simulator and the viewer needs to be sent through a network connection, and while usually this shouldn’t be a bottleneck, it sometimes can also cause lag.

First of all, the kind of connectivity necessary for OpenSim and Second Life is at least 100 kbyte/s downstream (i.e. receiving data from the server in the viewer). Consequently, if you’re running OpenSim, you need at least the same amount of bandwidth upstream (sending data from the server to the viewer) for every avatar that’s supposed to visit your simulator at the same time. So a regular DSL home connection might be sufficient for yourself and an occasional friend visiting, but for a public installation that’s open for all, you might need your own server in a datacenter.

One of the most prominent times you’ll experience network lag is whenever you connect to a simulator and the whole environment data, as well as your avatar and inventory needs to be sent to you, which may well take several minutes to complete. During that time, large amounts of data are being received and cached, and the viewer will be busy rendering the scene and loading and displaying textures all around you, which will result in a combined viewer/network and, if the simulator is very busy, even server lag.

Outside of that startup phase, network lag typically only occurs when either a lot of data needs to be sent simultaneously to a number of recipients (for example when a complex object is being rezzed with many people around watching, or many avatars in one place changing their appearances/outfits). If it occurs during your regular interaction with the simulator (for example things take a very long time for you to rezz) and other avatars around you don’t experience the same, your connection might be too slow for OpenSim/SL to be enjoyable. (I’ve been running the viewer on a tablet with a mobile connection of 25 to 30 kbyte/s and, while it didn’t crash and everything worked alright, it wasn’t really the kind of experience you want to have unless you need to.)

While there’s not much you can do about network lag (other than getting a better internet connection), many solution for either viewer  or server lag will affect the network as well, as it will require less data to be sent and received.

Conclusion.

Lag is a very common problem in both OpenSim and Second Life, and it requires education and effort on all sides to handle it. Because of their open and user-centric nature, there are no “consumers” in the traditional sense in the Metaverse; we all shape and affect the environment constantly simply by being there the way we are. Along with this freedom and control comes a great deal of responsibility to make this shared environment more enjoyable for everyone else.