Second Life on GitHub
community.secondlife.com
community.secondlife.com
The Firestorm viewer sources are at "https://vcs.firestormviewer.org/". It's a major fork.
Technical documentation of the client-server protocol is at "https://wiki.secondlife.com/" It's out of date, but it helps.
The real progress is that the build procedure has been cleaned up. It used to require ancient versions of Visual Studio on Windows. Firestorm can be built on Linux. I've built it and submitted patches that went in.
This is just the client side. The server is proprietary. But there's a compatible server, Open Simulator. It's not a clone; it's in C#, while the original is in C++.
There are meetings for third party viewer developers. Both inside Second Life, and on OSGrid, which is a third party grid of simulators. That really is the metaverse - a federated system of 3D worlds. It's sluggish, because most of the people running regions don't have enough server power. Think Mastodon for 3D.
I'm been writing a new client in safe Rust for the last two years. I'm trying to see how much can be done in parallel. The viewer isn't ready for public use yet. I've posted videos made with it.
There are thousands and thousands of active regions and thousands of users (https://opensimworld.com/). It's not millions, but it s the best self-hosted virtual world ecosystem.
Is a "viewer" a Second Life term for "game client"? And is Firestorm a particular FOSS client for the game?
If so, that's really interesting... like you can access the same shared Second Life world but through a custom client? Does it reuse existing game assets or you free to reinterpret character models, textures, effects, etc.?
Assets are downloaded by the viewer from the SL servers.
Viewers are "game clients" yes. The viewer is intended to mostly be a dumb receiver. The server does most lifting.
Firestorm is a fork of the official viewer. It forked about the time I played SL but seems to have grown into it's own these days.
Assets are downloaded from the servers. Users can upload and optionally gift / sell their own.
Quick overview:
Second Life / Open Simulator worlds are divided into "regions". Each region is 256m on a side on Second Life, and can be other sizes in Open Simulator. The world is about the size of Los Angeles. The viewer starts out by talking to a login server for a grid, which authorizes it to connect to a region. The viewer then talks to the region. There are short messages over UDP, and some bigger stuff over HTTPS.
Once a viewer is talking to a region, it asks the region server about adjacent regions. The viewer can then talk to a few adjacent regions, too, showing the user a big, seamless world. As the user moves around the world, connections are made to new regions, and connections to regions out of range are dropped. This is how the world scales. There are about 27,500 live regions today, each with its own simulator program, always on even if nobody is using it. Each simulator program takes about one CPU and under 4GB on a server. Second Life is currently hosted on AWS, with dedicated servers. This scales well; the world could be scaled up substantially if there was demand.
The always on feature runs up costs, but the business model is that users rent land. So costs and revenue stay in balance. Regions really are always on - trains run through regions where no one is around, for example. This architecture made more sense when Second Life had their own servers in a co-location facility.
The region servers talk to their direct neighbors. This allows movement across region boundaries. If a region goes down or is restarted, the viewer will show a water-filled hole with steep sides where the region is supposed to be. So the overall system is fault-tolerant.
Content is stored on asset servers, separate from the region servers. These are ordinary AWS web servers, front-ended by Akamai caches. Content includes textures (images in JPEG 2000 format, which is a pain to decompress fast), meshes (a documented but nonstandard format), sounds, etc. There's petabytes of content. The region servers tell the viewers what hashes (UUIDs) to ask for, the viewers get the content from the servers with ordinary HTTPS requests, and all this is assembled in the viewer into 3D images for the user.
Any user can upload new content. Uploading an image costs about US$0.05. Uploading a 3D model might cost $0.50. This is a one-time charge. As long as someone, somewhere, is using that content, it's kept on the servers. There's a garbage collection run every month or so.
Voice is outsourced to Vivox. (Vivox has problems. Not recommended for new designs.)
Coming soon is "puppetry". Second Life is now testing full body and face tracking, for those who want their avatars to run more than canned animations. This involves an entirely new data path by which viewers talk to other nearby viewers via a server that just forwards data. Second Life is getting VRchat features.
So that's what a viewer is. And that's what the architecture of a big metaverse looks like.
If they support VR now I might give it another look.
The idea behind "puppetry" is that not everyone will use it, but performers and people who want to be looked at will. The audience doesn't have to wear trackers. I'm looking forward to live theater in Second Life.
I've been to puppetry virtual meetings in Second Life. Someone rigged for full body tracking and with a good microphone dominates the meeting without effort.
I had no idea this existed, and have been looking for it [conceptually] for years. Thanks!
At the time most people thought the open ecosystem of interconnected endpoints was destined to dominate the global information storage, because smtp had outcompeted proprietary messaging systems, and the url was looking to displace localized hypertext systems and the open web looked unstoppable.
That was before the reactionary trend where gmail overtook the smtp momentum and actors such as Facebook expanded their walled gardens by absorbing or straight up acquiring forums and smaller players. Facebook wasn't unique in their business model of locking up as much of the world's information as possible into their platform, but they were certainly the most successful.
That made it the more telling when Facebook made their pivot into assisted reality public, starting with naming their walled garden the metaverse, a term taken right out of the most visionary futurists of yesteryear! Let's see if the time has come for the pendulum to swing back. It'd certainly be most fitting if the term itself gave new life to the open interconnected ecosystems to show us again what's possible with permissionless innovation.
You might want to take a look at the Matrix.org approach to this: https://thirdroom.io/
Fun fact: Apparently Matrix (the protocol itself) was made after a brainstorm session on how to create an improved and decentralized / federated Second Life game :)
Out of all the metaverse projects being worked on, SL coming out on top would be an incredible development
Good a peak as any.
Virtual worlds are not entertainment media. There are entertainments within them, but the worlds themselves are neutral. That's true of Decentraland, Second Life, Sominium Space, and even VRchat. You have to make your own fun. Or find some place where someone else has made something fun.
The world itself is completely indifferent to you. It's not like playing a game, where the game forces you to do something immediately. It's like arriving in a new city on a bus and being dropped off at the big bus station. (Second Life's Social Island 10 is the Port Authority Bus Terminal of Second Life.)
This completely throws many new users. They want to be entertained, or told what to do. If your idea of a good time is being given access to a good workshop, Second Life can be very satisfying. Minecraft players do fine in SL. FPS gamers, not so much. If you're not very creative, it can be extremely frustrating. A majority of the population is not that creative.
Second Life has a social scene, and until you figure the place out, you're on the outside looking in. This drives many new users nuts. Some try being annoying. Then they discover the hard way how Second Life handles that. They get banned from a club by the club owner. They find that some clubs share ban lists, and now they're locked out of multiple venues. Anyone can ban anyone from their land for any reason, and Linden Lab will not interfere. Linden Lab itself does do some rule enforcement, but you have to file a complaint and wait a few days until the tiny Governance unit gets around to it. This is for situations where you'd call the cops in real life. There's no outsourced army of minimum wage moderators armed with ban hammers. It's all social pressure.
You can get a vehicle and just drive around. You'll pass houses, stores, gas stations, art galleries, coffee shops, farms, malls, vacant lots... Just like real life. It turns out that if you let people build, mostly they build familiar stuff. There are castles, fantasy areas and space stations, but mostly, people replicate suburban America.
Much of Second Life is rather banal. Linden Lab decided to build an island of unfurnished suburban houses with nice landscaping and rent them out, for people who wanted a pre-built lifestyle. After five expansions, there are now over 60,000 of those, with each user paying about US$100/year. People have parties, BBQs, and other suburban stuff. Really.
The technology is not the problem. Figuring out what to do once you have the technology is the problem.
I used to play MUDs back in the day and they had a protocol that, in theory, could allow you to move from MUD to MUD and transfer your character. I don't think it ever got really popular though, due to all the problems you can imagine from trying to seamlessly move between completely different MUDs. This is all from memory, so I could be mistaken about specifics.
When second life first came out I think I logged onto it a few times and then concluded it was a gimmick and haven't really been back since. I think your post explains very nicely why it never drew me in, I thought I was logging into a game.
Perhaps it was just a marketing person writing this who googled "Big open source github".
Perhaps some of these issues have been fixed by now, but the Linux project has been going on just fine without GitHub (git was even invented for the Linux kernel project itself even before GitHub existed) so there doesn't seem to be any reason to switch even if all the mentioned problems were to be fixed
Just a theory.
For example, his take on security bug prioritization. His argument is that all bugs have potential security ramifications in the kernel, therefore calling something a security bug is not useful, it's just a bug.
"Security people" want special treatment of their reports and he refused to give it to them.
It's not like Linux never has security vulnerabilities, and if he's actively driving people away with his hostility... what happens when someone actually catches a mistake of his but doesn't want to deal with reporting it to him because he's so thorny? What happens when he decides to retire or dies? Is Linux going to fragment into petty warlordism because the only core contributors who managed to thrive in that atmosphere were equally tough-skinned and vicious? It reminds me of dictators' empires that collapse after the strongman goes away, unless another strongman can immediately take over and consolidate power.
I'm not saying that he needs to be all nice and accommodating and accept crap PRs all over the place, but there are ways of gentling correcting people and encouraging them to fix mistakes while not driving them away altogether. Some of those might've turned out to be valuable contributors with some coaching, rather than being turned off altogether.
Now I wonder how much of the abysmal adoption of Linux on the desktop is because of hostile attitudes like that. Could Linux have altogether obliterated Windows and macOS if it had a more welcoming, inclusive community that took different perspectives and user needs into account? Maybe they never wanted that, preferring a clear kernel/userspace separation? I dunno. Of course Linux is super valuable on servers and in embedded devices, but how much of its greater potential was limited by the arrogance and hostility of one man?
shrug On the other hand, look at more "community-driven" FOSS efforts that inevitably fragment and diverge from their original core mission... the modern Web is a mess (looking at you, ECMAscript), Mozilla has its tentacles everywhere while Firefox goes down the gutter, Wikipedia was taken over by a cabal of elite admins that care more about power than users, the BSDs are mostly niche now... maybe there IS an argument to be made for a strongman, dictatorial vision?
Just food for thought.
So, while your thought experiment is probably fun, we already have the answer to it.
instead of your text renderer is really frustrating to
read because of all the seemingly random line breaks
throughout his comment.
- word-wrapping (the correct approach is to indicate preformatted text in markup and allow the viewer to control wrapping for all other text)
- his own demeanor (Linus stepped down in 2018 from his role as BDFL of Linux to "get some assistance on how to understand people’s emotions and respond appropriately").
Linus wrote the git version control system specifically to suit the development of Linux.
GitHub made git mainstream, and they made lots of great features for the masses.
But there is no reason for the Linux project to change the way that they like to work just because of that.
Still, a copy of the Linux source tree is also hosted on GitHub. So it seems to me that everyone gets what they want. Linux get to continue their development in the way that they like to do it. People on GitHub can easily find the copy of Linux on GH, and browse the sources there. And they can create forks of the Linux sources on GH too.
It's an easy mistake to make, the code is there and there's nothing to suggest it's a mirror.
I've also never understood why GH allows turning off issues, but not turning off PRs
EDIT: the viewer code is LGPL , but forked Third party viewers are subject to particular terms
Terms DO have meanings whether contrarians and the argumentative want to acknowledge those meanings or not.
> Open-source software (OSS) is computer software that is released under a license in which the copyright holder grants users the rights to use, study, change, and distribute the software and its source code to anyone and for any purpose.
https://en.wikipedia.org/wiki/Open-source_software
Citations for this sentence (many more in the full article):
- https://books.google.com/books?id=04jG7TTLujoC&pg=PA4#v=onep...
I personally prefer to be effective at communication, so I don't just use words to mean something different than what everyone else thinks.
And “what everyone else thinks” is mostly different from the OSI definition.
Sorry, I meant: "everyone else except 3 or 4 people on ycombinator"
> Many people think that Open Source simply means availability of the source code of a project, but that does only tell part of the whole story.
> The Open Source Initiative (OSI) provides a commonly accepted definition of what constitutes Open Source. To summarize that, in order to be constituted Open Source,
>> a work has to allow free redistribution,
>> the source code needs to be made available,
>> it must be possible to create further works based on it,
>> there must be no limitations of who may use the work or for what purpose (so something like "no commercial use" or "no military use" won't fly with Open Source),
>> the work must not require an additional license on top of the one it comes with,
>> and finally, the license must not depend on a specific distribution format, technology or presence of other works.
> So, you see, it goes way beyond "the source code is available", in fact, a whole lot more requirements are stated that must be fulfilled in order for a work to really be considered Open Source.
https://github.com/readme/guides/open-source-licensing
Also, GitHub's most starred repo is freeCodeCamp (359k stars - https://github.com/freeCodeCamp/freeCodeCamp). Linked right in the repo's README, freeCodeCamp defines open source as:
> Open Source Software is code that is publicly available for people to view, modify, and share.
https://www.freecodecamp.org/news/what-is-open-source-softwa...
Maybe you're thinking of "looky loo" software or something? But if you can't change it and if you're not allowed to share your changes then it is NOT open.
The Open Source Initiative is one of many groups that define open source software as including the right to not only view the source code but also to modify and redistribute it. Others include
Merriam-Webster Dictionary:
> of software : having the source code freely available for possible modification and redistribution
https://www.merriam-webster.com/dictionary/open-source
New Oxford American Dictionary:
> denoting software for which the original source code is made freely available and may be redistributed and modified
https://subscription.packtpub.com/book/web-development/97817...
And the authors of all of the academic literature cited in https://en.wikipedia.org/wiki/Open-source_software
Those terms don't apply if you use the viewer to connect to non Second Life servers, such as Open Simulator. For Open Simulator, you have to deal with a somewhat grouchy lead developer who does not suffer fools gladly. (He's helpful if you don't waste his time. He ran a test region server for me and looked at what I was sending to get something to interoperate.)
Some viewers turn on extra features when talking to Open Simulator. The main one is "varregions". Second Life is divided into squares 256m on a side. Open Simulator supports other sizes. Also, there are several competing money and asset systems for Open Simulator grids, and third party viewers support those.
This is the metaverse at the level where it actually works.
Of course, this policy changed a few months after I stopped playing with OpenSim. I think RealExtend had something to do with it. I remember playing around with their viewer and being impressed, but never doing much else with it.
[msg_type variable_length_message]
Most game systems use
[msg_type msg_length variable_length_message]
The primary technical designer of Second Life did a very good job. Then he had a disagreement with management and was fired. So he went to Facebook, developed their mobile client, became a Facebook VP, made lots of money, and was semi-retired for a while.
Scaling to a big world is really hard. Big has two dimensions - area and density. The Second Life architecture scales well in area but not in population density. More than 20-30 users in a region will choke it. More are possible if most users sit down and doesn't move, because sitters are not getting physical simulation. So audience-type events work.
Improbable was working on large crowds, but their solution is really expensive to run. Which is why they just do demos of Otherside for a few hours at a time every few months. Also, they use very simple avatars and do not, as yet, support vehicles. Took them over US$500 million to get to that point. They have dynamic regions - more people, divide the world into smaller chunks. That introduces a whole range of new problems. What if I have 20 people on my boat, and a region boundary moves under the boat while I'm crossing it? Stuff like that. Fixed region crossings involving multiple avatars and vehicles are troublesome in Second Life. People do try bus tours, but sometimes an avatar gets Left Behind at a region crossing. Linden Lab's devs have been trying to fix that for at least 8 years, without success. It's a tough problem in real time distributed system design.
It seems like there's an easy way to retrofit the protocol without breaking it for older clients (as long as older clients truncate packets once a message type is unknown). Put a new message type at the end of the packet, with a list of message start offsets in it. By including its own offset as the last element, the start of the list is easily found. The first message offset could be omitted, since it is implicitly zero.
Presumably they could have a negotiation process that newer clients invoke to upgrade the message stream to a new format, and support both in parallel on the backend until an insignificant number of sessions were using the old version.
I remember a post from him where he said something like "Note to self: next time spend more than a week(end) on the scripting language". :-)
I tried SL in 2011. I was able to get some avatar add-ons for free. My favorite add-on was a Tux (Linux mascot) doll. Due to poor 3D graphics, the doll looked like its semi-detached to my avatar and moves with my avatar without any animation. It was ridiculously irrelevant, but I enjoyed the game.
Curious if anyone else had that same add-on.
I dabbled with it many years ago and ... Christ, can it really be 20 years since release?!?!
EDIT: Well it does something...
```Sure! Here is a basic client for Second Life written in Python:
import requests
# Set the base URL for the Second Life API base_url = "https://api.secondlife.com"
# Set the endpoint for the login API login_endpoint = "/login"```...
Silly ChatGPT.
On first connect, it used to be practice to go to a busy scene (a mall) and then go do something in real life for an hour or so while the viewer grabbed everything and cached it.
Regions can be very low lag or amazingly laggy, depending on a bunch of things (particulary misbehaving scripts).
I've been working on that with a multi-threaded viewer in Rust.[1] After I started posting videos like that, there was a lot less "can't do" attitude from the Linden Lab devs. Now there's a "performance viewer" project out of Linden Lab, using many of the same techniques. Key concept: the backlog of assets to be loaded needs to be on a priority queue which gets reordered as the viewpoint moves. Otherwise you bottleneck loading stuff that you were previously near, not stuff near where you are now. There's a nice B-tree solution to this. Once you have that, everything in close-up is present and at high resolution.
Hardware helps. 100mb/s networking and putting the viewer cache on an SSD will help a lot.
There's still a big problem server side with the transient load when a new user enters a region. A mall or event with a lot of traffic can slow way down. The server devs are trying for more concurrency, but it's hard in an old single-thread 32-bit C++ program.
It's striking that, despite all the "metaverse" hype, there are very few people, even among game devs, talking about the nuts and bolts of making this stuff work. The metaverse conferences are mostly about branding, NFTs, and moderation/censorship.