Malleable Messages
Sunday, March 4th, 2007 at 6:06 AM by: babbagelindenA couple of months ago Zero talked about the new capabilities infrastructure that has been added to Second Life. The viewer requests capabilities from the simulator and receives a URI which it can use to invoke the capability. When the capability is invoked a proxy maps the URI to a private URI and passes the request on to a web service which processes it.
One of the nice things about capabilities is that they make the Second Life back-end much more loosely coupled. Although most of the current capabilities still map to web services implemented by the simulator, the MapLayer capability is implemented by the dataserver. When the viewer requests the MapLayer capability it receives a URI which maps to a service on the dataserver. When the viewer invokes the capability it is routed directly to the dataserver. The simulator is no longer bothered by MapLayer requests and the dataserver doesn’t need to bother the simulator to ask whether the request is valid: if the viewer can invoke the capability then the dataserver knows that it is authorised to do so.
As time goes on we will be moving more functionality out of the monolithic C++ simulator application and in to a cloud of web services implemented until the simulator is doing just that: simulating physics, running scripts and routing updates to viewers. All the house keeping required to manage groups, inventory and events will be implemented as web services. The capabilities system makes this all transparent. One day your viewer will be invoking a capability implemented by the simulator, the next it may be implemented by a web service, the day after the web service is upgraded and you don’t notice the difference.
The logical conclusion is that eventually everything is implemented as a web service, everything is loosely coupled and every piece of the Second Life infrastructure can be upgraded without taking the grid down.
It’s an attractive prospect, but there are a couple of practical problems: there are hundreds of methods that send or process messages using the current LLMessageSystem API and there are a few messages that really should be sent over UDP rather than HTTP. So, instead of refactoring the whole of Second Life to send messages via capabilities we’ve instead refactored the message system to optionally build LLSD messages and send them over HTTP instead of always using message templates and UDP.
The message templates used by LLMessageSystem are the cause of much of the current tight coupling between parts of Second Life. The messages themselves include little structural data, so receivers of messages must know the structure in advance: when messages change new versions of all the applications that communicate using templates are needed. In contrast LLSD messages are self describing, requiring no additional information to decode, so applications can cope gracefully when they receive messages from different versions of the sending applications.
In addition to making Second Life more loosely coupled, moving to LLSD/HTTP messaging also promises performance improvements: on Friday Tess and I connected a viewer to a simulator that was sending the majority of message types as LLSD messages over HTTP and we saw a perceivable reduction in lag.
There’s still a long way to go with this work, but we’re heading towards a future in which Second Life is far more loosely coupled, allows many different versions of viewers and simulators to work together and requires far less downtime than the grid does today.
March 4th, 2007 at 6:16 AM
This is excellent news. Anything that increases the enjoyment of Second Life while reducing the downtime and bottleneck loading has to be good. Yay Lindens! =)
I guess with the simulator being freed up to do more of the tasks that it is supposed to be doing, we can look forward in the future to improved physics and core capabilities, such as improved flexi prims…
The future’s bright….. the future is digital! Keep up the good work
March 4th, 2007 at 6:23 AM
WTG Lindens!!.Great news.
March 4th, 2007 at 6:28 AM
woohoooo YEA!!!!!!!!!!
March 4th, 2007 at 6:41 AM
Great work! Always good to hear performance and reliability are being worked on.
March 4th, 2007 at 6:45 AM
I didnt understand a word of this except the word “web”, and then i have to just say im having faith that this is a good thing, although make sure you keep spam and viruses at bay now before they even start. Yes i would say freeing up the grid for tasks not related to every day living. Im actually getting used to the idea (skeptical at first) to having that open source interface out there because this is going to get so huge and i want you lindens to focus on why my island keeps crashing instead of wasting your time trying to fix up or not fix up whether you can teleport your friends or not your friends(sigh…
bravo lindens anyways you know i wanna have your babies!
March 4th, 2007 at 6:49 AM
although make sure you keep spam and viruses at bay now before they even start.
Huh?
March 4th, 2007 at 6:58 AM
Who are you and what have you done to the Lindens?
Seriously, your candid nature and forthright adddressing of challenges facing the grid are GREATLY appreciated! Thank you for this change in approach to your communications with your customers.
March 4th, 2007 at 7:09 AM
I’m not completely sure what any of this means, but…
It sounds to me like LL is finally getting away from a model that regards the grid as anything real, but rather a data distribution system. We’re not buying land or items, here.. we’re buying data and hosting space.
And as we are all aware: data you need, and are supposed to have access to. that is stuck on a server that can’t communicate with your workstation is of no use at all.
The next step would be for Linden Labs to re-organize themselves in a way that actually assigns people to maintaining these things, rather than waiting for someone to see a problem and decide whether they feel like getting involved in a solution.
March 4th, 2007 at 7:27 AM
As many times said, we love the speed improvement and we know you are doing a hard job to fight the lag.
But please be carefully with with moving services from in-world to off-world.
Some services may have the chance to harm the special charakter of SL, if they are moved to off-world.
Ok, the inventory may not one off these services which may have an impact on the SL-character.
March 4th, 2007 at 7:33 AM
Yuppers- good stuff. Also worth reminding readers that Zero holds office hours where he enjoys discussing this stuff at length. Sometimes we even get pretty pictures.
March 4th, 2007 at 7:48 AM
I Have a big problem every time i log in i get an error that says: Unable to connect to second lofe the system may be down.
Please try again in a few minutes,
i already checked my firewall bit id don’t make sense what the …. can i do? please help me
March 4th, 2007 at 7:58 AM
I didn’t really understand a whole lot of that except for “less lag,” which sounds good to me.
March 4th, 2007 at 7:59 AM
I Have a big problem every time i log in i get an error that says: Unable to connect to second lofe the system may be down.
Please try again in a few minutes,
i already checked my firewall bit id don’t make sense what the …. can i do? please help me
March 4th, 2007 at 8:01 AM
[...] (more…
[...]
March 4th, 2007 at 8:17 AM
Great stuff, Babbage!
Keep it up!
I am just wondering why it’s more performant to use HTTP but then again I also haven’t thought about it more deeply why it could be
But the loose coupling alone is already worth it anyway and this seems also be the thing software engineering is more and more moving to in general.
And I should not forget Zero’s office hours, should put them into my calendar.
(is there a ical feed or so for all the office hours btw?)
March 4th, 2007 at 8:19 AM
As a programmer, I can tell that this will increase the amount of data being passed during login. This may also increase the problems logging in.
For example I can’t log in reliably today. Sometimes it works and sometimes it comes up with the “Despite our best efforts, Something unexpected has gone wrong” message.
It will also make users more dependent on the QOS(quality of service) system to control the data rate and keep the movement commands responsive. Some XP users might not even have the QOS system installed and Microsoft is changing the interface in Vista.
March 4th, 2007 at 8:19 AM
ok, found the feed
Hope to show up more then 
March 4th, 2007 at 8:24 AM
Thanks for keeping us updated on this sort of thing - it’s always interesting to get a bit more technical detail on infrastructure development (well, to be fair, a lot of people won’t find it interesting in the slightest, but I do). The new setup sounds like a much more upgradeable and agile one, and probably a lot more fun to work with, even if it’s a pain trying to fix it while SL still has to keep running….
Tao: yes, the Google Calendar for the office hours has an iCal feed.
March 4th, 2007 at 8:31 AM
Sounds good in general. If you’re planning to use XML to communicate with your web services, my experience has been a bad one for any system that needs to scale. The flexibility provided by XML is valuable, but it doesn’t require XML to achieve, where-as XML requires a whole lot of parsing and copying of the data as its marshalling and unmarshalling of the data on both sides. Additionally, there is really no reason to include self describing information other than a version number in every message sent, this just ends up wasting a lot of bandwidth and resource on both ends of the communication.
If you have full control of both ends of the communication, XML is not the way to go, if you want to expose it to some low volume third parties/the internet who might need an XML Web service interface to get it up and running quickly you can provide a separate front end, you’ll probably end up doing this anyway for security reasons instead of exposing the backend service directly to the internet.
Also note that even worse than sending data via XML, is sending binary data via XML such as images since these are base64 encoded, which makes them 1.3x bigger as well as more marshalling and unmarshalling required.
Anyhow, just my 2 cents on web service implementations. Even AJAX doesn’t use XML. I don’t know what your plans are, or if you have someone with some web service experience on large scale systems that need to scale well or if you’re even planning on using SOAP/XML messages for your web services, but if you are DONT.
Not saying that moving to web services isn’t good, just don’t think you automatically should default to using XML messages to communicate with them.
March 4th, 2007 at 8:31 AM
grumble Loudon Says:
March 4th, 2007 at 8:19 AM PST
As a programmer, I can tell that this will increase the amount of data being passed during login. This may also increase the problems logging in.
As a programmer I say this is nonsense.
March 4th, 2007 at 8:38 AM
Great news! Please continue to try and improve SL. But will all of these improvements happen in time to save SL from itself?
I’ve only been here for four months. But due to the increasing lag and problems with SL, I was ready to dump my three premium accounts last week and give my 20K m2 of land in three regions away.
My plans for my own sim have been shelved until I see improvements in SL.
I love SL, but when it becomes more frustration than fun, I’m out of here. There’s a large number of us premiums who feel the same way and are willing to leave SL to the corporations, griefers and no-payment info freebies.
March 4th, 2007 at 8:47 AM
While I actually understood that whole post (not sure if everyone did), and hope it improves performance, I have to ask how security will be affected by these changes. There’s already lots of security problems with SL, hence griefing, account hacking, easy-made no-payment-info alts for any nefarious purpose, low priority given to server failsafes, the ShoopedLife ban-immune griefer client, etc. Please address how security will be impacted by this shift, positive or negative. FI: will going to http sessions make it easier for man-in-the-middle type attacks, sniffers, and other traditional hacking techniques? Please elaborate.
March 4th, 2007 at 8:52 AM
There are competing pros and cons to using web services rather than a more proprietary message structure.
For those lost in the programmer speak, a web service amounts to using http — the same stuff your browser uses to request a page, or send stuff on a form — to a server and get a response back that contains data stored in xml. XML is another one of these acronyms that isn’t as hard as it sounds. It just means some text that is written in a way that can be separated back into parts by a computer. Just like html only with tighter rules. There’s other details of course, like how you define what can be sent and what comes back, but mostly it’s just a set of rules for how you send stuff back and forth.
XML is “big” — anything you want to send XML is going to be way more data than just the thing itself. It’s the nature of it that as much as half of what you’re sending is the stuff that defines what you’re sending. It’s also pretty “chatty” — there’s a lot of back and forth between you and the server. Finally, there’s such thing as “logged on” with http. Each request and response is a single complete transaction. We “simulate” a logged in “session” by doing tricks like including with each of those requests a kind bit of data that identifies us as the same one who made the last one, so the system can remember stuff about us - but that’s a trick, not really a session in the old sense of a modem call (remember “NO CARRIER” ?) or a phone call which is “hung up”.
On the plus side, even those this method means sending more data and it’s big, and less specific, there are advantages. Because it’s used by a great many kinds of systems, there is a whole industry around making very high performance tools for handling this kind of data both on the server side and at your local workstation. It’s true that xml is big and bloated, but because there are a bunch of really good tools that lots of people put lots of effort into, it turns out that it’s very flexible and easy to process quickly. Since you don’t have to make your own server to accept and respond to the requests, you can use proven technology that is very high performance and not worry about that part of your project.
In theory, it would be faster to send these messages (the requests for information and the responses) using your own shorthand. In practice, you’d have to build every single part of that process yourself. That leads to bugs in software you otherwise wouldn’t need to be dealing with. Most people — even the best programmers — are just not going to be able to build their own server for their own shorthand messages as well as big product systems for handling more generic messages.
So, moving to xml based web services is a good thing for SL.
March 4th, 2007 at 8:53 AM
Yipes. At this time, anything that depends on Vista-related implementation is a bad idea. The OS still has too many holes in it, and is unlikely to be fully stabilized until Q1 of 2008 (if MS works as typical.)
March 4th, 2007 at 8:54 AM
#14 Azrael B. Says,
LOL! That’s the first thing I thought of as well when I read grumble Loudon’s comment. What is he going on about?
March 4th, 2007 at 8:55 AM
sorry for the typos. Composing in a text window is what it is.
March 4th, 2007 at 8:55 AM
While that is all very fascinating, I was wondering about the changes being done to the scripting using the Mono system to compile them. I heard about this a few times a while back in the blog, and I would like to know if it has been implemented yet. It sounded promising and like a good anti-lag measure being taken.
March 4th, 2007 at 9:01 AM
I Have a big problem every time i log in i get an error that says: Unable to connect to second lofe the system may be down.
Please try again in a few minutes,
i already checked my firewall bit id don’t make sense what the …. can i do? please help me.
March 4th, 2007 at 9:02 AM
I Have a big problem every time i log in i get an error that says: Unable to connect to second lofe the system may be down.
Please try again in a few minutes,
i already checked my firewall bit id don’t make sense what the …. can i do? please help me….I Have a big problem every time i log in i get an error that says: Unable to connect to second lofe the system may be down.
Please try again in a few minutes,
i already checked my firewall bit id don’t make sense what the …. can i do? please help meI Have a big problem every time i log in i get an error that says: Unable to connect to second lofe the system may be down.
Please try again in a few minutes,
i already checked my firewall bit id don’t make sense what the …. can i do? please help me
March 4th, 2007 at 9:02 AM
Another benefit that may not be apparent is that moving away from UDP toward HTTP improves wireless connection performance for those of us with laptops. I usually don’t have alot of problems, so I am looking forward to some great grid performance. This is exciting news!
March 4th, 2007 at 9:08 AM
Bravo!
Sounds like some very thoughtful change to a more distributed, grid computing infrastructure. This is what you have to do, exactly, to make a computer grid, and this is the first blog I have seen that indicates this is where the network structure is heading.
I hope you are keeping in mind that although you are taking these functions off the simulators, the will most likely need to be holographic on the web so these calls will execute smartly and independently through the nearest grid nodes riding on the web.
This is grid architecture people. Yay!! (gonna get some more ram for my little machine though…
I cannot wait until even more of the structure can be more distributed. And flops shared among the client machines with the shortest packet transport times, etc., sharing some of what would ordinariily be server side logic across less-loaded clients in close packet transport proximity to server equipment holding work in que.
How soon?
March 4th, 2007 at 9:16 AM
@16 - Security isn’t going to be a function of the tech you used to send the messages back and forth — it’s a function of what you send, and how you validate what you get back from the client side. Let’s hope data send back and forth includes some kind of encrypted hash of it’s content so that it can still be compressed, but can also be validated at each end using some kind of public/private key encryption. That tech exists. In fact, using standard web services for moving the data means you can use standard servers, and plug in SSL and other kinds of encryption without major rewriting projects.
@11 — Moving to vista just to use QOS is the dumbest thing I’ve heard in terms of someone claiming knowledge in a really long time. First of all, many systems properly support QOS including nearly any linux implementation and Apple’s OSX. Second, most user workstations are behind nat routers (like the dlink and linksys routers at home, or more complex ones at offices) some of which support QOS and some do not. QOS at the workstation is a very small part of the overall equation.
Chattiness on the data is much talked about here. This is a really big issue and impacts high latency connections (like satellite providers) a great deal. Latency is also easy to understand. Think of bandwidth has the width of the pipe you’re sending data throw. The bigger the pipe, the more data you can send at once. Latency is how long it takes data to move from the start of the pipe to the end, regardless of how much data that slice is. Latency is why when you hear tapes of people talking between earth and moon there is a delay. It takes time for the sound data to travel up and back. That’s latency. Latency makes computer programs that are “chatty” (have a lot of data going back and forth) run very slowly.
What SL can do to reduce problems with latency is to make the data sent back and forth less chatty. To do that, you do things like including the hash needed to validate something along with the thing you’re sending., so you send one slightly larger hunk of stuff instead of sending a hung, getting an acknowledgement back that it got there along with a hash of what they got, so you can validate they got what you sent, and then sending back an “ok”. All you really need to send is the data and the hash, and get back a request for retransmission if they don’t match. That’s one block of data going one way, not a back and forth chatter. The other thing you have to do is anticipate what else will be needed and start sending it before it is requested. If they send a request for details about a building fascade, you can assume that once they get it they’ll also want details on all the parts of that fascade and start sending them before you get a request for them since to get that request you’d have to wait for the first response to be received, parsed, and new requests be made and that to transition the high latency connection.
Thus — choice in protocol makes WAY less difference than how you use that protocol.
Oh, and FSCK Vista. Vista is the very first operating system ever where the key design goal is to PREVENT THE USER DOING THINGS. Up until Vista, the job of the OS was to provide services from your software to the hardware and the data stored on it. Now, all of a sudden Vista needs to decide what data you can read directly with your software, so someone else’s financial interests can be looked after. It is there for the benefit of someone else, not you. All that programming and all the bugs and performance problems it may create — have no benefit to you, the person who bought it. The benefit is there to prevent you from cheating on licenses even if you want to.
March 4th, 2007 at 9:16 AM
I Have a big problem every time i log in i get an error that says: Unable to connect to second lofe the system may be down.
Please try again in a few minutes,
i already checked my firewall bit id don’t make sense what the …. can i do? please help me………
March 4th, 2007 at 9:43 AM
Great job moving to a transactions model that keeps non database necessary transactions out of the database!
As for Vista, nobody needs it. It is another *Bob*. Eventually it will be relagated to the boneyard since it will never make the 2 year evaluation criteria for profit at msft. Anyone else who has ever been involved with msft at the group management level will know what i mean.
March 4th, 2007 at 9:46 AM
I too eagerly await the time when I can use Python instead of LSL to script my stuff. Any updates on Mono? Can we fool with it in a beta grid? >3
-
This news of stepping away from a compact system to a more diversified one is good news. One wonders who was responsible for thinking of the “all-in-one, make the sim do it” approach. That does show a bit of short sightedness, because as we’ve seen the model doesn’t work very well on a large scale. >
To really simplify it for the non-techies out there, this update is kind of like an office environment. There’s the boss, there’s underlings, and there’s lots and lots (and lots and lots and…
of files. A client (in this case literally our SL client) will request something from the boss, like a texture, and expects to get it back.
In the current model, this amounts to the boss walking over to a filing cabinet and looking for the texture itself, then sending it back to you via standard mail. Sometimes that mail gets lost, usually it takes a while because it has to go through all kinds of stations, and generally it’s a big waste of time. If you end up waiting too long for your texture, you call back and complain to the boss, who in a tizzy goes and does it again until it actually succeeds. The underlings don’t really have any specific role, they just do work making sure every thing is filed properly so the boss can find it.
In the future model however, you call up the boss and tell him you want something, then he puts in a call to a department (the texture department for instance) and tells his underlings to ship off the texture to you via… We’ll say FedEx. If you’ve ever shipped with a private courier, you’ll know there’s a lengthy form to sign (that’s the XML part). However the form ensures that data’s recorded properly, and FedEx doesn’t lose packages (in theory anyhow). You promptly receive this package, perhaps in the same day, and the form you get on the box will tell you about the contents of the box.
-
On the technobabble side, this will probably mean packet sizes will increase a good deal, and will probably be a little slower, but stepping away from UDP and LL’s contrived packet loss tracking is a good thing because they’re relying on a long proven standard (TCP, which does the same job very well and without possible bugs in LL’s current system). So downside, packets are bigger and go a little slower. This will certainly be an extra nail in the coffin for dialup users, should any of them still exist on SL.
However, by delegating the workload out to multiple systems, a sim won’t have nearly as much of a workload and will be able to support more active things at one time (more scripts, more avatars running around, and possibly more vehicles. I won’t say “certainly” until we get a Havok upgrade. >
). More servers in different locations sending packets means less of a bandwidth bottleneck, less processing strain on the back end, and for us that shows up as a reduction in lag.
March 4th, 2007 at 9:56 AM
There is an easy answer to the XML issue.
Use a binary template messaging system. Instead of wordy XML, you stick with binary. Instead of a “fixed” compiled binary, you use a wordy binary-describing XML template.
It’s the best of both worlds. Tiny, bnary message packets that are succinct. and maleable message packets that can be changed by sending a new template xml to the client.
March 4th, 2007 at 10:05 AM
Web services, XML, loosely coupled component architectures….welcome to the 21st century LL.
In all seriousness, a sustainable grid that can be updated and maintained without the need to bring the entire SL world and economy to a halt once a week will be a giant leap forward. I’d like to shake the hands of your architects.
Being able to host sims outside of Linden Labs on private servers (following the distributed web server model) would be the next step in positioning SL as the 3D internet leader.
March 4th, 2007 at 10:07 AM
ugh, shouldn’t post before the first cup of coffee is fully consumed…
What I meant to say is this…
Instead of boing pure XML and wordiness in every packet, upon conenction the client downloads a dictionary of packets. Thisdictionary is an XML document that describes every packet type in detail. If this is large, a “diff” dictionary can be sent based on the dictionary that the client tells the sim it already has.
Now, the second thing is that upon connecting the client tstarts up a pool of conenctions to the middle-man proxy-routers. These connections are capable of multiple sequential requests, or even async requests. The pre-connecting to the proxies removes the inherent lag of a tcp connection to speed things back up to nearly the speed of a UDP stream without loosing the guarantees of a tcp stream on a good connection.
Packets returned by the http connection are still in binary format. Every packet has a Dictionary version as it’s VERY FIRST 64 bits. The second 64-bit field indicates family and message type of packet. The client then translates this into the binary objects that the viewer needs in order to operate quickly.
I’ll explain more if anyone doesn’t understand what I’m describing.
March 4th, 2007 at 10:17 AM
Whilst I appreciate all the effort being put into all these new little things. I for one am not a techno wizard and speak plain English. please if you are going to spam us with this sort of thing can you make it so we can understand
March 4th, 2007 at 10:18 AM
Heh, Azrael, poor janie matahari simply makes the same assumption most of the general public does when they hear the word “web” and “internet” and thinks “viruses” and “spam”. She doesn’t know those depend on a completely different client. SL has always been, and will always be, subject to the same hack attempts and techniques used from the beginning.
Andrew, nice ideas. I use precisely that technique for a system I designed used in emergency response and life safety. “Blind” packets are used to start the processes and only the ACK/NAK is needed to end the transaction.
And I am not sure that the impact on dialup users will be any worse than the problems they have presently. My own system can behave as badly and slowly as my fiancees system, which is on dialup, when the network is being balky (frozen avatar, object creation taking several minutes) and *I* have a cable service provider second only to fiber based connections and top of the line hardware (well, not bleeding edge, but nVidia 7950 SLi and a duo core 2.8G Pentium mobo). Her system is quite capable of playing several web based games that use a very large number of very chatty http requests.. as Andrew has said (to paraphrase), it ain’t the size of the packets, it’s how you use them.
March 4th, 2007 at 10:26 AM
is SL down or sumthing as it aint loadin up for me, it seems to keep bring up a message then cuttin me off
March 4th, 2007 at 10:31 AM
ryu — interesting that you’re in the public safety sector. I’m doing work in that space as well. Ping me in world (Completely Obscure) or drop by my own site http://www.thenorth.com/apblog and send me mail. We should compare notes.
March 4th, 2007 at 10:34 AM
@29 Hurly — Explain to me how this is spam. It’s a good discussion even if you can’t be bothered to figure out how to understand the big words. (despite those of us making an effort to explain them).
March 4th, 2007 at 10:38 AM
Thanks to Andrew Pollack’s comments (#17, #23), for putting this in a language I can understand. Well done, Linden, seems a good move then!
March 4th, 2007 at 10:45 AM
I know this is not to spot to post this BUT I’ve tried every other channel to reach LL - the SLURL site has been broken for monhts now - it DOES NOT work on IE7 - PLEASE FIX!
March 4th, 2007 at 10:58 AM
Keeping us informed about this kind of back-end changes is really a great idea. Loosely-coupled system components are the way to go for a more scalable infrastructure, that may well lead (particularly using WS) to a next generation of SL - third party hosting of sims.
March 4th, 2007 at 11:04 AM
What?
coco
March 4th, 2007 at 11:13 AM
As a programmer I say this is nonsense.
As a programmer, I say “another glass of wine please, and some of those little biscuits as well, the cheesy ones”.
March 4th, 2007 at 11:25 AM
Count me as another user who doesn’t understand what is being announced. But I applaud LL for providing the information and the tech oriented users who help explain it.
March 4th, 2007 at 11:48 AM
34 Yo Brewster Says: I know this is not to spot to post this BUT I’ve tried every other channel to reach LL - the SLURL site has been broken for monhts now - it DOES NOT work on IE7 - PLEASE FIX!
It DOES work on IE7 and has worked fine for me for months, maybe you need to adjust some security settings or something, LL aren’t going to come around to your place to fix it.
March 4th, 2007 at 12:02 PM
I’ve just joined this world, not seen any of it yet, and have been trying to log on for an hour now. I hope this new development will work soon as this is very disapointing for a new person looking forward to a new life!
March 4th, 2007 at 12:02 PM
If loose coupling allows LL to spread out their server infrastructure and rely more on direct client to service traffic, instead of huge amounts of back-end traffic… *maybe* it will help with the packet loss problem…
The ability to down and work on specific features, like the map, without bringing down the whole grid, is certainly appealing, for proactive or bug fix work. Grid content problems like future more insidious replicating attacks may still require grid downtime though.
March 4th, 2007 at 12:41 PM
Having done some XML based web services that pass pretty good sized chunks of data around, I’d suggest you take a look at some of the “binary XML” protocols. You’re in a completely proprietary space so you don’t need to be compatible with every browser that comes along and XML is really a pig if you’re sending a lot of numeric data.
March 4th, 2007 at 12:48 PM
Thats nice… but will I be able to log on ?
March 4th, 2007 at 1:01 PM
i think this is good
March 4th, 2007 at 1:09 PM
That’s fantastic, for so many reasons given and more. Does this mean we’ll also be able to script with net-reliable or streaming transports in the scope identifier? Basically, define a state, a function, or a whole script that communicates with other clients or web serves as requiring an HTTP, TCP or UDP transport?
March 4th, 2007 at 1:09 PM
[...] auch den Link auf dem Linden Lab-Blog zum Thema [...]
March 4th, 2007 at 1:29 PM
Of course, island owners will be required to pay $6000 a month for this “added value.” I’m just sayin.
March 4th, 2007 at 1:31 PM
Sounds like a blast to work on, and sounds like a blast to see a sim do the job it’s supposed to be doing, and do just that. Sometimes I wish i knew more C++ and C so I could jump in on this, sounds like fun :).
March 4th, 2007 at 1:39 PM
Couldn’t you just connect the fabrillicator to the XUP-port and let the KMY stream directly through the mass inverter box? That would help with reducing… oh heck, who am I kidding… keep up the good work!
March 4th, 2007 at 1:58 PM
Very interesting post, Babbage, as well as rather stimulating discussion afterwards. I understand this is not really “news” but an ongoing struggle in redefining the whole protocol, the “under the hood development” that has been going on for perhaps two years or so to make the grid a bit more distributed and less monolithic.
Personally, I’m admired with the daunting task and the huge changes that are being done, literally “under our noses”, and that are completely invisible to the end users. Last week, a very knowledgeable programmer (although still a recent resident) asked me when LL would launch “SL 2.0″ (an old dream of the past). The answer is that there will not be — or at least, not for ages — a “SL 2.0″ in the sense of a new product which is incompatible with the current one — instead, we’ll have an “evolutive” approach to all subsystems of the grid, some of those changing very dramatically over the months, but it will always be the “same SL”. What will people notice? More performance, more features, very likely different clients (SL viewers) that will show a slightly different world (and will definitely render it in a completely different way!) depending on what your hardware is able to support. And on the server side, you’ll see several more functions to be distributed among different sets of servers, sharing the growing load that is currently mostly focused on the central servers (asset, login, user/presence server, etc).
This will obviously be gradual and in most cases very little of it will ever be noticed. Well, perhaps in 2009 we’ll suddenly see a SL viewer on the iPhone or the XBox or the PS4… and people will ask “how did they manage to do that? The hardware is so different!” The answer, of course, will be that the whole concept of how you retrieve data from the “grid” and display it for users will change so much that you’d have ways of creating very specific viewers dealing with the same data grid-wide, but display it differently depending on its hardware.
But similarly the server-side grid will also be distributed and changed to allow completely unthinkable things these days. For instance, loading textures: today, you depend on LL to store all textures on their simulators, compressing them when you do the L$10 upload, and streaming them to your computer after the asset server had a chance to figure out where the textures actually are. However, things like video streaming (or possibly HTML-on-a-prim) do not rely upon the “standard” model of retrieving “textures”: in effect, your client is instructed to replace a “face” of a prim with data that comes in a different format from servers completely outside the grid. Now imagine you’d be able to do the same with textures as well — being able to upload them on a high-speed storage system like, say, Amazon S3, and design a front-end that gets a texture UUID and retrieves the appropriate texture in the appropriate format. All you need is some “glue code” at the asset server that tells you “this texture is not on LL’s grid, but on a separate, external, third-party provider”. Obviously this needs the (long-awaited) redesign of the whole concept of “asset server” to make it become like the DNS system — hierarchical but distributed — but it will be slowly possible: retrieving a texture from an external server should be as easy as sending a message using the SL protocol today and getting it streamed back from the sim servers. There are permission issues to deal with, obviously, but this should be “doable” — I would imagine that the biggest issue for not doing it right now is mostly that people would complain about all those users thinking to install their own home-based “texture servers” and generating lots of “missing textures” or “grey textures” — and incorrectly blaming LL for a “laggy environment”. However, we have a precedent: people are slowly understanding that when there are problems on the audio or video stream, it has nothing to do with LL. When we get in-world HTML, they will also slowly understand that they’re retrieving pages from 3rd party servers (since these days most people understand how the Web works…). So residents are being educated and learning about “fully distributed models”. In time, they will also understand that there is no real reason for not having everything distributed across several different parties — object data on some place, texture data elsewhere, messaging systems on a further service, and so on. LL’s role in this will ultimately be the ones providing “the root nodes” on a very complex hierarchy, and things like universal authentication (acting, in fact, as a certification authority, guaranteeing that “this avatar has a valid name in the whole grid”), and, obviously, defining the SL protocol that “glues” all together.
This is for me rather exciting — since the closest we have to this model is, indeed, the World-Wide Web. When you see a page on your browser — and it can be IE7/FF2, but also a text-based browser, or one on your phone/PDA — you have no idea where all the bits have come from. Images can come from one page, text from others, RSS feeds from even remoter areas, and ads from elsewhere. But the user only knows that a particular page is rendered by your browser by typing an URL on the browser — the way all the bits come together is actually not really important. This, I think, has been seen in the past 14 years or so as the “best” way to deal with 2 billion web pages for an Internet population of a billion users — it works well, it distributes the load well, and seems to be infinitely scalable. The more the SL grid resembles that conceptual model, the bettter we, as users of this amazing technology, will be better served.
You might remember the days when both AOL and MSN thought they could develop “their own Web” using a centralised model. Both have very long ago given up that idea. Instead, they use the same tools, protocols, and services, that every other service/content provider does on the Web. I can only imagine that SL will benefit way more from thinking in a similar way, and, most important than that, slowly implementing the changes, step by step, without anyone noticing what goes “under the hood”.
Since the Web was “invented”, the protocol for communication between clients and servers has changed just twice: from HTTP/0.9 through 1.0 and now with 1.1. But that certainly doesn’t mean that the Web of 1992/3 looks like the one in 2007! Still, the server infrastructure and the concepts behind it is exactly the same. It was simply an amazing act of a very well-planned model that is immensely flexible and incredibly simple to implement… and resisted “aging”.
March 4th, 2007 at 2:33 PM
Is this why I can’t log in to SL? Been having problems all weekend.
March 4th, 2007 at 2:43 PM
Remember the defect in the sim host side where if your client crashed your account was stuck in the sim until it was restarted? even for days? Well that defect is back. So whatever was changed on the back end recently neeeds to be rolled back to the previous fixed state and more attention be paid to full software configuration management.
March 4th, 2007 at 2:49 PM
Lindens!
Can we have a message about being unable to log in most of the weekend please!
What is going on for Gods sake
I thought that there was a back up plan for when the Grid was overwelemed.
Quant
March 4th, 2007 at 2:49 PM
To help all the new people to understand what this all means,,,i want you all to go to your dvd collection and pull out Titanic.Pop some popcorn and get a glass of Linden Labs Kool aid and start the show.Now,Picture yourselves as Jack,you know,the guy who thought he was soooooo lucky to have caught a ride on the newest most fantastic ocean liner(metaverse)Watch as he explores the ship with wonderment in his eyes,so innocent.Now,as the ship begins to sink,i want you to note the parts where the ships staff try to calm the people and tell them lies about the fate of the ship…………hmmm perhaps a feeling of deja vous is hitting you now as you read this blog…….Now,sip your Kool Aid,relax,the ship isnt sinking,listen to the band play
March 4th, 2007 at 2:52 PM
Yeah Rock
I knew that I had heard that tune somewhere
Quant
March 4th, 2007 at 3:04 PM
I saw LLSD and I broke out laughing.
But seriously I look forward to seeing this happen.
March 4th, 2007 at 3:16 PM
Jopsy, if they switch from UDP to TCP, there shouldn’t be any packet loss. If there is, it’ll be less a problem on their end and more a problem in the route to your computer.
Rock, that’s not a fair comparison. I’d use Shadowrun’s net crash scenario instead. >
(ah the dreams of data jacks and UV servers…
To everyone experiencing login issues, I believe this might be your problem. Remember, it IS a weekend, and there are a good many people besides yourself who want on.
March 4th, 2007 at 3:19 PM
Just don’t put them on any odd ports. The biggest problem I have is that I use SL in a corporate environment and I’m hosed anytime you allocate a function to a new or different port because it take 2-4 weeks for IT to update our firewall software to allow it.
March 4th, 2007 at 3:24 PM
I don’t think it’s that cause im premium and own private islands, and still can’t get in.
March 4th, 2007 at 3:25 PM
And grid status is not restricted
March 4th, 2007 at 3:25 PM
Dont forget to consider JSON … XML is pretty heavyweight…
March 4th, 2007 at 3:26 PM
I’m wet.
March 4th, 2007 at 4:07 PM
Re - logging in: I logged in a few times today, albeit briefly, and so no trouble. I didn’t go anywhere other than my space in Pruni, so perhaps that wasn’t impacted as much. Who knows.
March 4th, 2007 at 4:25 PM
Blah blah blah. More techno drivel. I’m sure this is all well and good but if I cant log in , who really gives a flying f***! What happened to the so called contingency plan for heavy load? Talking out your butts again Lindens?
March 4th, 2007 at 5:16 PM
Got to love it. Pages of techno-babble that even I couldn’t understand. Basically it seems you are using one method and are considering using another method because it would reduce load on the servers. The new method’s philosophy is to avoid doing anything un-needed. Why load the whole kitchen when you only need the sink? Am I correct?
March 4th, 2007 at 5:17 PM
That is absolutely excellent news babbage, I have already completed the lab and stasis tubes complete with regeneration & reproductive systems, self contingency over-rides and organic feeding modules. Lipesky has almost completed the synapse integration code and it looks like we’ll be able to go underground by years end. Also, the crystal has been deciphered from pX and we are on the right track and ahead of schedule. Our dream is a reality, our existence in SL 24/7 will be complete and we’ll be able to usher Earth into the next evolutionary process. Thanks again for all the Lindens help.
jk ;O)-
March 4th, 2007 at 5:39 PM
Hi i know this is a bit off topic but i was wondering if anyone has been having issues with uploading textures,( including taking snapshots) in just a few sims?, it seems my home Sim will not allow anyone to upload a texture ( torllhaugen) if anyone knows a work around i would love to know about it
March 4th, 2007 at 5:52 PM
Why all these bitter posts attacking Babbage’s announcement as “techno-babble”? It’s written very well, and while it is dealing with a technical subject, it’s written in a very explanatory and educational way. All people begin life ignorant on such matters. It is people like Babbage (and Zero) that help the ignorant to become knowledgeable. Step up, don’t ask people to step down to your level. If you want people to talk down to you, you probably think Bush is a great man and read USA Today.
March 4th, 2007 at 7:19 PM
Still having issues today logging in…and no word about why so many of us are having issues with this. I’m glad LL is making progress, but I’d like to see some kind of feedback or fixing of the issue. Perhaps we’ll all have to wait until Monday to see something happen.
March 4th, 2007 at 7:48 PM
In order of importance, and speaking as a Software Engineer and Bon Vivant:
@Ordinal - the cheese distracts from the wine IMHO, please consider the plain ones, and don’t turn up your nose at Merlot no matter what they say.
@Babbage - Keep it up, it’s all very encouraging to hear.
@Bucky - JSON is better than XML, but it’s still text and it still has to get parsed.
March 4th, 2007 at 8:31 PM
will resetting the sim/server still have the same positivie affect? seems to me sending stuff all over the web is just asking for trouble. Granted I barely understand… but can take a good guess… at what LL is going to do… almost almost like P2P (please dont post a comment saying it’s not P2P i know it’s not lol).
I’m not skeptical, just very very curious as it sounds very intersting and positive at the same time.
Anyone got a Powerpoint on this topic?
cheers
March 4th, 2007 at 8:34 PM
62 JT Dagger
I read the comics in US Today… is that ok? And have you seen that Disney cartoon… Over the Bush… or is it Over the Hedge.. oh heck i can’t remember
March 4th, 2007 at 9:40 PM
WHAT DOES IT MATTER IF YOU CAN’T GET INTO SL?????
THE PAST TWO DAYS I’VE BEEN GETTING THE “DESPITE OUR BEST EFFORTS, SOMETHING UNEXPECTED HAS GONE WRONG.” MESSAGE. I’VE FOLLOWED THE SUGGESTIONS OF CLEARING ALL CACHE AND REINSTALLING SL!! I’VE DONE IT 4 TIMES.
HOW ABOUT MAKING THE BASIC STUFF WORK FIRST??????
March 4th, 2007 at 10:04 PM
@ 66 : I fully agree there… Been trying to get onto the Grid for 48 hours, tried logging on for over 125! times… (I need my daily shot SO badly… *starts shaking* )
Finally I gave up, I will try again after the update…
Meanwhile take a look at :
http://www.getafirstlife.com/
Its very funny if ya ask me ^_^
(Note: I still AM a Diehard SL lover :p )
March 4th, 2007 at 10:35 PM
I had no problems getting in to the game? when did this start?
March 4th, 2007 at 11:29 PM
I am looking forward to the day that Wednesday die-in parties are a thing of the past.
March 5th, 2007 at 1:26 AM
The past dayz I’ve been getting the “Despite our best efforts…” message and can’t really connect to 2ndlife server even when status sayz server is open…
I hope to be able to connect soon…
help!
March 5th, 2007 at 2:10 AM
[...] In a technically dense post on the Linden Blog, future development plans for Second Life. For the layperson, what it essentially means is that over time, a lot of things that the main grid needs to deal with (messaging for instance) will move to an aligned Web Services framework, which should ease a lot of the burden on the grid with a subsequent improved user experience. Or as put by Babbage Linden: [...]
March 5th, 2007 at 2:51 AM
I fully endorse this product and/or service.
March 5th, 2007 at 3:08 AM
“Why all these bitter posts attacking Babbage’s announcement as ‘techno-babble’?”
Because if someone posts something that you personally either can’t understand or just aren’t interested in, it proves you’re not the center of the universe. Many people find this very upsetting. Every moment LL spends satisfying the needs and curiosities of others is one moment less that they’re focusing on you. If you think this is bad, you should see how upset they get when LL fixes a bug that wasn’t even affecting them.
March 5th, 2007 at 3:24 AM
Agreed Gaius.
I did not even remotely understand the original post by Babbage Linden, however SOME of the replies sort of indicate to me it may be quicker to load http language to perform various tasks AND because of the fact http can function in an environment where not everything needs to be working, we may see:
1) Less downtime for upgrades as it will not be wholly necessary to take the entire grid off line for updates
2) A general improvement in performance coupled with the fact that as Linden are not re-inventing the wheel with http language it may be a further move towards open source software
3) A swifter interface between the web and SL
Please keep the posts coming for the benefit of non-programmers such as myself
Regards
John
March 5th, 2007 at 3:49 AM
Lots of talk about XML being heavyweight disregards the fact that the gzip algorithm is pretty fast, free (as in speech and beer), and works well on text (which is what XML mostly is). In a sane world the heavyness of these XML transactions will be solved by compressing on transmission, and decompressing on receiving.