Rolling(sim) Update Completed
Wednesday, November 1st, 2006 at 1:31 PM by: Jesse Linden[4:32 PST] The Rolling Update is now Complete
While we’re not doing a full deploy today requiring extended downtime of the grid, we do want to push a new version of the simulator and offer an optional viewer that correct several known issues affecting many Residents in Second Life.
The rolling update will begin at approximately 12:45pm. It will correct some physics problems associated with script energy as it relates to hovering/buoyancy which were introduced with last week’s release. There are a few known remaining physics problems that will be addressed in a future update, including wind effects on small objects and jointed objects. It is also likely that this release will not address all of the causes of reduced simulator performance but we believe it will provide a significant improvement over the current condition. We have added some new instrumentation to this code allowing us to better monitor performance and will make additional improvements in the near future.
The viewer update release notes will be posted in a separate blog post. That will be made available later this afternoon.
November 1st, 2006 at 1:36 PM
Sounds good I’m glad its being look at maybe check burns out it has like 1fps most of the time.
November 1st, 2006 at 1:54 PM
Physics fixes sounds good, but i’m curious - is anything being done to fix the physics bug that allows people to break Havok and get around the push restriction? I’m asking because i got hit by one of those things earlier and it forces you to log out and back in to fix your client. I’m hoping a sanity check on physics forces would fix it.
November 1st, 2006 at 1:54 PM
What a coincidence that we are having a rolling restart just as the land store opens. Gee, I wonder why they did that?
M.
November 1st, 2006 at 1:55 PM
Why no blue box warning then. Just laggier and laggier until Crash no login.
November 1st, 2006 at 1:55 PM
This is good news , hopefully this will fix the problem that caused several of my products to die , last thing i need is 200 people asking me for new ones , thanks lindens , hope this works .
November 1st, 2006 at 1:56 PM
>>> There are a few known remaining physics problems that will be addressed in a future update, including wind effects on small objects and jointed objects.
November 1st, 2006 at 1:57 PM
A rolling restart just as the land store is about to open? Gee, I wonder why they are doing that? And, I also wonder why my previous post just seems to have disappeard.
More stuff that makes you go, hmmmm….
M.
November 1st, 2006 at 1:59 PM
yea burns is pretty brutal, it’s been running between 1 and 50 sim fps for the past few months.
November 1st, 2006 at 1:59 PM
Hmmmm… apparently the backspacing arrows cut off postings…. interesting…. Anyway I look forward to getting my windchimes back in order…. So do my carbon-based avatar co-dependent and his significant other (you know those carbon-based SOs are suckers for wingchimes….)
November 1st, 2006 at 2:16 PM
Is this the version that has been in the preview grid?
Does this address the apparently common packet loss problem?
November 1st, 2006 at 2:17 PM
hm… wait and see…
i hope a real better improvement.
November 1st, 2006 at 2:18 PM
ya’ll are just a riot to read! No mater what it is, it is a conspiracy about this that our the other thing. Its always the same people over and over. Ya’ll need RL jobs! You have to much time to think about this.
November 1st, 2006 at 2:30 PM
Too soon, Arthur. Wait for Lewis.
November 1st, 2006 at 2:34 PM
HOT DAMN, THESE 5-MINUTE RESTARTS RULE!
AND a warning at each minute during the countdown.
(I’m in adidas right now.)
FINALLY I have some reasonable time to pack up my builds and teleport!
Huzzah for freakin’ courtesy. *super big smile*
See you guys on the flipside.
November 1st, 2006 at 2:44 PM
Ok. So…when are you fixing the bloody lag?
November 1st, 2006 at 2:54 PM
Ummm, the best way for you to monitor performance is to disconnect from your direct server, go down to BestBuy, buy a pc off the shelf, take it home, hook it to your cable internet connection, download, install, and log into SL like the rest of us peons. Hit your search button, the popular places tab, TP to a few of the places on that list, walk around (if you can - kind of like wading through mud you’ll see), ask a few people how they are doing, “how’s the lagg” is a great conversation opener these days. TP several more places, and see how long it takes before you start having to force rezz textures, (oh tip - sometimes you can fidddle with your network bandwitdhth slider and make them come in better for a few min.) count the ava’s you see in various states of missing image disarray, (be nice and tell them though, because they dont know, they look fine to themselves - that way they can rebake and change skins and outfits and relogg whichever works first). Come play with us and see if you would be willing to pay MUCH higher tier for what we are getting.
November 1st, 2006 at 3:11 PM
I think Torley forgot to put on the “I’m joe average user and I love LL” costume before posting.
Looks really funny, a linden backing the lindens.
November 1st, 2006 at 3:12 PM
FYI, we’re just passing the halfway point in the restart, about 40 minutes remaining on the wall clock
For those keeping score, previous rolling restarts took 4-6 hours to complete. Thanks to some infrastructure work I’ve hinted at before on the blog about simstates being temporarily saved on their local hosts, we’re looking at just over an hour for the whole thing to complete.
Wait, you ask - how is 40 minutes = halfway through a process that takes an hour? Here’s the math:
* We ask each region to restart, with a one second delay between regions. 3900 regions @ 1 minute each is about 1 hour, 4 minutes
* Each region gives a five minute warning, then restarts
* The restart takes a few minutes to complete
As I post this, there are 28 minutes remaining until we’ve asked all of the hosts to restart. About 5 minutes later every region will have begun the restart process. And a few minutes after that the process should be done. We may have to restart some crashed regions, but otherwise the grid should be back to normal.
November 1st, 2006 at 3:18 PM
@Sue Baskerville:
Yes, this has the fixes that have been on the preview grid with the addition of the physics fixes and some crash instrumentation.
November 1st, 2006 at 3:27 PM
I liked having the warnings, even if I only managed to tp to another region only to see it was rolling too with too little time to get somewhere else first, lol. I’d much rather know whats going on, than not. So Grats on the warning system LL. ^-^
November 1st, 2006 at 3:28 PM
Femina Street wrote:
Why no blue box warning then. Just laggier and laggier until Crash no login.
I’d have left you a message in-world, but you’re not findable in people search… You may not have gotten the blue warning due to your own client’s lag issues. I *did* get the warnings, I clicked through them after I came back from the kitchen, so I know they went out.
And I *like* the extra warning time.
November 1st, 2006 at 3:31 PM
Instead bragging Joshua Linden
Could you personally check out Burns for us?
I know you got stats and info you can see what I mean.
November 1st, 2006 at 3:34 PM
I hope this improvement is literal and not just a font. The last 2 builds or so i have noticed improvements in sim performance, more stable time dil (relative to 1.11), less hitching, but still moments of complete uselessness. I think it’s getting better o_O. It has almost made me forget how long we had to endure the mess that 1.11 made.
Improving performance is the surest direction to gain customer support. I look forward to todays release.
Releasing crap, closing forms and price gouging sure the F aint
Best of luck to the Lindens working on performance issues!
November 1st, 2006 at 3:34 PM
Torley still is a resi at heart, and Lindens get no more warnings than regular resis do when sims go down (except for interoffice gossip, etc)
November 1st, 2006 at 3:37 PM
Haha, well it was nice to get feedback from Torley about the 5 minute warnings but I was wondering if anyone else has any strong positive/negative opinion about it. Is the every-minute warning annoying? Is 5 minutes too long or too short? Let me know.
November 1st, 2006 at 3:50 PM
Sabin,
What would be wonderful is a map of the progress posted on the website, not in-game. When I read about rolling re-starts on the blog, more often than not I delay login until the “All Done” is blogged. Just my 2 L.
November 1st, 2006 at 3:57 PM
I think warning should come up 5 times
1 hour
30 min
15 min
10 min
5 min
and please please would a linden or 10 check burns I mean theres serious lag there for weeks now not just for a day something is killing the sim.
November 1st, 2006 at 3:58 PM
This rolling update seems to be coincidel with anyone that has just logged in by that i mean how stupid is it that when you log in to your home location you dont even have enough time to rezz let alone finish loading before your kicked offline or have to tp to another region also if you ask me this rolling update has caused more problems than dealing with the problems already seeing as i cant withdraw my funds from the slxchange terminalls or from the ginko terminalls or even pay lindens to vendors or people this isnt a dig at linden labs or anyone its just my cents worth on the fact that if your going to deploy new updates you should at least warn residents a day in advance because of those of us thats in uk also consider the fact that a few weeks back you all mentioned something about nomore 2 week updates on a wedsnday well as far as i can see that has not been brought into action whatsoever all these rolling updates are doing is causeing more lag and more scripts to fail you should wait get a good version of sl working on the beta grid first go through it all with a fine tooth comb iron out all the bugs alltogether and then satisfy your customers who pay good money for this game by releasing them a reliable client that doesnt crash every time yet those who pay money for the game each month like me a yearly payment and monthly payment dont get any response from lindens when asking questions regarding sl or live help yet any newbie gets respected and answerd so far all my questions i have had only torly linden and char linden and a couple of others answer in time all the rest of the lindens i message are eaither coming back with the answer there too busy on the teen grid or just dont answer at all but thats not my point here my point is as follows
1) test all the sl client and server on the beta grid before releasing the updates with bugs in it
)
2) give the customer what they pay for ( a reliable client that works
3) the more you mess customers about the more they take there services else where and trust this from a person thats been building and designing 3d worlds for the last 5 years not once has the company i designed for had any problems deploying there code and having lag issues and bugs its called reliability tests you should try it some time
November 1st, 2006 at 3:58 PM
“…and offer an optional viewer that correct several known issues affecting many Residents in Second Life.”
Uh?
Where’s the viewer. This is my second login since this “announcement” and haven’t seen it yet?
November 1st, 2006 at 4:01 PM
The new client isn’t working in wine on a Debian (ubuntu) kernel. Are the windozers having any problem with it?
November 1st, 2006 at 4:12 PM
Marzinpan Maladay writes:
“A rolling restart just as the land store is about to open? Gee, I wonder why they are doing that? And, I also wonder why my previous post just seems to have disappeard.
More stuff that makes you go, hmmmm….”
really kind of scraping the bottom of the barrel for your conspiracy theory here…you can do better than this. I BELIEVE in you.
November 1st, 2006 at 4:18 PM
FYI, we’ve paused the restart while we investigate a DB load spike that caused some in-world issues about 45 minutes ago. So all my nice predictions about being done go out the window.
The DB load spike was not caused by the ongoing restart process, but we didn’t want to aggravate things. For those who are curious, we’re paused at about the 85% mark. We’ll start things up again to complete the remaining restarts as soon as we’re confident that we’re not going to hit another spike.
November 1st, 2006 at 4:31 PM
… and we’ve restarted again. Should be 9 minutes until the restarts are complete.
November 1st, 2006 at 4:38 PM
@Kane Kent:
I’ve been keeping my eye on Burns. It’s had some sim FPS spikes over the last little bit, but nothing lingering. Anything particular you’re concerned about?
November 1st, 2006 at 4:52 PM
Yeah its bad there not now but check back every few hours tomorrow and so on you will start to notice it. Many times the server there comes to a grinding 1-2 FPS.
Many people in the area that live would tell you the same. I’m not sure if there is a way upgrade that sim cuz theres a big new player lot there for freebie items and I don’t mind having them there I guess them dropping scripted items all over the sim and also being there puts a heavy load on the sim its self.
Thanks for getting back to me
November 1st, 2006 at 5:47 PM
Just a few quick replies here:
Getting the status on the website is probably not a high enough priority to get done before heterogeneous grid (updates that won’t kill what you’re doing). However, there may be a way to check to see if the sim you are on has been updated or not, which could probably lead to a resident-made map.
Having a 1 hour warning is way too much, exception being a blog post about the update. Whatever wait time you have, we have as well in order to see if anything goes wrong when the rolling update begins. If there was a 1 hour warning, it would mean 1 hour of waiting to see if the grid goes to hell or not. If it did, that would be yet another hour of waiting after we fix it. Perhaps a better idea would be to give a grid-wide hour/half-hour warning that we’re going to commence an update?
Finally, logging in only to be kicked off instantly due to a rolling update should be an extremely rare occurrence. My apologies if this happened to anyone, but we don’t have anything in place right now to prevent that kind of coincidence.
November 1st, 2006 at 6:34 PM
Have you considered just moving them to another sim instead of kicking them off?
“Oh look, there’s still people inside me and I’m shutting down. Why don’t I teleport them to the next nearest sim”. And so on until they’re out of the rolling update wave. Nothing annoys the heck out of me more than being logged out instead of just moved. My sim’s restarted while I was building, and it’s more of a hassle to log back in, find out I’ve been teleported to another place because my requested simulator is down, then I’m just sitting there hitting the hotkey for teleport home until my sim comes back up, then having to wander all the way back to my build and continue.
Not having to deal with that whole unload/reload process using up large amounts of time, it would be much more efficiant to just shunt someone a couple sims over.
Could even be as simple as rethinking the way you run a rolling restart — fire off the northwestern-most servers to the northeastern, and just continue your ‘ripple’ south, teleporting people a couple sims north when the wave hits them. (Warn them first, obviously, but force a teleport instead of a logout at the end.)
Also, a 5 minute warning is awesome — but you should consider a 30 minute and 10 minute window as well, when you have the time to. I say 30 minutes since that’s the longest time someone can stay idle, normally, so they’d be booted anyway.
November 1st, 2006 at 7:50 PM
Well nice try , my products are still dead after this update however they do manage to fly up about 10 meters before they crash to the ground . This means that there are countless thousands of vehicles that use this common flight script to fly the skies of secondlife and they are on the ground as useless junk now , I will have to deal with all the fallout these updates have caused and I’m about ready to throw in the towel as a content creator , it’s far safer to just make a unscripted box , give it a cool name then stand there and look at it , oh wait , can you believe it , the box is broke . . . .
November 2nd, 2006 at 2:47 AM
Well Im really impressed People. It is nice to have (once every thing finishes streaming) Movement that isn’t jerking every thing around all the time…. SL actually feels as stable as when I started almost a year ago, so concidering all that has been added in the last year, all I can say and please dont make me regret it. Way To Go Lindens, this place is an experiment in progress, But I no longer feel that the complexity of the place has gone past your ability to manage
Having fun again
SlimD
November 2nd, 2006 at 2:50 AM
@ Lindens: The warnings are okay.
I went to another sim, got a new warning a few minutes later, went to third sim, got a warning there, too. Finally went back to original place which was up again.
Jerry: that is a pity.
All residents should be able to look up existing bugs before bugging the creator. If i have a bug in mozilla or Safari, I go to bug tracking system and loo, if it is a known (unsolved) bug and enter a new report only, if there is no.
Here I have to enter a report always and I can only hope that a Linden reads it and does something about it - in a week, in a year of whenever. Is it solved? Is it a new bug? Can I do the work I put back because of a bug? Can I work again on projects, I have unready in my inventory?
The known issues page cannot be a replacement for a good bug tracking system with customer access.
November 2nd, 2006 at 6:43 AM
The 5 minute warnings were really helpful, perhaps I am biased because that’s how we work when we are taking our servers down (and getting sworn at by customers).
What I’d like to know is why after nearly every issue of this type my av is thoroughly Ruthed. I am sick of looking like Ruth. Ruth is simply not sexy.
November 2nd, 2006 at 7:50 AM
Cherry: rumor has it that Ruth was modelled on a LL employee. Hopefully she’s not reading your comment!
November 2nd, 2006 at 9:00 AM
I went to another sim, got a new warning a few minutes later, went to third sim, got a warning there, too. Finally went back to original place which was up again.
Jerry: that is a pity.
All residents should be able to look up existing bugs before bugging the creator. If i have a bug in mozilla or Safari, I go to bug tracking system and loo, if it is a known (unsolved) bug and enter a new report only, if there is no.
Here I have to enter a report always and I can only hope that a Linden reads it and does something about it - in a week, in a year of whenever. Is it solved? Is it a new bug? Can I do the work I put back because of a bug? Can I work again on projects, I have unready in my inventory?
The known issues page cannot be a replacement for a good bug tracking system with customer access.
You the best.
Oh my good…
November 3rd, 2006 at 1:31 AM
Regarding the 1hr warning, I think that something along the following might not be a bad idea:
“A rolling update has been started in this grid. This sim is scheduled to be restarted but has not yet done so”
Considering that the rolling restarts now take “about an hour”, a 1hr warning would basically broadcast grid-wide that a rolling restart was beginning. Which is probably a lot better than randomly bumping into the blog.
And as for posting current progress on the website, I hear you restart the sims in “server order”. So if the website had the server ID number that the restart was currently at, you could combine that number with something like “Sim Neighbors” to figure out which sims had already been restarted.
November 3rd, 2006 at 5:28 AM
“Torley Linden Says:
November 1st, 2006 at 2:34 pm
HOT DAMN, THESE 5-MINUTE RESTARTS RULE!
AND a warning at each minute during the countdown.
(I’m in adidas right now.)
FINALLY I have some reasonable time to pack up my builds and teleport!
Huzzah for freakin’ courtesy. *super big smile*
See you guys on the flipside. ”
Nice large corporation plug.
*rolls eyes
May 27th, 2007 at 9:05 PM
[URL=http://www.dapatente82.org/shell] shell [/URL] shell [URL=http://www.dapatente82.org/portatili] portatili [/URL] portatili [URL=http://www.dapatente82.org/2006] 2006 [/URL] 2006 [URL=http://www.dapatente82.org/user] user [/URL] user [URL=http://www.dapatente82.org/player] player [/URL] player [URL=http://www.dapatente82.org/cerca] cerca [/URL] cerca [URL=http://www.dapatente82.org/1993] 1993 [/URL] 1993 [URL=http://www.dapatente82.org/abbiamo] abbiamo [/URL] abbiamo [URL=http://www.dapatente82.org/volta] volta [/URL] volta [URL=http://www.dapatente82.org/vittorio] vittorio [/URL] vittorio [URL=http://www.dapatente82.org/perche] perche [/URL] perche [URL=http://www.dapatente82.org/trasporto] trasporto [/URL] trasporto [URL=http://www.dapatente82.org/connessione] connessione [/URL] connessione [URL=http://www.dapatente82.org/pocket] pocket [/URL] pocket [URL=http://www.dapatente82.org/pascoli] pascoli [/URL] pascoli [URL=http://www.dapatente82.org/rele] rele [/URL] rele [URL=http://www.dapatente82.org/hotel] hotel [/URL] hotel [URL=http://www.dapatente82.org/donne-foto] donne foto [/URL] donne foto [URL=http://www.dapatente82.org/oops] oops [/URL] oops [URL=http://www.dapatente82.org/forse] forse [/URL] forse
July 27th, 2007 at 2:59 AM
Nice!
August 21st, 2007 at 1:19 AM
Nice
September 12th, 2007 at 5:17 AM
Cool!