Rolling Update - 11:30 AM to 1:30 PM PST
Friday, November 17th, 2006 at 11:23 AM by: Joshua LindenIn just a few minutes we’re going to initiate an update of the simulator code that may help us address one of the frequent causes of server crashes in recent weeks. It may not address the crashes, but should give us additional information about the nature of the crashes.
Each region will be restarted, with a five minute warning given in-world in each region warning residents that they’ll need to move or be logged off. Most regions should come back online in under five minutes of the restart. The process should complete in about an hour and a half.
For those who don’t track Pacific Time, this corresponds to a window of 19:30-21:30 GMT.
[Update: 12:30 PST - we paused for about half an hour to investigate an issue that turned out to be unrelated to the restart; the restart process will complete about half an hour later than originally expected.]
November 17th, 2006 at 11:27 AM
GMT timezones
save me working them out 
thanks!
November 17th, 2006 at 11:44 AM
glad to see our fees being put to good use !
November 17th, 2006 at 11:53 AM
hehe YAY for GMT times!
(and of course YAY for the update if it helps fix things!)
November 17th, 2006 at 11:55 AM
i got the 5 min warning but was logged out of system in 15 seconds before i could go anywhere.
good thing i had already saved my build.
November 17th, 2006 at 12:24 PM
“i got the 5 min warning but was logged out of system in 15 seconds before i could go anywhere.
good thing i had already saved my build.”
Things you don’t want to here from your server admin: “CRAP! EVERYBODY SAVE, NOW!”
November 17th, 2006 at 12:31 PM
@Uber Stein: I suspect the region you were in crashed coincidentally right after the warning was given. We saw that happen to Boulevard Mystique.
November 17th, 2006 at 12:48 PM
…update of the simulator code that may help us address one of the frequent causes of server crashes in recent weeks. It may not address the crashes…
Which one is it? This makes no sense. May address the crashes, or may not… hmm…
November 17th, 2006 at 12:54 PM
Linden-speak for ‘oh hell! It might work guys, let’s do it’ Chance
November 17th, 2006 at 12:54 PM
@Chance Unknown: Rather than a fix it’s another instrumentation-type change, meaning it gives us more data about the cause of the crash, or at least eliminates a few possibilities. I admit my wording was vague - it will “help us address” the crash (i.e. help the investigation), but won’t actually fix it itself.
November 17th, 2006 at 1:16 PM
Thanks for the GMT time!
November 17th, 2006 at 1:17 PM
If WordPress supports it, displaying the SLT on blog pages would be nice. Doesn’t have to be a dynamic java/flash thingy, just something that showed the SLT when you went to the page you’re on.
November 17th, 2006 at 1:23 PM
Yay for the GMT time!
And I agree with A Resident.
November 17th, 2006 at 1:44 PM
I was in two different locations that were going down. The warning system worked perfectly. MUCH better than “You have been logged out. Tough noogies.”
November 17th, 2006 at 1:56 PM
Joshua,
Thank you for the updates along the blog-trail.
They are much appreciated.
Especially to Uber Stein’s comment–I’ll bet your quick response saved you tons of grief mail.
I wish the rest of the Lindens would learn that fast and frequent communication is a NET time and grief saver for everyone.
November 17th, 2006 at 2:06 PM
@Sterling Whitcroft: Thanks for the kind words.
During a rolling restart, I actually have cycles to poke the blog a few times. Once we’ve initiated the process there are usually 2 or 3 Lindens watching various parts of the system to make sure everything is going as expected. Apart from taking notes on how long things are taking and the occasional moments of “OMGWTF!?!?! Oh wait, that’s normal…” terror, it’s pretty boring. Until something goes wrong.
November 17th, 2006 at 2:38 PM
So is most of the company just sitting down twiddling their fingers and staring at the big board as this small group of Lindens feverishly restarts the system so they can get back to work or something?
What do Lindens do during these rolling restart times when they can’t help out with the process itself?
November 17th, 2006 at 2:39 PM
Thanks for the GMT time! what with coming out of BST to normal GMT i’m forever lost calculating timezones!
November 17th, 2006 at 2:42 PM
LOL@ ‘Occasional moments of “OMGWTF!?!?! Oh wait, that’s normal…” terror…’
Hehehe! Now you know what the average Windows sysadmin goes through everyday. “Onoes!! …wait, it’s supposed to do that…”
Hope this will give you what you need to finally fix some of these issues. Take care, and good luck!
- SDW ^..^
November 17th, 2006 at 3:05 PM
Why is it whenever the sim crashes or sometimes rolls over my traffic numbers restart.. if my numbers stayed i would be showing an enormous turnover of traffic but with so many crashes of the sim lately i keep going back to the beginning which is not good to encourage vendors to come to my mall. They see numbers like 400 instead of in the thousands….This is very frustrating.. can this be resolved
November 17th, 2006 at 3:10 PM
“Occasional moments of “OMGWTF!?!?! Oh wait, that’s normal…” terror…”
what’s worse is trying to explain to an “OMGWTF’fing user” that what they did is in fact normal…
if you don’t know what I mean, see:
http://ars.userfriendly.org/cartoons/?id=20030128
November 17th, 2006 at 3:29 PM
@Odysseus Fairymeadow:
That’s actually a really good question. We warn everyone in advance, especially Liaisons, and give regular status updates on our internal IRC channel. But most of the developers and QA team, for example, are plugged into isolated development grids working on their separate code, so it doesn’t affect them. Our operations team usually has someone watching various monitoring systems to make, but otherwise there isn’t much of an impact on the Lindens, other than the usual in-world disruption as the restart occurs.
November 17th, 2006 at 11:03 PM
I was having my big product unveiling at Love Thyself when, wouldn’t you know, SL crashes. Four or five more crashes this evening. Then, just now, just about to close a sale with a rep and her client, SL crashes again - this time taking her demo unit with it (and of course the client didn’t come back either).
FIX SOMETHING! Is this really the way business is supposed to work? No backups allowed, no uptime assurances at any price, no recourse for losses imposed?
!@#$!@! !!!
November 18th, 2006 at 12:39 AM
*thumbs up*
November 18th, 2006 at 11:55 AM
Is this the reason I cannot log in?
Every time I try I get a message saying my account will not be available for approx 5 minutes.
When I try after the given time I get the same message but giving another 5 minutes. This has been happening for half an hour
November 19th, 2006 at 10:43 AM
Woohoo, wish Linden Labs would use GMt in game as well, it would make life so much easier for those of us who live this side of the Atlantic….so how about it Lindens?????
November 22nd, 2006 at 5:47 AM
yes! yes! yes!!!! to GMT please… at least the whole RL world knows what it is and how to adjust to their local time if necessary. Personally, I have no idea at all what PST means, so how am I supposed to translate SL event times into my local time???
If SL has aspirations to be a truly global concept, then they should accept that a global reference time already exists… it just doesnt happen to be centred on America!… (omg! yes! Something in this world is not centred on America!!!)
February 19th, 2007 at 12:20 AM
Administrare server