Michael Linden's Blog
A big part of the 1.15.0 release was a revamp of the systems behind the Group (and Conference) IM mechanisms, moving from a single server to a distributed set of systems. Obviously, this migration didn’t go without a hitch.
This bug summarizes both the problem and the pain pretty well:
We have fixes for the following issues undergoing testing internally, and they should be deployed early next week. Unfortunately, to deploy them will require a short grid downtime (20-30 minutes). Once we have tested the fixes, we will announce a time for this update.
- “X has left the session” is not seen when someone has spoken then left the group IM session
- “X has left the session” message sent when it shouldn’t
- Group IMs take up to 5 minutes to get through to you after a teleport.
There is one more case that will unfortunately require both a server and viewer update. We will evaluate the timing of releasing this based on feedback - kicking everyone off the grid for 1-2 hours and requiring a new download is not something we take lightly, and while the problem is irritating, we don’t want to make the cure worse. The issue is:
- ”Group chat IM reopens and error is received when attempting to send a message after user has closed group chat session”
Please report additional issues seen in the 1.15.0 release at our Issue Tracker [ http://jira.secondlife.com ].
We anticipate making an optional viewer update available within the next week or two with many accumulated fixes.
Our apologies in advance for the inconvenience, and our thanks to the Residents while we continue to improve Second Life.
[UPDATE 9:25 PM] Charity and I found and fixed one cause of vending machines and llEmail failing. We’re aware there are still problems. The problem is usually with in-world objects that receive email. In general, sending llEmail is not affected and newly created/rezzed objects are not affected. In some cases, restarting the server of the affected region can address the problem. We’re still working on it. I’m sorry it’s still broken. James Linden
[UPDATE 6:19PM] The problem is not related just to email. We are experiencing continued login issues, the friends list loading slow and IMs and are not always successful after a teleport. There may also be issues with group IMs failing and the map not updating as quick as it should. We have a operations working on it and it should be all resolved soon. We’ll keep everyone updated as it develops. Thanks! - Chadrick Linden
[5:44pm] — Linden Lab has received reports of trouble with e-mails between scripts, and from scripts to outside of Second Life — all using the llEmail function. The engineers are looking into it; if there is a problem, we’ll fix it as soon as possible! — Michael Linden
12:01pm - After yesterday’s update, we had reports of some problems in Second Life subsequent to the version 1.15.0 update — issues with land transfers, logins, IMs, and the LindEx.The land transfer problem was quickly fixed; but to apply the other fixes, we needed to shut down Second Life for about an hour (or so), starting at 10am PDT on Thursday, 26 April. No new client download was required.
The downtime ran until just after noon. Very sorry about that!
KNOWN ISSUE: a remaining problem will cause Residents to not get group IMs/conferences for a minute or so after teleporting. The engineers are looking into this issue. - Michael Linden
It’s another update-ey day in Second Life. While the regular flavor Grid is inaccessible, feel free to leave comments here!
[If you're in need of an SL fix, you can still log in to the Beta grid!]
12:30pm - Grid’s back up and accessible; log in and have fun! - Michael Linden
As we mentioned before, Second Life will be unavailable tomorrow, Wednesday, April 25 from 6 AM-12 PM PDT for the version 1.15.0 release. A viewer update will be required.
You can test-fly the new release on the Beta Test Grid.
Update @ 5:30pm - by Joshua Linden
Final release notes for 1.15.0 appear below the fold…
Downtime Status Updates - Wednesday - by Joshua Linden
7:00am - The release is in progress. The grid is currently shutting down. (Yes, it takes a while.)
8:50am - Things are on track. Most of the button pushing is done. We’re finishing up a bit of hardware work. After that we will start the grid up again and commence testing.
10:40am - We’ve brought the grid up (but still staff-only) and we’re doing our final validation testing now. - Michael Linden
12:04pm - We’ve identified and fixed a bug with outgoing XMLRPC calls. We’re doing final testing of the fix now, and then will open up. - Joshua Linden
12:25pm - Second Life’s Main Grid is open again; welcome back and thanks for your patience!
(more…)
We’re planning for scheduled downtime on Wednesday, April 25 from 6 AM-12 PM PDT for a new Second Life release. A viewer update will be required.
This brings us to version 1.15.0; please give it a try on the Beta Test Grid. See Josh’s earlier post at for details of what’s changed.
As we continue to improve the Second Life experience, we’ll keep you updated.
The Second Life Terms of Service have been tweaked to make it clear that open source browsers can connect to our server infrastructure. Also, the Privacy Policy was edited to state that we comply with applicable laws regarding personal data.
Please take a look at the Terms of Service at http://secure-web9.secondlife.com/corporate/tos.php
We ran a rolling restart from 3PM to 6PM, to address database load. Thanks for your patience!
[Edit: the rolling restart is done now - Michael]
During and after the 1.13.3.2 deploy today we noticed several things that did not appear during our internal and beta test cycles. We determined that a set of changes made to remove load from the central database associated with scripted email was not working as designed in some simulators and we decided to roll back those changes completely. That required that we restart all the simulators and was the cause of our 2 hour downtime extension.
Due to changes made in the way Search -> People is performed (as described at http://blog.secondlife.com/2007/01/26/changes-coming-to-searchpeople/ ) the offline state is always presented in the results and prevents the teleport button from being enabled. This will be fixed in an upcoming viewer update but the temporary workaround is to use the Offer Teleport button in the Friends list.
Also, this release made some changes to the results of llfrand(). Per LSL reference: llfrand returns a pseudo random number in range [0.0, mag) or (mag, 0.0], depending upon the sign of mag. This change caused a negative mag to be clamped to zero, hence the function will always return zero if a negative range is passed. The fix will require a patch to the simulator and the simulators to be restarted. A temporary workaround is to change calls to llFrand( -mag ) to 0 - llFrand( mag ).
[Edit: repaired an incorrect spelling of llFrand - Michael]
[Edit 2 Feb 07: comments closed, thanks for your input! Collating and distributing now - Michael]
[update] The query load on the database servers has been eased, and grid performance is returning to normal.
—————
We’re seeing heavy load on the database servers, and the associated problems in-world with failed teleports and difficulty rezzing objects. We’re looking into the cause and will let you know as we get more information.
|
12