First  <<>>  Last
Forum Main>>Announcements>> Note: Lag and problems *Updated*

ID: 1
Level: 35
Posts: 7241
Score: 12652
Chedburn [1]Reply | Quote | Report

Thread created on Thu Nov 08, 2012 18:05:08
Last replied to on Thu Nov 08, 2012 18:05:08
Hey everyone,

So we've been hard at work this week fixing the problems with the servers. (Well, at least our server guy Davz has). As you may have noticed over the past few minutes, the main culprit is the new icon system.

For some reason this system worked completely fine for over a month, and is now only now suddenly starting to cause problems.

We'll be adding serious caching to the icons system very soon to resolve this problem. This shouldn't effect the system as you see it at all, but should almost certainly remove the lag it causes.

We've also ordered a new dedicated server which will solely be used for logging purposes, helping us to find the culprit of any issue on Torn very quickly. This has often been very difficult in the past due to the complexity & size of the site and the amount of different servers we need to manage, but thanks to new logging software (the same one that facebook uses) this will be a thing of the past.

Feel free to view the server status by clicking the server name on the sidebar. This will bring you to a list of all dedicated web servers currently in action. Once this issue is resolved, we'll do our best to keep each server below 0.35 which is blazingly fast.

Update: The icons system has now been rebuilt ready for the new caching method to be implemented. I imagine this will be added over the next 1-2 days.

Update 2: The icons system rebuild is now in place and is working very well (as you can see via the server status page). However... The new system is now overloading our mongoDB servers which is causing the locks on some pageviews causing delays. It's very frustrating, but at the same time we have (and continue to) vastly speed up some of the sidebar scripts which run on every view, so once this is fixed Torn will be faster than it's been in a very long time. Sorry for the inconvenience. I can't wait for this all to be dealt with once and for all.

Update 3: We've discovered the icons actually have no part in the current lag at all (Although moving them over to a new system has sped things up a little). With the new system, icons now take 0.0159 seconds to load so they are now checked off of the list. It's all based on user sessions writing to disc. This session system was previously changed over to resolve the random logout issue, but seems to have created this lag we're seeing now. We're now working on moving this over to a new system stored on RAM. We expect to have this resolved over the next few days.

Update 4: The session system change is now complete, I believe there is only one issue which is with the 'logout' button not working. This will be resolved ASAP.

Update 5: We've fixed the 504 lag errors on messages, it was due to the previous conversation logs, so they have been temporarily removed.

Update 6: We sacrificed 3 goats to the server gods, they seem to be pleased, for now...


Last Edited: Mon Dec 03, 2012 18:21:43
Forum Main>>Announcements>> Note: Lag and problems *Updated*
First  <<>>  Last

This thread has been locked.