Forums
First  << 1  2  3  4  5  6  7  8  9  10  11  12  13  14  15  16  17  18  19  20 >>  Last
Forum Main>>Announcements>> VERY IMPORTANT - PLEASE READ
Chedburn

ID: 1
Level: 35
Posts: 7240
Score: 12650
Chedburn [1]Reply | Quote | Report

Last replied to on Thu Oct 09, 2008 02:49:50
Although it could have been a hell of a lot worse, some user data has been set back by one month. This does not include quite a few things. Bear in mind, this was in no way our fault at all, and will NEVER be happening again ever.

The worst part: All user accounts created in the past month have been removed from Torn City. Their donator packs however ARE recoverable by using donator issues.

-Just prior to an upgrade (literally an hour before), a RAID controller failed
-During this failure one of the drives died - probably killed by the controller. Since we were using RAID-0 in the short term for performance, this is fatal. (again, this was going to be changed to RAID-1 approximately one hour later).
-This combination of failures killed the partition
-Despite 98% of these failures being recoverable, the array has been examined professionally and it is not recoverable
-We therefore have to rely on our backups. We have offsite backups of the database, however on examination these have only been taking the first 80% or so of the database. This is entirely the host's fault for not checking them, although the method used was standard (a mysqldump) and there was no error produced during the automated backup process.
-We therefore have to use old data for the last few tables - basically the users table.
-This is a combination of worst case scenarios that it is hard to foresee. We shall of course be manually checking offsite backups regularly from now on.





Heres where we stand.

Firstly the things which are recoverable and will be exactly as you left them 4 days ago:
Items
Factions (although users will have to be reinvited/kicked)
Donator packs (you will have to use them again)
Battle stats
Auctions
Awards
Forums
Pets
Friends/Enemy lists
Some but not all crime EXP


Things which are not recoverable and will be approximately one month out of date:
Your level (set back one month)
Education courses (set back one month)
Your points (set back one month) (but DPs will be recovered meaning a lot more points back in to the system)
Your property (set back one month)
Bank times (set back one month)
Completed missions (set back one month)
Merits (set back one month)
Password (set back to what you had one month ago, please change them)



I would like to make three promises to you for the future.

1) We will do everything in our power to produce at least one large update every month from now on.
2) In future We will ALWAYS give you at least 24 hours notice before a downtime to the hour (not including emergency downtimes)
3) We will make full backups daily and check they are all working fine to avoid anything like this again
4) We will make further manual full database backups to our home desktops twice per week
5) Long downtimes like this will never happen again
6) Data will NEVER again be more than 24 hours out of date if anything like this happens again.

I feel Torn City has grown a hell of a lot stronger over the past 4 days. I promise you, by march 1st, everything will feel back to normal again and with some nice new nifty upgrades.

I sincerly apologise, but promise nothing like this will ever happen again.

I will post a short blog tomorrow answering all questions and how far along the process is.

Thanks very much for understanding.


Edited by: Chedburn on 20/02/07 at 1:14:35 PM

Ched.jpg
Super secret reinforced spam barrier 2.0
Aethwynn
ID: 9900
Level: 41
Posts: 2481
Score: 0
Aethwynn [9900]Reply | Quote | Report

Posted on Mon Feb 19, 2007 18:13:03
bleh.. at least stats are safe.


Edited by: Aethwynn on 19/02/07 at 6:18:22 PM


f1.jpg

Super secret reinforced spam barrier 2.0
Mix

ID: 21666
Level: 74
Posts: 4760
Score: 504
BOX Mix [21666]Reply | Quote | Report

Posted on Tue Feb 20, 2007 13:24:39
Thanks for the info about what happend..

bam2.gif
Super secret reinforced spam barrier 2.0
04doolant
ID: 49405
Level: 50
Posts: 2813
Score: 555
04doolant [49405]Reply | Quote | Report

Posted on Tue Feb 20, 2007 14:04:55
Thanks for getting tc back up

49405-Sig.gif
Super secret reinforced spam barrier 2.0

ID:
Level:
Posts:
Score:
 []Edit | Reply | Quote | Report

Posted on Tue Feb 20, 2007 14:05:19
Glad its back and all. And not too much is lost.

Super secret reinforced spam barrier 2.0
aurigus
ID: 37524
Level: 45
Posts: 921
Score: 231
aurigus [37524]Reply | Quote | Report

Posted on Tue Feb 20, 2007 14:05:28
Thanks for the update Chedburn!



Super secret reinforced spam barrier 2.0
Brown0
ID: 44666
Level: 47
Posts: 422
Score: 0
Brown0 [44666]Reply | Quote | Report

Posted on Tue Feb 20, 2007 14:05:50
At least stats are good, I'm most upset about the bank

You bought 1 points for $1 and a total of $1 from the market. They will be delivered to you when I can be bothered.
Super secret reinforced spam barrier 2.0
-Adam-
ID: 120693
Level: 33
Posts: 270
Score: 0
-Adam- [120693]Reply | Quote | Report

Posted on Tue Feb 20, 2007 14:05:58
are you going to fast forward bank times?? Lol

nadzcarnageanime0cs.gif
Super secret reinforced spam barrier 2.0
Hutton

ID: 46405
Level: 42
Posts: 380
Score: 0
Hutton [46405]Reply | Quote | Report

Posted on Tue Feb 20, 2007 14:05:59
urgh urgh urgh urghh i think im gunu have to buy my palace again! =(

Super secret reinforced spam barrier 2.0
GodzGift786
ID: 176558
Level: 52
Posts: 2656
Score: -156
GodzGift786 [176558]Reply | Quote | Report

Posted on Tue Feb 20, 2007 14:06:06
i lost like 500 points i thikn

man i have lost 500+ points:(


Edited by: HaNk_K00ldude on 20/02/07 at 2:08:08 PM

You have 0 casino tokens and a current streak of $86,495,865 :p


TO the GUy who keeps hospitalising me .If u r man just leave ur name once!!!
Super secret reinforced spam barrier 2.0
Ardoneye
ID: 258526
Level: 50
Posts: 1098
Score: 43
Ardoneye [258526]Reply | Quote | Report

Posted on Tue Feb 20, 2007 14:06:11
......ive been downgraded to a mansion lost 4 mill cash all my donator days 2 levels and f ing loads of stats and so on if this doesnt get sorted sorry but im quitting


Edited by: MaTrIxFrEaK on 20/02/07 at 2:13:51 PM

SIG-1.png
Super secret reinforced spam barrier 2.0
JimbobBEng
ID: 27610
Level: 65
Posts: 3369
Score: 679
JimbobBEng [27610]Reply | Quote | Report

Posted on Tue Feb 20, 2007 14:06:13
ok so are you going to put the bank forwards again I have been waiting for a long time to get my money out.... doesnt seem fair that you just set the times back one month!

1000_days_forum.png

Buying Stat Enhancers. Mail me your prices
Super secret reinforced spam barrier 2.0
kyrannblake
ID: 267055
Level: 20
Posts: 1
Score: 0
kyrannblake [267055]Reply | Quote | Report

Posted on Tue Feb 20, 2007 14:06:14
Thanks to all staff for fixing this

Super secret reinforced spam barrier 2.0

ID:
Level:
Posts:
Score:
 []Edit | Reply | Quote | Report

Posted on Tue Feb 20, 2007 14:06:15
try http://www.ibas.com/

Super secret reinforced spam barrier 2.0
FallenOne
ID: 13634
Level: 54
Posts: 442
Score: 9
GAFallenOne [13634]Reply | Quote | Report

Posted on Tue Feb 20, 2007 14:06:38
Thanks for the info Ched, sad to hear that this happened, but it's not your fault.

Fallen One.

Super secret reinforced spam barrier 2.0
Mumra_2790

ID: 269622
Level: 55
Posts: 1149
Score: -61
BAMumra_2790 [269622]Reply | Quote | Report

Posted on Tue Feb 20, 2007 14:07:00
thanks for the info ched:)

dj_logo-1-2.pngplaygroundradio.png
Super secret reinforced spam barrier 2.0
Sweeney_Todd

ID: 27468
Level: 63
Posts: 46411
Score: 20044
BoDSweeney_Todd [27468]Reply | Quote | Report

Posted on Tue Feb 20, 2007 14:07:10
We missed you guys...

sweeneyeb2.jpg


The day I start taking this seriously is the day I quit...
Super secret reinforced spam barrier 2.0
Sir_Hugh_Felch
ID: 201349
Level: 25
Posts: 35
Score: 0
Sir_Hugh_Felch [201349]Reply | Quote | Report

Posted on Tue Feb 20, 2007 14:07:11
Call me old fassioned but surely raid 5 is a better option, especially for a server that provides a service....

Super secret reinforced spam barrier 2.0

ID:
Level:
Posts:
Score:
 []Edit | Reply | Quote | Report

Posted on Tue Feb 20, 2007 14:07:16
awesome, will i now get my lvl 30 merit and can i use it again( used it on stats, stats are same as they were, but i will get new merit? will i? )

nothing more :=)

Super secret reinforced spam barrier 2.0
Elusive
ID: 420114
Level: 55
Posts: 3088
Score: 184
Elusive [420114]Reply | Quote | Report

Posted on Tue Feb 20, 2007 14:07:21
had to sign up again, seein as i was only 25 days old *sigh*

lost 3 months worth of dp's which i can find noway to claim back as i cant go to the donate page as i am not over level 3 ...

lost all my stats.. gone from level 7 back to level 1 ...lost everything the whole darn lot, that i spent a solid month gaming for....... REALLY peeved about the DP's ...lets see if i actually get them back tho, with me claiming them back under a NEW & different account id




Edited by: chilledXheart on 20/02/07 at 2:34:55 PM


Edited by: chilledXheart on 20/02/07 at 4:13:30 PM

30stmc.png
Super secret reinforced spam barrier 2.0
Maxx-
ID: 233792
Level: 31
Posts: 225
Score: 0
Maxx- [233792]Reply | Quote | Report

Posted on Tue Feb 20, 2007 14:07:33
:( i had just made about 6 mill on roulette... oh dear

barthing.jpg
Super secret reinforced spam barrier 2.0
DrSeussFreak

ID: 131293
Level: 69
Posts: 2255
Score: 94
CgDhDrSeussFreak [131293]Reply | Quote | Report

Posted on Tue Feb 20, 2007 14:07:37
omg, I dropped 3 levels... No way this only went back 1 month.,..

Damn, well at least it is back up (sort of)... I can post to the forum anyways...

DSF_TC_Forum_Sig_2.gif
CD_Leader.gif
Super secret reinforced spam barrier 2.0
Crushproof

ID: 365494
Level: 27
Posts: 190
Score: 51
Crushproof [365494]Reply | Quote | Report

Posted on Tue Feb 20, 2007 14:07:42
i lost my house ah well could of been worse

Father_TedGunB1.jpg
Super secret reinforced spam barrier 2.0
spiritwolf
ID: 420101
Level: 32
Posts: 15
Score: 0
spiritwolf [420101]Reply | Quote | Report

Posted on Tue Feb 20, 2007 14:08:48
so the not recoverable stuffs will be gone permanently

Super secret reinforced spam barrier 2.0
JimbobBEng
ID: 27610
Level: 65
Posts: 3369
Score: 679
JimbobBEng [27610]Reply | Quote | Report

Posted on Tue Feb 20, 2007 14:09:00
i am also confused as to why you arent using raid5 with at least one redundant disk, then this would never happen (unless you had 2 drives simultaneously fail which is pretty unlikely!)

1000_days_forum.png

Buying Stat Enhancers. Mail me your prices
Super secret reinforced spam barrier 2.0
HHawk
ID: 185678
Level: 53
Posts: 26
Score: 18
EDMNHHawk [185678]Reply | Quote | Report

Posted on Tue Feb 20, 2007 14:10:10
Damnit 4 levels down. Just when I was planning to become a donator.

Oh well, maybe in 6 months I will think about it again.

Super secret reinforced spam barrier 2.0
Z_junior

ID: 247674
Level: 96
Posts: 7853
Score: 3850
JFKZ_junior [247674]Reply | Quote | Report

Posted on Tue Feb 20, 2007 14:10:45
Where do we report lost banks? My entire bank is gone (banked 1 week ago before downtime).

JFKbanner.gif
Super secret reinforced spam barrier 2.0
Tkama
ID: 72301
Level: 55
Posts: 1561
Score: 71
Tkama [72301]Reply | Quote | Report

Posted on Tue Feb 20, 2007 14:11:49
what about crimes and busts? will they be recovered or do i have lost over 1k crimes and 200 busts forever?

"I see dumb people... they're everywhere... they walk around like everyone else...

they don't even know that they're dumb."
Super secret reinforced spam barrier 2.0
-Clansdancer

ID: 65306
Level: 85
Posts: 12169
Score: 3742
~V~-Clansdancer [65306]Reply | Quote | Report

Posted on Tue Feb 20, 2007 14:12:32
I lost 200 million for the house I purchased ...

Lovely

Super secret reinforced spam barrier 2.0
LabeL-LoveR

ID: 199077
Level: 69
Posts: 207
Score: 141
JFKLabeL-LoveR [199077]Reply | Quote | Report

Posted on Tue Feb 20, 2007 14:12:49
i lost ..99 mil that ar in bank..i won at RR...:(..can i get them

Super secret reinforced spam barrier 2.0
DemonSkye

ID: 37124
Level: 79
Posts: 2474
Score: 1084
DemonSkye [37124]Reply | Quote | Report

Posted on Tue Feb 20, 2007 14:12:52
Im going to be the first to say that im a little annoyed about this... in the 30+ days i lost i hit 2 stat goals, did the lev 35 mission, hit lev 35, and some other things... I mean its kinda an area where i just think that its ubershifty that things got rolled back an entire month. you have a TON of people working on the game, and people consulting on it (davz) but somehow you didn't get an accurate backup of a mysqldb for over a month? and this guy is a published author about mysql clustering?

sorry if i seem critical but you must understand my extreme dissatisfaction with tc and its reputation of delivering on its promises (lag free... oops, j/k. new server, lag free.... j/k.... lag free beta... lag free real server.... j/k.)

Im not even bothered by the fact that I lost x days of my donator pack on this.. im grumpy because i lost 25-30k of stats that i now have to retrain... and thats just a hassle. im sure you can all relate to that.

anyways I digress... i hope this catastrophic failure taught us something about the importance of routine, accurate backups.
Fyrestorm.

p.s. edit
======================
didn't read post properly.
- some rage about battle stats not being recoverable.


Edited by: Fyrestorm on 20/02/07 at 2:20:19 PM

demonskye.png

If you like something I've done on the forums send me a morphine, I'm running out!
Forum Main>>Announcements>> VERY IMPORTANT - PLEASE READ
First  << 1  2  3  4  5  6  7  8  9  10  11  12  13  14  15  16  17  18  19  20 >>  Last

This thread has been locked.