Critical RAID server data recovery from Manchester fire.
This case study relates to a job CDR received a few months ago. The circumstances were particularly unfortunate with regards to how failure/data loss occurred. During the summer of 2013, there was a large fire on Oldham Street, Manchester city centre. It received significant media coverage not only due to the size and location of the blaze but also because a fireman attending the scene died during attempts to extinguish it. Details regarding this story can be found on this BBC News report.
Above the shop in which the fire was started there were offices which held 12 servers for a large UK based company.
Above the source of the fire, there were a number of offices. Inside the offices were a total of twelve servers all owned by a large UK company.
Reasons for data loss
There were a total of 5 tape backups. One for each day of the working week. 4 were kept off-site at any point. The fifth during the day after the backup was performed was kept in a fireproof safe.
Faulty Backup Tapes
A system administrator had accidentally turned on ‘incremental’ backup. This meant that one tape held a starting copy of the data, which was required if data from any of the other 4 tapes needed to be restored. This master tape was the one which had been kept in the office safe on the day of the fire. It was retrieved and had suffered a significant amount of heat and smoke damage. It was not possible to recover any meaningful data from it.
The fire damaged server HDDs
Immediately after the evacuation of personnel from the building the system administrators were able to remotely login into the server and start extracting current data set. They were able to do this for 10 of the 12 servers before the electricity was turned off for the building. It was not possible to extract data from 2 of the servers. These contained critical business information relating to online purchases of event tickets for UK and international events and festivals.
The HDDs from the servers arrived at CDR – Manchester Data Recovery Services – after we confirmed that it was not possible to extract data from the heat damaged LTO tape backup. The HDDs had suffered from heat and smoke damage. Components on the printed circuit boards (PCBs) had melted, suggesting that the temperature in the office exceeded 350 degrees centigrade.
Some of the 12 HDDs which had been damaged in a major fire in Manchester city centre in the summer of 2013.
There were two 6-disk RAID 5 arrays using Seagate SAS Cheetah 15K.5 ST3146855SS HDDs. All HDDs were in a similar condition. Careful handling, cleaning and repair were required for each of the HDDs. Fortunately all but one of the HDDs responded well to recovery attempts and it was possible to obtain a full clone of each HDD. Once this was complete a standard RAID recovery procedure was followed which involved emulating the failed RAID controller hardware which existed in the server.
On successful extraction of the data a file listing was sent to the customer detailing the recovery so they could ensure that the data they wanted had been recovered OK. The file listing was approved and the data was returned. All of this work was completed on a 24 to 48 hour emergency service over a weekend to ensure the data was ready for 9.30am on Monday morning.
After-sales support
In this case, various exchange in-boxes were required. Although the large EDB file had been extracted OK the customer did have an easy way to mount the exchange database server. At no additional cost, CDR mounted the EDB within our own specialist recovery software an extracted each of the mailboxes into individual browsable folders which could be imported as PST files to any version of Outlook.