Nightmare - External hard drive crashes to floor

[RTSDaddy2]

Not sure where this goes, but didn't know who else to turn to for honest help.

Damned if I didn't do a stupid thing about three weeks ago. I was moving video files, via my laptop, from our old Iomega external HD.  Both were sitting on the top of a short bookcase in our bedroom, maybe 3-4 ft tall (yes, another bonehead idea, but anyway). My wife had gone to work, and my kid had just about driven me crazy. Without thinking I went to move the laptop off the bookcase to the bed, and....yep, you guessed it - the HD was still wired up.  On a hardwood floor, no less.  Sounded like a damn bomb went off when it hit, but it landed flat (like that matters), not on its side.

I have little doubt that the arm is out of sync, with the clicking noise I heard when I powered it back up. I've done research via google, and it's not overly encouraging.  I have tried various "freeware" data recovery programs, but have not found anything that finds the drive - or if it does, I don't know what to do next with it.

I am still out of work, so a data recovery service is completely a useless idea for now. Thought about opening it up, moving the arm back, and then trying to copy the data, but that's a bit out of my complete comfort zone.

We are just wondering at this point if I totally toasted it and there's no hope of getting the data back.  The only truly valuable things we will have lost are pictures of our little one that my wife had not yet moved from it...everything else is "redoable" as most of it was downloads....it'll just take forever and a day.

Any help - your suggestions for data recovery program, a cheap data recovery service, etc. is greatly appreciated.  I've been  at myself for three weeks now - and like I said, while I see info on it online, can't make heads or tails of some of it after getting started.  Thanks ahead of time!

[protokatie]

Can't be certain if I am not there to look at it, but it may just be ---fouled up beyond all recognition---. One suggestion would be to use a program called SPINRITE to recover data. It has been around since the 80's (even a computer chronicles ep about it) and is updated every few years. It costs like 80 bucks, but is considered a must have by every computer nerd on earth (PC, MAC, Amiga, Linux, etc nerds). It only runs on PC, but can read and recover data on almost any formated drive. One thing you may have to do tho, is to remove the HDD from the case and use an adapter to connect it directly to a full fledged PC.

Outside of that, I cannot stress enough how much you need to BACKUP data, not just move it. Simply buying a stack of 40 DVD-r's for 20 bucks and making 2 copies of all of your most valued stuff (on top of the copy on the HDD) is more important than anything. Not to chide you, but backing your data up is a pro-active thing and is cheap in comparison to recovery (if possible).

Anyways, check out SPINRITE, it is the only program I know of that will do low level reads and can even read over "destroyed" data multiple times to get a heuristic "idea" of what the 1's and 0's are on a damaged track.

Sorries.

[ark_ader]

External hard drive eh?  They are capable of withstanding falls from 6-8 ft.  Heck I had one bounce down some concrete steps in our old apt and it survived.

Well for a start I would open her up.  Sounds like power is not getting to the drive (clicking sounds).

Find out what the interface is and try to power it via your laptop/PC.

I know its late saying this, but when you have some pennies or a spare PC kicking around - convert it to a NAS, and BACK EVERYTHING UP.

If it is truly shagged, then find the same drive and manufacturer and buy another one. 

You might just be lucky enough and find that its just the main board is damaged.

Swap the main-board and get the data off.

Spinwrite wont help you.  But it is at www.grc.com 

Good luck!

[SavannahLion]

Simply buying a stack of 40 DVD-r's for 20 bucks and making 2 copies of all of your most valued stuff (on top of the copy on the HDD) is more important than anything.

Uh... it's a little more complicated than that. A DVD's half life is meager at best. Bought a stack of 100 HP DVD+R and they noticeably lighter than the last stack of TDK DVD+R I bought. Would've bught another TDK spindle, but no one carries it anymore. I expect the HPs to delaminate or get some bit rot before the TDKs. The HPs look and feel more like CDs of yesteryear.

But I digress. If you're making two copies of your files (as you should) use two different manufacturers, not just brands. Go and download DVD Identifier (If you have to pay for it, you're being ripped off). Use the tool to identify your spindles until you find two that are from different manufactures. There are websites that help clue you in, but it's hit and miss. Maxwell changed manufacturers and I got a crap spindle of black archivals once. When you have two spindles from different manufacturers then make your backups.

Having two different manufacturers for your discs reduces the likelyhood that both will delaminate or experience bitrot at the same time.

If you don't think you'll experience bitrot, believe me, you will. I have burned CDs from the early days that are already failing. I lost about 20% of my old custom Carmageddon files because of this ---smurf-poop---.

[saint]

I have completely abandoned optical media for backup. I keep copies of my data in multiple places (original on each machine, duplicate on the server in the house, another duplicate on my work machine), but I think the most brilliant backup addition I did was to buy thumb drives. Stick important stuff on a thumb drive, and put it in my fireproof locking safe in the house. Small, relatively cheap, and getting bigger all the time.

[HaRuMaN]

Small, relatively cheap, and getting bigger all the time.

That's what she said!  Wait... wot?

[Havok]

Saint's got it right - optical media is too volatile. Best to keep your data on multiple devices in multiple places. I'm a little skeptical on how long jump drives would hold their data however. I personally have two mirrored 1TB drives where all my pictures and crap resides.

RTS - sent you a PM...

[saint]

Do you have those elsewhere as well Havok? Mirror will protect you from a drive failure, but not something happening to the volume or disaster in that machine, house fire, etc...

I'm uncertain as to the longevity of jump drives myself. Eggs in multiple baskets for sure!

[Havok]

That's the thing - I have them both in my home office. They're external for easy access, but you're right - I probably should periodically grab an image of the drive and store it offsite in case of home disaster. I like the idea of "borrowing" space on my work computer, I think I'll be using that hard drive on my laptop a little more fully now...

 

[MonMotha]

Was the drive on when it fell?  If so, it's probably toast.  If not, it probably survived, but the little board that's hooked up to it to do the USB or whatever side of things may have gotten loose or damaged.  If it's doing a "click", "click", "click" thing with a regular interval after spinning up, then it's almost certainly gone beyond any recovery you can attempt.  Generally that means that the firmware can't find its position for the armature.  Cause is usually a "head crash" or a bad position/limit sensor.

The thing to do is yank the drive out of the enclosure and hook it up to a PC directly.  If it at least shows up in the BIOS as present, you've got a good start.  You can then attempt various recovery methods on it such as spinrite.  Even just using a Linux live CD can sometimes let you see things Windows/DOS will choke on.  Probably a good idea to keep everything mounted read-only if you do get any access, just in case.  If you do get access, you may want to image the drive to another drive or file then attempt more complicated recovery "offline".

If you can't get any of that to work, you have a couple last ditch options.  These options are mutually exclusive since the cheap option tends to destroy the drive entirely if it doesn't work, rendering the expensive option impossible.

As a cheap option, you can chuck the drive in a freezer overnight then attempt software based recovery methods again.  Sometimes the temperature change is enough to let the drive find all its alignment stuff and work *for a while*.  If it does work, copy off everything you can as fast as you can.  You generally only get one shot.  Only do this if the data is not worth spending the money for professional hardware recovery.

As an expensive but more reliable option, call up OnTrack or a similar company.  If there's data to be recovered from that drive, they'll get it.  Expect a bill in the upper 3 to mid 4 figures.  Some complicated recovery processes may even hit lower 5 figures.  Don't attempt to freeze the drive then send it to OnTrack when that doesn't work.  That generally just costs you more money or results in them being unable to recover anything.

Whatever you do, do NOT attempt to open the drive.  That will generally just kill it.  Tolerances on hard drives are so tight that a speck of dust can destroy them.

[Xiaou2]

 
 You are better off keeping the drive closed until a day when you can afford a recovery specialist.
 
 The drive heads are small as a human hair, and once they hit the surface of the drive, it
destroys them, and even sometimes scratches the surface of the discs.

 I believe Ive heard of some people transplanting part from one drive to another drive of the
same model... however, if Any dust gets on a surface, you will just have ruined another drive.
As said, a single spec of dust can ruin an HD head.

[ark_ader]

When do you open a drive casing to replace the main-board?

There is a plastic trace connecting the main board to the drive assembly.   

[RTSDaddy2]

Protokatie and others:

You didn't hurt any feelings by reminding me to do backups.  The really stupid, sorry thing we did was when we bought the external drive, we bought it to " make room" by moving files to the EHD from our desktop....but the issue is, if we'd have only copied the pictures of our kid - instead of doing a move - we'd still have them.  Had that happened, this wouldn't even be an issue; I'd just chunk that drive and start over.

One thing you cured me of is thinking of opening the hard drive itself.  I had read online that sometimes if you open up the drive itself I could reposition the arm, but that sounds like a really, really bad idea.

[saint]

So here's a thought - have you tried undeleting on the machine's internal hard drive to see if you can recover any of the files you moved?

[Comtek]

i do not recomend what i done but it worked
had a internal hard drive in and external enclosure fall
all i got after that was a clicking
knew i wasn't supposed to but opened it up anyway the data on it was something i could collect again
i opened it up and turned it on and seen that the platters were not spinning i twisted them heard a crunch they started to spin after that
with a noticable scraping a vibrating noise but windows recognized it and it showed up in explorer
i started with the most important data first but there was a portion on the drive i could not access and when i tried it froze explorer up
tried it several times and could not get at that part it was about 2gb out of 200gb
 AGAIN I DO NOT RECOMEND THIS
just thought i would share it

[SavannahLion]

I have completely abandoned optical media for backup. I keep copies of my data in multiple places (original on each machine, duplicate on the server in the house, another duplicate on my work machine), but I think the most brilliant backup addition I did was to buy thumb drives. Stick important stuff on a thumb drive, and put it in my fireproof locking safe in the house. Small, relatively cheap, and getting bigger all the time.

Like what was mentioned elsewhere, I haven't seen any reliable data on using thumb drives for long term storage. (I had to assist someone on that research about three years ago. I'm sure there must be some research? bleh) There's been extensive studies on optical media dating all the way back to the hideous LD stuff. You see it all the time when someone bitches about bitrot on the LD games.

There have also been studies done on the long term viability of HDD. The current accepted method for "storing" HDD for archival purposes is to actually disconnect the drive and store the computer and the HDD in a low temp environment. Obviously no studies have been done on bad caps 

But Saint nailed it on the head. The key to storing digital records is duplication duplication duplication and to do it across as many different types of media as feasible.

[danny_galaga]

I'm not an expert, but RTS is saying it is probably physically damaged. I can't imagine ANY recover software is going to help...

[protokatie]

I would actually suggest optical (and this is the reason why I suggested it) was for yearly updated off-site backup. That DVD-R isnt going to die in a year.

My basic thoughts for optical: Make two optical backup copies (this is on top of having another copy on your NAS so total of 4 copies) and keep one at home and another offsite (safe deposit box). Every year, recopy all the data to be backed up, label them, and store them. This way you are refreshing the data each year. BTW this is only for the MOST important of your data, which should normally fit on a few DVD-r's at a low cost per year. This sort of backup is so your most precious memories have a backup somewhere (and the yearly update will keep it up to date) (add to that any new data that should be copied immediatly)

[SavannahLion]

I would actually suggest optical (and this is the reason why I suggested it) was for yearly updated off-site backup. That DVD-R isnt going to die in a year.

My basic thoughts for optical: Make two optical backup copies (this is on top of having another copy on your NAS so total of 4 copies) and keep one at home and another offsite (safe deposit box). Every year, recopy all the data to be backed up, label them, and store them. This way you are refreshing the data each year. BTW this is only for the MOST important of your data, which should normally fit on a few DVD-r's at a low cost per year. This sort of backup is so your most precious memories have a backup somewhere (and the yearly update will keep it up to date) (add to that any new data that should be copied immediatly)

When you generate backups, you have to consider a key human nature, laziness. I can guarantee that after three to five years of creating "yearly refreshes" of that data, a person is going to postpone the next refresh to next year. Then it'll be five years. Then you're going to work on that massive project which postpones it further and next thing you know, ten or fifteen or even twenty five years have passed.

You can't setup a archival program requires additional work and expect anyone to keep that up on a consistent basis. The best practice is to set up a "create it and leave it alone" kind of thing.

I always thought it would be an interesting project to build a disc juke box that does its own scheduled "maintainence" and backup of archived discs. For example, a database that earmarks DVDs for scheduled examination and data verification automatically generating new backups as necessary. But with the cost of HDDs and SSDs dropping faster than one could build such a juke, it's kind of a moot point.

[protokatie]

SL, I see your point but another one that enforces periodic refresh is not only that HDDs die over time (electrolytic caps for instance) but that all standards are replaced about every 10 years. Luckily we will have USB and ATA/IDE/EIDE/PATA/SATA for awhile, but you cannot count on it. Even if the periodic backup was only once every few years, it would still be valid. On top of that, as far as laziness goes; we all make our own beds.

[Xiaou2]

saint's note - nasty bit removed...

http://www.fat-ntfs.com/articles/data-loss-problem/physical-errors/What-causes-a-Hard-Disk-Clicking-Sounds-323.html

[Xiaou2]

CD Standards change far faster than any other standard.  They also are the Worst when
it comes to backwards compatibility.  On top of that, burnable discs are not made to last
more than a few years.  The coating on them usually flakes off between 3 to 6 yrs.

 If you burned them at 50x today... and your burner stops working...  The new 100x burner
may not ever read the 50x discs.

 To keep on top of Optical Media BURNERS is far more expensive than to keep up with
direct Hard Drive backups.  Also, HD backups are far easier and 500x faster.
They are also a LOT cheaper per GB than Optical Media.

 Data can also be recovered from most any HD problem.  However, a single scratch on
the coating side of a disc, and its a coaster.

[Hoopz]

saint's note - Nasty bits removed.

saint's note - Response nuked.

RTS,
Sorry about your drive and this thread.  I agree that you should leave the drive alone until you have the funds to properly retrieve the data.

[SavannahLion]

SL, I see your point but another one that enforces periodic refresh is not only that HDDs die over time (electrolytic caps for instance) but that all standards are replaced about every 10 years. Luckily we will have USB and ATA/IDE/EIDE/PATA/SATA for awhile, but you cannot count on it. Even if the periodic backup was only once every few years, it would still be valid. On top of that, as far as laziness goes; we all make our own beds.

I believe that's the reason why a PC is typically stored along with any HDDs in professional long term archivals. I know it would've been easier to have the equipment at hand instead of spending months trying to figure out and track down a multitude of antiquated media players on eBay and in museums.

The last research I dealt with prior to leaving my last job was cloud based storage. I don't necessarily agree with the concept for long term storage purposes (too many factors outside ones control and "live" data is always at risk for malicious corruption). However, that tech does neatly sidestep the issue of cap leak and bit rot. It's got enough support that a federal system is up and running designed to store federal and state archives.

[ark_ader]

saint's note - nasty bit removed...

http://www.fat-ntfs.com/articles/data-loss-problem/physical-errors/What-causes-a-Hard-Disk-Clicking-Sounds-323.html

You get that sound too if you are using a USB External drive (2.5) and USB 1.1.  you need adequate power for the drive to work, if it doesn't have it, the drive will click.  Thus my original post about opening the drive - meant opening the enclosure and testing the USB interface, by plugging the drive into a PATA or SATA cable.

[shmokes]

External hard drive eh?  They are capable of withstanding falls from 6-8 ft. . . .

. . .  Sounds like power is not getting to the drive (clicking sounds).

An external hard drive is almost always just an internal hard drive screwed into a rigid case with no kind of shock absorption.  What kind of twisted logic makes you think external hard drives are likely to withstand a 6-8ft drop?  Also, if no power is getting to the drive, what in god's name is making the clicking sound?  Squirrels?  Elves?

[Xiaou2]

 Ive experienced externals failing, however, to hear a click at all, you had to place your ear
on the drive itself.. and even then it was barely audible.  More like a light "tick".  I did however
hear a high pitched whine from even a distance away. 

 People who usually say they hear a Click, means they are probably hearing it Loudly and
from a distance.  ITs a very pronounced noise.

 I agree, that the drive could and should be tested outside of its external enclosure.
(but not opening the drive itself)

 I have the impression that it was already tested outside of the enclosure.

[ark_ader]

An external hard drive is almost always just an internal hard drive screwed into a rigid case with no kind of shock absorption.  What kind of twisted logic makes you think external hard drives are likely to withstand a 6-8ft drop?  Also, if no power is getting to the drive, what in god's name is making the clicking sound?  Squirrels?  Elves?

Google: Western Digital 60GB Passport 

I have two of these and I can toss them down the stairs and they still work.  Got a Lacie 80gb that is the same but the USB interface is dying.  I might attempt throwing that out the window and see if that works.   

The click is not demons or bats in the belfry.  It is from the drive motor.  Its real quiet.........

Is Shmokes having a bad day? 

[SavannahLion]

An external hard drive is almost always just an internal hard drive screwed into a rigid case with no kind of shock absorption.  What kind of twisted logic makes you think external hard drives are likely to withstand a 6-8ft drop?  Also, if no power is getting to the drive, what in god's name is making the clicking sound?  Squirrels?  Elves?

Google: Western Digital 60GB Passport 

I have two of these and I can toss them down the stairs and they still work.  Got a Lacie 80gb that is the same but the USB interface is dying.  I might attempt throwing that out the window and see if that works.   

60GB? 80GB? Are we comparing laptop drives with desktop drives? And regardless of its size, those external drives are exactly what shmokes describes them as, internals with a nice jacket.

[Xiaou2]

I have two of these and I can toss them down the stairs and they still work.

 You can "IF" they are not in the middle of a write operation.  An HD that is not active will Park
its heads so that they will not be able to get damaged due to vibrations.

The click is not demons or bats in the belfry.  It is from the drive motor. 

 A Click is from the swing arms that holds the read/write heads.  Mostly, when the heads have
Crashed into the surface and jammed up.  Such a click is Very loud.  You can hear it from inside
a closed computer tower at times.

 If its so quiet you can barely hear it even against your ear... then its not really a click.
More like a skip/tick... which is probably the heads starting move, then getting parked again.
 
 The motor itself, is just about completely silent.  An HD is only heard, when its reading and writing.
When its spinning idle, there is no real noise made, because HDs use very high tolerance
bearings... and operate brushlessly.

[Grasshopper]

Like what was mentioned elsewhere, I haven't seen any reliable data on using thumb drives for long term storage. (I had to assist someone on that research about three years ago. I'm sure there must be some research? bleh) There's been extensive studies on optical media dating all the way back to the hideous LD stuff. You see it all the time when someone bitches about bitrot on the LD games.

I looked into backup options a year or so ago (I posted a thread here about it) and noticed exactly the same thing. There is no information whatsoever out there about the suitability of flash drives for long term storage. If the manufacturers know the answer, they're not telling.

Flash drives are undoubtedly an incredibly convenient and durable method of storing data in the short term but there's a big question mark over whether they're suitable for long term storage. If I understand correctly, they use electrical charge to store their data. However, electrical charge (just like magnetic charge) tends to dissipate over time. And I'm guessing that the problem of charge leakage is likely to get worse as capacities increase.

As far as the durability of DVDs is concerned, I've seen estimates varying from 10 to 30 years for DVDs that have been properly stored. However, as the technology hasn't been around for that long, no one can really know for certain.

Personally, based on my own experiences, I'd say that the fragility of optical media has been somewhat overstated. The problem is that people tend to treat optical disks like crap and then wonder why they fail.

Most people I know think nothing of leaving disks in direct sunlight, and the disks I borrow from other people are invariably badly scratched.

I've also noticed that people often bend optical disks, sometimes quite severely. This is partly the fault of badly designed cases that won't release a disk without significant force being used. But still, people don't seem to care.

Anyway, as others have already pointed out, whatever storage method you choose to use, there's really no substitute for making regular copies of your backups.

[ark_ader]

Well all this talk about backup has made me rethink my own backup strategy, and since 500gb drives are under £35 its better than two weeks of DVD burning and cheaper too.

I wonder what strategies the normal every day user has, if any?

[SavannahLion]

As far as the durability of DVDs is concerned, I've seen estimates varying from 10 to 30 years for DVDs that have been properly stored. However, as the technology hasn't been around for that long, no one can really know for certain.

We can make educated guesses.

LDs have been around since 1978 and exploded onto the market in the mid 80's (remember Back to the Future?) I don't have statistics at hand, but delamination and bitrot is extremely common with these. I'm not sure when the problems started, but it's been about twenty-five years for most of these discs.

CDs were introduced to the market sometime in the 80's (IIRC) and really took off in the 90's. I feel manufacturing qualities really changed. I had CD's from the late 80's early 90's that felt more substantial than current CDs sold today. That's about twenty years.

Finally, most companies cite accelerated weathering tests (similar to tests applied to cell phones and auto paints to determine product life expectancies).

It's generally accepted that pressed discs last longer than burned discs by nature of the requirements of both discs.

So..... given the studies on the accelerated weathering tests and the history of previous, similar, technologies, 10-30 years seems to be right around the mark.

Of course that is barring any extreme conditions, fire, mold, etc.

[shmokes]

Whether from the motor or the arm, it needs power to move, and it needs to move to click.  The drive is getting power.

As for backup strategy, check out Microsoft's SyncToy.  This is a great utility for a basic, informal backup solution.  It allows you to keep two folders synced up, whether it's the My Docs folders on your laptop and desktop or the My Photos folder on your desktop and an external hard drive.  Changes made in either folder will automatically be reflected in the other folder in real time (or as soon as possible if the connection between folders is lost).

One thing I intend to do relatively soon, ie when I have a real job and can afford it, is start backing up to an online location for example with Amazon's S3 cloud storage offering.  I've never used it but Microsoft offers 25 GB of free cloud storage with Windows Live SkyDrive.  It requires a LiveID, but I'm pretty sure that by now everyone already has one.

As for photos, there's really no excuse to lose digital photos.  How many unlimited free services do you need before you get your photos backed up online?  Flickr, Picasa, Shutterfly, Photobucket . . . even Facebook.  Apps exist for all of these to automate the process so you can just mass upload your photos.  Get.  It.  Done.

[MonMotha]

FWIW, quality SLC flash devices (bare chips) generally specify 50-100 years retention at 25C going down to 10-25 years retention at 85C.  These are the guaranteed parameters.  You'll usually do much better.  MLC, the type used in most inexpensive, high-capacity flash storage devices, is usually only good for 10-25 years at 25C and 5-10 years at 85C.  It depends on the quality of the device.  Your high end Micron and Samsung devices will generally last longer than the cheapo no-name Chinese stuff.

Quality CD media stored in a controlled environment (dark, room temp, low but non-zero humidiity) should last 10 years or more.  You can buy special "archival" discs that will actually give you specs.  The Verbatim AZO (dark blue) are usually decent.  Quality DVD media is comparable.  All these specs assume you have a quality burner, which most people don't.  For your cheapo $35 CD/DVD burner, quarter the lifetimes.

As an anecdotal data point, I do have CDs burned more than 10 years ago that I can still read just fine, and they've just been stored in a standard jewel case in a dry basement, but they're on good media and from the era when you couldn't get crappy CD burners because they didn't exist.  I also have flash devices from the very early days of flash memory (mid 90s) that still work fine.  Again, they've just been stored indoors in a standard household environment.

You can pay more to get devices that have specified lifetimes and are tolerant of elevated temps or other environmental abnormalities.  Just expect to do just that: pay more.

In my opinion, the best backup system for your money right now is 2 external hard drives.  You do hard-link based incrementals using rsync (which are actually full snapshots in and of themselves, just only updating the stuff that's changed - this lets you go back in time while still making recovery easy) to one disk for a period (days or weeks) then swap in the other drive to do the same.  The drive that's not being used lives off-site.  Every year or two, you copy all the files to a new drive.  This builds in capacity increases and ensures that you have a "fresh" (from a physical point of view) copy of the data.  It will also let you spot any corrupted data (so make sure you check checksums when you do that) so you can attempt to recover that backup from your hopefully still good primary hard disk.  Total cost is a couple external hard drives every couple years, or about $200-300/yr.  Not bad.

[Xiaou2]

Whether from the motor or the arm, it needs power to move, and it needs to move to click.  The drive is getting power.

 As a tech, Ive ran into a few external drives in which a component failed inside the device,
causing improper voltages going to the drive... resulting in it being useless.  Spins up, but
wont read/write/detect.

 You can test this easily with a multimeter.  First at the power supply... and if that passes,
then at the rear pins of the drives power connector.

FWIW, quality SLC flash devices (bare chips) generally specify 50-100 years retention at 25C going down to 10-25 years retention at 85C.  These are the guaranteed parameters.

 There is no such Guarantee.  Partly due to the fact that the tech has Not been in place for 25yrs.
Also, Ive read reports of people reading and writing constantly on purpose, killed flash drives in
less than a day of time.  Of course, there isnt much reason to argue about it - as flash drives of
any real size will cost a Lot more than other means.

 

Quality CD media stored in a controlled environment (dark, room temp, low but non-zero humidiity) should last 10 years or more.

 I hadnt opened the CDRs I had bought over 7yrs ago, in room temperate, in a room with
blocked off windows (no sunlight, only indoor lighting).   When I finally did open them,
the coating flake off on my fingers from the lightest touch.

 The Discs they use to make your CDs and DVDs for commercial sales... and not the same as
the ones they sell at the stores for PC burners.  A commercial DVD probably will last a lifetime -
unless you scratch it badly.  Not the same for consumer burnable discs.

All these specs assume you have a quality burner, which most people don't.  For your cheapo $35 CD/DVD burner, quarter the lifetimes.

Heh.  Thats Scammer Info  IMOP.  Prove it with any form of Logic.

[Hoopz]

Hmmm....

Monmotha v. X.  I'm putting all my money on Monmotha being right and being able to prove it unlike our resident know-it-all.  X never proves it.  He just blusters and tells people if they don't like it not to read it and that the masses here want to keep him quiet.  Seriously, can you document what you say AND have it be from a credible source?

Monmotha is a tremendous help to others here with (seemingly) obscure technical data that's always on point.  X just likes to talk about how he worked in an arcade so he knows everything about it. Funny how being a change boy makes a genius even smarter. Now he's a hard drive tech too!

[Xiaou2]

 I was a change guy for only a few weeks.  Then I became the store
manager for 3 years.  Which involves well more than counting money...
and included teaching myself how to fix 42 arcade & mechanical redemption
games.

 As for SSD last-ability, and almost all last-ability that is listed by MFGs,
are general specs that are guesstimates based on percentage of usage,
ideal conditions, and short range tests.  Rarely ever long range test,
(how could a company test a drive for 25yrs before its sale?!), or pure
factual results.

Monmotha is a tremendous help to others here

 In case you didnt notice, I gave plenty of helpful advice throughout my
many years here..  Including in this very Thread!, for which me and
your buddy agree on HD backup as the better solution.

 Instead of throwing your hate for me around... maybe you should
concern yourself with much more grown up things in life.  You will live
longer, and happier that way.

[Hoopz]

   X, I don't hate you.  You aren't worth time or effort like that.  I just don't think you're the god that you claim to be. 

My suggestion is both physical and cloud storage.  With Google docs taking any type of file now, dropbox being a great asset, and other sites that can take multiple gigs of pictures with no fees, there's no reason not to use some of these services.   Granted, no for sensitive information, but for general off-site storage, it's a free, relatively safe and secure path to choose.

[shmokes]

All these specs assume you have a quality burner, which most people don't.  For your cheapo $35 CD/DVD burner, quarter the lifetimes.

Heh.  Thats Scammer Info  IMOP.  Prove it with any form of Logic.

It's not really a fair challenge cos you are limiting him to logic and not evidence, when the obvious method of proof here would be citation rather than argument.   

[ark_ader]

All these specs assume you have a quality burner, which most people don't.  For your cheapo $35 CD/DVD burner, quarter the lifetimes.

Heh.  Thats Scammer Info  IMOP.  Prove it with any form of Logic.

It's not really a fair challenge cos you are limiting him to logic and not evidence, when the obvious method of proof here would be citation rather than argument.   

+1 

[MonMotha]

You asked for an explanation, so here's one, albeit a bit technical at times.

As a data point on flash memory retention, Google a datasheet for the MT29F1G08.  This is a 1Gb (256MB) SLC NAND device from Micron that is reasonably mature (it's been out for several years).  Micron recently placed all their NAND flash datasheets under NDA for some reason I can't comprehend, but this one's old enough that copies not covered by NDA are still floating around.  The datasheet says 10 years over full temp range, which goes to 85C on the extended temp parts or 70C on the commercial temp parts.  I've not known Micron to lie about things like this.  They've been making memory devices (volatile and non-volatile) pretty much as long as such devices have existed.  Can they 100% guarantee those specs?  No; the parts haven't existed long enough.  However, they do accelerated aging tests to arrive at those numbers, and I'd say they have the data to back those aging tests up.  They spec these things out for military/aerospace use too, and those guys run their own independent tests to validate.

Do be aware that the higher density devices, even in SLC, sometimes have slightly lower reliability due to the smaller geometries.  Generally, they make up for it by applying lots of ECC.  MLC devices also can have lower reliability since they have to discriminate between 4 or 8 levels per cell rather than just 2.  Again, these larger devices tend to have lots of ECC to mitigate this.

As another data point, the microcontroller I'm currently using from Atmel specifies 15 years flash data retention across operating temp range (-40 to +85C).  Atmel's also been at this a while.  Reliability of microcontrollers is generally taken to be a pretty freaking big deal, much more so than in the consumer market segment.  These things get used in applications where failure could result in injury or death.  Obviously, the designer of the device should (and generally does) take precautions to make the device "fail safe", but even then, failure is often costly.

In talking with my reps from various companies that do non-volatile memories, there is apparently something of an exponential relationship between temperature and retention.  A device that retains 10 years at 85C may only last a few hours at 150-200C.  There are often characterization charts that exceed the specified operating ranges kicking around inside semiconductor companies for many devices (yes, I've worked for one, and yes, I've seen such charts for various devices, though not flash memories).  If you ask nicely (and often sign an NDA), you can often get access to those charts with the caveat that the data is not "guaranteed".  Semiconductor companies take electrical performance characteristic guarantees very seriously.  Guaranteed characteristics are generally valid all at once, so worst case across the board.  I've never had a semiconductor device fail to meet basic electrical performance characteristics in a manner that wasn't disclosed as an errata.  In general, devices VASTLY exceed their specified performance.  I once ran a micro specified for 6MHz@5V at 20MHz@3V, though it couldn't go any higher, and I was operating it at room temp only.  Generally, reliability problems are design related, but I have also found functional errata (the device does not behave entirely as described) before.

Most of the industry reports and technical literature I read seems to agree with these numbers.  There are also flash device integrators out there who make SD and CF cards that will sell you devices with a warranty.  These guys will say, for example, "5 years minimum retention at 85C or your money back plus losses incurred up to $X".  Expect to pay a substantial premium for such devices just because of the insurance they have to carry, let alone to more rigorous design review and testing that goes into such devices, but such integrators do exist.  The fact that people will put their money where their mouth is again says to me that the numbers are probably at least reasonable.  Flash memory first started showing up in the mid to late 90s, and most of it is still happily chugging away, so a number of "10 years" is at least pretty well validated at this point without any accelerated testing.

As to optical media, my experiences are much more anecdotal.  I do have media burned in the late 90s that still seems to read fine on various readers.  You can get special "archival" media.  Memorex has a "gold" line that they claim 300 years in highly controlled storage.  I'd expect (though without any data to back it up) that a typical controlled home environment could get at least within an order of magnitude of that, so 30 years.  I've got friends who have pulled out CDs from the early days of CD burners who can also still read them just fine.  That's generally 10+ years.  Your media lifetime will also be affected by data mode: CD-ROM or CD-DA.  CD-DA has far less error correction on it due to the assumption that an incorrect bit here or there does not render the disc totally unusable (it just creates an audio artifact), while a single bad bit CAN render a CD-ROM useless.

Now, as to writer quality, that really is just based upon my experiences, I'll admit.  The "quarter the lifetime" suggestion was an attempt to be conservative in the resulting media lifetimes.  After all, if you expect it to last 10 years but it lasts 40, you probably won't complain.  If you expect 40 but only get 10, you may be in trouble.  However, my reasoning on doing this is based upon actual experience.  My discs burned on quality burners all still seem to work, but I do have discs burned with cheap writers that are unreadable after 3-6 years of reasonable storage (though I'll admit that they are in general also on crappy media).  I've also noticed that newer, cheaper writers are more likely to create discs which have trouble being read by other devices (especially standalone CD players) even when writing at slower speeds.  My old Plexwriter 16/10/40A seems to be able to write the cheapest media I throw at it at 16x and produce something that reads on pretty much everything I chuck that disc into (from standalone players to other burners to CD-ROM drives that have seen 10 years of arcade duty) with no issues.  Again, anecdotal I know, but it at least suggests that there is a difference.   I've read some articles to this effect as well, but nothing scholarly (though I'm guessing something is available).

[Xiaou2]

I just don't think you're the god that you claim to be. 

 Heh.  I dont think Ive ever claimed myself as any form of Deity lol.  But man, Im somewhat
flattered that you credit me so high, to be that jealous.  Im just some guy whos average at some
things, and have been dedicated enough to get very exceptional in other areas.. much the same as
many others out there who you may be jealous of.

 I do have gifts in some areas, such as design/art, spatial awareness, heightened awareness
thru ability to make associations, and mechanical ability... but for all my gifts, I have a slew of
problems which darken my days.  Poor memory retention, a hair of OCD (which causes mild depression at times), poor level of ability to fully understand complex information... so will never be
a programmer, do complex maths, etc. 

 Health isnt the greatest.  Tired too much. (poor ability to process food to energy?)  Horribly crooked teeth I cant afford to fix. Larger pores which cause me to often get 2 whiskers growing
out of one pore... ugg

 Basically, Im Human, like everyone else.

It's not really a fair challenge cos you are limiting him to logic and not evidence, when the obvious method of proof here would be citation rather than argument.

 Heh.  I guess so    But...

 I seriously doubt anyone will be able to dig up actual proof that one burners 'Burned Cd' will last
less years than a higher priced burner, which is exactly why I asked for some sort of Logical
reason for that.

 There IS physical proof that the MEDIA you burn on your plays the biggest role in data lastability.
If you buy a DVD movie at the store, that DVD will have been burned with a different type of
media than what they sell at the stores for your PC burner.  Of course, the burners they used to
burn those discs are different too..  But if you put the PC media discs into them, them will
fail just as fast as a cheap home burner.

 A burner is simply a laser that burns pits into a surface.  The pits are no different from
a pro burner, than a home burner,  ...else they would not be able to be read by the same types
of players.

 A pit does not change over time based on the quality of Laser that burned it. The only thing
that can change, is if the surface the data is on, is scratched/flaked off. (which happens on
cheaper media, or simply media meant for home burners)

 So, If Im missing something here... then correct me... but logically, that theory that a
$200 burners burned cds will last longer than a $50  burners cds... is IMOP, seriously flawed.

[MonMotha]

The issue is that some burners may create a better transition in the media than others (the media quality obviously has a lot to do with this, as well).  The higher quality transition has more margin for error than does the lower quality one meaning that it is more likely to be able to be recovered properly given the same amount of media degradation.  This is generally true of any storage medium (optical, magnetic, electrical e.g. flash, ink on paper, chiseled wood, etc.).  The better your SNR to start, the more margin you have for recovery.  Now, whether or not cheap burners have less margin to begin with than more expensive/higher quality models may be questionable, but my experience has been that there is, indeed, some sort of phenomenon along these lines going on.

Imagine my goal is to burn a hole in a piece of quad-rule graph paper using a laser.  If a square has a hole in it, that's a 1.  If it has no hole, that's a zero.  I need a laser powerful enough to actually burn a hole in it, but not so powerful that it just sets the whole piece of paper on fire.  Furthermore, I need to control the size of the hole.  Over time, the holes will either get bigger or they may get plugged with debris.  I need a hole that's big enough to be reliably seen but no so big that it causes adjacent squares to also look like they have a hole burned in them even if they were not intended to.  You can see how my laser power would have an effect here.  Same goes for CD media, but the squares are way smaller, and you're just causing a change in reflective/diffractive properties by deforming or ablating a dye layer rather than burning a hole straight through the thing.

[shmokes]

I don't know much about this, but I can at least hypothesize about scenarios in which you would be wrong.

First, it's important to understand that commercial DVDs and CDs (i.e. movies and music) are NOT burned.  They are pressed.  An image of the data is tooled and physically stamped into the polycarbonate to create the permanent pits that make up the little 1s and 0s.  Since pressing DVDs at home is not feasible they needed another method, hence the burners.  But the burners don't burn pits right into the plastic to mimic what's going on with the commercial DVDs and CDs.  The laser inside a burner isn't nearly that powerful.  Instead, DVD-Rs and CD-Rs have a layer of dye in them that reacts to the laser; the laser turns the dye opaque if I'm not mistaken.  This dye isn't nearly so robust as a physical pit pressed into the plastic, which is what makes a commercial disc so much sturdier and more reliable than one burned at home.

So, with that in mind I would suggest (mind you without any basis) that perhaps the dye layer reacts differently to different intensities of laser.  For example, maybe the spec calls for the dye to be heated to 200 degrees in order to trigger the effect that turns the dye opaque.  This is a spec that produces highly consistent and relatively permanent results.  But maybe using a cheaper to manufacture laser that only heats the dye to 180 degrees turns out discs that work nearly as often, but the dye is more likely to revert back to its previous state, or simply fade into something unreadable in a shorter period of time.

I have no idea if this is the case, but it or something like it seems at least plausible.  In other words, without more I see little reason to dismiss the possibility out of hand.



edit: heh . . . I hadn't read MonMotha's post when I posted, but it's pretty close to what I was guessing at

[CheffoJeffo]

Basically, Im Human, just not like everyone else.

Fixt.

[CheffoJeffo]

Xiaou, the last time you asked for citations in a storage-related discussion and somebody gave you references, you dismissed them, insisting that there was a conspiracy between Google and hard drive manufacturers ... or something equally entertaining.

And  to MonMotha for good explanations of his position.

[protokatie]

...And to actually add NEW ideas to this thread;

If the data you want to protect is of a photographic nature and is important enough, there is a new technology out that uses processed wood scrap to make a medium, then it is impregnated with salts that specifically absorb or re-emit certain frequencies of light. You can only store one image per medium, but it has been proven to last atleast 130 years.

Just a thought. Also, it is highly portable, no OS is required and electricity is only required if it is dark out and there is no fire nearby.

[saint]

Just don't get it wet...

...And to actually add NEW ideas to this thread;

If the data you want to protect is of a photographic nature and is important enough, there is a new technology out that uses processed wood scrap to make a medium, then it is impregnated with salts that specifically absorb or re-emit certain frequencies of light. You can only store one image per medium, but it has been proven to last atleast 130 years.

Just a thought. Also, it is highly portable, no OS is required and electricity is only required if it is dark out and there is no fire nearby.

[protokatie]

Just don't get it wet...

Nor get it too close to the fire, but how is this any different from modern media? Submerse an Hdd, and it is likely toast. A CD-r will fair better with water, but heat will kill it. This just goes back to the idea of storing data on a multitude of media (recommended) as opposed to a view of only one medium (not recommended). Besides, paper is really useful, and for many applications, it is durable and long lived. I remember them(c) saying we would be a paper-less society by 2015. Doesn't look like that will happen. The stuff is too useful.

[saint]

You put entirely too much into my reply

[SavannahLion]

MonMotha, I'm a little confused at your post there. At first you write about individual components, then it seems like you're talking about integrated devices.

From an archival standpoint, the fact an SLC or any other component lasts 10 years across a temperature range matters jack when another component on the device lasts only say.... three to five. You're going to end up going with the smaller value and paying out the ass to recover if that ever happens.

(I would kill to find out where my 16/10/40A is. I'm stuck using this Pioneer DVR-116D in the meantime)

[MonMotha]

MonMotha, I'm a little confused at your post there. At first you write about individual components, then it seems like you're talking about integrated devices.

From an archival standpoint, the fact an SLC or any other component lasts 10 years across a temperature range matters jack when another component on the device lasts only say.... three to five. You're going to end up going with the smaller value and paying out the ass to recover if that ever happens.

(I would kill to find out where my 16/10/40A is. I'm stuck using this Pioneer DVR-116D in the meantime)

Data on discrete components is much, much easier to obtain since most device manufacturers just plain don't give it out (or sometimes even know it!) or will only give a MTTF/MTBF which isn't a particularly useful number.  Device integrators don't seem to take "product specifications" as seriously as semiconductor makers.  You have to pay a LOT more than normal consumer products to get warranties that cover data loss due to product failure.  That's just the way the industry has gone.  However, there are some integrators that do publish specs and who are known for consistently meeting their specs (and often exceeding them) while others can frequently fail to deliver.  Caveat emptor.  You do often get what you pay for.

One other thing to consider is that there are a LOT of flash media integrators but not many flash semiconductor makers (at this point, there's only 2 or maybe 3).  Therefore it's easier to speak generally about the bare semiconductors than the integrated devices.

Integrated device lifetimes DO vary, and that's what you care about in the end.  The cheap crap sometimes barely makes it a year.  Mostly, this is due to terrible firmware (bad block replacement or lack thereof, bad wear leveling or lack thereof, etc.) but sometimes also cheap components or improper design (bad power is a common issue).  If you intend on archiving on flash media, make sure you get something that doesn't suck.  Sandisk has some products that they give real specs on (5 years minimum lifetime non-operating from -55C to +95C or 43,800 power-on hours is a spec I just pulled for one of their SSD products, but at the prices they sell them normally, there is no warranty to that effect), but generally only on their OEM products.  Sandisk does have a reputation for meeting their specs, at least.  I also note that their standard SD cards include a 5 year warranty.  While they may not be on the hook for data loss, they would have to deal with the cost of replacement and all the logistics of failures, so I'd guess they're pretty confident that the devices will last at least that long.

When people write up warranties and such, they tend to be rather conservative.  It's expensive to pay out on a warranty (it generally costs a lot more than the cost of the product).

As another anecdotal data point, I do have a few CF cards from ~2003-2004 that still seem to read fine, but it's been a couple years since I tried them.  I did have one really cheap card that died about a year after I got it, but it wasn't due to flash retention issues.  It just completely stopped working.  A failure nonetheless, but again, it was a real cheapie.  I think all my Sandisk and Kingston branded media still works.

As an unusually topical example, there are lots of arcade games floating around that store their data on flash memory.  In many cases, a standard PCMCIA or CF card is used.  Compared to everything else that goes wrong in these things, the flash devices seem to be very, very reliable.  These things sit around for years powered up and operating every day.  Flash memory is EVERYWHERE.  It's been used to store firmware on many production devices for the past 10-15 years.  On the whole, it sure seems pretty reliable.

My experience is mostly with discrete devices as that's what I'm normally specifying when doing designs.  Retention and temp range is definitely something I look at.  I have been known to request additional characterization data from vendors, and they're generally happy to provide.  If you do plan on archiving using flash media (and I can't say I wholly recommend it), you should get a device from a reputable integrator, and, of course, you should always have more than one copy.  Overall device failures tend to be sudden and catastrophic due to some outside influence or design flaw, while bitrot due to floating gate charge degradation on flash happens over time at a fairly steady pace (you just don't notice it because, as long as the sense amp can still figure things out, a nice, clean 1 or 0 comes out).  Multiple copies guards especially well against the former, while it doesn't really help much with the latter.  Of course, multiple copies on different media types is even better.

Nothing is immune to failure.  Unexpected stuff happens.  That's why you keep multiple copies.  Heck, that's why you're keeping backups in the first place.  If you've got reasonable assurance that something will last 5-10 years with the potential for it to last several dozen more, I'd say that's a pretty good medium term storage solution.

[Grasshopper]

One thing I intend to do relatively soon, ie when I have a real job and can afford it, is start backing up to an online location for example with Amazon's S3 cloud storage offering.  I've never used it but Microsoft offers 25 GB of free cloud storage with Windows Live SkyDrive.  It requires a LiveID, but I'm pretty sure that by now everyone already has one.

As for photos, there's really no excuse to lose digital photos.  How many unlimited free services do you need before you get your photos backed up online?  Flickr, Picasa, Shutterfly, Photobucket . . . even Facebook.  Apps exist for all of these to automate the process so you can just mass upload your photos.  Get.  It.  Done.

Personally, I think the whole cloud computing idea has been a bit overhyped. It requires a degree of trust in large faceless corporations that I simply don't have.

Online backup is fine as long as it's being done in addition to local backup, and the data being backed up is either of a non-confidential nature, or strongly encrypted. But if those conditions are not met then I wouldn't touch it with a bargepole.

The free services are the worst. They can (and sometimes do) pull the plug at any time without warning and you have absolutely no redress. Just look at Geocities for example. Even the paid for services give a false sense of security. Sure, you can sue the company if they lose your data but if they go bust you're still screwed. And even if you can sue them that doesn't actually get your data back. This might sound paranoid but events of the past year have shown us that even large household names can go bust overnight without warning.

There's also the issue of data security. Even if the company is well known, there's so much outsourcing and outshoring going on these days that you can never be really sure who you're dealing with.

[shmokes]

But as far as free services are concerned I'm mainly talking about backing up photos.  And by backing up, I mean you have a local copy, and your backup is uploaded to your Flickr or Picassa account.  Sure those companies could go belly up (Picassa is Google ), but the chances that the company happens to go belly up at exactly the same time that your house burns down or you accidentally drop your computer in a tub of water are not very high.  But even for other data, for example backing up to Microsoft's SkyDrive . . . there are definitely concerns and caveats, but no matter what you have to work within your means.  There is ALWAYS a better backup option.  ALWAYS.  ALWAYS.  ALWAYS.  But the better solution may not be feasible, whether because of finances, technical capabilities or motivation/discipline.  You have to ask yourself, what can I do, and what am I willing to do.  Then see what options correspond with the answers to those questions.  For some people the free services are very likely the best/only realistic option.