Let's talk about data backup options! Share your favorite ideas.

[smalltownguy]

I'm realizing that after 5 years in the IT industry supporting client solutions for data backup, mirroring, raid arrays and the like, I'm realizing that my own personal data backup situation is pretty terrible at best. I've already experienced 2 lost data situations, resulting in time consuming software based methods to recover all important documents and pictures. They say the plumber's toilet is always broken -- so true, so true.

That said, what is everyone using for their own personal backup options? Has anyone found any effective methods that are low cost (free?) Don't get me wrong, I'm willing to pay for a system that provides value, but I'm also wise enough to know that the community at large must know about SOMETHING that works well for an inexpensive investment.

I've tried several online methods that install services on your machines like carbonite, jungledisk, etc. Each has it's pluses and minuses.

What do you guys suggest?

[drventure]

At one point, I was using an Iomega SAN disk array (a plain 2 drive mirrored system was pretty inexpensive), but it got fried in the lightning hit.

Fortunately the drives (well one of them) was still good, so I was able to recover it's content completely.

Iomega units can be pretty pricey for the bigger systems (the last was a 1tb system, mirrored so I got 500gb storage).

So this time, I just built a machine, and put 2 2tb drives in it and mirrored them. A little cheaper, plus I can run other stuff on the machine.

I use FileBack PC for backup purposes. I like it because you can setup triggers such that it'll start a backup as soon as you plug a USB drive in.

So, then I have third 2tb usb drive that I regularly just plug in. Fileback kicks off and backs up everything to it, complete with several previous versions of the files. That drive goes into a firesafe when it's not plugged up.

And in between those backups, I have fileback nightly reach into each of my machines on the network and copy important files up to a "nearline" backup on the NAS server.

It can take some time to set up, but I've never lost any data.

I've been pretty intrigued about the online backup services, but the monthly cost per gig makes them a bit out of my reach.

[Howard_Casto]

I wouldn't worry about it.  Simply put us IT people don't need to backup our data as much because we know what we are doing.  If our computers start acting up we attend to it before they get to the breaking point.  If they DO break it's very unlikely that the harddrive itself will be completely unusable and we are competant enough to pull important data off the HDD of a dead computer. 

Ghost and similar software isn't a good option because unlike in a business situation, generally when we need to recover data it's because the pc has failed and the data will be put on a new computer with new specs.  I mean yeah you technically can put the image on a different pc, but then it's a pain in the butt getting windows back in shape if the specs are too different. 

Online backup is equally useless.  I don't count it as backup if you have less than a gig of data to backup and if you do it'll take FOREVER to upload all of that to an online account.  I've got like 100+ gigs of unique data so online backup is worthless. 

So basically you've got two viable and cheap options that'll actually work. 


1.  Seperate data hdd.  I"m not saying get a dedicated backup drive, I'm saying put all your data on a seperate hdd.  You can setup all of window's folders to re-direct to a seperate hdd as well.  Then when you upgrade your pc or if it fails all you have to do is re-direct your windows folders after installation and you don't have to do anything.  Now you could use a dedicated backup hdd with an auto backup system on top of this, but again, restoration after something fails can be time consuming.

2.  Good old dvd backups.  I generally do this about once a year or so, putting more recent data on flash drives.  It takes a long time, it's a pain in the butt, but it's one of the few cheap, reliable methods out there.  You keep the directory structure in tact on the disc so it's a simple matter of dragging the whole mess back on a hdd if something fails.  This is also the best method for the average joe because most people aren't freaks with 80+ gigs of ACTUAL DATA to backup. 

[patrickl]

What's wrong with mirrored hard disk's though? Obviously that's not enough (against virusses, accidental deletes or fire), but I personally think adding an extra HD is an easy way of dealing with disk failures.

[Blanka]

2.  Good old dvd backups.

There is nothing good at DVD backups. My old ones (2 years and older) have lot of read errors already. DVD's are very sensible to disc rot.

Today I do everything with the biggest economic HD's available. I have around 800Gb of data, so at the moment I have 3 separate 1.5 TB external drives (cases with esata, USB and FW, with room for 1 SATA 3'5 inch drive), in 2 locations, and I sync the content every week with Synkron. RAID is a waste of money, as it needs way more disks than separate drives, and the discs are unreadable when mount separate in a case. When the 1.5 Gb limit is reached, 3Gb will be cheap and widely available and I migrate the discs to a higher capacity. 10 years ago I had 20 seagate removable HD's, 5 years ago I had everything on 10 DVD's, yesteryear I needed 2-3 hard drives for everything, and now 1 disk is enough. So number of disks needed went down to 1!

[jimmy2x2x]

just curious about the amount of data you guys are talking about.

Is the bulk of the data user created content?

I suspect it isn't, a newsgroup subscription and membership to a couple of decent tracker sites would recover most of that data in a shorter time and at a lower cost, with the side effect of newer collections in some cases.

I agree with separate data drives and a common file structure on those drives, it makes them interchangeable between systems, easier to reinstall the operating system if you need to.

My backups are consist of personal data (documents, photos, passwords etc) which are relatively small, backed up on site and online.  Everything else can be reconstructed easily, either by reacquisition or duplicating from another system.

[drventure]

That'd work unless you:

1) have a ton of movies or mp3s, in my many of which were ripped from Vinyl that you can't buy in digital form
2) have a ton of pictures or home vids of the kids and such that you couldn't reacquire in any event.
3) have a ton of other stuff, things like Installations, Dev stuff (I'm a programmer), and whatnot that would be difficult to "reaquire"
4) have a spent a bunch of time cleaning up image files, rom sets, etc. Sure, I could probably re-acquire all the files later, but when you're recovering from a nasty disaster like a downed machine, the last thing I want to do is ALSO redo a ton of work I did cleaning everything up in the first place.


USB drives are so cheap, it will often make more sense to pick up 2, backup regularly and just rotate them offsite (say to my parent's house, or a safe deposit box) on a monthly basis or so.

Even as an IT pro, systems can go boom to the point where they aren't recoverable. I got lucky with the lightning strike recently in that, in my NAS, which had two mirrored drives, one drive survived intact and I could recover it. And it had the lions share of my recent changed data on it. A backup disk had all the older stuff.

[Silas (son of Silas)]

I have about 4tb of music and movies sitting on a logical volume (so I can expand it when it starts to fill by adding more disks) and a second identical 4tb logical volume that I back up to.

As my media is already compressed to hell and back, there's no point trying to compress it further when backing it up so I use LuckyBackup (front end for rsync) that runs every day and keeps my live LV and backup LV sync'd.

The way I look at it, the amount I save in not having any TV service in my home (and therefore not having to pay for a TV license, cable or satellite fee) easily covers the cost of my stonkingly huge amounts of storage in my media server.

[Howard_Casto]

There is nothing good at DVD backups. My old ones (2 years and older) have lot of read errors already. DVD's are very sensible to disc rot.

Good lord, I thought that urban legend died 20 years ago.   

Agreed.  Dvds get read errors due to scratching and/or bad burns.  By the time the dvd wears out the data will be too old for you to care.

[Howard_Casto]

What's wrong with mirrored hard disk's though? Obviously that's not enough (against virusses, accidental deletes or fire), but I personally think adding an extra HD is an easy way of dealing with disk failures.

Absolutely nothing IF by mirroring you mean the drive simply copies what's on the parent and that mirrored drive is perfectly readable without the mirroring software.  But if software is required then you've made a brick basically because all manners of badness can happen if somethign goes wrong with windows, the software becomes outdated ect....  Also some of the mirroring stuff is too DUMB and they do a straight binary copy.  If the parent drive is good then all is well, but if it has physical errors which result in a funky write, said funky write can be copied over to the mirror drive and if it's something crucial like the file table then you've just partially corrupted your backup data. 

They are definately a good option, but for me at least, more trouble and expense then they are worth.  They don't hurt anything though, so if you have spare hdds around put em in an enclosure and have at it!

[Howard_Casto]

just curious about the amount of data you guys are talking about.

Personally I've got about 40 gigs of music and I dont' believe in stupid e-commerce vaporware like itunes so my mp3s are physically ripped from cds.  It's technically backed up in that regard, but man... it'd take forever to re-rip it all.

I also write software (a lot for this hobby) so I've got about 20 gigs of irrereplacable source code.  Also I do a lot of artwork and all of the family photos are on my pc.  So there's gigs of that stuff as well.

Then of course mame chds and daphne mpegs.  Those are just huge!  I have em backed up on dvd of course but it'd be a pain to transfer them back on the pc so I like to keep those safe. 

[patrickl]

What's wrong with mirrored hard disk's though? Obviously that's not enough (against virusses, accidental deletes or fire), but I personally think adding an extra HD is an easy way of dealing with disk failures.

Absolutely nothing IF by mirroring you mean the drive simply copies what's on the parent and that mirrored drive is perfectly readable without the mirroring software.  But if software is required then you've made a brick basically because all manners of badness can happen if somethign goes wrong with windows, the software becomes outdated ect....  Also some of the mirroring stuff is too DUMB and they do a straight binary copy.  If the parent drive is good then all is well, but if it has physical errors which result in a funky write, said funky write can be copied over to the mirror drive and if it's something crucial like the file table then you've just partially corrupted your backup data. 

Never had this happen, but indeed I always wondered how they detect bad blocks on a disk and which disk is actually bad.

They are definately a good option, but for me at least, more trouble and expense then they are worth.  They don't hurt anything though, so if you have spare hdds around put em in an enclosure and have at it!

I was actually asking because I though someone said that mirrored disks are a bad idea, but I guess I just misread.

Personally I have found they save a lot of time for practically no cost. When a disk goes bad I just take it out and replace it. It also works for off site backups. I simply pull one disk out and replace it with the spare.

I use offline backups too. My webservers synchronize to my office file server overnight. Only problem with that is if I actually need to put it back. To do that over an internet connection would take forever. Primary backup is an extra disk in the webserver though.

[AtomSmasher]

Does anyone have any opinions about Carbonite?  My mom has a home business and has to keep all of her stuff backed up.  I used to do it manually every couple of months, but now everything gets backed up on a second harddrive automatically, although I still back it up on an external hd every 6 months or so.

Frankly I'd prefer to have something simple and secure so I no longer have to worry about it, which is why I'm now looking at online backup such as Carbonite.

[Blanka]

For me online is too slow. If I've done a shoot with 16Gb of photo's, then it would take tons of time to upload that. With 2 fire-wire drives, it is 5 minutes work on Friday.

[smalltownguy]

Does anyone have any opinions about Carbonite?  My mom has a home business and has to keep all of her stuff backed up.  I used to do it manually every couple of months, but now everything gets backed up on a second harddrive automatically, although I still back it up on an external hd every 6 months or so.

Frankly I'd prefer to have something simple and secure so I no longer have to worry about it, which is why I'm now looking at online backup such as Carbonite.

Carbonite has saved ---my bottom--- at work a few times. There have been a few crucial spreadsheets that I had to recover using it. My boss used it to migrate to a new laptop when the old one was destroyed. She had about 40 gigs of user content, and it took 48 hours to download it all. Sure it took a long time, but she's and average joe user, and the process was very easy for her to use. For me, that solution is too long -- and at $50 PER COMPUTER a year, I could have a couple of decent sized hard drives after 4 years for that money.

Me, I regularly email myself files that I need to keep tabs on regularly. Sure it's not a very secure way to backup files, but it does create a multiple redundancy that keeps me satisfied. I now have a copy of the file on my laptop, in my email, and also saved on the 3rd party email provider's servers.

I'd really like to find some sort of program that will allow you to plug in an external hard drive and it will automatically 'sniff' pre-selected folders you choose and backup updated content. Anything that has not changed will be left alone, i.e. it doesn't need to be re-written. Then I'll just get in the habit of plugging in the external HD about every few days or so.

Anyone find a method like this that works?

 

[Hoopz]

Instead of emailing myself documents, I've started uploading them to Google Docs.  My office uses the enterprise version which allows for greater storage capacity, but even a personal account comes with a gig.  That's small cookies compared to the sizes that you guys are talking about, but I've found it to be great for smaller files.  Heck, even opening a file with Google docs will allow you to open it again at a later time.

[patrickl]

I'd really like to find some sort of program that will allow you to plug in an external hard drive and it will automatically 'sniff' pre-selected folders you choose and backup updated content. Anything that has not changed will be left alone, i.e. it doesn't need to be re-written. Then I'll just get in the habit of plugging in the external HD about every few days or so.

Anyone find a method like this that works?

I use rdiff-backup and rsync. Those synchronize two file systems and even keeps extra backups of changed files (if you want). That way you even keep a history of the file.

It even works over an internet connection (at least from my webservers it does)

If I remember correctly it calculates the CRC on both sides (server and client) and then only copies the files that actually changed. Saving a massive amount of network traffic. Of course it also works on two disks inside the same computer.

I usually have a RAID set for regular storage and add a single cheap SATA disk for rdiff backups.

I'm not sure if they have Windows or Mac versions of this yet. It's prety common on Unix based systems like file servers (NAS) and webservers though.

[stace]

I have a couple of 2TB NW drives.

They hold my Photo's, Music, Docs, etc.  One is a copy of the other (I do it manually when I feel the need).  The second drive is kept at my sisters house.  At least in the event of something serious, she's a few miles away.

I work in IT so am not worried about automation and the headache that goes with it.  I am happy to spend a few hours, when I know there is a need, making the copy.

[DaveMMR]

I've never worried too much about back-up because I don't have much unique data.  Maybe a couple of Gigs of pictures, that are duplicated across two hard drives and backed up on disc.   My music collection is only really 30 Gigs so that's on two separate drives and an old iPod.  Any unique movies I may have (i.e. stuff I edited myself) has at least 15 copies made into DVD-Video -- store bought DVDs can always be reripped if needed.  My Roms are duplicated for the sake of convenience (it wouldn't be a catastrophe if they were lost - only annoying).   Finally, all of above also gets put on an external drive occasionally.   

Looking at the thread's title, I'm thinking the OP is looking for something groundbreaking but it's really not so complicated anymore when you can buy thumbdrives and extra HD's for cheap.  It's almost a non-issue nowadays.  I do see some posters' have extensive data needs for their jobs and/or Hobbies, but for the average Joe (e.g. people I do computer jobs for), even a couple of burned data DVDs goes a long way.   

[Malenko]

I used to run a mirriored RAID but meh, it was over kill. Now I just have a second drive in the MAME cab full of junk , a second drive in my PC full of junk, and a second drive in my media PC full of junk, so its just a different kind of redundancy

[ark_ader]

I used to backup to tape, in the old days of half inch Blackwatch tape to the recent T1000s.  Tape has been a real issue for me, as I am really impatient and want to get data in and out quickly.

Therefore I use 500gb Sata drives, and just use a Sata dock.  I can get my drives dirt cheap, and it is the best way to backup and archive data.  I tend to backup in tandem, so there is very little risk, and I do not do increment backups anymore.  Compression is a no no too.  After a complete backup is done I vacuum seal all the drives in plastic and store them in a controlled storage environment.  Forget about CD and DVDs - the old bugbear of rot exists today as it did with the old 12 inch WORMs of the late 80s.

I still like flash for simple file swaps, but do go for the odd floppy for the older kit.

I dug out a 1gb Jaz drive with SCSI-2, going to see if it plays with W7. 

[Endaar]

I've gotten to the point where every PC I have at home has all its drives mirrored in a hardware RAID1. I would not use software RAID, but with most motherboards these days including SATA RAID, and with the cost of drives so minimal, it's pretty much a no-brainer. Drives do fail, and having a perfect mirror copy makes those failures pretty uneventful.

All of my (non-MAME) data, music, etc., is on one server, and for that, I also use Carbonite (offer code Glenn!) for online backup. Yes, it took quite a while for the initial sync, but once that's done, the incrementals are pretty manageable. $50 a year is well worth it. Even with the RAID, I'd rather spend the few bucks to know the data is there if my basement floods, etc.

Endaar

[mmb]

I've a six drive SATA Raid 6 setup in the media server in the house and that rsyncs with a duplicate system off site.  I follow a 3-2-1 backup system for my personal data.  Three copies, on at least two different media and one offsite. 

This is for pictures, home video projects, documents and the like.  I don't rsync all content offsite and I don't duplicate things like roms/chds that are easy to get a hold of.  Configuration of mame/emulators/frontend I do duplicate, but that's maybe a max of 200MB.

[danny_galaga]

I am very crude with my backup, but it's rock solid. I keep everything I would hate to lose in one folder, and copy that onto a removable USB drive, which i only connect to the PC when I back up. I backup when I've made some significant additions or changes, otherwise once a month or so...

[Blanka]

I keep everything I would hate to lose in one folder, and copy that onto a removable USB drive

Sounds like one instance too little, and just one location?
Also manual copying is very prone to human errors. With a good syncing app you are less likely to destroy your data yourself.

[danny_galaga]

I keep everything I would hate to lose in one folder, and copy that onto a removable USB drive

Sounds like one instance too little, and just one location?
Also manual copying is very prone to human errors. With a good syncing app you are less likely to destroy your data yourself.

Well, everything precious is on an internal slave drive (it would have been simpler if I had just made a partition for it, but whatever). Then I back that folder up (two actually, one is called music, the other is called 'stuff worth keeping'!). On the USB drive. I merely change the name of the folder on the USB drive so that when I copy and paste onto it, it's a completely new copy. therefore bypassing all that tedious 'folder already contains blah, do you want to replace'. Later I delete the old folder off the USB drive, leaving the latest. Like I say, sounds crude, but there's almost no way I can ---fudgesicle--- it up. At any time, there's at least two up-to-date copies (one on the PC and one on the USB drive) and one slightly out of date copy on the USB drive until I delete it.

So that's two completely physically different locations. I keep that USB drive away from the PC when I'm not using it. If there's a fire for instance, I can grab that in two seconds and run like hell 

If I'm burgled, there is a chance one copy might be stolen but not both, since they are two different things. I kinda wish someone WOULD steal this piece of poo PC though...

[shateredsoul]

I'm with Danny_galaga

I find that doing it manually is the only fool proof way to do this.  My most important stuff is about 1gb of stuff (papers, files, work stuff), that I rar and upload to megaupload and my flash about once a month. 

I back up my game and music stuff at cirtical points on an external that's put away when not in use. 

I tried doing that online backup with 200 gbs in music, ---fudgesicle--- that

[patrickl]

A client of mine bought an external harddisk which came with some software that automatically starts running backup software once it's connected.

[Cakemeister]

I use Syncback and I backup my wife' computer to mine and mine to hers. I also email myself irreplaceable files (encrypted, you never know who's going to be snooping) for offsite backup. Music is not irreplaceable. I'm too lazy to rip MP3s, I just listen to my CDs. If a CD goes bad I guess I can invoke fair use and get a backup copy. Some of my VHS and audio cassette tapes are going bad, especially the audio tracks on the VHS, so I may need to do that for those, too.

[massive88]

I use Syncback and I backup my wife' computer to mine and mine to hers.

I also use SyncBack.  I have any data that I want to back up backs up to my Nas weekly, and then to my webhost weekly.  Both through FTP.  I used to just back up to my arcade machine, but then my wife got worried about a potential house disaster and wanted her stuff backed up offsite as well.  Makes sense I suppose.

The first backup was a ---smurfette--- (we have about 40 gigs worth of stuff we back up), but after that it only backs up data if it changes, so its usually not a ton in any one sitting.  It starts itself at 1am on weeknights anyway, and my upstream is 1.5 Mbps.  Course, I dont take 16 gigs worth of pictures at a time.

[danny_galaga]

  I used to just back up to my arcade machine

Hehe. Just recently I accidentally reformatted my secondary drive, and my computer is playing up, so I've been backing up on my mame machine (",)

[Santoro]

I wouldn't worry about it.  Simply put us IT people don't need to backup our data as much because we know what we are doing.  If our computers start acting up we attend to it before they get to the breaking point.  If they DO break it's very unlikely that the harddrive itself will be completely unusable and we are competant enough to pull important data off the HDD of a dead computer. 

Gotta strongly disagree with that sentiment Howard.  Nothing can save an IT guy from a disk head crash for example - short of a $2000 professional recovery job you are hosed.   God forbid you have a housefire!

I go to the other extreme, probably overboard.

1) Every PC in the house is backed up nightly to a Windows Home Server.  I have recovered important files at least 6 times from the WHS. It's awesome.
2) The WHS shares are duplicated on separate disks in case of failure.  It's not RAID, but it's close enough for me.
3) The documents on my pc (financial, work, and photos) are backed uo to Mozy every night.  (I have FIOS so this is reasonably fast.  The first backup took a few days, and the nightly incrementals go quickly
4) Every year or so I burn a few DVDs (blu-rays now) of vital stuff and throw it in my safe deposit box. This is the only step that takes my own time.

I highly recommend a Windows Home Server to anyone with multiple PCs to back up.

[danny_galaga]

  Nothing can save an IT guy from a disk head crash for example - short of a $2000 professional recovery job you are hosed. 

This happened to me once

Hence the habit of having a secondary drive.

[patrickl]

I once got hired to recover as much as possible from a crashed disk. He didn't wan't to bring the disk to some unkown firm (it had confidential files on it).

It's amazing how much you can actually still get off of these disks if you find the right software. The software cost a fortune, but luckily they gave me a trial key so I could use it for free for a month.