• Please use real names.

    Greetings to all who have registered to OPF and those guests taking a look around. Please use real names. Registrations with fictitious names will not be processed. REAL NAMES ONLY will be processed

    Firstname Lastname

    Register

    We are a courteous and supportive community. No need to hide behind an alia. If you have a genuine need for privacy/secrecy then let me know!
  • Welcome to the new site. Here's a thread about the update where you can post your feedback, ask questions or spot those nasty bugs!

External hard drive for mirror/backup

Doug Kerr

Well-known member
My current "safety" situation for files comprises the following (all the systems discussed run Windows XP):

1. I mirror most of the data from the two hard drives on my machine (one for programs, one for data) onto space on one of the hard drives on Carla's machine. I mirror all the date from her two hard drives (except of course for the mirror of mine!) onto a separate partition on one of my hard drives. Because this operates on a "mirror" basis, it provides no protection should I accidentally "lose" (or screw up) a file (unless I realize it before nightfall).

2. I backup all the data onto DVD's using a "full-incremental scheme".

The amount of data has become so stupendous that the backup to DVD plan is just impractical. A "full" backup is overdue, and I face the prospect of recoding onto 20 or more disks!

I think I need to look into an external hard drive. This of course raises a lot of questions, including:

1. What interface? USB2 rigs are readily available. But I guess there is also the prospect of accessing it over our LAN (100baseT). What are the pros and cons of both? One matter would be just that of transfer speed. Another is that LAN-capable external drives seem harder to find; maybe I mean "harder to find inexpensively". If I used a USB2 rig, I would host it on my machine, and for mirroring the data from Carla's machine the software would read it over the LAN (as happens now for both mirror and backup to DVD) and write it out over the USB port to the external HD.

2. What data management strategy? A mirror strategy is perhaps the simplest, but gives me no "recall" of accidentally deleted or accidentally badly bungled files. But of course some version management system will consume more space. A compromise would be a scheme in which all files are written to the "safety" drive, so that a bungled one would displace the earlier "OK" one, but files that were deleted on my system would not be killed on the safety drive (as happens under the backup software I currently use for mirroring.) And what software packages would be good for administering this?

I'd appreciate any observations you might have on this matter.

Thanks so much.
 

Michael Fontana

pro member
Hi Doug

I can' really talk about the best option on yours OS, but here's what I use to have alwith every data twice, ending up with totally 13 harddisks, from 300 GB to one TB, each:

- Mirror-RAID for instant/shorttime backup within the computer.

- excremental autobackup of it on a daily base, on a exernal disk, softwarebased. The box is a S-ATA-based Firm/Seritek, with a hand full of drive trays.

This covers my needs against potential electric/lithning hazards (destroing the computer) as well as accidentally deleted files. With the external autobackup I alwith can go back, within a days period.


- when projects are finished, its entire data is migrated - with drag and drop!! - but I din't found a better solution yet, to at least 2 different harddisks, beeing at the studio and at home.
 

Cem_Usakligil

Well-known member
1. What interface? USB2 rigs are readily available. But I guess there is also the prospect of accessing it over our LAN (100baseT). What are the pros and cons of both? One matter would be just that of transfer speed. Another is that LAN-capable external drives seem harder to find; maybe I mean "harder to find inexpensively". If I used a USB2 rig, I would host it on my machine, and for mirroring the data from Carla's machine the software would read it over the LAN (as happens now for both mirror and backup to DVD) and write it out over the USB port to the external HD.

2. What data management strategy? A mirror strategy is perhaps the simplest, but gives me no "recall" of accidentally deleted or accidentally badly bungled files. But of course some version management system will consume more space. A compromise would be a scheme in which all files are written to the "safety" drive, so that a bungled one would displace the earlier "OK" one, but files that were deleted on my system would not be killed on the safety drive (as happens under the backup software I currently use for mirroring.) And what software packages would be good for administering this?

I'd appreciate any observations you might have on this matter.
Hi Doug,

I don't have much time for a longish answer right now, but I can go on writing pages later if you want me to ;-)

1) Interface: USB is more than fine, but it can be an issue accessing it from various PCs. So I'd recommend a SATA II enclosure (a so called NAS) which has an GB Ethernet interface and room for 2 SATA II 3.5" drives. Most of these enclosures sell for between 100-300$. You have to install your own disk drives yourself (1 or 2 disks). I recommend using 2x 1TB disks from Samsung, Seagate or WD. If you use two disks, you can configure the NAS to use Raid 0 (not recommended for a NAS) or as Raid 1 (I am neutral on this but I do not recommend it per se). In order to get any decent read/write speeds over the Ethernet, you have to get hold of one of the better NAS enclosures, such as the Synology Disk Station DS207+, the D-Link DNS-323 or the QNAP TS-209 Pro II. These units achieve a throughput of some 40-50 MB/s when connected using a 1000Mb/s (Gigabit) Ethernet infrastructure. 100 Mb/s Ethernet speeds are around 10-12 MB/s max. I don't know anything about the Drobo units, they seem to be popular in the states. If you buy one, check the real life transfer speeds published by independent reviewers on the net. USB speeds are usually between 25-35 MB/s, BTW.

2) Data management / backups: I'd, in your case, mirror the files from the PCs to the NAS. Use a synchronization program such as the MS SyncToy, Backer, MirrorFolder or any other equivalent. If you don't set-up Raid 1 in the NAS (no need to), then you can use one of the HDs in the NAS for this purpose. I'd then use a backup program to create full and incremental backups from the NAS HD which contains the mirrored data to the 2nd HD in the NAS. I'd also back up the 2nd NAS disk to an external USB drive (use a set of two) and keep one copy locally and keep the other one at an off-site location. You can rotate these weekly or more/less often.

HTH,

Cheers,
 
Last edited:

Doug Kerr

Well-known member
Michael, Jack, Cem,

Thanks so much for your insights. I may have to press for more details as my thinking develops.

Best regards,

Doug
 

Cem_Usakligil

Well-known member
Michael, Jack, Cem,

Thanks so much for your insights. I may have to press for more details as my thinking develops.

Best regards,

Doug
Please do so :)
PS: I see that you don't have a 1000 Mb/s Ethernet infrastructure in place yet. So maybe you can start with 2x USB2 drives plus one for off-site until you can upgrade the PCs and your switch to GB Ethernet. 10-12 MB/s throughput over the 100 Mb/s Ethernet can be painfully slow, albeit manageable for incremental backups overnight.

Cheers,
 

Asher Kelman

OPF Owner/Editor-in-Chief
Hi Doug

I can' really talk about the best option on yours OS, but here's what I use to have alwith every data twice, ending up with totally 13 harddisks, from 300 GB to one TB, each:

- Mirror-RAID for instant/shorttime backup within the computer.

- excremental autobackup of it on a daily base, on a exernal disk, softwarebased. The box is a S-ATA-based Firm/Seritek, with a hand full of drive trays.

This covers my needs against potential electric/lithning hazards (destroing the computer) as well as accidentally deleted files. With the external autobackup I alwith can go back, within a days period.


- when projects are finished, its entire data is migrated - with drag and drop!! - but I din't found a better solution yet, to at least 2 different harddisks, beeing at the studio and at home.
I ROTWL!

The word to use, I believe might be "incremental" as excremental implies that the material being stored you never want to see again! source of definition.

However, it's really a novel way of thinking of all the data we keep pumping out!

Asher
 

Daniel Buck

New member
I've been using a drobo, I don't work with huge volumes of images, but for me it seems to be doing pretty good. I use it as one of my main backups (but not my only)
 

Michael Fontana

pro member
hmm Asher

I know the floor-version as ROFL

meanwhile ROTWL - I had to go through Poland as well - Rolling Out The Window Laughing, which is new to me too.

Ah ok, I see that you Asher used "I ROTWL!" and not "ROTFWL " first, therefore the difference.
 

Doug Kerr

Well-known member
Hi, Michael,

hmm Asher

I know the floor-version as ROFL

meanwhile ROTWL - I had to go through Poland as well - Rolling Out The Window Laughing, which is new to me too.

Ah ok, I see that you Asher used "I ROTWL!" and not "ROTFWL " first, therefore the difference.

I roll
You roll
He rolls/she rolls/it rolls
We roll
Y'all roll (youse roll in the Northeast) (reseparating the collapsed second person plural)
They roll

I just thought it would be nice to do something conjugal.

Best regards,

Doug
 

Mike Bailey

pro member
Doug,

The simplest solution - without change to your current network/computers - might be to just buy external USB drives. I've found the Seagate 500g drives to be reliable and fast (about 1.2 gig a minute transfer rate, depending). If you had two, you could put one on each machine, so no need to worry about the slower network. Currently they're selling for about $90 at places like Best Buy and some office supply stores. From your 20 DVD, that probably means you're talking a total of 100 - 120 gig per full backup, so that would allow you a number of backups before you had to start deleting the oldest ones.

The more expensive external Seagates have eSata; the cheaper just USB 2, which it sounds like you already have on your computers.

Keeping the size of the external drive down seems sensible since you aren't stockpiling all your data on one even huger drive. With the increasing size of drives it seems when failure happens too much more (i.e. ALL) is lost. Always a compromise, but these 'medium' sized 500g drives should manage nicely.

Mike
 

Michael Fontana

pro member
Watch out for the Seagate Barracuda's and Diamond Max at the moment - they've some problems:


1_m.png
 

Doug Kerr

Well-known member
Hi, Mike,

Doug,

The simplest solution - without change to your current network/computers - might be to just buy external USB drives. I've found the Seagate 500g drives to be reliable and fast (about 1.2 gig a minute transfer rate, depending).

Actually (and I didn't get a chance to report to the gang on the development of my thinking) I ordered a 1TB Western Digital USB2 rig yesterday, and it is reported to be on the truck headed here at present (of course there is a bad storm here, so who knows what that will do!).

I haven't yet figured out exactly what strategy I will use for exploiting it, and I expect a second unit will be in the picture (but I wanted to make sure I liked the particular machine I got before ordering a second one).

Thanks a lot for all your insight into this matter. I'll keep you up-to-date.

Best regards,

Doug
 

Doug Kerr

Well-known member
Hi, Michael,

Wach out for the Seagate Barracuda's and Diamond Max at the moment - they've some problems
Thanks for the heads up.

We've pretty well concentrated on Western Digital drives here (although I did in fact have one go sour recently!), and as you will see from other my recent note here we have ordered a 1TB WD USB2 external. So we'll see what develops.

Thanks again for the heads up.

Best regards,

Doug
 

Jay Hoss

New member
Doug,

Another option would be to use one of these. I have two of them at work and use it as a back-up solution using Microsoft's free software SyncToy. I find that full size internal SATA drives are much more affordable per gigabyte then other external enclosure solutions.
 

Doug Kerr

Well-known member
Hi, Jay,

Doug,

Another option would be to use one of these
Interesting. Thanks for the lead.

My 1TB WD external came this morning! It's on the air, and I am now making the initial load (using the same software I formerly used for mirror backup from each of our machines to the other.

That's not the final arrangement - just wanted to get back on the air.

Thanks again.

Best regards,

Doug
 

Jeremy Lawrence

New member
Doug - you can probably buy a decent HD enclosure for the same amount of money as one of those gadgets. They are designed for people who swap HDS around a lot or computers shops doing repairs/upgrades, not for the sort of thing you need.

A handy programme I've used for Windows backing up is Clone 2.1 http://newtonsoftware.co.uk/clone/
Very simple, but very good. I use it to keep the data synced on the 2 external HDs I use on on my window's laptop, whilst travelling. You tell Clone what folders/HDs to to keep synched, how you want it syncing [replace with newer usually] and how often to do it and that's it really.
I have fancier software on the Mac machine, but nothing quite as good as this little gem.


I'm surprised no-one has mentioned WHS [WIndows Home Server], this was produced by the good + clever coders at MS whilst all the newbies made Vista! ;-) .
WHS is a safer bet than Drobo too - Drobo is a proprietry and simplified version of WHS really, even though they got to market first as WHS was delayed for some time. I have a HP WHS, nice little machine which sits in cellar head behind a fire door and connects via the Home network+ethernet.


To help you decide what to do.
Mirroring safeguards current data on a single computer against the one drive failing.
Copying all this data to another disk incementally, is backing up.
Copying all new Data as it is created up and simply adding it all to another disk is archiving.
Archiving and backup are two different things. If you get a corrupt file and it is backed up, the back up is corrupted. The Archive however should be untouched as data is only added to the Archive with no replacing.
Archiving is a complete PITA, as files worked on need archiving just as much as the original RAWs, but how do you add them easily into your normal file structure?
 

Asher Kelman

OPF Owner/Editor-in-Chief
This is good Jeremy.

What software do you use for your Mac? There's need for backup of files, finding duplicates and making sure one doesn't corrupt the archives.

I've just boiled down 27 drives to 5 TB of archived data. Cataloging is an issue. Iview Media Pro crashes when the more than 10,000-30,000 files are stored. My one Drobo will make a dynamic incremental update of what's new for me beyond (i.e. that's not represented as yet on the archived file. I still need to get another set of drives to put off site. Ideally this would be a second Drobo or similar set up to work automatically at the other end of a modem far away.

I met a guy in the big photo store in S.F. who painstakingly scanned all his film to digital files and made DVD backups too and before he arranged an offsite place, a fire destroyed all his work, everything film and digital of 15 years creative work! Of course this was a big blow to him and he has not recovered and is now selling gear instead!

So having a mirror is not good enough! Need something off site too!

Asher
 

Jeremy Lawrence

New member
I use Chronosync on the Mac.
It's like a graphically fancier verion of Clone2.1 for the PC.
Though there have been some issues when people upgraded to 4 from 3, but if starting from scratch.....


I'm just trying to work out how to do off site storage that can be updated frequently and easily. It has to be easy or it won't get done.
I'm thinking two external HD enclosures with 4 Sata drives RAIDed. One is at a friend's house and one incementing during the week at my place. If I swap enclosures once a week, that limits me to only one weeks data loss.
Not cheap though. Another eSATA card for Mac, £120, 2x eSATA 4drive enclosures £350, 8x 1.5Tb HDs £800 and it's still not big enough for all my data either as with RAID5 that's only 4.5TB of total backup. Plus a UPS to make sure data isn't corrupted whilst writing due to power glitches.

If Broadband was actually fast, then I could use online storage, but upload speeds are pitifully slow for this sort of thing.
 
Top