• Please use real names.

    Greetings to all who have registered to OPF and those guests taking a look around. Please use real names. Registrations with fictitious names will not be processed. REAL NAMES ONLY will be processed

    Firstname Lastname

    Register

    We are a courteous and supportive community. No need to hide behind an alia. If you have a genuine need for privacy/secrecy then let me know!
  • Welcome to the new site. Here's a thread about the update where you can post your feedback, ask questions or spot those nasty bugs!

Longer term storage

Alan T. Price

New member
Nik, I'm not sure how to help you out on that one. 2,000 GB is a lot of storage. What does it represent for you ? i.e. is it the total storage of backups, or the on-line storage, or the combined storage ? Is it for a single backup, or multiple backups ? Is a single backup in one device, or multiple devices ?


My own requirement is quite modest at this stage. Because of that I have not kept up with the latest available HDD holders / RAID controllers. Even if I had I would doubt that I would find most of them here in Perth. For example, the biggest HDDs at the popular store here is 500GB SATA2 (already superceding SATA). Their external cases are for single PATA or SATA drives. At a camera shop I found a 2-disc RAID box with LAN connection but only 10/100. So with what is readily available here I can't set up a stand-alone TB storage solution. I'd have to import one of the expensive options that obviously will have little or no local support.

Another shop has a Thecus Gigabit LAN 5xSATA2 RAID controller box with cpu and ram for $22, but that's gotta be a misprint. You can't buy anything for $22. Especially Australian dollars.

I'll enquire further when I get back to Perth, but in the meantime all can say is to repeat my advice to keep it simple and cheap so you afford to replace it early.
 

John_Nevill

New member
Have any of you guys tried the Buffalo Terastation. They are available upto 2TB with RAID 5. Its a cheap Linux SANS solution and provides both network and USB connectivty. I use them at work for moving masses of GIS/Telemetry data around between clients / suppliers.

For personal use, I use Lacie USB/Network drives and just add to them when needed. I keep one live and do differential back ups to the others. That way i'm not wearing out the backup drives through constant use.

I also keep DVD copies of everything.
 

Alan T. Price

New member
none of those things are particularly cheap. You pay a hefty premium for what is basically a clever box to hold and manage the drives, and one of my concerns was what happens when that box dies - will Buffalo still be around ? Will the drives be readable in another box from a different company ? Do you just reformat and start again ? Do you maintain two of the boxes just in case one dies ?

The individual drive approach works too but has risks associated with it. Consider using them with Windows XP and formatting them with NTFS for maximum security and robustness. Now you have a choice of allocating a specific drive letter to each one so you can manage them better, but you will run out of letters and have to double-up eventually and that will cause grief for windows or you because no more than one drive partition can have a drive letter. Or you can not assign drive letters and let the system do it on the fly, in which case you risk losing track of your on-line drives.

It's still manageable but it takes more effort and awareness on your part.

DVDs are a good backup too (although Bluray will be more practical), but you need to create two sets in case one DVD goes wonky and loses data. It probably pays to duplicate it again every so many years just to improve longevity of the data. I use different brands for each copy in case one is worse than the other.

Differential backups are a risk in themselves. I once lost a whole set because something in the process trashed the master catalogue. I wasn't happy to lose not only the new backup but also the old one. Apart from that type of error, the master could be lost to defective media and then there may be difficulty extracting the rest of the data from the differential backups. It depends a lot on the software you use.
 

Nill Toulme

New member
Something that should be factored into all this is that drives just keep getting cheaper. My data currently lives on a 1.1TB internal SATA RAID. It backs up nightly to a couple of external Seagate 400GB firewire drives (which I like a lot better than the chronically problematic Maxtors that preceded them).

Inspired partly by this thread and partly by general angst, I just buzzed over to Best Buy and picked up another Seagate external, this one 750GB, for $450. For the time being, that one will be used to back up the backups weekly, and it will live offline at my next door neighbor's house.

One of those nasty old Maxtors does still get used also. When I download a card to the PC, Downloader Pro automatically puts a second copy of the download over on that drive. I dump stuff off of it periodically as things make their way onto the other backups.

Nill
~~
www.toulme.net
 

ChrisDauer

New member
Josh Liechty said:
There are tons of enclosures on Newegg, but the reviews there border on useless, and I would also like to hear what's working for people who use external hard drives already.

Some cost data that I found while reading about Quantum's Govault system (removable hard disk cartridges, like Iomega's Rev system) might be of interest. A 120GB Govault cartridge (excluding the drive) is US$2.16 per GB. 400GB external hard disks that come in an enclosure tend to be around $0.50/GB, with 250GB external drives costing approximately $0.44/GB. If one were to take a 300GB drive sans enclosure (~$99) and put it in an enclosure (~$30), it wouldn't be much different than a pre-built 250GB external drive, at $0.43/GB. This is just a sampling of a few options on Newegg, so it's quite possible that you could do something for less than what I've estimated here. These results do call into question, however, my thoughts that a removable disk cartridge system's advantages outweigh the cost penalty.


I just picked up a 400 GB drive for $100 (+tax). I bought a very nice enclosure for $50 that does IDE and Sata. That brings me to $.375/GB. I should mention that the enclosure is nice because I can swamp new drives in when the current one gets full, and I don't have to invest in another enclosure. I'm not as worried about the drive degrading or falling apart when I put the next one in. I've been very fortunate in the past to let drives sit for 3-5 years and then pop them in and just have them work.

My current game plan is to move everything off the very old 160GB onto the new 400GB. Then, turn around and back up the important images on the 160 (second copy). Then remove all photos/backup photos off of all 3 of my internal system drives EXCEPT for current works in progress AND the best of the best which I send to friends now and again.

It has been very educational reading this thread. Thanks to all who contributed. I have been very curious as to what others do for there photo storage solutions. I know my solutions is extremely simple and not as involved as some of the more complex devices using the various RAID options.
 

Asher Kelman

OPF Owner/Editor-in-Chief
Chris,

This is such an important step you have taken! congrats on keeping the cost down. Ideally you need to have one more such setup so you can backup the other drive each night.

Us the previous drive for off site storage of you key files or else as a scratch disk for Photoshop.

I'd love to know what make drive and box you ended up getting.

For one thing, I steer clear of getting LaCie big drives with more than one drive inside as that mayincrease the chance of trouble and the spanning of the drives may not be dealt with for you when trouble occurs after the short warranty period is over. I learned the hard way.

Asher
 

Ray West

New member
I don't know if its been mentioned here, or elsewhere, but the 'buffalo linkstation' is a useful device. It sits on the network, has a reasonable sized inbuilt hdd, and also 2 usb 2 ports, one for a printer, one for another external hdd. It runs a cut down version of linux (which you never have to get into) and you can run it ith macs and windows. It also has automatic backup built in. Not expensive, takes up little power or space, and its quiet. They do bigger units too, but this system is simple.

Best wishes,

Ray
 

Asher Kelman

OPF Owner/Editor-in-Chief
Hi Ray,

What connections does it have and are the drives removable? One things that is nice is when the box monitors for SMART fn of the drives.

I'm not sure I really understand how the link station differs from a regular hard drive or other simiar boxes.

I have moved away from firewire since it is problematic with a bunch of them causing G4 and G5 Macs to become unstable. It may be a combination of that with 3rd party memory.

Currently I buy 250 GB drives and have 5 removable and a bunch of Macsales.com (OWC) closed cases which work very well. each closed case has 2x500 GB and I have a spare one.

My filing is going slowly as I rename files according to year and move them to the new drives.

The big issue is finding some files are present with 5 copies and others just one with no backup.

Asher
 

Ray West

New member
Hi Asher,

here's a link http://www.buffalotech.com/products/product-detail.php?productid=106&categoryid=22

As I said, it is a network device with print server, and a built in hdd, and the ability to have another external usb drive. It is not that much more expensive than a prebuilt external usb drive. As its a network device, it can copy files from its internal hdd, to its usb external one, or between other link stations. Some of the reviews on some other sites are out of date, some are wrong.

Buffalo do other, larger storage devices, too.

Best wishes,

Ray
 

Asher Kelman

OPF Owner/Editor-in-Chief
Seems good, Ray. Just that the auto backup software is for PC users.

How would one utilize all the functions with a Mac? I guess one would need print server software? I know /mac has it for their own print servers. Who knows?

Asher
 
Asher Kelman said:
Seems good, Ray. Just that the auto backup software is for PC users.

How would one utilize all the functions with a Mac? I guess one would need print server software? I know /mac has it for their own print servers. Who knows?

Hi Asher,

This tech is old school. Way older than OS X or XP. And OS X derives from the Mach Unix kernel via a few generations. The files could be shared via Appleshare (may take a Linux extension), NFS (Network File System), or Samba (windows based network server protocol, a.k.a., SMB). On OS X you should be able to simply open a command line and type [URL="http://www.openphotographyforums.com/forums/"]lpr[/URL] with the appropriate command line flags and print away. Albeit, this is old school (not really old school like punch cards) stuff and requires a command line. I cannot imagine Apple not supporting such in OS X. Last time I needed to network an OS X box (I think it was OS 10.2 IIRC) 10 minutes on the net told me where to look under finder to do it. Nonetheless, this technology was common (on large networks) in 1993 (5-7 generations ago in computing time).

Nonetheless, IEEE 1394 (Firewire) is faster than Fast Ethernet or wireless networks making such a solution low performing unless you have a Gigabit Ethernet LAN in the house. Add to that that you can unplug a Firewire or USB drive and have it disconnected from the mains/power very easily and you have a much better solution in discrete drives than by a network solution where data safety is your primary issue. An unplugged drive will be lost in a fire, tornado, earthquake, or hurricane, but lighting strikes will not hurt it nor will viruses when it is not connected to power. External hot-swappable (Firewire/USB/E-SATA) drives are the next best thing to offsite backups.

As to corrupt copies, that is easily corrected by validating copies via things like MD5/SHA1/CRC32/... hashes or direct bit for bit validation.

some thoughts but no answers,

Sean
 

Tony Panzica

New member
This is a great thread and very informative. Of the 42 posts, there are 42 different solutions that everyone seems to have developed on their own. I would like to see a committee of say 5 of the heavier contributors develop some step by step solutions. Now you have to develop a different solution for an event shooter, a deadline conscience sports shooter, a wedding shooter, a nature shooter... Get the idea? Maybe at a convention, get a bunch of folks together to work. I dont know if this will work, but on paper it sounds cool. The problem that we have is as soon as you develop those solutions, the manufacturers change their product line up, pricing, availability.
My problem was reading how several of you bad mouthed the Maxtor drive. My brother, a top exec quality control guy who works for a chip manufacturer recommended it to me. I have the Maxtor III, 1TB drive, in fact I have 2 of them. Its fast and quite. I started out with film, then had the original 30D, used a Travan tape drive, have over 50GB on tape, then burned about 100 CDs, then burned 30 DVDs.
There are definite gaps in the Photo Industry for sure. But I wouldn't trade where I am at now compared to how I used to shoot with film. Can you imagine shooting a sporting event with only 36 shots on a roll? Then have to wait til you go to lab and hope you nailed it. You just have to stay on top of whats going on, then decide what fits your budget and workflow. Good luck...
 

Cem_Usakligil

Well-known member
Hi Tony,

You probably will have realised that a committee of 5 experts will never be able to come up with a single solution no matter how long they delibarate on this (LOL). You should digest the information given here and there and decide for yourself what works in your particular case and what does not.

Re. the Maxtor drives, this is how I see it:
They are one of the best (and biggest) brands out there and they produce good/affordable drives. Some of the products will inevitably be bad or wil have QC issuues. Some people will bad mouth the brand in general, some will praise them. Mind you, you can replace the brand name with Seagate, IBM/Hitachi, Samsung, whatever. Each and every one of them have people who either love or hate them. It is a bit like the silly Mac vs PC wars, really. Having used hundreds of PC hard disks from all major manufacturers in the past 25 years, I can only conclude that they all have been good on the average, even though many of them have crashed or sub-performed in that same period. What I'm trying to say is that one should ignore the negative feedback as long as it is not based on statistical and objective evidence. Maxtor drives are very good, and I'd certainly recommend one. Heck, I use many of those myself without problems so far. Take your pick, and use it. Don't forget to make BACKUPs though ;-).

Cheers,

Cem
 

Dierk Haasis

pro member
Tony Panzica said:
I would like to see a committee of say 5 of the heavier contributors develop some step by step solutions.

Design by committee - in the best of cases we get a camel [for those not knowing the expression: A camel is a horse designed by a committee]. IME - and I am heavily biased - committees are not very good since they look for the lowest common denominator (= compromise). That's a good thing when we are talking soft issues like how to organise schools and universities; it's decidedly bad if you need a workable result on hard issues.

Politicians, for instance, tend to look for a compromise even when talking about facts or mathematics. Two opposing parties come up with clearly opposite plans for taxing, one says we need a VAT that's 2 points higher, the other says they will not increase. After the election they have to come up with a compromise:

(0+2)/2

Curiously the committees came up with a surprising solution to this rather simple problem - 3 points increase.

My problem was reading how several of you bad mouthed the Maxtor drive.

I use a Maxtor One Touch II and a Maxtor One Touch III, and both seem to be very good.No problems, yet apart from a strange interaction over my Firewire connections [the III automatically disconnects when I switch on a Plextor DVD burner].
 

Ray West

New member
In most circumstances, a camel performs far better than a horse.

I am coming to the conclusion, that in many ways it is best to use usb2 for everything you can. Cheap, fast enough for most things, portable, allows off site backup, easy to upgrade hdd's, etc. etc. Keyboard, track ball, hdds, cd/dvd r/w, network, etc. everything onto usb. As well as buffalo, linksys do a network usb hub. Also simple enough to use a cheap pc good enough for a file server.

Best wishes,
Ray
 

Dierk Haasis

pro member
Ray West said:
In most circumstances, a camel performs far better than a horse.

Hence my 'in the best case'. The adage was obviously coined by some big city Victorian who was into looks and elegance more than into performance. Today it's often teh other way round, many folks go for over-powered, over-sized cross-country cars (SUVs and such nonsense), which are never ued off road, never driven remotely close to their limits. A camel is very good in deserts and near-deserts, a llama performs better in mountainous terrain, a horse is brillinat on relatively even savannah/prairie ground. It obviously looks better.
 

Cem_Usakligil

Well-known member
Dierk Haasis said:
... It obviously looks better.
Now we are treading into your favourite territory Dierk, i.e. philosopy.
A horse obviously looks better to another horse, definitely!
Possibly not from a camel's perspective though ;-)

Cem
(LOL)
 

Asher Kelman

OPF Owner/Editor-in-Chief
Sean DeMerchant said:
Hi Asher,

This tech is old school. Way older than OS X or XP. And OS X derives from the Mach Unix kernel via a few generations. The files could be shared via Appleshare (may take a Linux extension), NFS (Network File System), or Samba (windows based network server protocol, a.k.a., SMB). On OS X you should be able to simply open a command line and type [URL="http://www.openphotographyforums.com/forums/"]lpr[/URL] with the appropriate command line flags and print away. Albeit, this is old school (not really old school like punch cards) stuff and requires a command line. I cannot imagine Apple not supporting such in OS X. Last time I needed to network an OS X box (I think it was OS 10.2 IIRC) 10 minutes on the net told me where to look under finder to do it. Nonetheless, this technology was common (on large networks) in 1993 (5-7 generations ago in computing time).

Nonetheless, IEEE 1394 (Firewire) is faster than Fast Ethernet or wireless networks making such a solution low performing unless you have a Gigabit Ethernet LAN in the house. Add to that that you can unplug a Firewire or USB drive and have it disconnected from the mains/power very easily and you have a much better solution in discrete drives than by a network solution where data safety is your primary issue. An unplugged drive will be lost in a fire, tornado, earthquake, or hurricane, but lighting strikes will not hurt it nor will viruses when it is not connected to power. External hot-swappable (Firewire/USB/E-SATA) drives are the next best thing to offsite backups.

As to corrupt copies, that is easily corrected by validating copies via things like MD5/SHA1/CRC32/... hashes or direct bit for bit validation.

some thoughts but no answers,

Sean

Sean,

I like these points:

1. We have already fast independant drives that become moderately virus-protected by unplugging.

2. Hot Swap SATA gives us "off-site" backup capability: and it's cheap.

3. Data is checkable. Here I'd like to have more detail.

What "out of the box" software is their available for Mac and Windows that can be added to our work flows? Is this already inherent in some of the smart boxes or file servers that won't break the bank?

Asher
 
Asher Kelman said:
3. Data is checkable. Here I'd like to have more detail.

What "out of the box" software is their available for Mac and Windows that can be added to our work flows? Is this already inherent in some of the smart boxes or file servers that won't break the bank?

For simple data verification on PCs one can use CDCheck:

http://www.kvipu.com/CDCheck/

This is about $50 for a commercial license and you can validated copied data easily before deleting it from your system and relying on your backup copies*. Such validation may take extra time, but it also leaves one a lot more comfortable deleting tens of GB of image files from a drive.

As to already included software there are likely md5, sha1, or direct difference (i.e., diff) executables on most *nix installs or they are available for free download. But they will require command line scripting. I do not know of any simple GUI based tools for OS X, but they likely exist.

enjoy,

Sean

* Note explicit plural.
 

Eric Hiss

Member
lot's to think about-here's what I use now

Hi All,
I've been enjoying this thread and lots of points I had not thought about have been raised. I had been learning about this stuff the hard way - through 3 hard drive failures over the course of 6 months. I also know that DVD backups are not 100% :-(

Okay so what I have now. I run a highpoint 1.2Tb RAID which is connected via their pci-x card to my main working computer a Mac Quad G5. A fast connection is essential for a working volume. I am using Aperture to catalog and process most of my files from Canon 1Ds and 5D, but also use lightroom beta right now for my Leica DMR images until Apple can get organized to add compatibility for that. The main computer and working RAID are protected against power outages with a UPS. Everytime you loose power while the computer is operating you can corrupt a small portion of the disk. If that happens to be the header volume information...well you are screwed or are in for at least spent time in recovery. If your computer is not on a UPS then you should consider it. Laptops of course are protected. UPS also protects against lightning too.

Now my backup data is stored on a NAS server. I use the ReadyNAS NV with the same 4x400gb sata drives. This is much slower but I can let it run while I am away. I am backing up at irregular intervals right now because of the slow speed. The nice thing about having the back up on the internet is that you can provide remote access to data if needed and also you can locate the backup volume away from the main working volume to protect against loss to fire etc. What I am doing is to unplug the back up and take it to my studio so not really using the NAS features now. But the ReadyNAS has its own but smaller UPS.

It cost me a bundle for all this, but then I already know the cost of replacing lost images!
Eric
 

Paul Spatafora

New member
Buffalo Tereastations

I finally ordered my 2Tb Buffalo Terastation. After much consideration, I decided to get this unit over all the other options mentioned in this thread. The reasons for this are:

1. I can run in a RAID 5 mode.
2. Buying the compatible hardware and assembling a RAID myself would have saved me a $100-$200 without any warranty or tested and proven software.
3. The box is ready to use and tested with easy to use software.
4. It has 4 USB2 ports and a print server port for networking.
5. DVD's or Dual Layer DVD's are a PIA and I'm not going to get sucked in the Beta vs. VHS fight that seems to be playing out between Blu Ray and HD. Besides point of entry is going to be very expensive. Consider that a DL DVD cost 4 times that of a single layer DVD and you can imagine the cost of the high capacity disks when they come out.
6. Single USB external drives provide no safety other than shutting them off.
7. HD's are still the cheapest, most backward compatible solution still today and within a RAID.

I think for now, a hot swappable system might be the safest of all the solutions above. And the management of the array looks simple. Even I can administer it, hopefully.

Great thread.

Paul
 

Amy Goalen

New member
The topic of Digital Asset Management is such a big animal to get around in the digital age. A few months ago a read a wonderful book on this subject that address many of the concerns on this thread, "The DAM Book" by Peter Krough.

I can't say enough about this book and I also recommend it to all my students in my workshops. The one thing I have found is that Digital Asset Management is not cheap, but very necessary...which is why I finally took a deep breath and plunked down the business charge card.


Amy Goalen
Leaf/Phase One software trainer
Digital Technician
 

Asher Kelman

OPF Owner/Editor-in-Chief
Hi Amy,

For me at least, the DAM Book is something that should be in everyone's list to reread and rethink one's current storage system. It's just storage 101!

I BTW, don't follow Peter's advice fully, but I like his overall ideas.

The main thing is to have a standard system and to have upgradeable to any new storage medium and software system.

I do not run huge iview media Pro catalogs, since they get klutsy can crash. I am generous in having multiple catalogs by month organization and type of work or even camera.

So I have scores of specific catalogs, for example: an M8 catalog, a 2006_12_14_Smashbox catalog and say a 2006_Art Master catalog.

There is absolutely no functional worth for me to have my art projects in the same catalog as a wedding or gala social.

Now maybe I'll get closer to Peter's methodology on my next rereading.

I'm still working on my organization of files from before I started a logical DAM system.

BTW, OWC.com is selling a Terrabyte (2x500 GB Seagate SATA) external kit in a beautiful black aluminum case wit a very quiet fan, 4 about $430.

It goes with Macs but I cants see why it wouldn't work with PC's too.

Asher
 
Top