Open Photography Forums  
HOME FORUMS NEWS FAQ SEARCH

Go Back   Open Photography Forums > The Gear > Macintosh

Macintosh All Power PC and Intel platforms.

Reply
 
Thread Tools Display Modes
  #1  
Old December 21st, 2006, 08:10 PM
Eric Hiss Eric Hiss is offline
Member
 
Join Date: Nov 2006
Location: San Francisco
Posts: 275
Default backup or sync software for aperture

Hi All,
I'm using both aperture and lightroom now on a mac G5 Quad with two highpoint RAID systems. I want to create basically redundant drives that contain my image libraries from both applications. Aperture has the vault feature and lightroom has some kind of backup but I'm not wanting to use either. Aperture's vault system is strange to me because it takes about 12 hrs to check and back up my library even if all I did was add 500 images and this is with a hardware controlled sata II RAID! Way too slow since I can't do anything in aperture while its backing up.
What I really want is a way to duplicate and sync the two RAID arrays so that I can take one to my studio and work on whatever files I need, then bring it back and sync up the data so that I can keep changes at home and vs versa all while having a duplicate copy in case something goes wrong with one of the Arrays. They are both set to RAID 5 which provides some security but I prefer to keep a backup. I'll intend to have both on a UPS as well.

So my question to OPF guru's is what software will allow me to sync the RAID's and capture the changes made to either volume, plus do this inside package files such as the aperture library?

On my list of things to check out is SuperDuper, NetVault, and rsyncbackup so any feedback on these tools would be great. I understand that superduper does backup changes to packaged files, but may not sync but do not know for sure.

Thanks in advance,
Eric
ps...if you understand what I am trying to do and have a better idea on how to accomplish this than cart the raid back and forth to the studio feel free to make a suggestion.
__________________
Eric W. Hiss
www.eh21.com
Reply With Quote
  #2  
Old December 21st, 2006, 10:30 PM
Asher Kelman Asher Kelman is offline
OPF Owner/Editor-in-Chief
 
Join Date: Apr 2006
Posts: 33,941
Default

Eric,

Let's see what answers we get. First why it should take 12 hours to check and back up your library?

Could you provide the details of your G5 Quad computer, the amount of RAM, is is Apple RAM? and the number of files you have altogether and how full the drives are?

Asher
Reply With Quote
  #3  
Old December 22nd, 2006, 12:11 AM
Ken Tanaka Ken Tanaka is offline
Senior Member
 
Join Date: Nov 2006
Location: USA
Posts: 1,300
Default

I've no experience backing-up an Aperture library. But I do use Lightroom and back its library up daily with Qdea's Synchronize X Plus product. I've used this inexpensive product to back-up selective folders for several years with flawless results. Perhaps it will also do the trick for Aperture.
__________________
- Ken Tanaka -
Reply With Quote
  #4  
Old December 22nd, 2006, 07:55 AM
Eric Hiss Eric Hiss is offline
Member
 
Join Date: Nov 2006
Location: San Francisco
Posts: 275
Default Aperture uses packaged files

Asher,
I'm not exactly sure why it's taking so long but I believe the time is due to something about the way the application works (which I surely hope their engineers will fix). I mean I could write the entire file from one drive to the other in less time so a back up should only need to do the new things right? Should take just a few minutes in my opinion. I have a Quad G5 2.5ghz with 4gb ram and the sata drive card is the highpoint 2322.

Aperture however uses packages files so this may foil some backup programs. It appears as only one file in the finder, and one must control click on it to get the option to view package contents. Inside this is a bunch of folders that can be drilled down into all the way to individual image folders. Its kind of a mess actually.

The lightroom file structure is something more straightforward so most backup programs should work with it.

I'm hoping that Apple will build functionality into their next OS release to include the Leica DNG files because Lightroom/ACR though great with my canon files sucks for the leica files and Aperture other than a few quirks seems to be a better cataloging tool.
__________________
Eric W. Hiss
www.eh21.com
Reply With Quote
  #5  
Old December 22nd, 2006, 04:13 PM
Stan Jirman Stan Jirman is offline
Member
 
Join Date: Jun 2006
Location: Santa Clara, CA
Posts: 189
Default

Your easiest solution - which also comes for free: rsync.

Open a Terminal window, and type: rsync -va [source path] [target path]

Note that you must NOT include the trailing / in the path. So for instance, if your photo folder (containing Aperture stuff and all your other pictures) is rooted in /Volumes/Photo/Camera, and you want to back it up to the volume Backup, the command would be:

rsync -va /Volumes/Photo/Camera /Volumes/Backup

If you want to capture resource forks as well, make the options -vaE. But there's currently a bug with that and then it doesn't do an incremental update, which sucks. I think that resource forks are evil and deserve to be stripped.

If you want to delete files in the backup that have been deleted in the original file tree, add the --delete option (note the two dashes).

This is how I've been doing backups for the past five+ years and had no problems with it.
__________________
Stan Jirman
http://www.jirman.com/
Canon & Apple equipment
Reply With Quote
  #6  
Old December 22nd, 2006, 04:54 PM
Eric Hiss Eric Hiss is offline
Member
 
Join Date: Nov 2006
Location: San Francisco
Posts: 275
Default great...but need more info

Hi Stan,
Thanks for the help and this might work. Does Aperture use resource forks in their library? How can I check?

Now suppose that they don't and I have two RAID volumes Apollo and Venus and both are connected to the same computer named Tobago:

If I make changes to Venus, then I would type into the terminal

rsync -va /Tobago/Venus/aperture library.aplibrary /Tobago/Apollo/aperture library.aplibrary

I noticed that apple names their library with two words separated by a space so will I have to use quotes for that so it doesn't mess up the terminal command?

Thanks,
Eric
__________________
Eric W. Hiss
www.eh21.com
Reply With Quote
  #7  
Old December 22nd, 2006, 09:17 PM
Stan Jirman Stan Jirman is offline
Member
 
Join Date: Jun 2006
Location: Santa Clara, CA
Posts: 189
Default

Quote:
Originally Posted by Eric Hiss
Does Aperture use resource forks in their library? How can I check?
No they don't. Take my word for it.

Quote:
Now suppose that they don't and I have two RAID volumes Apollo and Venus and both are connected to the same computer named Tobago:

If I make changes to Venus, then I would type into the terminal

rsync -va /Tobago/Venus/aperture library.aplibrary /Tobago/Apollo/aperture library.aplibrary

I noticed that apple names their library with two words separated by a space so will I have to use quotes for that so it doesn't mess up the terminal command?
Well, two things: if you use path completion in Terminal with the Tab key (depending on your shell), it will automatically escape the spaces. The easiest way to type a space in a shell is backslash-space.

Also, to clarify my example (sorry, it was misleading), your command would be:

rsync -va /Tobago/Venus/aperture library.aplibrary /Tobago/Apollo

However, typically, all external and internal volumes on OSX are mounted into /Volumes, so it's more likely that it would be

rsync -va /Volumes/Venus/aperture library.aplibrary /Volumes/Apollo

You would have to be doing some quite fancy mounting to get them mounted the way you wrote it - and then I don't think you'd be asking this question :)

Also, you don't specify the name of the target folder if it is already named in the source. Rsync is *incredibly* picky and complex when it comes to the paths, and pretty much anything you can come up with will have a different meaning. My specific instruction to omit the trainling / was mentioned because many shells will automatically include it when using the Tab path completion, and you need to explicitly backspace and nuke it.

I rsync bewteen my two RAIDs regularly, and this works like a charm. As for the resource forks: they are inherently evil, and I don't know of a photo related app that stores *relevant* info in them; you may lose stuff like Finder thumbnails (which are different from e.g. embedded JPG thumbnails - these are in the actual file), and some other random stuff. Quicken is the biggest offender, it stores huge amounts of relevant data in a resource fork. Probably not even Zeus knows why.

Given that the performance mis-behavior with the -E flag is so significant I could imagine that it would be addressed in a software update. Or at least one can hope :)
__________________
Stan Jirman
http://www.jirman.com/
Canon & Apple equipment
Reply With Quote
  #8  
Old December 22nd, 2006, 11:45 PM
Eric Hiss Eric Hiss is offline
Member
 
Join Date: Nov 2006
Location: San Francisco
Posts: 275
Default thanks and a few more

Hi Stan,
Thanks again for the info. I just played a little with the terminal window and discovered I can actually drag the source file into it and get the correct path with the \ for spaces already.

You didn't say if you are doing your rsync with an aperture library or if you were going back and forth between the volumes...but that is what I was thinking of doing. Whichever volume I worked with I would then rsync to the second but this post on the apple aperture forum seems to warn against doing that (http://discussions.apple.com/thread....1012&tstart=15) and suggests only moving the syncs one way to avoid corrupting the database.

What are your thoughts - do you think it would be best to do it this way?

Finally I am wondering why Apple uses the vault structure as a back up in the first place since its different that a working library? Is there some kind of advantage to doing it that way that I should keep in mind? I guess if I were only moving data one way, I could let the darn thing run over night/weekend and make the backup that way?

Thanks,
Eric
__________________
Eric W. Hiss
www.eh21.com
Reply With Quote
  #9  
Old December 23rd, 2006, 10:54 AM
Stan Jirman Stan Jirman is offline
Member
 
Join Date: Jun 2006
Location: Santa Clara, CA
Posts: 189
Default

I can't speak for the Aperture vault system. I would hope the documentation would explain it; I don't use the vault because I use rsync, as described.

I wasn't aware you were trying to sync two ways; you wrote "back up": to me, that's a one way thing. Syncing two ways is far harder to do correctly, be it a photo library or your address book. However, using rsync -va --delete should do a perfect two-way sync as long as you always have one master library. It will not merge data. As in, you work on Library A, and when you finish that work you sync A->B. Then you work on B, and when you finish you sync B->A. However, it will not work when you work a bit on A and a bit on B - there's no way to sync this, unless you write your own tool. This would require internal knowledge of how the Aperture DB works, which I doubt you have :)
__________________
Stan Jirman
http://www.jirman.com/
Canon & Apple equipment
Reply With Quote
  #10  
Old December 23rd, 2006, 01:57 PM
Eric Hiss Eric Hiss is offline
Member
 
Join Date: Nov 2006
Location: San Francisco
Posts: 275
Default thanks again

I'll just cart the main working RAID to and from the studio and leave the backup volume here. It's not that hard to do the updates one way only.

Yes, you are correct that I don't know much about how aperture structures their database and I'm not sure I really want to know but the manual does not do much of a job explaining anything technical. I believe the assumption that Apple makes is that one can't do anything else since they sort of force the vault so why does anyone need to know.
__________________
Eric W. Hiss
www.eh21.com
Reply With Quote
  #11  
Old December 23rd, 2006, 04:37 PM
Asher Kelman Asher Kelman is offline
OPF Owner/Editor-in-Chief
 
Join Date: Apr 2006
Posts: 33,941
Default

IOW Erik, you can only do new work on just one of the libraries and then you must synch before you work on the other library.

As long as you synch A to B or Bto A after each us, there should be no problem.

Asher
Reply With Quote
  #12  
Old December 23rd, 2006, 09:08 PM
Stan Jirman Stan Jirman is offline
Member
 
Join Date: Jun 2006
Location: Santa Clara, CA
Posts: 189
Default

Using rsync gives you the peace of mind that the copy is an exact replicate of the original. So whatever is in the Aperture library, you got at the other end as well. The Aperture vault has the benefit that it knows what was backed up, when, and where, and it can use that knowledge in some contexts. The trade-off is that you need to trust it, and do it on its terms. Personally I am a control freak and trust rsync because of its simplicity and me knowing exactly what I get - nothing more and nothing less.
__________________
Stan Jirman
http://www.jirman.com/
Canon & Apple equipment
Reply With Quote
  #13  
Old December 25th, 2006, 08:19 AM
BobSmith BobSmith is offline
New Member
 
Join Date: Jun 2006
Location: Waco, TX USA
Posts: 16
Default Chronosync?

I have Aperture but I've done little more than experiment with it a bit. I'm not sure if Chronosync would be appropriate for Aperture but it sure has served me well for all sorts of other bi-directional syncs for years. Chronosync is essentially a GUI frontend for rsync and other utilities/filters working in tandem. It handles bi-directional syncs very logically and safely. Whether or not all of this works with an Aperture vault is beyond me, but might be worth investigating.
__________________
Bob Smith
Accurate Image
Reply With Quote
  #14  
Old December 25th, 2006, 09:24 AM
Andrew Rodney Andrew Rodney is offline
Member
 
Join Date: May 2006
Posts: 467
Default

I've been somewhat successful in keeping two LR databases sync'ed and doing the switch between them after updating with a product called ChronoSync.
__________________
Andrew Rodney
Author "Color Management for Photographers"
http://www.digitaldog.net/
Reply With Quote
  #15  
Old December 25th, 2006, 10:30 AM
Asher Kelman Asher Kelman is offline
OPF Owner/Editor-in-Chief
 
Join Date: Apr 2006
Posts: 33,941
Default

Quote:
Originally Posted by Andrew Rodney
I've been somewhat successful in keeping two LR databases sync'ed and doing the switch between them after updating with a product called ChronoSync.
Hi Andrew,

Happy Christmas to you and to everyone else calebrating today!

Do you have any particular routine to do the bi-directioanl synching or is this just straightforward "out of the box", so to speak?

Asher
Reply With Quote
  #16  
Old December 25th, 2006, 10:32 AM
Stan Jirman Stan Jirman is offline
Member
 
Join Date: Jun 2006
Location: Santa Clara, CA
Posts: 189
Default

Quote:
Originally Posted by BobSmith
I have Aperture but I've done little more than experiment with it a bit. I'm not sure if Chronosync would be appropriate for Aperture but it sure has served me well for all sorts of other bi-directional syncs for years. Chronosync is essentially a GUI frontend for rsync and other utilities/filters working in tandem. It handles bi-directional syncs very logically and safely. Whether or not all of this works with an Aperture vault is beyond me, but might be worth investigating.
That's a key observation: it's a GUI on rsync. I know rsync inside out, and I can assure you that it will not be able to correctly do a two-way sync of the Aperture database. It would take me about 5 minutes to construct a counter-example of two databases that would get destroyed by a two-way sync.

So you may get away with it, but in the general case, it won't work and you'll be hurting.
__________________
Stan Jirman
http://www.jirman.com/
Canon & Apple equipment
Reply With Quote
  #17  
Old December 25th, 2006, 10:55 AM
Andrew Rodney Andrew Rodney is offline
Member
 
Join Date: May 2006
Posts: 467
Default

Pretty much out of the box. I have all images and folders in one location (Pictures). That's ALL I have in that folder, the stuff I want in my LR library.
__________________
Andrew Rodney
Author "Color Management for Photographers"
http://www.digitaldog.net/
Reply With Quote
  #18  
Old December 25th, 2006, 10:57 AM
Stan Jirman Stan Jirman is offline
Member
 
Join Date: Jun 2006
Location: Santa Clara, CA
Posts: 189
Default

Quote:
Originally Posted by Andrew Rodney
Pretty much out of the box. I have all images and folders in one location (Pictures). That's ALL I have in that folder, the stuff I want in my LR library.
What you are describing doesn't sound like a two way backup, though, does it? Do you then go and mess with the LR ribrary, and try to copy it over to the master, while that one has changed as well? I doubt it :)
__________________
Stan Jirman
http://www.jirman.com/
Canon & Apple equipment
Reply With Quote
  #19  
Old December 25th, 2006, 11:34 AM
Asher Kelman Asher Kelman is offline
OPF Owner/Editor-in-Chief
 
Join Date: Apr 2006
Posts: 33,941
Default

It seems then a simple thing to do is to use this Chronosynch GUI of rsynch to backup the Aperture library and then back the copy up and store offline.

Asher
Reply With Quote
  #20  
Old December 25th, 2006, 12:13 PM
Stan Jirman Stan Jirman is offline
Member
 
Join Date: Jun 2006
Location: Santa Clara, CA
Posts: 189
Default

Quote:
Originally Posted by Asher Kelman
It seems then a simple thing to do is to use this Chronosynch GUI of rsynch to backup the Aperture library and then back the copy up and store offline.
Sure. While I don't quite understand why doing a 2-level backup before offlining it, there's certainly nothing wrong with it.

All I want to do is caution people from the misbelief that a 2-way sync of an Aperture library is possible with rsync or related GUI tools; it is not. Aperture will do its best to resolve the mess that would result from such a 2-way sync, but information would get lost simply by the way rsync works, and there's no substitute for missing information. Nor do I know of any tool that can actually do a 2-way sync of Aperture library, which I believe was the original question in this thread.
__________________
Stan Jirman
http://www.jirman.com/
Canon & Apple equipment
Reply With Quote
Reply

Bookmarks

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are Off
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -7. The time now is 01:55 PM.


Posting images or text grants license to OPF, yet of such remain with its creator. Still, all assembled discussion 2006-2017 Asher Kelman (all rights reserved) Posts with new theme or unusual image might be moved/copied to a new thread!