Gathering, consolidating, sorting, keywording, backup and temperature control of massive photocollections

[Asher Kelman]

More than a few of us have accumulated massive collections of photographs and filled countless drives. I only sorted based on the current project and delivering what was promised. Now I am scraping every source for files and sorting by dates and keywords.

I have an M4 Max Studio Mac connected by
Corning Gas’s Thunderbolt cables to my 100 TB NAS in the much cooler garage some 30 ft away. In thr coming days I will share my setup and reasons for my choices.

Today I drilled many holes through the house structural 2x4 framing to accommodate Corning Thunderbolt optical cables, (1/8” thick shown as a pair traveling together and obviously more delicate than the two thick shielded ethernet 6A cables m.

A miniaturized magic laser generator, (hidden in the ordinary l-looking but slightly longer USB-C type male connector), magically converts the electric Thunderbolt 5 signal from the Mac or NAS to a series of light pulses.

This transmission is very reliable and not as limited by cable length as regular copper TB cables. This is 40 Gb/sec and two cables can be multiplexed to 80 Mb/sec.

However at nearly $400 per 45 ft length, I can only afford one such cable to my NAS and another a 10 bay SATA JBOD enclosure to feed in files, (from drives filled over the years), to make sure that my NAS has at least one copy of each file.

 

[Asher Kelman]

Tomorrow I will start to explain choices of computer, software, challenge of consolidating 2 decades of studio and street photography and how I selected storage, backup and most importantly viability of drives subjected to heat load 24/7 outside of a refrigersted data center.

Then I will cover keywording, collections, backup and AFFORDABLE offsite safe backup with a commercial service. Then I will address use of Mylio to make everything available anywhere you happen to be!

FEEL welcome to add your comments!

Asher

 

[Doug Kerr]

Hi, Asher,

Quite a project!

Is the white stuff I see some sort of filler you put around the cables as they passed through the studs? Or perhaps it was just to make a smoother hole after the holes had been bored through the studs?

I am fascinated by the 4x4 box at the upper left. Just what is that?

Best regards,

Doug

 

[Robert Watcher]

Wow. You must have an unbelievable body of work.

1 have a 16TB WD standard hard drive that sits beside my MacMini, for storage and access to all of the photography I have shot digitally since 2004 when I went fully digital —- which included all my pro jobs through the years, and years of shooting almost daily in Central America —- and tons of duplicates in various sizes, and still have 1-1/2 TB left on it. That drive is backed up to 4 small 2-1/2 inch inexpensive 4TB hard drives that sit in my desk drawer, that I used to store files while travelling. - and the full 16TB drive is backed up to the cloud daily for $15/month using Backblaze. I have lots of video and web design files on the drive as well.? That’s about as complicated as I want things to be for me LOL. My calculation is that 16TB store around 1,600,000 10mb files (that’s about the largest for my jpeg files, although I have lots bigger). I don’t know how many files I have on the drive, but already way more that I want to sort through and find out. A person can really go down a rabbit hole managing storage for their files.

All the best on this project

 

[Ted Cousins]

Wow. You must have an unbelievable body of work.

1 have a 16TB WD standard hard drive that sits beside my MacMini, for storage and access to all of the photography I have shot digitally since 2004 when I went fully digital ...

All my stuff is on a 1 TB conventional hard in my oldish 4-core Dell desktop.

I've IPTC/XMP key-worded a good few using XnView MP ... a work in progress ...

 

[Asher Kelman]

Your choice if the Olympus was correct. The file size, like computer software has bloated for no justifiable reason but wisely you didn’t fall for it.

It reminds me of the small fast efficient English Royal Navy, (of Queen Elizabeth the First), who out manured the massive war galleons the Spanish Navy.

Her privateers had captured several previously laden with gold stolen from South American native peoples. That’s gold financed her modern disciplined highly trained navy.

The Spanish we blasted from all sides set on fire and then forced Centuries the English ruled the waves.

So essentially the Olympus, your camera, warned to all living, allowed you to celebrate the magnetism, artistry, joy and dignity of the South American peoples in beautiful color and enough detail.

Yet you did it in less than 16 TB, a fabulous achievement!

While I am burdened With 100 TB of files such that I have to deal with the heat as an issue and the far greater cost of safety and backup! Yes, the GFX, 50 MO and 100 MO files are the most robust and the lens are out of this world but I dream of the wonder of using a much lighter weight 300 mm Olympus optic, behaving on your camera aspect 600 mm, and enjoying birding.

This is a proof that size doesn’t always overcome mobility!

Asher