Reading Time: 4 minutes

Over the years I have used Aperture, Picasa and the Apple Photos Apps. In that time they have organised my files chronologically, automatically, as soon as I took pictures, in some cases.

What They Do

Aperture was well behaved. It would organise photos by year, by month and by day, so it’s easy to migrate a library from drive A to drive B. Apple Photos on the other hand makes a pig’s breakfast. It renames the files with a chaotically huge number, and then moves files into folders from 0-9 and then from A to F or some similar chaotic mess.

The Issue.

If you want to migrate an aperture or Photos library from one volume to another it will take hours, despite there not that being much data. That’s because Aperture and Photos create preview files of different sizes, caches and plenty of other files. The result is that you’re not moving x number of photos and one or two json files with the appropriate metadata. You’re moving 200,000 files within that library folder.

The 500 Gigabytes of photos that you want to move, and that would take up to five hours to move, if they were just photos in folders then take 24 hours or more.

The Cause

There are two principle reasons for this collection of aperture libraries. The first is that for a while I had a mac book air for daily use, and a mac book pro for video editing. As a result I had two libraries simultaneoulsy. The second reason is that I would backup the laptop to external hard drives every so often, and in doing so I would have several versions of my photo libraries.

Time machine is also partially responsible because it creates multiple copies so you need to reconcile the differences between versions, to avoid losing files that are not backed up.

The Solution

If you have aperture libraries that have not been converted to Apple Photos folder structure then you’re in luck. In my case I opened up each library and moved the folders containers from within the package to an external folder structure where I kept the chronological organisation. I methodically worked my way through several years of photos within half an hour to an hour, and then told Finder to move the files from Drive A to Drive B. It told me “about one hour remaining” so I took the time to write this blog post.

In the Mean Time

One of the funniest things I have noticed, while playing with my video and photo archives is that I have not seen some people that I have forgotten many of their names. It is for this reason that I need to keep at least one Photos photo library, until I have renamed faces that are recognised on Immich, or PhotoPrism, before deciding what to do with the old Photos libraries.

The Next Stage

Out of curiousity I tested to see whether I could import the experimental photo folder structure into photos and I saw that I can, and that duplicate detection works.

Combining Old and New

At the moment I have three photo libraries. I have the Google Photos and mobile phone based on on PhotoPrism. The next photo library is the one that I got out of extracting photo galleries from Aperture and Apple Photos. The final library is the one that is based on the files and folders that I have from storing files manually, outside of Photo management apps.

The next step is to clear a four terabyte drive. It will be dedicated to photos and audio books. PhotoPrism will take care of the photos, and AudioBookShelf will take care of the books.

Why I Chose Four Terabytes

I want room to expand. When experimenting with one terabyte I found that my photo library immediately fills the entire drive and when I tried with two terabytes I feel that with audio books I will be tight on space. With four terabytes I can have one terabyte for photos, one terabyte for books, and two terabytes for the libraries to grow, without having to swap the drives.

The other reason is easy backup. I plan to free storage space on at least two four terabyte drives with the newest being the primary drive and the older one being a backup. If one fails the second one will take its place.

The final reason is price. Four terabyte drives have the best price. They’re cheaper than smaller and larger drives per terabyte.

And Finally

In the past we would go out, take photos and video and when we got home we woulc create a folder with the name of the activity. Over time we would have plenty of folders but everything was organised, by default.

In the modern era our phones and cameras do all of this for us. They add the date, the location and more automatically so we don’t organise anything ourselves. The result is hundreds of folders organised by year, month and day, but without any further information. That chaos makes it so that we need Photos, PhotoPrism and other solutions. They “organise” our files.

Conclusion

Photos, by Apple, and other apps ingest our images and organise them out of sight, which is great when we’re using their apps, but awful when we’re trying to use another software solution. It makes sense to have a drive with two folders. Photos and videos, with everything organised by year, month, day and project or activity name.

In so doing we can see in the finder, which files are duplicates, or missing, within seconds. We need to organise our files, and software should just help us look through our archives.

https://www.main-vision.com/richard/blog/of-photos-aperture-and-sliding-between-volumes/

#aperture #Apple #dataMigration #Google #mediaAsssetManagement #timeConsuming

Reading Time: 4 minutes

Imagine for a second that you have six hard drives filled with Data. Some are four terabytes. Others are one terabyte each, and you’ve already sorted personal videos and photos from other media files. Each drive is moved to its own folder

Two Terabyte Seagate has gone from being a drive to a folder. Bob 2017 has also become a folder, rather than a drive. Samantha 2018 is also a volume, rather than a drive. There is a reason for congregating the media from smaller files to a large volume and that is speed.

If you want to organise photos into a “photos” folder, videos into a “videos” folder and then organise those photos by year, initially, then moving files and folders from location A on a drive to Location B on a drive takes seconds.

To be more specific, you have volume A, B and C with files in them, and there is a lot of overlap of files. You could compare the files in three locations to make sure that there are no duplicates across a spaguetti junction of physical drives but by centralising everything you have a single volume, whether it’s a raid or a hard drive.

When everything is on a single volume organising files becomes as easy as creating folders, and moving files to the right folder. Video files from 2024 all go into videos/2024, videos from 2016 go to videos 2016. You may notice that I’m not going through the year-month-date yet. That’s because I find that itteration is faster. The aim is to identify the duplicates fast, and delete them from the aggregation/consolidation drive. Having three or four copies across three or four drives makes sense, for data recovery. On a single drive it’s a waste of space.

Once duplicates have been identified and got rid of, then time can be spend in uniformising date format and file names. Remember, eventually you can move the well organised files back to a smaller drive as a low cost backup.

10 Terabytes Become Four

This figure is an imaginery one. The point is that if you have data across 6-8 drives, and their storage amounts to 10-12 terabytes but a lot of that data is duplicates then the real space needed and used is lower. You should have two local backups and one off-site backup.

By copying data from multiple drives to a single drive it becomes easy to get everything organised, and one it is organised you can dump that data, in an organised manner, back to external drives.

For example you could have a drive for photos from 2010-2024, and another for videos from 2020-2024, and so on. Usually I don’t print labels for drives so it can get confusing. That’s where I like to use post-it notes. They’re cheap, and versatile. They need to last only until you finish organising your files. In theory you could print proper labels for drives but post-its are quick, cheap, and easy to use.

If I had a label maker I might print labels. I would consider printed labels once things are finalised, rather than when they’re in flux. Post-its are good for constant change.

Knowing What You Have and Where

If you backup from your laptop to drive A when you run out of space, and then drive B when you run out of space you have duplicates, triplicates, maybe even five or six copies of the same file. The problem is that because it’s decentralised it’s easier to back everything up and be safe, than assume you have a file or folder backed up when it isn’t.

By aggregating smaller drives to a big drive you gain control of your former chaos. You go from thinking you need ten terabytes to realising you need four to six terabytes instead.

Two Motivations

The first motivation for finally doing this properly is that I noticed a few years ago that I had lost track of hundreds of files from the uni years and I wanted to recover them. Between Picasa, iPhoto, Aperture, Google Photos and other solutions I lost track of these files. Now that I have regained track of them I can take advantage.

My second motivation is that PhotoPrism and Immich look like interesting solutions. In the good old days I had so few photos that they fit on my laptop’s drive with ease, but in the age of having a camera with us everywhere we go we end up with thousands of photos. These take up space, and by having a self-hosted solution like PhotoPrism and Immich we can keep track of these images with ease.

Estimating Cost

For the sake of argument let’s say that you have twenty drives. and they vary from 750 gigabytes to 8 terabytes in size. In theory you would assume that you need a 30+ terabyte raid to backup all that data. The issue is that this 30 terabyte raid costs hundreds, if not thousands of francs. If you regain control of how much space you need, on smaller volume drives then you get a better idea of how much storage you need.

An empty four bay synology device costs over four hundred francs, and that’s before you get disk drives. With the disk drives you get to 1500 CHF. I am not against getting a Synology or other device. I’m encouraging people to get into good habits, to ensure that there are two local copies, and a third off-site backup, rather than 15 drives that all have similar files.

 And Finally

I chose to write this blog post today because yesterday I suddenly felt overwhelmed by the amount of data I felt I still had to consolidate and the little amount of space I had, relative to the requirement. Initially my idea was to dump all the data from the smaller drives to the central server but I don’t have that much storage.

By organising files by photos, documents, and videos, as well as then going down to organising them by year I can quickly detect and delete duplicates. This helps me streamline how much storage I need, and I can then backup that data by photos on one drive, video on another, and documents on a third, for example. I could also organise them chronologically.

By organising the video and photo files on a single volume though I prepare it to be indexed and catalogued by either PhotoPrism, or Immich, or both, to see which one I prefer over time.

https://www.main-vision.com/richard/blog/organising-terabytes-fast/

#comfort #control #data #mediaAsssetManagement #photos #videos