WriteFreely + PhotoPrism

Współpraca z Immichem pod kątem wrzucania jakoś z automatu zdjęć z udostępnianych albumów szła jak po grudzie. [...]

https://blog.argilus.online/czach/writefreely-photoprism

Using Photoprism and Immich in Tandem Across Three Machines

I have an HP laptop running Photoprism, and two Raspberry Pi 5 running Immich. One of the Pi devices is running Immich, solo, and is made accessible via Tailscale when I am on the move. Photoprism runs on the HP laptop and is not available at all times. When the laptop sleeps, Photoprism is not available. This means that I have to consciously decide to ingest the photos to photoprism.

Propagating from Photoprism to Immich

I setup Photosync on my phone to upload the most recent photos and organise them by year and month. This is easy to forget, so I consciously set that up. The reason for this decision is two-fold. The first reason is that by organising photos in this manner it is easy to mirror them to multiple drives, as well as to make them accessible to Immich, and Photoprism on the second Pi I mentioned earlier.

At the time when I set this up I was using the Ubuntu machine daily so every morning I could upload the photos with photosync.

Woupsie Daisies

Recently I allowed the HP laptop to run out of battery, and when it rebooted everything seemed fine, so I went to sync between devices and that’s when the “woopsie daisisies” incident occured. The “Woopsie Daisies Moment” was both catastrophic, but luckily recoverable. The SD card that I use within the laptop didn’t mount properly but I didn’t notice. When I told rsync to sync from the HP SD card to the external hard drive locally, and to Immich on the Pi it deleted all the files on the Mac spare drive. I suddenly noticed that Nextcloud had no more photos so I killed the sync within seconds. When a delete command is sent it goes for a while before the kill command was received.

The impact was that my secondary backup was wiped, and that my cloud backup, as a result was also in danger. When I realised I unplugged drives, safely, to isolate them, until I understood what happened.

Drive not Mounted

When I looked at the mounted drives on the HP machine I checked to see if the external hard drive was mounted. It was, but the photos were gone. It occured a few weeks ago so I don’t remember if the Pi5 was also wiped, but I don’t think it was. I then realised that the primary drive that I was using had not mounted after the dead battery incident. I mounted the drive, and the photos were intact.

A Sync Shell Script

To avoid this mistake from occuring again I made two key changes. The first is that I removed –delete from the rsync commands. If a file is not present rsync will not delete anything. Accidental deletion is almost instanteneous. The second change was to make sure that the shell script checked that the drives are mounted before doing anything. If the drive is not mounted it aborts the transfer.

Now I sync from the local SD card to the local external hard drive. When that’s done I sync the external hard drive to the Pi5 currently running Immich before then syncing from Linux to MacOS and my kDrive cloud instance. Within two or three minutes I have things backed up across the HP machine, the MacOS machine, a Pi5, and Infomaniak.

The Standalone Immich Backup

I have an entirely seperate workflow for my second Pi5. I synchronise to this Pi5 via the Immich app directly. In theory, within seconds of taking a photo it can be synced and indexed by this Pi5. If the Photoprism and Immich suffers an issue such as a describe above, this one is entirely seperate and isolated. A mistake with one worfkflow only affects one workflow.

The Tertiary Backup

The primary worflow goes from Photosync to Photoprism, and then Immich across multple machines and drives. The secondary worflow is from the Immich iPhone app to the Immich instance on a seperate Pi. The tertiary backup is kDrive. All of the photos are automatically backed up by kDrive in the year/month folder structure, so if the primary and secondary workflow fail, I can still revert to the tertiary solution.

Each library uses the same structure therefore migration data from one library to the other should be seamless. This gives me the opportunity to continue learning, and if I make a mistake I have several recovery solutions.

And Finally

The key mistake was not to check that the volume was available. If I had done that, then I would not have run rsync. I have removed that risk by replacing manual operation with a shell script. The second mistake was to have a delete flat in an rsync workflow by default. I removed it. The third mistake was to allow the battery on the laptop to run flat. That’s why when the machine was rebooted it did not mount correctly.

If you see that files are missing, the first thing to do is check that the volume is mounted. If I had done that, then I would have seen it wasn’t and I could have mounted it, and the issue would have been resolved within seconds.

The current workflow is now more resilient, and automated.

The "convenience" of Google Photos often comes at the cost of your privacy. If you're tired of big tech scanning your family memories to build a data profile, it's time to switch.

PhotoPrism uses AI to tag your photos and recognize faces locally on your own instance. Same smart features, zero data mining.

Host it on PikaPods from $6.20/mo (sponsor features included): https://pikapods.com/pods?run=photoprism

#OpenSource #Privacy #PhotoPrism

The latest PhotoPrism update introduces enhanced Ollama configuration options, improved security, and multiple bug fixes related to indexing, folders, and metadata.
https://linuxiac.com/photoprism-ai-powered-photos-app-brings-better-ollama-integration/

#photoprism #opensource

Migrating Photoprism From One Machine to Another

Due to the Raspberry Pi 5, and older, having issues with heat throttling and more it makes sense to build a Photoprism on a "normal" laptop before migrating towards the Pi. The process is an interesting one.

Photo Consolidation

The First step is to consolidate your photos from Google Photos, Apple Photos, Flick and any other source you might have. The simplest method is to organise them chronologically, and then to spend time removing as many duplicates as possible. There are tools for that. They will help you add exif data back into the photo exif fields, as well as look for duplicates.

Photo Ingestion

If your library is organised and ready there are three folders that are interesting to us.

• ./photoprism/originals:/photoprism/originals
• ./photoprism/import:/photoprism/import
• ./photoprism/storage:/photoprism/storage

Originals

The originals folder is where you put your chronological library. Photoprism will automatically index all the files in this folder. If you have decades of photos this takes time. It applies machine learning to catalogue dates, locations, people, objects and more.

Storage

The storage folder is critical because this is where json, yaml and other metadata files. This is also where the thumbnails are stored and this is key for migrating from a high spec laptop to a limited spec Pi. When I migrated this folder I had 1.8 million files for 110,000 photos.

Import

The import folder works as a stepping stone. It allows you to import photos and videos at a later date. If you select "move" then it will import photos, and then move them out of the folder leaving it empty.

Phone Ingestion via Photosync

Photosync is a partially free app. If you pay once you can unlock more functionality. With Photosync you can setup photoprism syncing. You add a configuration title, for example "laptop" and then you select the destination folder, and make sure to use "create sub-directories yyyy/mm/dd to preserve the hierarchy you spent hours preparing earlier.

Be careful, because usually the default is Device Name + Album Name. There are other options but they are out of scope for now.

The advantage of Photosync is that once it is set up correctly, it will keep uploading photos according to the hierarchy you want, eliminating the need to do things manually.

Moving to the Pi 5

Once all the heavy lifting has been done, and the logs say that tasks are completed you can move everything to an NVMe card or external hard drive depending on budget, and convenience. With an NVMe card and a 400 gb library you still have head room. With a 2-6TB hard drive you have plenty of head room.

It's worth keeping in mind that the docker-compose file for an intel machine and a Pi are different so you will need to find a template for the Pi and ARM architecture.

I like to have a folder /apps/photoprism/ for the docker compose file on the SD card. I also have /apps/photoprism/database on the HD, that was prepared on the laptop, as well as /photoprism/photos, photoprism/storage and photoprism/import on the external hard drive. I prefer to keep the photos and DB separated.

The Thumbnail Mistake

When I migrated from the laptop to the Pi I didn't bother to copy the storage folder because I thought "the thumbnails won't take too long to generate". I quickly realised the error of my ways, shut down the Pi, rsynced the files, and then plugged the drive back in before rebooting the file. Within seconds Photoprism was happy.

If I had let the Pi regenerate all of the thumbnails and video files it would take hours, if not days and weeks.

The Vanished HEIC Files

According to the laptop library I have 119,000 photos but according to the Pi5 I only have 110,000. That's a huge difference. I noticed that the photos that were missing were those taken with a phone, specifically HEIC files. I tried uploading them as jpeg and they appeared almost instantly.

The quick solution is to upload the photos as jpeg rather than HEIC via Photosync. The slow solution, and this isn't a solution, since it makes the Pi unusable for hours is to reindex the library. I am attempting this now.

The Thumbnails Exist

the paradox, in this situation, is that Photoprism, on the intel based laptop already did all of the heavy lifting so Photoprism should just have seen that each photo had a thumbnail and reflected that in the index, instead of re-inventing the wheel.

And Finally

The Good

I was able to migrate my library from one computer to the other and it worked almost flawlessly except for the HEIC issue. If I had transferred jpeg images I would have had a flawless migration.

The Mistake

The mistake is to re-index the entire library, especially in light of the limitations of rRaspberry Pi devices.

The elegant Solution

The elegant solution is to revive an old phone with all these photos, make sure Photosync is installed and setup, and tell it to upload to the Pi5 as JPEG rather than HEIC. It takes more than five minutes, but it worked instantly with a few test photos.

Silver Lining

The Proof of Concept migration was a success. If I had moved between two intel devices it would have been flawless. It's because I migrated from one architecture to another that I hit a little snag. I simply noticed it while writing the blog post.

Frustrated by Google Photos' privacy trade-offs? 📸

PhotoPrism brings AI-powered organization—like face recognition and world maps—to your collection without the Big Tech tracking.

On PikaPods, you get the full experience with all Sponsor features included, starting at $6.20/mo. Keep your memories, lose the surveillance.

https://pikapods.com/pods?run=photoprism #PhotoPrism #Privacy #OpenSource