Okay, heeeeelp, how the fuck am I supposed to upload almost 230k photos from iCloud Photos to my NAS?

  • Keeping all metadata and original files (HEIC, HEVC, HEIF, etc)
  • Keeping live photos videos

Considering that:

  • Without using immich app because it's buggy as fuck and completely shits itself after a few uploads thousand
  • Without using the nextcloud app because it removes metadata from videos when autouploading
  • Without using iCloud Takeout because not only that separates all metadata from the files with the sole purpose of making your life harder requiring some script to fix which apparently only works on mac, but also that would require me to download God knows how many zips from Apples website totaling almost 3TB

Someone help should I just accept that I'll just lose 230k pictures because no one seems to make a software that works properly???

#immich #nextcloud #SelfHosting

PostgresError: duplicate key value violates unique constraint "UQ_assets_owner_checksum" Β· Issue #22742 Β· immich-app/immich

I have searched the existing issues, both open and closed, to make sure this is not a duplicate report. Yes The bug I have found similar issues, but they are closed. Unfortunately this is still hap...

GitHub

Trying this now, metadata seems complete!

Hope this works completely...

Apparently it'll need me to do one for the PrivateSync library and one for SharedSync, whatever are those (probably family shared library and personal one?)

https://github.com/icloud-photos-downloader/icloud_photos_downloader

GitHub - icloud-photos-downloader/icloud_photos_downloader: A command-line tool to download photos from iCloud

A command-line tool to download photos from iCloud - icloud-photos-downloader/icloud_photos_downloader

GitHub

Yeah, icloud-photos-downloaded worked well and is definitely the only way to do this initial upload. Re-enabled ADP now.

I’m now using only external libraries on immich, one for those and one for auto upload. For the auto upload, I’m trying using nextcloud to upload to an external folder that’s also imported on immich as an external library, let’s see if that works.

Nextcloud doesn’t seem to be actually removing the metadata from the videos, that was just a handful of old videos that seemingly lost their metadata when my father duplicated (maybe to take clips out of them? Idk) them.

This is what I used to run icloudpd, if anyone needs it. If using --xmp-sidecar you’ll also need to use a patched version of the program: https://github.com/icloud-photos-downloader/icloud_photos_downloader/pull/1290

icloudd(){icloudpd --size original --live-photo-size original --xmp-sidecar --set-exif-datetime --live-photo-mov-filename-policy original --align-raw original --file-match-policy name-id7 --password-provider console --mfa-provider console --notification-script /root/ntfy-icloudpd-2fa.sh --directory /mnt/pool1/PersonalFiles/Media/iCloud-dl --library PrimarySync -u '[email protected]' || icloudd}; icloudd

Run with --list-libraries so you can see all your libraries other than PrimarySync, you’ll have a second one if you have a shared iCloud library. Run that command one (or as many times as needed to completely succeed) time for each library.

Hmmm ok, already don’t like one thing about Nextcloud: icloud-photos-downloder is able to save screenshots with a β€œScreenshot” tag on the used lenses. Nextcloud doesn’t do this, which honestly will make filtering(out) screenshots on Immich a pain in the ass

Edit: oh and apparently it saves them without timezone data

And when a file errors it says it’ll try again but doesn’t and then doesn’t ever send anything else unless you clear cache. Ugh this’ll suck…

Ugh whatever I’ll get a postmarketOS phone eventually
Lmao Nextcloud saves screenshots without timezone data so they appear on a different time than the correct one

@luana
https://github.com/IsmaeelAkram/icloud-linux might help. I don't have any iCloud knowledge but a fuse filesystem was my first thought.

I'd aim to dump it into a physical drive and then copy from that into your final storage location. That way you don't have to go back to iCloud

GitHub - IsmaeelAkram/icloud-linux: Linux FUSE filesystem driver for iCloud Drive

Linux FUSE filesystem driver for iCloud Drive. Contribute to IsmaeelAkram/icloud-linux development by creating an account on GitHub.

GitHub
@luana i think most of these software things assume you have a mac device to begin the upload on because that's the only way to reasonably get access to the full image library
@niko I could use a macOS VM if it doesn't require downloading everything to the VM itself at the same time. Otherwise it'd require me to have a mac with at least 2TB free storage
@luana yeah it requires it on disk usually since then the images are stored in an easily accessible database

@luana I used this guide and it went smoothly: https://couteausuis.se/tinkering/2023/10/13/migrate-apple-photos-to-immich/

They use osxphotos for the Apple side and immich-go for the Immich upload.

Make sure, if you’re using a reverse proxy to access Immich, that you aren’t hitting an artificial upload size limit. That happened to me with Traefik until I disabled the limit.

migrate Apple Photos to Immich Β· couteausuis.se

@deancommasteven That'd require me to have a mac device with at least like 2TB of free storage
@luana You can export to a mounted network device. Sorry if I missed it though, do you not have a Mac available? With just an iPhone I’m not sure how else to get them that you haven’t tried.
@luana @selfhosting If you have a Mac, look into Parachute. It’ll copy your iCloud Photos to a NAS as a backup. I’d guess if you want to use immich, you’d just import from there.
@Aaron I could use a macOS VM if it doesn't require downloading everything to the VM itself at the same time
@luana I’m not sure about how much space parachute needs to use for downloads. But it may be worth a shot to try.