hey it's that time again.
do you have data? that you like? is it backed up? do you have automatic backups you've checked are actually happening?
in multiple locations?
have you done a test restore to make sure the backups work?
*even an extra copy on a flash drive is better than no backups*
@bob_zim there was no intent behind it perhaps but i can't help but think about how much data vital to computing history was recovered just because someone found a shoebox full of hard drives
if someone finds a box of m2 drives in 40 years they will just be unreadable e-waste
like if you chart the amount of "unintentionally preserved history" over time it will just plummet in a corresponding line to the adoption of flash storage.
that's kind of sad :(
@gloriouscow Most current hard drives will also be bricks over that time span. And most tapes. Outliers will exist, but that’s survivorship bias.
I think the prevalence of encryption is also likely to reduce the recoverable data from today.
the miniscribe hard drive in this IBM 5170 I bought spun right up and I was able to read this guy's creepy letters about his ex wife.
actually this is not a good argument for data preservation i'm sorry
@dalias @gloriouscow Even tapes don’t survive well unattended. The base gets brittle over time and fails. They last better than flash or spinning drives, but not *that much* better. I’ve been asked to digitize 30-year-old VHS and reel-to-reel tapes which just crumbled when I tried to examine them.
Long-term data preservation takes active work. If you want something to last decades, you need to build it to be fault-tolerant, you need to test it regularly to catch failures before they exceed the fault tolerance, and you need to fix the failures.
@dalias @gloriouscow SSDs stored in a controlled climate will generally also also retain data for decades. My point is the data loss concern for unattended flash is overblown (much like concern about flash wear), and other media types are also risky to leave unattended.
Basically anything but burnable optical media will last at least two years stored in a climate which doesn’t kill people. Store it with climate control, and it’ll last a decade, maybe two. Tape might last twice as long as an SSD, but it won’t reliably make it to the next bracket up.
@NanoRaptor I've got a daily backup to a USB external drive, a weekly backup to a NAS, a monthly-ish backup to a second external drive that's disconnected and stored in a more secure corner of the house when not being written to, and a once or twice a year backup to *another* external drive that I store at the house of a relative who lives in a different town. Plus another two full copies of my system on external drives intended less as a backup and more as a way of having the ability to get at anything I could possibly need while on my laptop, and another 2-3 portable SSDs with select items.
It's actually kind of a problem...
@NanoRaptor I actually have not!
...but during my university days I worked part-time in the wing of the IT department that offered tech support to students, and that campus was rough on laptop hard drives.
@lusrangifer Oh that's grand then!
I lost everything in 2000, and only got it (mostly) back more than a decade later.
Still have The Fear!
@NanoRaptor Answering my own question now that I understand the topic a bit better: The closest number to what I had in mind is UBER, Uncorrectable Bit Error Rate and for most modern HDDs and SSD is somewhere between 10^-13 and 10^-16. In practice the actual Raw Bit Error Rate on operations can be as low as 1/1000, but SSDs use a bunch of Error Correction Codes to validate and fix data as they read it, so you end up with the UBER number.
As a fun extra. the question as posed is kinda invalid: SSD longevity, even under ideal circumstances is heavily reliant on the load of read/write operations, so basically the amount of ops translates to SSD age, unlike HDDs.
Refs used:
* Data Longevity and Compatibility
* Data Retention in MLC NAND Flash Memory: Characterization, Optimization, and Recovery
https://web.ece.ucsb.edu/~parhami/pubs_folder/parh19f-ebdt-data-longevity-compatib.pdf
https://users.ece.cmu.edu/~omutlu/pub/flash-memory-data-retention_hpca15.pdf
This and other things about SSDs here: https://www.alkoclick.space/things-i-learned-about-ssds
These are the correct questions in a nutshell.
@NanoRaptor I store all my critical data* in the Fediverse!
*cat memes
Four mirrored RAID arrays, everything copied to an internal disk, an external USB drive, and a NAS, plus cloud backup. Yes, I'm paranoid. 😆
@NanoRaptor Glad when I switched the last bastion of Windows, my home desktop, to Fedora recently, I did so while doing a storage upgrade/expansion.
Installing Fedora on #ZFS was a challenge, but thanks to ZFSBootMenu it worked after figuring stuff out, and thanks to syncoid/sanoid I now have incremental and full backups that get synced daily.
Still have to find a solution for offsite part of the 3-2-1 rule that does not break the bank.
@elexia yes! and this is why i'll alway include just dumping stuff to a flash drive counts cos for some of us setting up more than that is functionally REALLY hard!
I lost everything in 2000 and it took until *2016* until I set up truly solid backups.
@NanoRaptor
Good post.
I have a Diskstation (called Daisy) with twin RAID HDDs.
I have a Diskstation (called Daphne) with a single HDD.
I have a Pi 5 (called Norah) with an NVMe on a hat.
All my vital data heads to all 3 of these via Rsync on a daily basis and I check it from time to time (no failures yet).
And I just got some Proton cloud storage so I'll send some stuff there too.
gwenh
Nanoraptor
gloriouscow
Zimmie
Mike Spooner
Cassandrich
ay
yetzt
Ra (Freyja) (it/its)𒀭𒈹𒍠𒊩
Kirtai 🏳️⚧️
Dragan Espenschied
Lady Lus, Rubber Bouiphant
Negative12DollarBill
Axel Hartmann
Andrew 
Ellie
ZorroX
Ari Sovijärvi
Alex Papageorgiou
snowyfox
Johan Sköld
fan boy 🪭
Cat 🐈🥗 (D.Burch) 
Lee Hauser
linear cannon
Ray McCarthy
Perrin42
👾👾👾
DaCool
Hugo Mills
Ms. Que Banh
Eric Peacock
Otis
Orca 🌻 | 🎀 | 🪁 | 🏴🏳️⚧️
Carlos Solís
refraction 
Jan Antoš
TheDyslexicKitchenwitch
ShadowInTheVoid
GrumpyDad 🇺🇦🇵🇸
Drew has moved to ieji.de