hey it's that time again.

do you have data? that you like? is it backed up? do you have automatic backups you've checked are actually happening?

in multiple locations?

have you done a test restore to make sure the backups work?

*even an extra copy on a flash drive is better than no backups*

reminder that flash storage is not long term storage
@gloriouscow If you’re planning long-term enough that flash data retention is an issue, then it’s not like spinning drives are long-term storage either. The only substantial step up from flash is a live system which you regularly scrub and service.

@bob_zim there was no intent behind it perhaps but i can't help but think about how much data vital to computing history was recovered just because someone found a shoebox full of hard drives

if someone finds a box of m2 drives in 40 years they will just be unreadable e-waste

like if you chart the amount of "unintentionally preserved history" over time it will just plummet in a corresponding line to the adoption of flash storage.

that's kind of sad :(

@gloriouscow Most current hard drives will also be bricks over that time span. And most tapes. Outliers will exist, but that’s survivorship bias.

I think the prevalence of encryption is also likely to reduce the recoverable data from today.

@bob_zim @gloriouscow Spinning drives do do better in the medium term, especially if they remain powered-on. As always, multiple copies with ECC or hash-check is wise.

@shelldozer @bob_zim

the miniscribe hard drive in this IBM 5170 I bought spun right up and I was able to read this guy's creepy letters about his ex wife.

actually this is not a good argument for data preservation i'm sorry

@gloriouscow It's still, what? Like plug it in every ten years or so for a few minutes?

I see a lot of fear mongering, but flash RAM doesn't degrade as quickly or as easily as people seem to think. It's not infinite by any means, but it really does take a lot longer to degrade than the claims going around much of the Internet.

Nothing is infinite. Magnetic drives lose magnetism over time and capacitors and such in each can degrade. Optical discs make claims of 100 years for good ones, 300 for super archive discs, but they need to be burned at a low speed in a high quality burner and stored very well. In 100+ years will they be able to read an optical disc though? (I mean, presumably whatever tech people have could adapt to it, but will people have it in their homes? Probably not.)

@nazokiyoubinbou

if you are put together enough and organized enough to have some sort of yearly flash-device-powering holiday (that implies you can even find what boxes they are all in) then I salute you on being someone who's brain I would basically find incomprehensible

@gloriouscow I didn't say yearly. It takes a whole heck of a lot longer than a year... That's my point. I forget how often you need to do it, but it's crazy high. Like five years to be super safe, probably 10+ to be realistic.

A couple of months ago I dug a MP3 player out that I last used in something like 2012, just before I switched to a smartphone instead. The old full sized SD flash card in it still was 100% readable with zero errors as was the internal flash (including its firmware of course.)

(As a side note, fun fact, but somehow the battery wasn't wrecked despite being below a safe voltage for who knows how many years... It's probably like 60% of what it once was, but it still holds a decent charge. Not bad for a player I bought in something like 2008...)

@nazokiyoubinbou

for some reason i find your singular anecdote completely convincing and retract all my arguments

@gloriouscow Then look up the whitesheets and etc if you like... I don't see what the rudeness has achieved.

All I'm saying is that it does last longer than just a few years as people seem to have gotten the idea and fearmongering that flash media is doomed to die horribly and quickly won't help anyone. As the OP says, a flash backup is better than no backup.

@nazokiyoubinbou

nobody is fearmongering, i'm making a point about decades, you're talking years. we're talking past each other, i'm just noticing it and choosing not to continue

@gloriouscow I'm talking about decades too.

@nazokiyoubinbou

so your solution is that everyone on earth keep track of every single flash device and power it on once a decade and this somehow makes my original statement worth objecting to

@gloriouscow You're seriously throwing a lot of hyperbole in there. My solution is "that drive with some important backups in the fire safe should be plugged in for a few minutes once in a while." That is not complicated.

My objection is to the claim that flash drives are useless for storage beyond a few years. I won't claim they'll last 100+ years. I will claim they'll last long enough to do really basic backup tasks. Enough so that someone shouldn't just give up on having backups because the Internet as a whole has been passing around claims that flash drives die really fast in storage but they don't have any M-discs handy.

@nazokiyoubinbou

man all i said was that they aren't long term storage. this does not need to be a whole thing

i am going to go solder some pins to some stuff and then probably post pictures about it

@gloriouscow Why? Technology wise, I know you're supposed to power the drive on from time to time to keep the charges from degrading, but every time a storage medium's failed me, it was always either a cheap USB flash drive or HDD. The first SSD I ever bought is still alive.
@NanoRaptor i read "do you have a date" and the more i read on the more confused i got.
@yetzt i have multiple tall muscular firefighter lasses with ukrainian accents stashed around the place. and i test them regularly!
@NanoRaptor counterpoint: I have 50TB of data. I can't afford to back it up
@freya similar tbh... but i do have a core of irreplaceable ones. My NAS is a little one, its backup drives little too, relatively.
@freya @NanoRaptor
Can you afford to lose it?
@kirtai @NanoRaptor I am on SSI. I do not have the money to dedicate to backing it up. it's not a tradeoff, it's a literal state

@freya @NanoRaptor i have on my personal pc ~500GB i chose that only 10GB of which are backed up with an exclude pattern: games, films, downloads or logs.

Look up if there is more important stuff in there and treat it with more special care 

@NanoRaptor I've got a daily backup to a USB external drive, a weekly backup to a NAS, a monthly-ish backup to a second external drive that's disconnected and stored in a more secure corner of the house when not being written to, and a once or twice a year backup to *another* external drive that I store at the house of a relative who lives in a different town. Plus another two full copies of my system on external drives intended less as a backup and more as a way of having the ability to get at anything I could possibly need while on my laptop, and another 2-3 portable SSDs with select items.

It's actually kind of a problem...

@lusrangifer if i'd hazard a guess i'd say you too have once lost everything due to no backups way back in the past...

@NanoRaptor I actually have not!

...but during my university days I worked part-time in the wing of the IT department that offered tech support to students, and that campus was rough on laptop hard drives.

@lusrangifer Oh that's grand then!

I lost everything in 2000, and only got it (mostly) back more than a decade later.

Still have The Fear!

@NanoRaptor
I have all my data backed up twice! Because it's in RAID, sitting next to my computer. Right?
@NanoRaptor - but remember to plug in and use the flash drives or SSDs every few months - I am seeing data corruption and loss on SSDs that I haven't used for only about a year. Spinning rust hard drives fare better than SSDs!
@NanoRaptor daily backups to a couple of USB hard drives I can grab and run with if the apartment’s on fire. Weekly backups to Backblaze, soon to be a 16tb drive sitting in a DC halfway across the world. I’d like to add an extra offsite copy when disk prices aren’t stupid.
@NanoRaptor I don't like it. I hope the disk fails and we can finally be free.
@ellie there is some aspect of this gods, yes.
@NanoRaptor I'm getting worried. Why is it "that time again"? I feel like it's been that time for months... What has changed?
@NanoRaptor "But Bob probably backed up his work and Sue backs up the shared folders. And the trainee said he sometimes backs up stuff too. You're just trying to sell us an expensive backup solution that we don't need."
@NanoRaptor My backups are great and tested tyvm (https://www.alkoclick.space/my-backup-strategy), but maybe some of the storage nerds in this thread can help me: How can I estimate bit rot on a modern SSD (I'll take any model) over a twenty year period of roughly ideal circumstances? I've been searching all day and I'm now down to academic papers proposing arcane optimization algorithms and still I fail. I must be missing something
My backup strategy

I am writing this document for myself, as well as anyone else that’s interested in taking a stab at backing up their homelab tools.

🔮 Enchanted Systems
@NanoRaptor No, but i will do it tomorrow.
@NanoRaptor also: do you have a way to get to that backup if you don't have access to what that backup holds? Any logins won't be helpful in restoring from backup if your password db is only in the backup. And 2FA can also get in the way if you're not careful.

@NanoRaptor

These are the correct questions in a nutshell.

@NanoRaptor
It's fine Google says they got me covered, even though it's a free account. 🙃

@NanoRaptor I store all my critical data* in the Fediverse!

*cat memes

@NanoRaptor in order:

yes yes yes and yes!

yes, but not enough. fixing that this weekend was already on my docket (for things arent local mirrors of easily available data, anyway)

yes, im actually running my home storage on the live backup as i write this (with "prod" as backup meanwhile) and will be switching back over to "prod" this weekend

@NanoRaptor

Four mirrored RAID arrays, everything copied to an internal disk, an external USB drive, and a NAS, plus cloud backup. Yes, I'm paranoid. 😆

@NanoRaptor

A timely reminder and honestly, one people don’t take seriously until it’s too late. Data care isn’t just about having backups, it’s about trusting they’ll actually work when it matters.

It does make me wonder: how do we get people to treat backups less like a chore and more like essential hygiene?

@NanoRaptor
I swallowed my flash drive and now I’m backed up.

@NanoRaptor Glad when I switched the last bastion of Windows, my home desktop, to Fedora recently, I did so while doing a storage upgrade/expansion.

Installing Fedora on #ZFS was a challenge, but thanks to ZFSBootMenu it worked after figuring stuff out, and thanks to syncoid/sanoid I now have incremental and full backups that get synced daily.

Still have to find a solution for offsite part of the 3-2-1 rule that does not break the bank.

@NanoRaptor darkling's first law: the user hates their data.

@NanoRaptor

Grab your chips and sing with me now the song of "7 Layer (backup) Dip":

- We do it Nightly
- We do it Weekly
- We do it Monthly
- We do it Quarterly
- We do it Yearly
- We do it Offsite
- We do it Offline

@NanoRaptor Good reminder, always 👍

@NanoRaptor I recently bought an adaptor so I can back up the box of hard drives I've saved from every single computer I've ever parted ways with - and a 22tb external.

I probably need a second 22tb external, now, don't I?

@NanoRaptor

How do you do a test restore unless you have two completely identical sets of hardware, one of which has nothing important on it?

(I did used to have two identical laptops, but that was in hopes it would help me troubleshoot for my dad from a thousand miles away.)

@NanoRaptor

I've been building a 3-2-1-1-0 system using a RaspberryPi and a SATA hat configured as an rclone mirror of an exFAT pair and a B2 offsite. My goal is to rotate through media every couple years making it easy enough a survivor will have the instructions of "Plug into USB adapter and any computer" or "Login to this website"

May be lacking in fancy ZRAID2, mdadm, hardware RAID mirror and tape system but I removed as many steps as possible to remove the need to get a separate technician or data recovery company involved.

@NanoRaptor Yes, it's that time again...