SD cards are the literal worst.
they've expanded to be the size of small hard drives, and devices like the rpi keep using them as boot media, but they:
- use garbage tier low endurance flash cells internally
- have little to no overprovisioning for wear
- perform only the most basic wear levelling
- have no protocol level integrity checking
- have few internal error correction features, if any
- decay comparatively quickly without patrol scrubs
- do not perform patrol scrubs
- cannot do PLP
something I've been thinking about a lot is engineering panic signals into embedded stuff.
put a rail supervisor on the incoming supply, and leave enough capacitance on the low side of your buck(-boost) reg to keep the device running for a few milliseconds when the power is pulled. treat the supervisor IC's PG output as a panic signal and have it trigger an IO interrupt on your MCU/SoC which tells the filesystem code to ensure integrity, flush, and halt IO.
@gsuberland That and ECC RAM.
Doesn't help that as laptops have gotten popular devs have gotten lazy at filesystem power loss support.
@gsuberland Yep this is something I've started to design into my architecture, prototyping on the trigger crossbar and then scaling to a next-generation system with more capabilities on my next project.
Luckily there's not a whole lot to worry about in terms of data integrity because MicroKVS is a) seldom written and b) designed to not corrupt data if interrupted at any point.
At some point I need to build a board where I can test that assumption: have one MCU that controls power to another and the target is just constantly booting up, doing KVS reads and writes, and being randomly reset/power cycled.
And see if I can make it break.
@apicultor what I mean is that if the goal is to get around bitflips and bitrot and write wear issues with ZFS, the correction itself will cause more write wear.
it's better than not using ZFS, but it accelerates the overall inevitability of complete failure.
@gsuberland Ah, legit, yes.
What about SD cards that don't suck — as in, ones that have their flash configured in SLC mode so you get bonkers durability like 5K TBW (for the largest ones, reducing proportionally by capacity):
https://www.transcend-info.com/embedded/product/embedded-memory-cards/usd230i
https://www.transcend-info.com/embedded/product/embedded-memory-cards/usd240i
Not quite as impressive, but still ~2.8K TBW if I did my math right (it's expressed in hours of HD video at 26 Mbit/s):
https://www.transcend-info.com/product/memory-card/usd350v
@kelpana @gsuberland USB-attached SATA or NVMe SSDs.
USB flash drives are the exact same (or often worse). Don't use them either.
@kelpana @gsuberland SD cards can be okay, if you use them read-only-only and buy from reputable vendors (e.g. Samsung).
If you really need R/W storage -- SPI NOR flash is a good option (together with proper wear leveling, etc.).
eMMC storage ICs are also available, often slightly more reliable and available from reputable vendors with a proper datasheet. They might work in some situations.
Otherwise, things get tricky...
No, NVMe won't work.
@raito @kelpana @gsuberland Yes, if you have a Linux kernel, you suddently have more good options.
"Microcontroller project" just didn't sound like Linux to me :)
@manawyrm @kelpana @gsuberland agreed; as we talked about RPIs in the start, I was considering the large spectrum of microcontroller projects
Nonetheless, I suppose that even ESP32 grade stuff could get access to a good implementation of something like UBIFS?
@raito @kelpana @gsuberland You can at least buy them from a proper source, with a datasheet and can be pretty much guaranteed to not get fake chips.
That helps a lot. Otherwise they're basically the same.
@gsuberland @raito @kelpana
Look at this datasheet: https://wmsc.lcsc.com/wmsc/upload/file/pdf/v2/lcsc/2007301503_Samsung-KLM8G1GETF-B041_C499918.pdf
You can also often configure/partition eMMC flash to work in pseudo SLC mode (trading size for reliability).
I know that at least some Micron eMMC has working built-in data refresh... Not sure about other vendors.
@gsuberland i wish they made small capacity (full physical size) sd cards that do proper wear leveling and use good quality and overprovisioned flash
maybe with a built in indicator for how healthy the flash is?
would be very useful to a lot of people
@ignaloidas @gsuberland SD and miniSD cards *could* do PLP. Just use a microSD card internally and fill the rest of the space with capacitors.
I know some microSD cards use a proper SSD controller with a single channel to a flash die. They have real wear leveling and page sparing. Not sure if they do internal patrol scrubs. SanDisk Industrial is one of the lines which works this way.
@ignaloidas @gsuberland It would be exactly as cost (in)efficient as it is with desktop SSDs, since that’s how they do PLP.
As for the added expensive of high-quality cards, basically every card of a given form factor from 64 GB down is the same price: about $10 US. I was recently shopping for a pile of cards. The most expensive I found were $12.70 for single industrial 16 GB cards. Oddly enough, 32 GB (the max capacity for SDHC) industrial cards were $11.
For 128 GB and higher, yes, high-quality cards start getting more expensive than the low-quality cards. I personally don’t use SD for anything except data which is trivial to reconstruct (like maps in a GPSr), or functionally disposable (like dashcam footage).
@ignaloidas @gsuberland An SD card only drives the controller and one die. It doesn’t need nearly as much power to complete in-flight writes. M.2 drives (e.g, Micron 7450) already typically have less volume in capacitors than the spare volume in an SD card.
Cost of shipping can definitely be a pain.
guess who’s backing it up as soon as they get home today?@carbontwelve oof. fingers crossed for you.
our wedding photographer gave us our photos on a USB stick. every single photo on it is now unreadable. (naturally I made backups immediately once we got them!)
Their merit is that they are cheap and easy to replace.
oink
Graham Sutherland / Polynomial
Jazz Rabbit 🐇🏳️⚧️
mirabilos
AMS
Andrew Zonenberg
Peter Bindels
Apicultor 🐝
John-Mark Gurney
Alanna 🏳️🌈🏳️⚧️
Manawyrm | Sarah
Raito Bezarius
julia "kallax" lnua,
Mrs Beanbag
Nicolas Dandrimont
1.3.6.1.4.1.61513
Xavier Knol 
Chris Combs (he/him)
dok
🇺🇦 haxadecimal 🚫👑
Ignas Kiela
Zimmie
Mx Autumn 
Chris Cunningham
Alexander Shendi
shironeko
rf