qualiawould it be rude to use a little atom board as a bitflip testbench with a wee smote of sealed radium nestled betwixt its sodimms
if i set up a 24/7 livestream of the side effects of a measured & controlled elevated ambient ionizing radiation field on a machine running continuous stability tests would you watch that
i would absolutely check that out
yeah maybe idk
wouldn't watch but findings would be interesting
i do not care
i actively do not want this to happen
qualia what the fuck
Poll ended at .
Andrew Zonenberg@qualia Unless your dose rate is extraordinarily high I would expect the live stream would be super boring.
But let it run for a year or two with good logging and I'd absolutely read a blog or paper summarizing the results
@azonenberg i'm halfway expecting to see failures almost immediately with it in as direct contact as i can muster to validate the whole idea, at which point i'll back it off, run a lap or two to make sure that excursion hasn't unduly damaged the dimms, and then pick some fixed distances of measured intensity and start walking it in forward
if it does turn out to be extremely boring, i'll definitely see about arranging a longer-term test strategy, since i have wondered about this for a long while
@qualia Well the other question is what you're irradiating (ram vs CPU vs chipset etc).
You're gonna see different natures of failures hitting caches, logic, main RAM, etc
@azonenberg true! i was specifically thinking ram, since it is, afaik, the most prominent/physically large domain where hardware error detection (if not correction) in consumer hardware is not ubiquitous. except storage, maybe? correct me if i'm wrong
this idea is acutely engendered by the firefox error report bit flip thread, if you've seen that floating around
🇺🇦 haxadecimal 🚫👑@qualia @azonenberg
Mass storage devices have universally had error correction since the early 1990s. Disk drive error correction was introduced by IBM with the 3330 drive (1970) though error detection was used by earlier IBM disk systems such as the IBM 2311 (1964).
As density has tremendously increased since the late 1980s, it has become technically infeasible to make reliable disk drives without error correction. The same is true of solid state drives.
1/
Mass storage devices have universally had error correction since the early 1990s. Disk drive error correction was introduced by IBM with the 3330 drive (1970) though error detection was used by earlier IBM disk systems such as the IBM 2311 (1964).
As density has tremendously increased since the late 1980s, it has become technically infeasible to make reliable disk drives without error correction. The same is true of solid state drives.
1/
@qualia @azonenberg
The error correction is internal to the drive. The drive attempts to present itself to the host as an entirely reliable device. Uncorrectable errors will of course be reported as such, but correctable errors are hidden, and only reported by diagnostic commands (e.g., SMART).
Both magnetic disk and solid state drives are dependent on "coding gain", where the error correction is used to achieve higher storage capacity than would be possible without it.
2/
The error correction is internal to the drive. The drive attempts to present itself to the host as an entirely reliable device. Uncorrectable errors will of course be reported as such, but correctable errors are hidden, and only reported by diagnostic commands (e.g., SMART).
Both magnetic disk and solid state drives are dependent on "coding gain", where the error correction is used to achieve higher storage capacity than would be possible without it.
2/
@brouhaha @azonenberg makes sense. i had suspected as much -- i know SSDs have a whole extra region of spare blocks for wear management, but I got to thinking about the phenomena of silent data corruption and second-guessed myself
but on reflection, with how aggressively disk I/O gets cached in the free RAM of non-ECC consumer hardware.. that must be the substantial source of most of it
this might make for another interesting test, if I can wind up the single-event-upset events to a practically noticeable level -- get a small ZFS mirror going and redline it with checksummed/deterministic garbage writes & reads; see if/how often it manages to catch-and-correct itself in spite of the RAM's unreliability
@qualia @azonenberg
If the error hits non-ECC RAM written by the application before a write, and before ZFS computes the hash, then of course ZFS won't detect any error.
Similarly, if data read from the drive into non-ECC RAM gets an error after ZFS has validated the hash, then no error is detected.
I'm amazed the commodity mass-market computers have successfully* ignored this issue for so long, as DRAM error rates have constantly increased.
*for some value of "successfully"
@qualia @azonenberg
There is a trend to embed ECC in the DRAM chips, and that is probably a good thing, especially if they do internal scrubbing, but that's not entirely sufficient for high reliability since it's not end-to-end to the memory controllers. The memory bus itself is also a source of errors.
Nevertheless, anything that reduces random bit flips in the DRAM is good.
There is a trend to embed ECC in the DRAM chips, and that is probably a good thing, especially if they do internal scrubbing, but that's not entirely sufficient for high reliability since it's not end-to-end to the memory controllers. The memory bus itself is also a source of errors.
Nevertheless, anything that reduces random bit flips in the DRAM is good.