lizzy مثلية الله ☭  
does fsync(2) do anything or is it placebo? my use case is making sure a file is properly written when i expect the system to crash within (potentially less than) a second or so
Apr 1 at 5:19pmWeb
Show threadmirabilosApr 1
@lizzy depends, also you often have to also fsync the directory it’s in
Show thread(eval 'Toast)Apr 1
@lizzy depends on the backing storage device, but USUALLY yes
Show threadgreeeeen  (corru edition)Apr 1
@[email protected] @[email protected] i know sync(1) does things, because takes a while, especially if the underlying storage is very slow
Show threadAthena L.M.Apr 1

@lizzy fsync functions as a barrier; it prevents changes after the fsync from being written before changes before the fsync. This is extremely useful for building reliable saving but does not suffice to achieve it alone. AFAIK it sometimes does not guarantee that the changes have been fully written to storage by the time it returns.

To build completely reliable writing of files, you need some way to roll back interrupted changes. There are several approaches; SQLite implements two fairly efficient ways of doing it, and has pretty good documentation on how.

Show threadإمي ❄️🏳️‍⚧️🇵🇸Apr 1
@[email protected] For that you need the O_PONIES flag.

More seriously, it's complicated and very filesystem dependent. See: https://lwn.net/Articles/351422/
POSIX v. reality: A position on O_PONIES

Sure, programmers (especially operating systems programmers) love their specifications. Clean, [...]

LWN.net
Show threadإمي ❄️🏳️‍⚧️🇵🇸Apr 1
@[email protected] The article is old so maybe things are better now? But I don't really know.
Show threadAmber Apr 1
@[email protected] i feel like there's some kernel facility that allows you to do what you're trying to do that's not fsync but i am completely blanking rn so i might just have made it up.
Show threadEmelia/EmiApr 1

@puppygirlhornypost2 @lizzy AFAIK fsync() works just fine for most cases, but for "must hit disk ASAP" writes, O_DIRECT will bypass the write cache for a given fd, so as long as the fd is blocking, writes will block until the data hits disk. (I had to do that to an imaging program to make the progress bar work, and given the workload in question, going through the write cache and potentially flushing more useful things was pointless)

EDIT: the biggest problem with fsync is if it fails for some reason, because the spec doesn't say what happens in that case.

Show threadAmber Apr 1
@becomethewaifu @[email protected] yeah. it's slowly coming back together. i remember one of the frustrations here is due to page caching with block devices. there's several layers in which write cache exists and you just need something that can cut through the bs
Show threadAmber Apr 1
@becomethewaifu @[email protected] bc u can have fs level (think something like zfs with ZiL), the page cache for block devices themselves and the controller for the drives. considering that this is for a system crash and not a power outage i dont think you have to worry about controller level (and/or drive level)
Show threadEmelia/EmiApr 1
@puppygirlhornypost2 @lizzy ZIL isn't a cache, it's only read back when repairing after an unclean shutdown. It exists to provide the "scratch space" for immediately fsyncing writes without interfering with other portions of ZFS's internals. fsync flushes to ZIL, but as the ZIL is stable storage, the rest of the FS is free to take however long it wants to do the 'normal' TXG write.
Show threadEmelia/EmiApr 1
@puppygirlhornypost2 @lizzy And disk controllers generally have write barriers so the host can at least know when they've actually flushed something, though a lot of cheaper SSDs will straight up lie about that "for performance" and trash your FS on a power cut.
Show threadAmber Apr 1
@becomethewaifu @[email protected] shit i didnt mean zil 😭
Show threadAlba 🌸 Apr 1
@becomethewaifu @puppygirlhornypost2 @lizzy my (not very well researched so possibly wrong) understanding was that O_DIRECT bypassed the page cache but not any disk controller caches (and perhaps even bio-level caches like dm-cache?) so you still need to fsync with that?
Show threadwyattApr 1
@lizzy usually yeah
might take more than one second for everything to be flushed though
Show threadlucasmz (en)Apr 1
@lizzy i thought u were the wine person
Show threadlizzy مثلية الله ☭  Apr 2
@lucasmz this is about the fsync linux syscall not the futex based sync system (fsync) in proton. they are basically unrelated
Show threadSteven BrudenellApr 1

@lizzy in particular, sqlite assumes that it works. from https://sqlite.org/atomiccommit.html#_hardware_assumptions :

"SQLite assumes that the operating system that it is running on works as advertised. If that is not quite the case, well then hopefully you will not lose power too often."

Atomic Commit In SQLite