I really don't get this interaction between "sync" and "rsync". Tried it a few more times and it doesn't happen always but sometimes the "sync" gets stuck until rsync is finished completely. Even if it has yet to touch e.g. 3TB of data.
WTF?!?
Anyone seeing similar things or is there something else at play here too?
@agowa338 You're likely hitting the kernel's dirty page writeback limit. With 4x HDDs in RAID 5 plus LUKS and Btrfs compression, your I/O wait is going to be massive.
When you run sync, the kernel tries to flush everything at once. On HDDs, that RAID 5 parity calculation + encryption is probably saturating your bus.
Have you checked iostat -x or your vm.dirty_ratio settings? Might be worth lowering the ratio so the flush happens in smaller, more frequent chunks rather than one massive hang.
Maybe I'm misunderstanding things, but doesn't "sync" normally only cause everything that is pending to be written right now but not things that enter the queue afterwards?
Klaus Frank
TheIPW