Fun Valgrind bugfixes of the week, number 2.
This time on illumos/Solaris amd64. I've been doing some testing on OmniOS and I noticed that the debug output (with the -d option) regularly contained corruption - a ctrl-b character - in long lines.
It seems like this is an old bug, git blame showing it as being present when the Solaris port was added to Valgrind back in 2015.
The issue was due to the debuglog function. This operates at a very low level - we want to be able to get debug logging very early on. Since we don't link with libc that means that we don't want it to even depend on our own heap manager. So roughly all it contains is a small 100 byte buffer on the stack, a vsprintf style function, a function to append 1 byte to the buffer and output the buffer when full and the function to write the buffer to file descriptor 2.
This last function is in inline assembler, and it is where the bug was. No registers were in the clobber list, but RDI is used to store the file descriptor, 2. Thart's STX or ctrl-b in ascii. Because RDI was modified it was changing the character in the first argument of the append to buffer function. After emptying the buffer the next character added was this corrupted ctrl-b character.
#valgrind #illumos #omnios