i blogged about the memcmp thing
*(a++) ^ *(b++)?@gsuberland i'm reminded of a weird bug i found in a compiler. in this case the vendor (green hills c compiler for coldfire cpus) converted a sequence like this:
uint8_t idx;
idx = <user input, a single character>;
char newbyte = lookup_table[idx];
The code was converting upper/lowercase letters iirc.
In this case the compiler actually converted idx to a signed value using a coldfire MVS.B instruction for some reason, so you could obtain values outside of the lookup table array if the user input a byte >= 0x80.
Not incredibly useful on its own in the app I was analyzing, but the bug was in the compiler itself so needed to be fixed...I never would have spotted the bug were it not for using Ida back in the day.
@dysfun essentially this:
int memcmp32(uint32_t* a, uint32_t* b, int words)
{
uint8_t sum = 0; // whoops
while(words--) {
sum |= *a++ ^ *b++;
}
return sum;
}
@dysfun ehehehe
the code in question lucked out by never actually using it in a path that matters, but it's in a core lib for something security sensitive so easily could've ended up being bad
Hmm… is the issue casting 64-bit memory to an 8-bit char?