Julia EvansJan 16, 2023

we talked last week about what can go wrong with floating point numbers, so -- what can go wrong when using integers?

so far I have:

* 32 bit integers are smaller than you think (they only go up to 4 billion!)
* overflow
* sometimes you need to switch the byte order
* ?? (maybe something about shift / bitwise operations? not sure what can go wrong with that exactly)

I'd especially love real-world examples of things that have gone wrong, if you have them!

Show threadGeorge DinwiddieJan 16, 2023
@b0rk
* underflow
* signed and unsigned may act differently
* loss of precision if the order of operations decreases the number of bits in the the intermediate results
Show threadJulia EvansJan 16, 2023
@gdinwiddie can you say more about "signed and unsigned may act differently"? (how?)
Show threadbazzarghJan 16, 2023

@b0rk @gdinwiddie one that has been biting me since the very first time I tried to use Java is that bytes in java are signed. So 0xFF is a negative number, and when you do a right shift with >> it gets promoted to a _negative_ int, then masking back with & 0xFF to get the byte result is incorrect

byte b = (byte) 0xFF;
System.out.println(b); // -1
S.o.println(b >> 1); // -1
S.o.println(b >>> 1); // 2147483647
S.o.println((0xFF & b) >> 1); // 127

Show threadJulia EvansJan 16, 2023
@bazzargh @gdinwiddie one big takeaway for me from this thread is that java doesn't have unsigned integers!
Show threadbazzargh
@b0rk @gdinwiddie it's not something that normally bothered me but the very first thing I tried to write was a Quake bot, and I needed to pull bytes out of the network packets...boom
Jan 16, 2023 at 8:04pmWeb