Julia EvansJan 16, 2023

we talked last week about what can go wrong with floating point numbers, so -- what can go wrong when using integers?

so far I have:

* 32 bit integers are smaller than you think (they only go up to 4 billion!)
* overflow
* sometimes you need to switch the byte order
* ?? (maybe something about shift / bitwise operations? not sure what can go wrong with that exactly)

I'd especially love real-world examples of things that have gone wrong, if you have them!

Show threadReed Mideke
@b0rk For the last one, sign extension can bite you
Jan 16, 2023 at 7:08pmWeb
Show threadJulia EvansJan 16, 2023
@reedmideke ooh do you have an example of how this has happened to you by any chance?
Show threadAshJan 16, 2023
@b0rk @reedmideke using an unsigned int in a for loop can lead to an infinite loop (while x > 0). Also, the differences between how programming languages support these. Java doesn't have unsigned int.
Show threaddeadalnixJan 16, 2023
@b0rk @reedmideke I ran into a compiler bug about them last week (was doing a signed shift when it should have done a logical one). Some hair pulling ensued.
Show threadReed MidekeJan 16, 2023

@b0rk Hmm, don't remember a specific case, though I'm pretty sure I've messed it up. Sticks in my mind because assembler usually has two different right shift instructions, for sign extending or not, and if using an unfamiliar language, I have to figure out which it does.

As https://learn.microsoft.com/en-us/cpp/cpp/left-shift-and-right-shift-operators-input-and-output?view=msvc-170 notes it's actually implementation defined in C/C++ whether right shift sign extends 😱 though AFAIK normal platforms do

Left shift and right shift operators ('<<' and '>>')

Learn more about: Left shift and right shift operators ('<<' and '>>')