Julia EvansJan 16, 2023

we talked last week about what can go wrong with floating point numbers, so -- what can go wrong when using integers?

so far I have:

* 32 bit integers are smaller than you think (they only go up to 4 billion!)
* overflow
* sometimes you need to switch the byte order
* ?? (maybe something about shift / bitwise operations? not sure what can go wrong with that exactly)

I'd especially love real-world examples of things that have gone wrong, if you have them!

Show threadGeorge DinwiddieJan 16, 2023
@b0rk
* underflow
* signed and unsigned may act differently
* loss of precision if the order of operations decreases the number of bits in the the intermediate results
Show threadJulia EvansJan 16, 2023
@gdinwiddie can you say more about "signed and unsigned may act differently"? (how?)
Show threadGeorge Dinwiddie

@b0rk
It's been a long time since I've worked at that level, but signed integers typically presume a twos-complement representation; unsigned not. That can affect shift operations, arithmetic between operands of different types, and probably other things, depending on how they're implemented.

If you divide by two by shifting right, you want to shift in a zero for unsigned, and repeat the sign bit for signed, for example.

Jan 16, 2023 at 7:22pmWeb
Show threadJulia EvansJan 16, 2023
@gdinwiddie why would you shift an signed integer? i always assumed shift was only used on unsigned integers
Show threaddeadalnixJan 16, 2023

@b0rk @gdinwiddie say you want a signed 31 bit integer and a bool packed into 32 bits.
int x; // the packed 32bits.
x & 0x01; // the bool.
x >> 1; // the int.

More generally, you want that to optimize, for instance, division by powers of two.

Show threadDan LykeJan 16, 2023

@b0rk @gdinwiddie shift is a great cheap divide by 2^n ('cause divides can be *expensive*), but... this in test.c:

#include <stdio.h>
int main(int argc, char **argv) {
printf("%d\n", -4 >> 1);
return 0;
}

$ gcc -o test test.c ; ./test
-2

And Node gives similar results:
$ node
Welcome to Node.js v16.15.0.
Type ".help" for more information.
> -4 >> 1
-2

But Perl....:
$ perl -le 'print (-4 >> 1);'
9223372036854775806

Show threadTill O'Rly Jan 17, 2023

@danlyke @b0rk @gdinwiddie

To be fair. That's just pearl doing pearl things.

Show threadDan LykeJan 17, 2023
@b0rk @gdinwiddie fair. I kind of assumed it was 64 bit losing the top bit, but above 32 bits the numbers aren't familiar to me, so it could be 53 bits or whatever the "overlay an int and a double on the same space" encoding ends up being. And I'm sure I've worked on architectures where the carry flag or zero of something got shifted into the high bit, so ya had to be careful.
Show threadJohn McFarlaneJan 17, 2023
@danlyke @b0rk @gdinwiddie then compare -3/2 to -3>>1. Effectively, the rounding may vary. This is a pitfall of underflow.
Show threadthinkbeforecodingJan 16, 2023
@b0rk @gdinwiddie arithmetic right shift can divide sign numbers by 2 and keep sign. Left shift multiply by 2 with no prb.
Show threadthinkbeforecodingJan 16, 2023
@b0rk @gdinwiddie arithmetic right shift ire-nsert the sign bit. While logical right shift always insert a 0.
Show threadHenk LangeveldJan 16, 2023

@b0rk @gdinwiddie

I recall that the 6502 instruction set has two types of bit shift operators. A rotate, or a pure shift. One just recycles all bits, the other shifts in a zero every time.

Show threadHenk LangeveldJan 16, 2023

@b0rk @gdinwiddie

Nice. A Google site has an overview.

https://sites.google.com/site/6502asembly/6502-instruction-set/6502-instruction-set-bit-manipulation

6502 Bit Manipulation Operations - 6502 Assembly

Show threadGeorge DinwiddieJan 16, 2023

@hlangeveld @b0rk
I never programmed the 6502, and find it odd they don't have an ASR (arithmetic shift right) which keeps the sign bit. That's what you need for 2's complement division by powers of 2.

Compare with the 6800 instruction set: http://www.8bit-era.cz/6800.html#ASR

6800 instruction set

MC6800 decoding table, full instruction set with description. Addition to my web based MC6800 assembler.

8bit-era.cz
Show threadHenk LangeveldJan 16, 2023

@gdinwiddie @b0rk

Fair.

The 6502 was a more cost-effective chip compared to the 6800. I don't think integer division was a design target. Low cycle count operations were a strong objective.