@b0rk The second point is a major understatement: on x86 for example almost every instruciton can and usually does act on multiple addresses at once, many implied by complex address-generation modes (PC-relative, scaled-indexed, segmented).

Further, a large amount of a CPU's machinery is dedicated to working with addresses in ways fundamentally different from the ways it works with non-address integers:

- calculating addresses implied by said address-generation modes of instructions
- reading and writing dedicated address-generation hardware like the stack pointer, frame pointer, instruction pointer, descriptor tables or segment registers
- loading and storing to addresses over the memory bus
- asking caches to load/store/lock/invalidate addresses
- scheduling out-of-order execution in the CPU in terms of the dynamic readiness of input data -- all reasoned-about by address
- speculating on addresses to prefetch, predicting addresses to jump to, etc.
- translating from virtual to physical addresses -- there's a hardware decoder for a kernel-managed "page table" data structure that stores nothing but addresses
- enforcing memory protection domains by address
- talking to IO devices memory-mapped at specific addresses
- deciding which instruction code to decode and execute based on the instruction-pointer address
- on newer CPUs, evaluating authentication bits attached to the address for control-flow integrity

I think it's not a stretch to say most of what the CPU does is about memory, which is spoken-of in terms of addresses. The residual stuff -- doing arithmetic on numbers that aren't addresses -- is the smaller / easier part.