@regehr

There are two different questions here:

• Is AArch64 a better ISA for modern microarchitectures than x86-64?
• Does the architecture limit the performance of an implementation?

The latter is trivially true. We have a load of examples of dead ends. Stack machines make extracting instruction-level parallelism really hard so lost completely to register machines.

Complex microcode makes out-of-order execution hard because you have to be able to serialise machine state on interrupt and decoded microops may have a bunch of state that isn't architectural. This one is quite interesting because it's a very sharp step change. Building a microcode engine that works is quite easy: serialise the pipeline, disable interrupts, run a bunch of microops, reenable interrupts. Building one that is efficient and allows multiple microcoded instructions to run in parallel is really hard, but if you do it then the complexity is amortised across a potentially large number of instructions. x86 chips took the first approach until fairly recently because there was a lot of lower-hanging fruit and microcoded instructions were rare. Having one instruction that requires complex microcode is absolutely the worst case.

Different ISAs favour different implementation choices. AArch32's choice to make the program counter architectural was great on simple pipelines, for example. It made PC-relative addressing trivial (just use PC as the base of any load or add) and made short relative jumps just an add to the PC. This became more annoying for more complex pipelines because you can't tell whether an instruction is a jump until you've done full decode (on most other RISCy ISAs, you can tell from the major opcode), which impacts where you do branch prediction. Similarly, all of the predication in AArch32 is great for avoiding using the branch predictor in common cases with simple pipelines, but you need that state anyway on big out-of-order machines. Thumb-2's if-then-else instruction provided a denser way of packing predication that scales nicely up to dual-issue in-order cores, but really hurts if you want to decode multiple instructions in parallel.

The question of AArch64 vs x86-64 is much more interesting.

Register rename is the biggest single consumer of power and the bottleneck on a lot of very high-end implementations. Complex addressing modes really help reduce this overhead, but so do memory-register operations where you avoid needing to keep a rename register live for a value that's used only once.

At MS, we did a lot of work on dataflow architectures to try to avoid this. This was largely driven by two observations:

• Around 2/3 of values are used exactly once.
• Around 2/3 of values (not the same 2/3, but an overlapping set) of values are used only within the same basic block where they are created.

The theory was that, by encoding this directly in the ISA (input operands were implicit, output operands were the distance in executed instruction stream to the instruction that consumed the result) you'd be able to significantly reduce rename register pressure. Unfortunately, it turned out that speculative execution required you to do something that looked a lot like register rename for these values.

AArch64 intentionally tries to provide a useful common set of fused operations. x86-64 does it largely by accident, but there isn't a clear winner here.

The one big win that we found has not really made it into any instruction set, which continues to surprise me. A cheap way of marking a register as dead can massively improve performance. I've seen a 2x speedup on x86 from putting an xor rax, rax at the end of a tight loop because the pipeline was stalling having to keep all of the old rax values around in rename registers, even though no possible successor blocks used them. If I were designing a new ISA, for high-performance systems I'd be tempted to do one of the following:

• Have a short instruction with a bitmap of dead registers that compilers could insert at end of the basic block for the back arc in a loop to mark multiple registers as dead.
• Put an extra bit in each source operand to mark it as a kill.

The latter hurts density, but would probably be a bigger win because it would let you rewrite a load of operations from allocating rename registers to using forwarding in the front end.

The other bottleneck is parallel decode. A64 ditched the variable-length instruction set that T32 introduced because it makes orthogonal decoding trivial. Fetch 16 bytes, decode four instructions. Apple's implementations make a lot of use of this and also have a nice sideways forwarding path that allows values produced in one instruction to be directly forwarded to a consumer in the same bundle without going via register rename (if the value isn't clobbered, they still need to allocate a rename register).

x86-64 is staggeringly bad here. As Stephen Dolan quipped, x86 chips don't have an instruction decoder, they have an instruction parser. Instructions can be very long, or as short as a single byte. Mostly this doesn't matter on modern x86-64 chips because 90% of dynamic execution is in loops and modern x86-64 chips do at least some caching of decoded things (at the very least, caching the locations of instruction boundaries, often caching of decoded micro-ops) in loops.

And this is where it gets really interesting. The extra decode steps and the extra caches add area, but they also save instruction cache by providing a dense instruction encoding (x86-64 isn't perfect there, it's not that close to a Huffman encoding over common instruction sequences, but it's a moderately good approximation). Is that a good tradeoff? It almost certainly varies between processes (the relative power and area costs of SRAM vs logic vary a surprising amount).

The thing that I find really surprising is the memory model. Arm's weak memory model is supposed to make high-performance implementations easier by enabling more reorderings in the core, whereas x86's TSO is far more constrained. Apple's processors have a mode that (as I understand it) decodes loads and stores into something with the same semantics as the load-acquire / store-release instructions but with the normal addressing modes. It doesn't seem to hurt performance (it's only used by Rosetta 2, so it's hard to do an exact comparison, but x86-64 emulation is ludicrously fast on these machines, so it isn't hurting that much. I'd love to see some benchmarks enabling it by default on everything).

Doing any kind of apples-to-apples comparison is really hard because things in the architecture force implementation choices. You would not implement an x86-64 and an AArch64 core in exactly the same way. There was a paper at ISCA in 2015 that I absolutely hated (not least because it was used to justify a load of bad design decisions in RISC-V) that claimed it did this, but when you looked at their methodology they'd simulated cores that were not at all how you would implement the ISAs that they were discussing.