Ok, that was a fun one.
Wanted to run some tests against my VGA implementation. Found a VGATEST program here: https://github.com/barotto/VGATEST
Try running it in DREAMM and it crashes in a DOS4GW exception handler reading a null segment selector?
Tracing back from the crash, I see that it did an FSTENV instruction and was reading the selector from there. FSTENV writes out the x87 floating point environment, including status and control registers, along with the segment+offset of the last floating-point instruction. DOS4GW was attempting to load from this pointer.
This is exactly what I would expect a floating-point exception handler to do. However, DREAMM never generates FP exceptions, so WTF?
Tracing back a bit further, I see an INT 10h instruction. Well, that makes sense, since the video BIOS calls are via INT 10h, and this is a VGA test program. But then it occurs to me that the #MF exception is defined by Intel to also generate an exception on vector 10h. So the code should be differentiating between a software interrupt (via INT 10h) vs an exception. But it's not even trying.
Eventually I trace back to the DPMI call that is installing the exception handler for INT 10h and see that it is loading the vector from memory. Hmmm. After much struggling, I got the DOSBox debugger running and when I get to the same spot, I see that the vector it is loading from memory is 2, not 10h.
Ah but wait: back when the FPU was a physical coprocessor, FP exceptions would come in via an NMI to the main CPU, and thus would use vector 2. So clearly in DOSBox DOS4GW thinks this is an older CPU.
Tracing back even further, I find the code that determines which vector to install the handler on, using a flag in memory. And even further back I find the code that sets the flag.
That code basically scans BIOS memory from F000:FFF5-FFFD and counts the number of ASCII digits or / characters. If it's less than 4 it assumes a newer machine using vector 10h, and if there are 4 or more it assumes an old machine using vector 2!
So I guess I put some digits in my BIOS?