Today's problem: I use an "AcceptanceTest" helper which I can just point at a non-public class and say "run the tests you find on this". This isn't really a thing you can do in AOT.
I'm not really sure what the equivalent is, except maybe pushing all those private types into a secondary pre-compiled test assembly that's only ever invoked for these acceptance tests (where the code can be public and get code-gen'd).
I'm hoping a background brain thread can come up with something better. Ugh.
Probably not a surprise that shelling out to an executable is **much** slower than just running the code under test in-memory. So I'm shifting my strategy to:
1. Only push the code out into a separate assembly for AOT, while still sharing the declarations between in-memory for core.tests and separate assembly for core.aot.tests
2. Just run the separate assembly once and cache all the results, and hand them out when asked for by the acceptance tests, to minimize the cost.
Don't have caching wired up yet, and the runtime for in-memory is 0.7 seconds vs. out of process at 21.8 seconds. Oof. To be expected, though.
I'm sure I'm not the only one who, when faced with multiple test failures, sets a breakpoint for each failure to tackle them one at a time.
Clearly I missed something fundamental here, right? 😂
Still don't have caching wired up, and I don't think I'm gonna bother, because things definitely got faster.
Also, count-wise I clearly have to make a pass to determine if there are more current "reflection only" tests that need AOT backfills.
Also need to fix an issue where I'm calling tests multiple times if they're defined in a partial across separate files.
Brad Wilson