I have a test that generates random numbers as input for an algorithm. I can verify some general properties that always hold (“all the input is accounted for”) but I can also test some conditions that are probabilistically true (this should be at least 10, to 5σ). Thoughts?
@saagar seed the prng so it’s a deterministic test. If you change the code it might give a false fail but that’s better than no test and it won’t be flaky
@gnachman I guess this could work. I was against seeding the PRNG because it’s always testing the same thing, which kind of negates the point of testing against novel random data. But if the alternative is not testing at all this is probably better.
Saagar Jha
George Nachman