Gave a talk yesterday:
"Property Based Testing to verify a pipelined CPU design (hardware)"
at the New England Programming Languages and Systems Symposium (NEPLS),
about verifying my Fife RISC-V CPU using U.Cambridge's TestRIG (which uses Haskell's QuickCheck) and RISC-V International's RISC-V ISA Formal Specification (written in Sail)

https://nepls.org/
https://nepls.org/Events/36/

#NEPLS #FIFE #RISCV #RVI #Haskell #QuickCheck #TestRIG #PBT #Sail

Started hacking on a shrinker (reducer) of floats for QCheck a while back - and struggled. Yesterday I finally made some progress and now opened a PR for it:
https://github.com/c-cube/qcheck/pull/357 ✂️ 🔢 🤓

#ocaml #pbt #quickcheck

Here's a preprint of the paper I wrote up for Olivier's festscrift 🎉 🍾

Property-Based Testing of OCaml 5's Runtime System:
Fun and Segfaults with Interpreters and State Transition Functions

We describe our effort on using property-based testing to test the OCaml 5 multicore runtime system. In particular, we cover three case studies of increasing complexity that utilize a model-based state machine framework: (a) Testing the Array module, (b) testing weak hash sets, and (c) testing the garbage collector, with the latter two behaving non-deterministically from the point of view of the blackbox testing process. We evaluate the approach empirically by analyzing the bugs found, and discuss both limitations and challenges we have met underway.

https://janmidtgaard.dk/papers/Midtgaard%3aOLIVIERFEST25.pdf

#ocaml #pbt #quickcheck #preprint

A few days ago, Edsko de Vries of Well-Typed published an in-depth article on property-based software testing, with a focus on the concept of “shrinking.”

In brief, property-based testing is sort-of like fuzz testing but for algorithms and protocols. Like fuzz testing, random test cases are procedurally generated, but unlike fuzz testing, the test cases are carefully designed to verify whether a software implementation of an algorithm satisfies a specific property of that algorithm, such as:

• “this function always fails if the index is larger than the array”
• “this function always returns a result in n*log(n) number of iterations for input dataset of size n“

• “the sequence of log messages is guaranteed to obey this rules of this particular finite-state automata: (connect | fail) -> (send X | fail) -> (receive Y | receive Z | fail) -> success .”

Shrinking is the process of simplifying a failed test case. If you have found some input that makes your function return a value when it should have thrown an exception, or produce a result that does not satisfy some predicate, then that input is a “counterexample” to your assertion about the properties of that function. And you may want to be able to “shrink” that counterexample input to see if you can cause the function to behave incorrectly again but with a simpler input. The “QuickCheck“ library provides a variety of useful tools to let you define property tests with shrinking.

Defining unit tests with such incredible rigor takes quite a lot of time and effort, so you would probably do not want to use property-based testing for your ordinary, every-day software engineering. If you are, for example, being scrutinized by the US Department of Government of Efficiency, you would likely be fired if you were to take so much time to write such high-quality software with such a strong guarantee of correctness.

But if you are, for example, designing a communication protocol that will be used in critical infrastructure for the next 10 or 20 years and you want to make sure the reference implementation of your protocol is without contradictions, or if you are implementing an algorithm where the mathematical properties of the algorithm fall within some proven parameters (e.g. computational complexity), property-based testing can give you a much higher degree of confidence in the correctness of your algorithm or protocol specification.

#tech #software #haskell #WellTyped #QuickCheck #UnitTesting #PropertyTesting

𝟭𝟬 𝘀𝗲𝗰𝗼𝗻𝗱𝘀 𝗼𝗻 𝗚𝗮𝗺𝗲𝗰𝗵𝗲𝗰𝗸 𝗲𝗾𝘂𝗮𝗹𝘀 𝗽𝗲𝗮𝗰𝗲 𝗼𝗳 𝗺𝗶𝗻𝗱

A quick search is all it takes.
Enter the casino URL. Look for the green "Real Games" badge.
If it is there, it has been verified by us. If it is not, Gamecheck before you play.

✅ Gamecheck before you play

#SmartPlay #QuickCheck #GreenMeansReal #GamecheckVerified