@michael_nielsen Very nice essay! I also appreciate discovery fiction as a tool for gaining insight and for its didactic value in teaching.
I had the same experience with teleportation - I understood it properly only after writing a discovery fiction story for myself. I have two different such stories that I use to teach it to my students now.
@michael_nielsen Btw, I consider any type of axiomatic characterization in mathematics as a form of discovery fiction.
For example, Shannon entropy was characterized by Faddeev as the only continuous function from probability distributions to real numbers that satisfies a very natural chain rule.
This is a simple way to motivate the entropy. The well-known formula with a logarithm then pops out of it by itself.
I first learned about this from a nice paper by Leinster:
http://arxiv.org/abs/1903.06961
Building on work of Kontsevich, we introduce a definition of the entropy of a finite probability distribution in which the "probabilities" are integers modulo a prime p. The entropy, too, is an integer mod p. Entropy mod p is shown to be uniquely characterized by a functional equation identical to the one that characterizes ordinary Shannon entropy. We also establish a sense in which certain real entropies have residues mod p, connecting the concepts of entropy over R and over Z/pZ. Finally, entropy mod p is expressed as a polynomial which is shown to satisfy several identities, linking into work of Cathelineau, Elbaz-Vincent and Gangl on polylogarithms.
@michael_nielsen This is not only of didactic value but has also led to new insights. For example, Leinster's paper introduces a curious (but similarly natural) notion of entropy modulo a prime. Here is a brief blog post about it:
https://golem.ph.utexas.edu/category/2017/12/entropy_modulo_a_prime.html
Leinster's paper also points out that entropy is like a logarithm on probability distributions since it is additive under the tensor product:
\[H(p_A\otimes p_B) = H(p_A)+H(p_B)\]
This by itself is a nugget of discovery fiction!
@michael_nielsen This paper by @johncarlosbaez, Fritz and Leinster is also effectively discovery fiction:
http://arxiv.org/abs/1106.1791
They consider a natural notion of information loss of a measure-preserving function, which has a simple and linear-looking definition. However, it turns out to coincide with the difference of entropies!
I find this much more satisfying that simply accepting the definition of entropy or trying to explain Shannon's source coding theorem to motivate it.
There are numerous characterizations of Shannon entropy and Tsallis entropy as measures of information obeying certain properties. Using work by Faddeev and Furuichi, we derive a very simple characterization. Instead of focusing on the entropy of a probability measure on a finite set, this characterization focuses on the `information loss', or change in entropy, associated with a measure-preserving function. Information loss is a special case of conditional entropy: namely, it is the entropy of a random variable conditioned on some function of that variable. We show that Shannon entropy gives the only concept of information loss that is functorial, convex-linear and continuous. This characterization naturally generalizes to Tsallis entropy as well.
@michael_nielsen Solèr's theorem is another example of discovery fiction along these lines (btw it was also brought to my attention by @johncarlosbaez):
https://en.wikipedia.org/wiki/Sol%C3%A8r%27s_theorem
It provides a purely algebraic characterization of Hilbert spaces over the real numbers, complex numbers or quaternions, without mentioning continuity in any way.
@michael_nielsen The whole area of reconstructing quantum theory from various simple-looking axioms is essentially discovery fiction!
Here are two prominent papers from this area, one by Lucien Hardy
https://arxiv.org/abs/1104.2066
and one by Chiribella et al.
https://arxiv.org/abs/1011.6451
This book provides a good overview of the subject:
https://doi.org/10.1007/978-94-017-7303-4
We provide a reformulation of finite dimensional quantum theory in the circuit framework in terms of mathematical axioms, and a reconstruction of quantum theory from operational postulates. The mathematical axioms for quantum theory are the following: [Axiom 1] Operations correspond to operators. [Axiom 2] Every complete set of physical operators corresponds to a complete set of operations. The following operational postulates are shown to be equivalent to these mathematical axioms: [P1] Sharpness. Associated with any given pure state is a unique maximal effect giving probability equal to one. This maximal effect does not give probability equal to one for any other pure state. [P2] Information locality. A maximal measurement on a composite system is effected if we perform maximal measurements on each of the components. [P3] Tomographic locality. The state of a composite system can be determined from the statistics collected by making measurements on the components. [P4] Compound permutability. There exists a compound reversible transformation on any system effecting any given permutation of any given maximal set of distinguishable states for that system. [P5] Sturdiness. Filters are non-flattening. Hence, from these postulates we can reconstruct all the usual features of quantum theory: States are represented by positive operators, transformations by completely positive trace non-increasing maps, and effects by positive operators. The Born rule (i.e. the trace rule) for calculating probabilitieso follows. A more detailed abstract is provided in the paper.
@MarisOzols Thank you for all these lovely examples!
Discovery fiction aside, an interesting point about taste: my first introduction to entropy was through various axiomatic approaches. But I didn't feel I understood it until I understood the source coding theorem!
@MarisOzols So: I felt I first really understood entropy for the first time as the answer to a question about source coding.
I'm not going anywhere particularly with this, I just find it interesting that our tastes in what is natural are so much the reverse of one another here!
Michael Nielsen
Māris Ozols