One thing that has piqued my interest is something based off of Hilbert's programme, applicable to proof assistants.
Suppose I write a proof assistant in Standard ML. Then I use it to prove some function can be added to the kernel without jeopardizing the soundness of the system. I would like to dynamically extend the running program by compiling this function, and adding it to the program.
Hilbert's programme had something like this (as I understand it, they called it the epsilon substitution method) whereby a theorem stated in "idealized mathematics" which states and proves a claim about "finitary mathematics" can be "imported" back into the finitary metatheory, even if the finitary metatheory cannot prove it directly.
I think John Harrison called such a scheme as I've outlined "computational reflection".
It'd be neat to see a toy model of it.