So, I've had a long look around #nontoxic #oilpainting. The easiest, most accessible way seems to just use this new kind of water-soluble oil paints, like the ones here (German site): https://www.kreativ.de/Kuenstlerbedarf/Oelfarben/Oelfarben-wasservermalbar/
Those are not completely watersoluble, just slightly more than traditional oil paints. But it seems like the solvents you can use (but don't have to!) are not based on toxic solvents like turpentine etc.