So apparently I discovered another curse of mine. Boards with a esp32-pico-d4 seam to just not communicate with me or rather my laptop is often unable to send anything over the on board uart. Now it turns out that no one else seams to have this issue, but I just discovered that my beaglebone blue can reproduce the error. Communication seams to work when using an external different uart, but there is still further research needed ^^
I also have problems with tinypicos which also use the same uC