I use #puredata, but I feel using a properly compiled language would definitely be a better solution for what I want to do (feel free to prove me wrong here), and also, more portable.

Oh, preferably C, even better Ansi C. If absolutely necessary, I can tolerate C++. I tried the #miniaudio C library, I got a simple clean sine out of it, both on Linux and Win, but I'm not sure it's the best approach to build a more complex system.
I read good things also about the #libsoundio library.