Mastodawn

aeva Mar 9, 2025

if you think about it, vulkan logical device implies the existence of vulkan illogical device

aeva Mar 9, 2025

I'm slowly working through the vulkan spec writing a compute-only vulkan program from scratch that doesn't render anything, and it's going pretty well because the spec is really well written and I already know more or less exactly what I want to do anyway, but I just want to say just how silly (fun) it feels to write a program like this because you get to just skip over large swaths of the API.

Like, I'm working from the spec because the tutorials all make it more complicated.

aeva Mar 9, 2025

also the tutorials I reviewed all did the annoying thing where the tutorial squirrels away the stuff you're trying to learn or reference into abstractions that only serve the needs of the tutorial writer, which given my goal is very specifically to *not draw anything*, there's really not much of a point to any of them lol. I'm really not the intended audience here though :3

aeva Mar 10, 2025

I think it's cute that practically every vulkan command has one or more optional args to let you enter Hard Mode

(sorry for the double post, I added this to the wrong thread)

aeva Mar 10, 2025

ok even with just the compute-only subset vulkan is a slog D:

aeva Mar 10, 2025

I wonder how many people have actually managed to knuckle down and write a complete, useful vulkan program from scratch (no copy pasting from tutorials and stack overflow, no offloading significant parts to 3rd party libraries like VMA)

To think if I power through and get this thing working I could potentially be like the 20th person to bother

aeva Mar 12, 2025

oh, update on my little vulkan compute project, last night I got as far as repeatedly dispatching an empty compute shader and allocating some memory 😎 I'm in the home stretch! I think I just need to figure out the resources / resource binding stuff and then I'll be able to start on my DSP experiment :3

which mostly means the next things are figuring out the least effort way of getting audio data into C++ (probably stb_vorbis?) and writing even more boilerplate for alsa...

aeva Mar 13, 2025

Success! I got the vulkan compute shader cranking out the fibonacci series and reading it back to the CPU through a 8 byte persistently mapped buffer. Should be smooth sailing from here.

aeva Apr 20, 2025

ok *whew* I finally did it! I implemented convolution reverb as a vulkan compute shader, and the results seem to be correct. I have it convolving the audio up front at the moment, but it seems to be reasonably fast and the results seem more or less correct. I'm using SDL3 to verify the output. It doesn't look like it'll be too crazy to rework it such that the stream is generated live.

aeva Apr 20, 2025

it turns out the main difficulty working with vulkan is accidentally breaking your laptop in half

aeva Apr 20, 2025

I reworked it so the convolution shader processes the audio in tandem with playback, so I'm *very* close to getting this working with live audio streams.

But more importantly, I used this to convolve my song "strange birds" with a choir-ish fanfare sound effect from a game I used to play as a kid and the result is like the grand cosmos opened up before me and I'm awash in the radiant light of the universe. Absolutely incredible.

aeva Apr 21, 2025

I want to power through and get this into a state where I can use it with live instruments, but I am completely exhausted 😴

I reworked some things and now my audio convolving compute shader can convolve ~11 milliseconds worth of audio samples with an average processing time of ~7 milliseconds. That's with one channel with a bit rate of 22050. When the bit rate is 44100, the average processing time is a paltry ~8 milliseconds.

aeva May 2, 2025

also sometime in the last week I made it so it can operate entirely on a live input stream from SDL3 rather than a wave file, so in theory I can incorporate this into a modular setup now, but the results are higher latency than I'd like, and SDL3 doesn't give you much control over audio latency.

aeva May 2, 2025

Apparently my best frame time can get as low as 3 ms. I think vulkan should let me VK_QUEUE_GLOBAL_PRIORITY_REALTIME this program, but sadly vulkan is being a coward about it.

aeva May 2, 2025

ok the problem I'm having with latency now is that the audio latency in the system grows over time and I'm not sure why. like it starts snappy and after running for a short while it gets super laggy :/

I'm guessing it's because SDL3 can and will resize buffers as it wants to, whereas I'd rather it just go crazy if it under runs.

aeva May 2, 2025

What I want to do is have a fixed size buffer for input and output, enough that I can have the output double or tripple buffered to smooth over hitches caused by linux. if my program can't keep up I don't want it to quietly allocate more runway I want it to scream at me LOUDLY and HORRIBLY, but it wont do that because I'll rejigger my program until it is perfect.

What actually happens is (sdl? poopwire?) just infinitybuffers so it never hitches and I get a second of latency after a little bit

cancel May 2, 2025

@aeva wouldn't it hitch the first time the game fails to fill the buffer, causing it to make the buffer larger?

aeva May 2, 2025

@cancel what game

cancel May 2, 2025

@aeva I'm using game in place of application. whatever the application is. the user of SDL.

aeva May 2, 2025

@cancel it's not hitching, but latency is growing. like, I start the system, and press a key on my instrument and it's ok. I noodle around for a bit and it's fine but the delay gets longer and longer until I press a button and there's this noticeable pause before anything plays. It is really weird.

cancel May 2, 2025

@aeva 🤔 my impression is that, if the layers outside of your code are dynamically growing in order to deal with some latency deadline not being met, you'd hear at least one glitch in the audio each time it has to grow. otherwise, how would it know the application/usage layer failed to fill the output audio buffer in time?

maybe something else is happening? which audio API are you using? it looks like SDL has multiple.

cancel May 2, 2025

@aeva alternatively, it's possible that it does something extremely paranoid like hidden double buffering, where it wants your usage code to keep *two* buffers full, and if it drops down to only one, it starts growing the chain or buffer size. but this seems crazy and i doubt it.

aeva May 2, 2025

@cancel it is almost certainly doing that because the docs say it does

cancel May 2, 2025

@aeva that's insane and i don't have any suggestion except to use something else for audio. if that's true they have made the problem way more complicated than it actually is.

(i am a professional programmer who does audio, including having done commercial embedded projects using linux with custom audio stuff)

cancel May 2, 2025

@aeva the last audio thing i worked on in linux i shipped to the client with 5ms of audio latency from user input to dac output. i used alsa and it was like 200 or 300 lines of code to set up.

aeva May 2, 2025

@cancel I will likely end up doing that. I don't suppose you know any good tutorial resources for working with alsa?

aeva May 2, 2025

@cancel ok this is wild. I decided to start calling SDL_ClearAudioStream right before putting data into the output stream. the result is distorted af, but the latency still accumulates. the problem might not be sdl3 or the problem might be on the input stream

cancel May 2, 2025

@aeva is the SDL audio system a push-mode design?

Josh Jersild May 2, 2025

@aeva as someone who's previous job was on realtime audio stuff this would drive me absolutely bonkers

aeva May 2, 2025

@JoshJers I'm hoping there's some way to tell it to not do that

Tom Forsyth May 2, 2025

@aeva @JoshJers I remember when we grew the buffers on our D3D8 drivers and the throughput increased by about 10% which was nice but also the latency grew to around 3 seconds which is sub-optimal when playing Quake but totally fine for the Quake benchmark.

Josh Jersild May 2, 2025

@TomF @aeva haha now I'm trying to imagine playing QuakeWorld where somehow the audio is *more delayed* than the controls

james is here May 2, 2025

@aeva i don't know the situation in sdl3 but i recall that the examples in sdl2 actually did the wrong thing and did not verify that the init call returned the buffer size or rate you asked for

mcc May 2, 2025

@aeva this is a surprise to me

aeva May 2, 2025

@mcc I've notice before that it will happily let me buffer minutes of audio in advance for output

aeva May 2, 2025

@mcc and the docs sorta say it can

aeva May 2, 2025

@mcc this also might be pipewire's fault though. I've seen something similar happen using pipewire to route stuff instead of just using alsa directly

mcc May 2, 2025

@aeva I'm used to audio systems using EITHER a buffer/watermark mode OR a realtime mode and you never want the former for low latency.

aeva May 2, 2025

@mcc I wish this had a realtime mode. The most I've found is a snide remark in https://wiki.libsdl.org/SDL3/SDL_HINT_AUDIO_DEVICE_SAMPLE_FRAMES about people who want low latency.

SDL3/SDL_HINT_AUDIO_DEVICE_SAMPLE_FRAMES

The Simple Directmedia Layer Wiki

mcc May 2, 2025

@aeva maybe i was right to stick with portaudio so long…

aeva May 2, 2025

@mcc maybe I should give that a try

mcc May 2, 2025

@aeva I've been leaning heavy on a Rust-specific solution of late so I'm not up to date on best practices. PortAudio feels like it's for people who Care About audio. I don't offhand remember what latency guarnatees it makes.

nex May 2, 2025

@mcc @aeva I care about audio and have fond memories of PortAudio from back when I wrote audio processing in C to get great performance on modest hardware. One of these days I want to do it again, with Zig instead of C.

I just realised I have no clue what the actual latency numbers were, but they seemed on par with professional DAWs.

Caveat: This was on Mac, not Linux.

ideaPDish May 3, 2025

@mcc @aeva
I’ve never thought much about rust before...
Is there a primer?

mcc May 4, 2025

@ipd @aeva …to … writing Rust?

ideaPDish May 5, 2025

Or reading Rust.
At the moment I'm just watching it.

egg boy color May 2, 2025

@aeva I was curious about this buffer-resize as I've done some audio code in the past, but not with SDL3(yet). that would be frustrating to write well-timed audio code if it dynamically resized things on the fly rather than doing audio dropouts on buffer exhaustion.

Iooks like there are hints you can set to force a buffer size in frames https://wiki.libsdl.org/SDL3/SDL_HINT_AUDIO_DEVICE_SAMPLE_FRAMES

Not sure if this could help? It doesn't seem to promise this will work, so may still need to fetch the actual buffer size gotten back.

SDL3/SDL_HINT_AUDIO_DEVICE_SAMPLE_FRAMES

The Simple Directmedia Layer Wiki

aeva May 2, 2025

@eggboycolor my program sets that hint. best I can tell it does nothing

egg boy color May 2, 2025

@aeva darn. hope you're able to figure out something there :< maybe audio backend specific, if you don't mind diving through sdl's code/statically linking stuff with symbols to do more debugging, but yeah these sorts of latency things are indeed messy if they (or the backend they wrap) don't give easy direct control over buffers.

either way curious to know what comes of it, since it's been really cool following your thread especially re:applying Vulkan for real-time audio!

aeva May 2, 2025

@eggboycolor I'll probably have to do something like that, though I might just rip out the SDL stuff and hammer ALSA directly

I’m 481 Phones May 2, 2025

@aeva do people usually use the GPU for audio? This seems like a really good idea for applications like live convolution!

aeva May 2, 2025

@benetherington I'm under the impression that it is somewhat unusual, but not unheard of

I’m 481 Phones May 2, 2025

@aeva Very cool, glad I ran across this thread. GPUs have magic smoke like all electronics but I’m sure it’s dark magic (smoke) after trying to work with shaders in any capacity. Smushing audio streams in there is delightfully forbidden.

aeva May 2, 2025

@benetherington there's a lot of fiddly bits to it for sure