Mastodawn

aeva Mar 9, 2025

if you think about it, vulkan logical device implies the existence of vulkan illogical device

aeva Mar 9, 2025

I'm slowly working through the vulkan spec writing a compute-only vulkan program from scratch that doesn't render anything, and it's going pretty well because the spec is really well written and I already know more or less exactly what I want to do anyway, but I just want to say just how silly (fun) it feels to write a program like this because you get to just skip over large swaths of the API.

Like, I'm working from the spec because the tutorials all make it more complicated.

aeva Mar 9, 2025

also the tutorials I reviewed all did the annoying thing where the tutorial squirrels away the stuff you're trying to learn or reference into abstractions that only serve the needs of the tutorial writer, which given my goal is very specifically to *not draw anything*, there's really not much of a point to any of them lol. I'm really not the intended audience here though :3

aeva Mar 10, 2025

I think it's cute that practically every vulkan command has one or more optional args to let you enter Hard Mode

(sorry for the double post, I added this to the wrong thread)

aeva Mar 10, 2025

ok even with just the compute-only subset vulkan is a slog D:

aeva Mar 10, 2025

I wonder how many people have actually managed to knuckle down and write a complete, useful vulkan program from scratch (no copy pasting from tutorials and stack overflow, no offloading significant parts to 3rd party libraries like VMA)

To think if I power through and get this thing working I could potentially be like the 20th person to bother

aeva Mar 12, 2025

oh, update on my little vulkan compute project, last night I got as far as repeatedly dispatching an empty compute shader and allocating some memory 😎 I'm in the home stretch! I think I just need to figure out the resources / resource binding stuff and then I'll be able to start on my DSP experiment :3

which mostly means the next things are figuring out the least effort way of getting audio data into C++ (probably stb_vorbis?) and writing even more boilerplate for alsa...

aeva Mar 13, 2025

Success! I got the vulkan compute shader cranking out the fibonacci series and reading it back to the CPU through a 8 byte persistently mapped buffer. Should be smooth sailing from here.

aeva Apr 20, 2025

ok *whew* I finally did it! I implemented convolution reverb as a vulkan compute shader, and the results seem to be correct. I have it convolving the audio up front at the moment, but it seems to be reasonably fast and the results seem more or less correct. I'm using SDL3 to verify the output. It doesn't look like it'll be too crazy to rework it such that the stream is generated live.

aeva Apr 20, 2025

it turns out the main difficulty working with vulkan is accidentally breaking your laptop in half

aeva Apr 20, 2025

I reworked it so the convolution shader processes the audio in tandem with playback, so I'm *very* close to getting this working with live audio streams.

But more importantly, I used this to convolve my song "strange birds" with a choir-ish fanfare sound effect from a game I used to play as a kid and the result is like the grand cosmos opened up before me and I'm awash in the radiant light of the universe. Absolutely incredible.

aeva Apr 21, 2025

I want to power through and get this into a state where I can use it with live instruments, but I am completely exhausted 😴

aeva May 2, 2025

I reworked some things and now my audio convolving compute shader can convolve ~11 milliseconds worth of audio samples with an average processing time of ~7 milliseconds. That's with one channel with a bit rate of 22050. When the bit rate is 44100, the average processing time is a paltry ~8 milliseconds.

aeva May 2, 2025

also sometime in the last week I made it so it can operate entirely on a live input stream from SDL3 rather than a wave file, so in theory I can incorporate this into a modular setup now, but the results are higher latency than I'd like, and SDL3 doesn't give you much control over audio latency.

aeva May 2, 2025

Apparently my best frame time can get as low as 3 ms. I think vulkan should let me VK_QUEUE_GLOBAL_PRIORITY_REALTIME this program, but sadly vulkan is being a coward about it.

aeva May 2, 2025

ok the problem I'm having with latency now is that the audio latency in the system grows over time and I'm not sure why. like it starts snappy and after running for a short while it gets super laggy :/

I'm guessing it's because SDL3 can and will resize buffers as it wants to, whereas I'd rather it just go crazy if it under runs.

What I want to do is have a fixed size buffer for input and output, enough that I can have the output double or tripple buffered to smooth over hitches caused by linux. if my program can't keep up I don't want it to quietly allocate more runway I want it to scream at me LOUDLY and HORRIBLY, but it wont do that because I'll rejigger my program until it is perfect.

What actually happens is (sdl? poopwire?) just infinitybuffers so it never hitches and I get a second of latency after a little bit

aeva May 2, 2025

I like that pipewire has an option to not be terrible ("pro audio" mode) and it doesn't work

aeva May 2, 2025

99% of audio problems on linux these days are just programmers refusing to just fucking use alsa. I'm part of the problem, because I'm using SDL3 instead because the API is simple. SDL3 is part of the problem because when I tell it to just fucking use alsa it uses pipewire instead! and pipewire is part of the problem because it's just completely terrible. like, wayland terrible.

aeva May 2, 2025

want to have low latency audio on linux? we have a tool for it, it's called STOP PILING LAYERS OF BOILERPLATE ON TOP OF ALSA YOU IDIOTS YOU ABSOLUTE FOOLS

aeva May 2, 2025

I'm like 30% sure SDL3 is not the problem or at least not the only problem because I tried resetting the streams every frame with SDL_ClearAudioStream and it still accumulates latency (in addition to also now sounding atrocious due to missing samples).

I've also seen this happen with pipewire before in other situations, and it was resolved by bypassing pipewire.

aeva May 3, 2025

*spaces out* so anyways, this is usually the point where I'd try to cut this down to a simple loop back with as few layers as possible and gradually build back towards my program until I either find where the fault is or or have something working properly. That would mean targeting ALSA directly, except that appears to not be possible without uninstalling pipewire-alsa, which I can't without uninstalling Steam :/

aeva May 3, 2025

so abnormally, this means starting with a pipewire loopback instead and seeing if all you brave defenders of the status quo are flickering my lights or not.

aeva May 3, 2025

this makes me unhappy, but the single silver lining here is pipewire's API docs seem to be a little more newbie friendly than ALSA's

aeva May 4, 2025

ok I did it. I've got a program that writes a pipewire stream of F64 audio samples where each sample is the total elapsed time since the first frame, expressed in mintues.

I've got a second program that reads that pipewire stream, and checks the offset against it's own elapsed time since the first sample processed. This program prints out the calculated drift ever second.

The results are interesting.

aeva May 4, 2025

In the first version of this, both programs just measured the time using std::chrono::steady_clock::time_point. This resulted in an oscillating drift that was well under a millisecond at its peak and nothing to be concerned about.

This is good! That means there's no place what so ever within pipewire on my computer for this specific audio setup where any intermediary buffers might be growing and adding more latency as the programs run.

This is not the interesting case.

aeva May 4, 2025

In the second version, I changed the first program to instead calculate elapsed time as the frame number * the sampling interval, and left the second program alone.

In this version, the calculated drift is essentially the difference between the progress through the stream vs the amount of time that actually passed from the perspective of the observer. In this version, the amount of drift rises gradually. It seems the stream is advancing just a touch faster than it should.

aeva May 4, 2025

The samples in the stream are reporting that more time has elapsed in the "recording" than actually has transpired according to the clock. The amount of drift accumulated seems to be a millisecond every few minutes.

I'm honestly not sure what to make of that.

aeva May 4, 2025

anyways, for the curious, I put the source code for the experiment up here https://github.com/Aeva/slowtime

aeva May 4, 2025

also interesting is the drift is faster if I have the second program's monitor pin hooked up to my sound card, but there's still drift either way.

Janne Moren May 4, 2025

@aeva
Feels like a rounding or off by one error. Are you sure FramesProcessed = 1, and then (FramesProcessed + frame) is correct? What if FramesProcessed = 0?

aeva May 4, 2025

@jannem the source is right there

Janne Moren May 4, 2025

@aeva yep. And no, that is correct.

@aeva My experience is the opposite. Most of the API docs are completely inexistent, and a large part of the API is an unholy abomination of header-only C inline defs that can't be used from other langs.

aeva May 3, 2025

@shinmera this is what I found https://docs.pipewire.org/page_tutorial.html

maybe newbie friendly isn't the right term, but it's probably sufficient for my needs

PipeWire: API Tutorial

@aeva Yes. The examples are... OK, though they're also very lacking in explanations. The actual API doxygen is 99.9% without docstrings or explanations on anything though. I had a horrendous time trying to get output working without it either eating all my cpu or hitching horribly. Ultimately had to look at SDL2's implementation to get it working right 🫠

@aeva And I still needed a fucking C shim library to handle those horrendous inline towers. That library, despite only having 2 (two) calls in it is 32KB big (!!!)

aeva May 3, 2025

@shinmera good to know

Lunaphied May 4, 2025

@aeva @shinmera yeah I'm very frustrated by pipewire both their documentation and base API choices are not my favorite. The API is extremely flexible but in a weird way where without examples and norms that don't appear to be defined anywhere as standards you can't actually do anything.

Like the magic that makes pipewire actually glue itself to things besides just being a bus that routes stuff is mostly/all in its plugins which are barely documented and other programs just kind of have to know what SPA configs do what. Very very strange interface imo and not good when coupled with no standards. At least Wayland has explicitly defined protocols and such.

That being said it'd be pretty trivial to make this better so I hope that it does get there eventually.

Glyph May 3, 2025

@aeva okay so this may be severely out of date advice (like … a decade or more) but if the problem is that pipewire is hogging all your physical mixing channels on your sound device, could you write an alsa.conf targeting them both at a synthetic dmix device instead of directly at the hardware?

aeva May 3, 2025

@glyph what I want to do is completely shut off pipewire temporary so I can have one program (mine) talking to alsa directly without any virtualization and experience my back and shoulders unclenching, so that I have a ground truth reference point for what I can reasonably expect my computer to do. once I had that in hand, then I would then restore pipewire and try see if it still works with pipewire-alsa and go from there until I isolated the real problem and banished it forever

aeva May 3, 2025

@glyph However, I did not install pipewire-alsa. As it happens, the reason why I have pipewire-alsa is because Steam wants it, and steam will uninstall itself if it is not available. I do not want to uninstall Steam. I am unhappy about this turn of events. I have been personally betrayed by Cave Johnson.

Glyph May 3, 2025

@aeva in any case I wish you luck and hope you conclusively figure out what is going on here

Glyph May 3, 2025

@aeva I opened up some documentation for present-day pipewire and alsa to see if I could say anything more specifically useful because I recognize this pain even if it’s been a while but then I noticed I was taking like 3 SAN damage per turn and needed to go back to my existing obligations

Glyph May 3, 2025

@aeva ah. I think you can also tell pipewire to talk to a null device, so you can functionally disable it without uninstalling anything

Sebastian Krzyszkowiak May 2, 2025

@aeva Sounds like what you need is just setting right quantum values?

aeva May 2, 2025

@dos sounds like what I need is to throw it in the garbage and just use ALSA

Evie 🏳️‍⚧️May 2, 2025

@aeva there should be a cmake option to compile out pipewire 🤔

@aeva *snores in pulseaudio*

aeva May 2, 2025

@lritter how do you make pulse audio laugh on monday

aeva May 2, 2025

@lritter you tell it a joke on tuesday 😏

jell May 2, 2025

@aeva Was surprised and horrified to learn the solution to latency in Resolve through pipewire is... ...to install pulse.

Like ffs, linux audio, disenfuck yourself.

aeva May 2, 2025

@joshuaelliott wait what really?

jell May 2, 2025

@aeva Annoyingly yeah. (Thankfully having both seems not to cause issues elsewhere, but... it just feels like a dirty install.)

Viraptor May 2, 2025

@joshuaelliott @aeva what? Pipewire completely crushes pulse with latency. It's like *the* reason to use it. What did they do in resolve to mess that up? 😭

jell May 2, 2025

@viraptor Right?! I remember messing with Reaper and Ardour post-pipewire and feeling like it was a new era. NO idea what they're doing over there in Blackmagic-land.

trustsafe May 2, 2025

@aeva once they hit "not a total pulseaudio shitshow" it was mission-accomplished.

cancel May 2, 2025

@aeva wouldn't it hitch the first time the game fails to fill the buffer, causing it to make the buffer larger?

aeva May 2, 2025

@cancel what game

cancel May 2, 2025

@aeva I'm using game in place of application. whatever the application is. the user of SDL.

aeva May 2, 2025

@cancel it's not hitching, but latency is growing. like, I start the system, and press a key on my instrument and it's ok. I noodle around for a bit and it's fine but the delay gets longer and longer until I press a button and there's this noticeable pause before anything plays. It is really weird.

cancel May 2, 2025

@aeva 🤔 my impression is that, if the layers outside of your code are dynamically growing in order to deal with some latency deadline not being met, you'd hear at least one glitch in the audio each time it has to grow. otherwise, how would it know the application/usage layer failed to fill the output audio buffer in time?

maybe something else is happening? which audio API are you using? it looks like SDL has multiple.

cancel May 2, 2025

@aeva alternatively, it's possible that it does something extremely paranoid like hidden double buffering, where it wants your usage code to keep *two* buffers full, and if it drops down to only one, it starts growing the chain or buffer size. but this seems crazy and i doubt it.

aeva May 2, 2025

@cancel it is almost certainly doing that because the docs say it does

cancel May 2, 2025

@aeva that's insane and i don't have any suggestion except to use something else for audio. if that's true they have made the problem way more complicated than it actually is.

(i am a professional programmer who does audio, including having done commercial embedded projects using linux with custom audio stuff)

cancel May 2, 2025

@aeva the last audio thing i worked on in linux i shipped to the client with 5ms of audio latency from user input to dac output. i used alsa and it was like 200 or 300 lines of code to set up.

aeva May 2, 2025

@cancel I will likely end up doing that. I don't suppose you know any good tutorial resources for working with alsa?

aeva May 2, 2025

@cancel ok this is wild. I decided to start calling SDL_ClearAudioStream right before putting data into the output stream. the result is distorted af, but the latency still accumulates. the problem might not be sdl3 or the problem might be on the input stream

cancel May 2, 2025

@aeva is the SDL audio system a push-mode design?

aeva May 2, 2025

@cancel I don't know

cancel May 2, 2025

@aeva do you fill the audio buffer from a callback in an audio thread, or does your application code just kinda chuck audio buffers at a sink when there's time to do so?

cancel May 2, 2025

@aeva everything I saw was trash, and I just read the official docs.

aeva May 2, 2025

aeva May 2, 2025

@cancel I believe SDL is using pipewire by default. If I set it to ALSA it prints an error but otherwise behaves the same, suggesting it falls back to pipewire. If I set it to jack it freezes.

FWIW I have also seen this behavior with pipewire.

aeva May 2, 2025

@cancel The scenario you are imagining only works if everything is running perfectly synchronously except the application. For example it could be that the recording device is feeding frames just a tad too fast. Or maybe SDL3 is gradually accumulating frames that do not exist via imperfect resampling. Or perhaps pipewire is complete dog shit. who knows