Val Packett 🧉5d ago
Natalie Vock

New blogpost! This time we're fixing VRAM memory management for VRAM-constrained GPUs, especially for 8GB VRAM and lower!

I've been working on this for a while, and it's finally in a state where others can try! If you run into issues where game performance ends up degrading over time due to your GPU running out of VRAM, this should provide a really noticeable improvement.

https://pixelcluster.github.io/VRAM-Mgmt-fixed/

Fixing AMDGPU's VRAM management for low-end GPUs | pixelcluster's GPU blog

Apr 9 at 2:48pmWeb
Show threadCampbell Jones5d ago
@pixelcluster Hey I remember talking about this with you! This would help a lot with my struggles with Stellar Blade
Show threadCampbell Jones5d ago
@pixelcluster Do you intend for a version of this to *eventually* be upstreamed into the kernel?
Show threadNatalie Vock5d ago
@serebit oh definitely! It's already on its way there in the mailing lists. 3/6 patches already have the necessary Reviewed-by tags, pretty much the only blocker is that the kernel, too, is short on reviewer time...
Show threadTheZoq25d ago
@pixelcluster Oh hell yeah! This problem has been plaguing me for a while, nice to see a potential solution!
Show threadМя ��5d ago
@pixelcluster maybe a dumb question, but is this work applicable for non-AMD GPUs? Or is the concept of GTT memory exclusive to AMDGPU driver?
Or is it the driver opensourceness that matters?
Show threadNatalie Vock5d ago
@mo Not dumb at all! Generally, every GPU will have some concept of system RAM that the GPU can access. For the GPU-specific side, the defining factor is whether the GPU driver is wired up with the dmem cgroup controller. For now, support is wired up in amdgpu and Intel's Xe kmd. Nouveau *might* be able to follow suit, but I'm not sure if you might run into bugs. NV proprietary driver might also technically be possible, but they could end up running into licensing issues since dmem code is GPL.
Show threadNatalie Vock4d ago
@mo eh yolo, here goes the nouveau patch: https://lore.kernel.org/dri-devel/2026[email protected]/T/#u :D
[PATCH] drm/nouveau: Wire up dmem cgroups

Show threadМя ��4d ago

@pixelcluster so, my 8gb gtx1080 can sleep peacefully, no updates for her  

(Proprietary driver dropped support, so I stuck with 580.xx, never tried nouveau since I heard legends about it being buggy as hell)

Show threadWizarro5d ago
@pixelcluster Just finished reading your blog post about the VRAM protection from eviction. This is awesome! Great work! Thank you for your contribution and I look forward to trying this out on my main machine as well as my sandbox. ❤️
Show threadBad Sector3d ago
@pixelcluster hey, just found this. By reading the post i get the impression this is (or could be) a "VRAM priority" model where you can assign priorities to different processes for their VRAM residency. Say i do some VRAM heavy thing in the background but also want to play a light game, so i want to have game +2, bgtask +1, all else 0. Ignoring the gnarly details, could this be used to write an app like amdgpu_top that lets you pick an process and put it in a "ultra/high/normal" priority group?
Show threadNatalie Vock1d ago
@badsector This approach intentionally doesn't use any numerical priorities, for quite a few reasons. cgroups pose a more elegant/flexible solution here, but require more intricate setup. Something like what you said would (theoretically) be possible by setting up cgroup hierarchy in a certain tree-like way. Generally, cgroups higher up that tree would be prioritized over cgroups further down in the tree, but this kind of setup allows you to do many different prioritization models, as well.
Show threadNatalie Vock1d ago
@badsector That is to say, if you wanted to write an app that lets you pick a process and choose a priority for it, that's possible but that app would also require cgroup hierarchies to be set up a certain way. I'm not completely sure if systemd is flexible enough for something like that, but that's probably the area you'd need to start working on first.
Show threadBad Sector17h ago
@pixelcluster i used numbers because of the limited text size, main idea is basically to set (in some way) at runtime that "process xyz should be at ultra priority, process abc should be at high priority, other processes are normal/default", etc. I guess it is fine for cgroups to be predefined (could be an install instruction), AFAICT the requirement is mainly to be able to assign the processes at runtime. I'll need to look into cgroups more though, never had an incentive about them before :-P
Show threadEskild Hustvedt3d ago
@pixelcluster Thanks for the nice writeup, a very interesting read!
How would this handle alt-tab-ing out to a browser for a few minutes mid game? Does it take longer for it to start shuffling memory around?
Show threadNatalie Vock1d ago
@zerodogg Should(TM) generally work fine. The idea at least is that when the priority changes from the alt-tab, the browser gets to fill up some of the VRAM again, and when you return to the game that gets undone.
Show threadlp0 on fire 1d ago

@pixelcluster, hmm… I had a look at the utilities' source code (not familiar with Rust…). The following looks like it works:

#! /bin/sh
sudo sh -c 'cd /sys/fs/cgroup/dmem
mkdir -p game-vram
cat dmem.capacity >game-vram/dmem.min
echo '$$' >game-vram/cgroup.procs'
exec "$@"

I'm assuming no inconvenient “max”s; I've not tested with the kernel patches in place; and hopefully I've understood it well enough that that script will help others who are also as yet unassimilated by the systemd collective.

(I don't plan to use exactly that. I'd use a daemon with a socket or I'd configure sudo specifically for that command, mainly 'cos I see sudo as “a bit risky”. Resisting systemd is not futile.)

Show threadNatalie Vock1d ago
@lp0_on_fire For launching a game this looks ok (save for some paths being not completely correct - you need to 'cd /sys/fs/cgroup' without 'dmem', and I'd heavily recommend using 'dmem.low' instead of 'dmem.min'), but the setup is not as easy. For one, I think all the desktop processes that should not be prioritized also have to be moved into some (non-root) cgroup, because memory protection is actually always relative to other cgroups. I heard OpenRC might do that, but I don't know for myself.
Show threadNatalie Vock1d ago

@lp0_on_fire Oh, also - you probably need to run 'cat dmem > cgroup.subtree_control' as root to actually activate the dmem cgroup controller. This probably needs to happen during login, because otherwise VRAM allocations made before the game was launched are not accounted in the controller.

Lastly, you might want to look into cgroup delegation, i.e. on login, you make an intermediate cgroup as root but change directory ownership to the user - then no sudo is needed to launch a game at all.