Mastodawn

Maxwell (it/its)Jun 14

I have a #webcam on a #Linux system and hypothetically, if I were to do the video game streamer thing, I might want to use it to feed video from that camera to two apps simultaneously, namely:

Head tracking software for a video game
Face tracking software for a VTube avatar

But from what I've seen, only one application on a Linux system using V4L2 can get a video stream from a webcam at a time.

The solution I've seen proposed to this problem is v4l2loopback, but none of the sources I've seen bothered to explain how to use it, and the wiki they link to is bare-bones and doesn't seem to explain much either.

Furthermore, I want to be sure that the camera is providing video to these apps at its maximum framerate and not just its maximum resolution, and I don't know how I'd configure this. Or... really anything at all.

Is there any actually relevant documentation I should be reading to solve this problem?

#Video4Linux #V4L2 #V4L2loopback

Maxwell (it/its)Jun 17

Uuuugh I apparently need to use ffmpeg but this is such a pain. I don't know what I'm doing.

Maxwell (it/its)Jun 17

This documentation (for v4l2loopback in particular) actually kind of sucks. All kinds of things are just missing.

Maxwell (it/its)Jun 17

I think the most painful thing about this endeavor is that I will eventually find the solutions to my problems and write them up here, but because I'm just some weird toy-therian thing on the Internet, many others with the same problems will not get to read about it.

Good documentation is invaluable and simply not available here.

Maxwell (it/its)Jun 17

There are so many examples of things someone might want to do, but none of them are what I want to do, and the essential information I'd need to figure that out for myself is not here. I don't know what option I should use in ffmpeg to get v4l2 to pass the right codec, resolution, and framerate info to my webcam. I don't know where I'd find that written up anywhere.

I can't be expected to read the manual if it's not there.

Maxwell (it/its)Jun 17

Is there complete documentation of v4l2-ctl anywhere? Because the man page I got with the package isn't it.

I saw the --list-formats-ext option recommended to list all the available formats for a device, which works, but isn't documented. I'm guessing that I need a corresponding option to set the format I want, but I have no idea because the documentation isn't there.

#V4L2 #Video4Linux #WhereIsTheManual

Maxwell (it/its)

Am I even supposed to be using v4l2-ctl to change the stream format for other applications?

I have no idea! None of this is documented! There's no one to tell me!

Maxwell (it/its)Jun 17

Was I able to find documentation of the commands? Yes, doing v4l2-ctl --help-vidcap, which is documented in the man page. That probably should have just been another man page.

Was I able to change the pixelformat on my camera to MJPG? Yes, by doing v4l2-ctl -d /dev/video0 --set-fmt-video=pixelformat=MJPG.

Does this make a difference when I do mpv /dev/video0 to check the video output? Nope! It gets set back to YUYV raw video at 5fps.

So I do actually need to figure out how to set the correct format in ffmpeg or whatever I use to stream to the loopback device?

#V4L2 #Video4Linux #V4L2loopack #WhereIsTheManual

Maxwell (it/its)Jun 17

Why is it that none of the examples I've seen assume that the default format provided by your webcam is a raw image slideshow and you probably want to change that to something more useful?

Maxwell (it/its)Jun 17

Also conspicuously missing from this help page: an option to change interval/framerate.

Maxwell (it/its)Jun 17

Can someone please explain the secret ritual I need to perform to get the stars to align and grant my wish?

Maxwell (it/its)Jun 17

I've reached the point with ffmpeg where I really need someone to explain something to me because I'm getting inconsistent results.

To get ffplay to show me the video I want to stream to my v4l2loopback device, I did this:

ffplay -f video4linux2 -input_format mjpeg -framerate 60 -video_size 1920x1080 /dev/video0

And it worked! It pops up a window showing video from my webcam at the correct resolution and framerate, in MJPEG as expected.

But when I do this:

ffmpeg -i /dev/video0 -c copy -f video4linux2 -input_format mjpeg -framerate 60 -video_size 1920x1080 /dev/video9

The stream goes back to raw video at 5fps.

Is there something wrong with my syntax? I've noticed that if I move -i /dev/video0 anywhere except as the first argument in the command, I get errors. For example I do:

ffmpeg -c copy -f video4linux2 -input_format mjpeg -framerate 60 -video_size 1920x1080 -i /dev/video0 /dev/video9

And the error I get is:

Unknown decoder 'copy'
Error opening input file /dev/video0.
Error opening input files: Decoder not found

#V4L2 #Video4Linux #v4l2loopback #ffmpeg

Maxwell (it/its)Jun 17

I'm going to put this aside for now. Continuing to bash my head against this problem without someone competent with ffmpeg to help me is pointless.

Maxwell (it/its)Jun 17

Solution to my ffmpeg issue!

Thank you so much woggle from FurNet.

The command I ran was:

ffmpeg -f video4linux2 -input_format mjpeg -framerate 60 -video_size 1920x1080 -i /dev/video0 -c copy -f video4linux2 /dev/video9

I don't know why the arguments need to be in this order, but it gave me the streamcopy I needed.

Well, it gave me one streamcopy. Turns out that only one application can access a virtual webcam, same as only one application can access a physical webcam. So next step is to make more loopback devices and streamcopy to all of them.

#V4L2 #Video4Linux #v4l2loopback #ffmpeg

Maxwell (it/its)Jun 17

Additional problem: The input stream is 60fps, but the output is 30fps. I noticed some latency and I'm guessing this is the reason. I'll need to fix this too.

Maxwell (it/its)Jun 17

Wow. #ffmpeg is such a damn mess.

Maxwell (it/its)Jun 17

... huh. If I pass -framerate 60 to ffplay, it gives me 60fps. And NOW LookPilot is getting 60fps too. So that problem is... solved?

Maxwell (it/its)Jun 17

Okay, so after creating the loopback devices, the next step is to do v4l2-ctl -d /dev/video9 -p 60 for each loopback device but with /dev/video9 changed.

That's all my technical issues solved. The only thing left is implementation. I might make a script and a user service to run ffmpeg when I need it.

I think I'll write a blog post that thoroughly explains my problem and solution and post it later.

Maxwell (it/its)Jun 18

If running ffmpeg with my webcam as the input doesn't significantly increase my PC's CPU utilization or power consumption, is there a reason I shouldn't leave this service running all the time?

Webcams don't have moving parts and the power draw via USB is less than a watt. I don't think it will suffer excessive wear. Right?

Dr. Gravitas Jun 18

@maxissakitsune
want to put a Split in your filtergraph to get multiple outputs the use map to map them to your output devices?

I've not worked with webcams before, and only read about multiple outputs, but I *have* worked with my own video files and the split filter to resize multiple inputs relative to another input at the same time then recombine them.

You're right about it being a hilarious mess tho

https://ffmpeg.org/ffmpeg-filters.html#split_002c-asplit

This doesn't cover the options in filter_complex but it works the same there.

I'd offer to share one of my unholy commands, if it'd help but it might be more confusing tbh >w>'

FFmpeg Filters Documentation

Maxwell (it/its)Jun 18

@DrGravitas What I ended up doing was to just specify multiple outputs, all with the same arguments. You can just do that in ffmpeg and it doesn't seem to use up any more CPU or memory even with four outputs.

I'll write up the full tutorial within the next few days.