Sean Tilley

Big milestone: everything on the @films account and all subchannels has been successfully migrated over to Object Storage for #VidCommons. This is a process that I have to go over manually, which is annoying, but the benefit is that every piece of media I go through gets reviewed.

Many of the entries were replaced with videos in higher resolution, with better metadata. Some entries got subtitles added and carefully aligned to the videos, and in the case of the John Wayne collection, custom thumbnails were to match the old movie posters.

https://vidcommons.org/c/john_wayne/videos

DSH Social

Feb 26, 2022 at 7:05amWeb
Show threadSean TilleyFeb 26, 2022

The final leg of this conversion process will involve the @documentaries account, which has approximately 55 different pieces of media associated with it.

It’ll probably take a while to get through, but once we’re done with it, our instance’s catalogue will 100% work for anybody that wants to view our stuff. This also will be a greater chance to more aggressively add Public Domain and Creative Commons media!

Show threadilja Feb 26, 2022
@sean This is pretty cool! I saw it took a while, but I didn't know it was that much work you put into it.
Show threadilja Feb 26, 2022
@sean btw for people who want to watch vidcommons videos, but the specific video doesn't work, my work-around is looking it up on spectra.video and it woroks 
Spectra Video

Spectra is a PeerTube instance dedicated to supporting original content creators, and building the video community within the fediverse!

Show threadSean TilleyFeb 26, 2022
@ilja Yeah, I dunno why that is the case sometimes, but it seems like the player experience feels better from one’s own instance. To be fair, a bunch of the videos have been fucked for a while, which is why I’ve been manually going through to fix things.
Show threadSean TilleyFeb 26, 2022

@ilja Hey, thanks! 😁 The downside is that it’s a super old instance, and the support for Object Storage for videos didn’t happen officially until like 4.0. This meant that everything had to be reencoded for HLS instead of WebTorrent, and a bunch of weird quirks needed to be worked through by hand.

But, it’s a good opportunity to clean stuff up, and it frees up a ton of server space.

Show threadclacke: exhausted pixie dream boy 🇸🇪🇭🇰💙💛Feb 26, 2022
How does this work in practice, is Object Storage exposed directly to webtorrent as web seeds or is something proxied through the instance?
Show threadSean TilleyFeb 26, 2022

@clacke @ilja So, the media files are encoded as HTTP Live Streaming (HLS), and then placed into Object Storage.

I’m uncertain as to how that part interacts with the PeerTube player’s supposed P2P capabilities, because the documentation is a little vague on how this part works. It looks like it uses P2P Media Loader. An uneducated guess is that the storage is kind of a persistent connection for loading, but that it’s “helped along” by peers when they are present?

One sad aspect is that the P2P part has to distinguish between WebTorrent and HLS - because they are effectively different files with different encodings, the P2P media-sharing bit can’t really mix the streams. You can in theory have a video activity that has both formats, but then it would theoretically use two P2P streams for each. PeerTube generally recommends just picking one.

HTTP Live Streaming - Wikipedia

Show threadclacke: exhausted pixie dream boy 🇸🇪🇭🇰💙💛Feb 26, 2022
Here's how I imagined it works and what it sounds like you're saying, let me know how close I am or might be according to your understanding:

I thought:
- PeerTube stores a bunch of qualities of the stream, perhaps separate audio and video streams.
- Your client uses webtorrent to load one of the streams and can switch up and down depending on net climate.
- The originating instance has one file per stream that is used as a web seed with http ranged queries.

I think you're saying:
- Actually PeerTube only offers one stream to webtorrent.
- It offers several qualities over HLS.
- The HLS fragments can't be used as web seeds for WebTorrent.
- For the most time WebTorrent just becomes irrelevant.

Am I close?
Show threadclacke: exhausted pixie dream boy 🇸🇪🇭🇰💙💛Feb 26, 2022
Does this happen on non-originating instances too? Do they offer HLS with the originating instance as source? Or do they actually store and/or proxy the fragments on their own host to offer HLS?
Show threadSean TilleyFeb 26, 2022
@clacke @ilja Yeah, I think it works across instances.
Show threadSean TilleyFeb 26, 2022

@clacke @ilja It does store separate frame resolutions for quality, and I think those might be separate streams too, but I honestly don’t 100% know?

But for the most part, yes. Some instances still use WebTorrent, but it’s sort of being phased out. HLS breaks up a file into several segments of equal length, and then uses an index file as a playlist to tell clients how to put the media back together again.

Honestly, I don’t put a huge amount of stock into the P2P aspect of PeerTube. A lot of instances are also capable of themselves kind of acting like dedicated peers, offering redundancy for videos of a particular instance host. I know of a handful of instances out there that do that for VidCommons as well.

Show threadclacke: exhausted pixie dream boy 🇸🇪🇭🇰💙💛Feb 26, 2022
So WebTorrent is kind of being phased out because in practice it doesn't offer much benefit?
Show threadclacke: exhausted pixie dream boy 🇸🇪🇭🇰💙💛Feb 26, 2022
This is very cool, Sean! Humanity now and in the future thanks you for your hard work and for being a conscientious nerd that cares about all these little details like metadata and subtitles.