Нейропоисковик на 17 миллионов картинок: OpenCLIP, Qdrant и ZeroTier вместо облаков
Как сделать свой аналог Pinterest, если у вас нет бюджета корпорации, но есть 4 ТБ картинок (17 миллионов штук) и желание искать по ним не просто по тегам, а по визуальному стилю? В статье — опыт построения распределенной системы поиска из подручных средств. Разбираем архитектуру: — Гибридное хранение: Backblaze B2 (холодное) + Домашний сервер (вычисления). — Сеть: Как ZeroTier объединил разрозненные ноды в одну Mesh-сеть без белых IP. — Векторный поиск: Как уместить 17 миллионов векторов OpenCLIP ViT-H/14 в 32 ГБ RAM с помощью Qdrant и квантизации. — Оптимизация: Почему поиск работает за 5 мс, а загрузка метаданных тормозила до 3 секунд, и как PostgreSQL спас ситуацию. Полный разбор ETL-пайплайна и экономики домашнего HighLoad-проекта.
https://habr.com/ru/articles/981330/
#python #векторные_базы_данных #qdrant #поиск_изображений #clip #zerotier #оптимизация #petproject #diy #selfhosting
you know what I want to be able to do with ZeroTier (or really any system like it that operates at layer 2)?
have a system on network A act as a host such that a system on network B (which might be mobile and move to network C or D at any time) can have a layer 2 interface connected to network A.
#ZeroTier currently gives the machines on network A and network B (or C or D) interfaces on virtual network Z by which they can talk to each other regardless of location. very useful, but not the same thing.
to expand even further, this probably doesn't matter in over 95% of cases, maybe over 98% ... both clients I'm testing with are behind an always-on VPN and don't have uPnP or NAT-PMP available.
I also suspect that it's because #ZeroTier doesn't always originate traffic froM its known port, whereas Tailscale does.
if I'M right about that, I'M not sure what their logic is, but the ZeroTier docs Mention listening on port 9993 and two pseudorandoM ports, so I wouldn't be surprised if it originated traffic on ports other than 9993 as well.
since My VPN bypass rule is based on source port, this would cause the bypass failure I'M seeing, which doesn't happen with #Tailscale.
I start cat-sitting tomorrow for about a week, and I may give ZeroTier a try while I'm there.
long term, it's a hard requirement for me to host my own coordination server, but for the week I will probably just use the one provided by ZeroTier Inc.
it looks like the client may use considerably less memory than Tailscale, which is nice if true - I'm likely to use #ZeroTier for my experiments building drop-in bidirectional proxy boxes for use cases like "I'm going to my friend's house / on vacation and want to proxy my DLNA server" or "I'm in a friend group of 4 households, two of whom travel 3-6 months a year, and we want to share network services with each other without opening them up to the internet," and Tailscale takes a lot of RAM on a router with <1GB memory.
Nueva entrada en el blog -> Acceso remoto a nuestro NAS y red doméstica
https://alt43.es/acceso-remoto-a-nuestro-nas-y-red-domestica/
そういうのがあるのか
5分でVPN接続完了なZeroTier #zerotier - Qiita https://qiita.com/RE-yura/items/3b750188e3cff05c55b9
Had a bit of trouble and lacked clear instructions on how to install ZeroTier on bazzite so wrote this guide
https://frans.today/zerotier-bazzite/
Documentation also lacked info on SSO via zerotier-cli so you can find that at the bottom.
Fedir Ustynov
Habr
Limping
selfhosting.couchsurfing
ALT43 
アルター
Frans