Meet the #StormCrawler users is back on our blog! We are delighted to share this Q&A with members of the OpenWebSearch.eu team. Come and read about their project and how they use both #StormCrawler and #URLFrontier to help deliver a truly open, transparent and legally compliant alternative to the big search engines.

#opensource #openwebsearch #opendata #innovation

https://digitalpebble.blogspot.com/2023/11/meet-stormcrawler-users-q-with-open-web.html

@openwebsearcheu

Meet the StormCrawler users: Q&A with the OpenWebSearch.eu project

It has been a while since our first “ Meet the StormCrawler users” blog and since StormCrawler is still going strong and used by a wide var...

Really proud to see both #stormcrawler and #URLFrontier used by OWLer
#OSSYM23

Would be sweet to have a native client for #URLFrontier (unfortunately I know only #Java).

Anyone here with experience in #grpc #golang or #rust who would like to contribute?

Recently released a new version of #URLFrontier.

https://github.com/crawler-commons/url-frontier/releases/tag/2.3.1

Fixes a couple of bugs reported by @anj

Release URLFrontier 2.3.1 · crawler-commons/url-frontier

Bugfixes See #71 Full Changelog: 2.3...2.3.1

GitHub

Fancy trying the new version of the #StormCrawler archetype which uses #URLFrontier as a backend?

https://github.com/DigitalPebble/storm-crawler/pull/1009

Using URLFrontier in archetype by jnioche · Pull Request #1009 · DigitalPebble/storm-crawler

A scalable, mature and versatile web crawler based on Apache Storm - Using URLFrontier in archetype by jnioche · Pull Request #1009 · DigitalPebble/storm-crawler

GitHub