Mastodawn

Quo Vadis, Crawlers? Progress and what’s next on safeguarding our infrastructure https://diff.wikimedia.org/2026/03/26/quo-vadis-crawlers-progress-and-whats-next-on-safeguarding-our-infrastructure/ #AI, #AIDataCrawlers, #Crawlers, #Infrastructure, #Knowledge, #KnowledgeAsAService, #Scraping, #ScrapingBots, #WebScraping, #WikimediaFoundation, #WikimediaProjects

Quo Vadis, Crawlers? Progress and what’s next on safeguarding our infrastructure

One year ago, the Wikimedia Foundation reported a significant increase in bot traffic to the Wikimedia projects, largely coming from crawlers who extract content to train generative AI systems. We …

Diff

Zyte 1d ago

What does the future hold for the tool some describe as “the gift that revolutionised web scraping”? https://www.zyte.com/blog/the-future-of-scrapy?utm_campaign=blog-2025-catchup&utm_activity=ORS&utm_medium=social&utm_source=mastodon

#webscraping #webdata #data #web

The future of Scrapy: Smarter, faster and ready for AI-powered scraping

What does the future hold for the tool some describe as “the gift that revolutionised web scraping”?

Zyte

tagxdata 2d ago

How to Choose the Right Data Collection Company for Accurate Market Research

This guide helps you evaluate providers based on data accuracy, scalability, compliance, and industry expertise.Discover how reliable data gathering services and research partners can deliver actionable insights, support better decisions, and give your business a competitive edge.
https://www.tagxdata.com/how-to-choose-a-data-collection-company-for-market-research

https://www.tagxdata.com/how-to-choose-a-data-collection-company-for-market-research
#DataCollectionCompany
#MarketResearch
#TagX
#webscraping
#dataextraction

Zyte 2d ago

With the emergence of managed data extraction vendors, businesses no longer need to gather web data themselves. https://www.zyte.com/blog/rise-of-the-data-vendor?utm_campaign=blog-2025-catchup&utm_activity=ORS&utm_medium=social&utm_source=mastodon

#webscraping #webdata #data #web

Rise of the Data Vendor: How Outsourcing is Transforming Supply and Fuelling Businesses

With the emergence of managed data extraction vendors, businesses no longer need to gather web data themselves.

Zyte

N-gated Hacker News 3d ago

Oh joy, another #GitHub repository rehashing the same #overhyped #AI tricks we've seen a thousand times before. 🚀👏 Now with 100% more #TypeScript to make sure your web scraping dreams are both verbose and complicated! 🏆🎉
https://github.com/lightfeed/extractor #WebScraping #HackerNews #ngated

Zyte 3d ago

The Strategic Case for Buying Web Data: Quality, Focus, and Scale https://www.zyte.com/blog/quality-focus-and-scale-three-ways-data-outsourcing-benefits-businesses?utm_campaign=blog-2025-catchup&utm_activity=ORS&utm_medium=social&utm_source=mastodon

#webscraping #webdata #data #web

Quality, focus and scale: Three ways data outsourcing benefits businesses

The Strategic Case for Buying Web Data: Quality, Focus, and Scale

Zyte

Zyte 4d ago

The generative AI gold rush is upon us, with astounding new products and capabilities emerging that are fuelled by web data. https://www.zyte.com/blog/what-ai-builders-need-to-know-about-the-training-data-copyright-debate?utm_campaign=blog-2025-catchup&utm_activity=ORS&utm_medium=social&utm_source=mastodon

#webscraping #webdata #data #web

Rise of the Data Vendor: How Outsourcing is Transforming Supply and Fuelling Businesses

With the emergence of managed data extraction vendors, businesses no longer need to gather web data themselves.

Zyte

Zyte 5d ago

See what 10 years of Scrapy 1.0 has produced — in milestones and metrics - as it became the most-used open source web scraping framework in the world. https://www.zyte.com/blog/ten-years-since-scrapy?utm_campaign=blog-2025-catchup&utm_activity=ORS&utm_medium=social&utm_source=mastodon

#webscraping #webdata #data #web

Ten years since Scrapy 1.0: The stats and stories behind your favorite framework

See what 10 years of Scrapy 1.0 has built — in milestones and metrics.

Zyte

Hackaday 6d ago

Hackaday Links: March 22, 2026

https://fed.brid.gy/r/https://hackaday.com/2026/03/22/hackaday-links-march-22-2026/

13 6d ago

Hackaday Links: March 22, 2026 https://hackaday.com/2026/03/22/hackaday-links-march-22-2026/
#HackadayColumns #Hackadaylinks #Amazon #ArtemisII #Electronicfrontierfoundation #Newyorktimes #Webscraping #X59

Hackaday Links: March 22, 2026

On Friday, Reuters reported that Amazon is going to try to get into the smartphone game…again. The Fire Phone was perhaps Amazon’s biggest commercial misstep, and was only on the market…

Hackaday