Reddit Tech VN BotMaxun v0.0.32 ra mắt với tính năng AI-native và ghi âm thời gian thực, mã nguồn mở, cho phép tự lưu trữ và trích xuất dữ liệu web không cần code. Hỗ trợ tích hợp với LlamaIndex, LangChain, OpenAI SDK, và nhiều framework AI khác qua SDK. Chế độ AI Extract tự động điều hướng, không cần URL. Ghi âm thời gian thực chính xác với hành động: gõ, click, cuộn, điều hướng. Phù hợp xây dựng workflow và agent thông minh. #Maxun #WebScraper #AIIntegration #OpenSource #DataExtraction #TríchXuấtDữLiệu #AI #MãN
Curtis McHaleCory – Blocking Countries because of scrapers
What the title says, Cory is blocking countries due to misbehaved scrapers. We do a bit of this at work, blocking misbehaving countries when they flood our sites with traffic. There is very little reason that anyone should be visiting the website of a city unless they live in the city, maybe the city next door, and maybe the originating country of the city.
99% of the time when a site gets DDOSed by something it’s coming from somewhere outside the country. The leading countries are India, China and North Korea. Sure a single person, or a family could be researching a city, but that doesn’t explain the traffic floods.
Many of our customers use Cloudflare so we just block them at the Cloudflare level and call it a day. I go back after a few weeks and remove the block because some valid traffic is reasonable.
I had to take a line like that on my own site as well, block a bunch of offending scrapers and bots from countries. It sucks to stop regular people from visiting my site but I’ve already dealt with a bill of $5k in a month that should have been $50 and I don’t need another one.
#webCrawler #webScraper
Curtis McHale
AI Daily Post
🍮 Vanilla(static)CMS
Dummy-X 🇮🇹
Philip Thomas
@reiver ⊼ (Charles) 