I have no idea why you'd want to do it but whilst fixing a @DNSHistory stats issue I spotted ~4500 domains have multiple (duplicated) SOA records.
For example https://dig.domains/soa/mrmmobilya.com has 7 SOA records (via Google Public DNS). Quad9 only returns one but Cloudflare, OpenDNS, and a local PowerDNS Recursor also return 7.
#OSINT, #BugBounty, #Pentesting, #CyberSecurity, #Infosec, #OriginServer, #CDNDetection, #DNSHistory, #SSLForensics, #FaviconFingerprinting, #AttackSurface, #InfrastructureAnalysis, #WebSecurity, #NetworkReconnaissance, #ThreatIntelligence, #SecurityResearch, #CloudSecurity, #ServerDiscovery, #DigitalForensics, #VulnerabilityAssessment
New Product
https://shoppy.gg/product/PvKQaT9
To fix this, the origin server should be restricted to accept traffic only from Cloudflare IP ranges.
Real log
Fewer domains have been crawled recently and will be for a couple of days as a unique number is being added to all domains and a full refresh is being forced (broken domains are normally checked with an exponential backoff and generally have slower DNS responses).
In future I hope this will allow the number of database writes to be reduced by caching some updates in RAM, freeing up capacity for more reads (maybe for that API).
It would also make it easier to bring back the old "Progress view" which I doubt many will remember as it's been gone for over 10 years IIRC but it's something I miss 😀. It showed visually how recent domains had been refreshed (and helped find a few bugs).
I've updated the numbers on https://dnshistory.org/ . It's now archived over 66 billion DNS records for 3.6 billion domains.
Sorry folks but https://dnshistory.org/ now has Cloudflare "Under Attack" mode enabled - someone decided they'd request one page per IP to get around my rate limits and then hammered the crap out of the server 😭 .
Less than a year later https://dnshistory.org/ now has data on 3 billion domains.
I'm gonna need a bigger disk 😁 .
When I first setup https://dnshistory.org/ I thought I'd be lucky to archive DNS records for a million domains but 15 years later it will soon exceed 2 billion 🥳 .
The recent increase is mostly from filling the gaps (a.b.example.com is in the database but b.example.com wasn't) or missing targets (example.org has an MX of mail.example.com but mail.example.com was missing) which should improve both navigation and looking up lost records. #DNSHistory
Chris Burton
ZugZang
DNS History