Christian Folini ⛑️ Apr 19, 2023

I've consolidated a list of security scanners user agents. Many modern one send a fake UA. But some don't. Here is the list - minus public security services such as ssllabs; all lowerkey and abbreviated to essential keyword:

betabot
bewica-security-scan
dirbuster
fimap
gobuster
havij
hexometer
jbrofuzz
jorgee
libwhisker
masscan
massscan
morfeus fucking scanner
nessus
netlab360
netsparker
nikto
nmap
nuclei
openvas
sitelockspider
sqlmap
sysscan
w3af.org
webbandit
webshag
wfuzz
whatweb
wprecon
wpscan
zgrab
zmeu

I've gone through many, many different sources. Did I miss anything?

Show threadMichela Marie 🇨🇦Apr 27, 2023

@Folini Hmm. That looks very good, but there must be more. 😄 I couldn’t find information on Shodan’s user agent string, but that’s probably a good one to put on the list.

They aren’t security scanners, but I also block HeadlessChrome', 'Colly', 'Python-urllib', 'python-requests', and 'Python-httplib2'. People often use those in scripts that scan for vulnerabilities or attempt to manipulate or breach web apps.

Similarly, one may block 'Wget', 'Apache-HttpClient', and 'Curl'.

Show threadChristian Folini ⛑️ Apr 28, 2023
@michelamarie We're doing a multi-level approach here. Via the CRS paranoia levels if you are familiar with the concept. So the list above are the worst, then a myriad of other bad UAs on PL2 and then finally some stuff you usually welcome on a server like search engine bots. They will also be blocked at PL4 and you need to allow them specifically if you want to have them on that security level.
Show threadChristian Folini ⛑️ 
@michelamarie Ah, and before I forget: Shodan is blocked at PL2, but we might want to consider it per default.
Apr 28, 2023 at 6:12amWeb
Show threadMichela Marie 🇨🇦May 1, 2023
@Folini It seems to me blocking Shodan by default is best. It must be rare that anyone actually wants their environment to be scanned by it.