Spider v1.0.0 released.

Spider is not just another web crawler -- it is a purpose-built wordlist and ngram processor for hash cracking workflows.

URL Mode:
Point it at a URL and Spider crawls the target, extracts words, and generates frequency-sorted wordlists and/or ngrams.

But, Spider does not stop at web crawling...

File Mode:
Feed it local files and it brings the same word-processing engine to your own datasets, scraped content, notes, dumps, configs, or any other plaintext source you want to turn into a targeted wordlist or ngram set.

More info:
https://forum.hashpwn.net/post/52

#spider #webcrawler #wordlist #generator #sort #ngram #cyclone #hashpwn #hashcracking

BILD-Bürgerstreiche & das Ende der Spaßgesellschaft

https://www.kritische-masse.de/logbuch/2026/03/bild-buergerstreiche-das-ende-der-spassgesellschaft/

les exercices sur #ngram sont chouettes aussi pour apprendre l'#ergol mais seulement je fois qu'on a bien potassé la mémoire mécanique ailleurs.

un an après avoir commencé à être en mesure de faire des phrases en dactylo en ergol, j'ai toujours du mal sur les enchaînements d'annuaire et auriculaire, mais que de la main droite.

probablement une histoire d'appui de longue date concernant une habitude de dessin où ces doigts me servaient surtout de support pour préciser mon trait.

Recently I've combined various functions which I've been using in other projects (e.g. my personal PKM toolchain) and published them as new library https://thi.ng/text-analysis for better re-use:

- customizable, composable & extensible tokenization (transducer based)
- ngram generation
- Porter-stemming & stopword removal
- vocabulary (bi-directional index) creation
- dense & sparse multi-hot vector encoding/decoding
- histograms (incl. sorted versions)
- tf-idf (term frequency & inverse document frequency), multiple strategies
- k-means clustering (with k-means++ initialization & customizable distance metrics)
- similarity/distance functions (dense & sparse versions)
- central terms extraction

The attached code example (also in the project readme) uses this package to creeate a clustering of all ~210 #ThingUmbrella packages, based on their assigned tags/keywords...

The library is not intended to be a full-blown NLP solution, but I keep on finding myself running into these functions/concepts quite often, and maybe you'll find them useful too...

#Text #Analysis #Cluster #KMeans #TFIDF #Ngram #Vector #TypeScript #JavaScript

Spider v0.9.0 released:

Updates:
-url-match flag to filter URLs by keyword
Several small bug fixes
Go bumped to v1.24.3

https://forum.hashpwn.net/post/606

#infosec #spider #urlcrawl #hashpwn #wordlist #ngram

Fellow finicky writers: Do you prefer "advance notice" or "advanced notice"?

Both are attested. But FYI, #ngram says that "advance notice" is much more common, even if it's in decline.
https://books.google.com/ngrams/graph?content=advance+notice%2C+advanced+notice&year_start=1800&year_end=2022&corpus=en&smoothing=3

🚀 Spider v0.8.0

New features include:

"-file" to generate n-grams from local plaintext files

"-timeout" for URL crawling

"-sort" to output n-grams by frequency

https://forum.hashpwn.net/post/52

#spider #webcrawler #wordlist #ngram #infosec #hashcracking #golang #hashpwn

Слушать некогда читать: где поставим запятую?

Узнаете, когда заглянете под кат.😉 Для затравочки: речь пойдёт про инструмент ЮMoney для транскрибации аудио с внутренних созвонов в тексты и про кое-что ещё для наших клиентов. 😎👇

https://habr.com/ru/companies/yoomoney/articles/896096/

#whisper #llmмодели #искусственный_интеллект #ai #саммаризация #диаризация #идентификация #транскрибация_звонков #ngram