Fix mojibake in Unicode text, after the fact

ftfy는 Python 패키지로, UTF-8 인코딩 오류(모지베이크)를 감지하고 복구하는 데 특화되어 있습니다. 여러 겹의 인코딩 오류나 HTML 엔티티 문제도 해결할 수 있으며, 잘못 디코딩된 텍스트를 원래 의도된 문자열로 복원합니다. AI 데이터 전처리나 NLP 연구에서 텍스트 정제에 유용하며, Apache 라이선스 하에 배포되어 사용 시 저작자 표기를 요구합니다.

https://github.com/rspeer/python-ftfy

#python #textprocessing #unicode #mojibake #nlp

ASCII Chessboard, No HTML Required - Sometimes, when I have absolutely nothing to do, I play with ASCII characters in vim. Today I made an ASCII chess board with black and white chess pieces. I'm pretty sure I'm not the first one to make an ascii chessboard and I won't be the last. I thought it looks pretty nice so I wanted to share it on my blog.

Full blog post at https://sava.rocks/blog/ascii-chessboard-no-html-required/

#ascii #unicode #chess

Curious what character limit can actually mean. On Twitter it's bytes, so two-byte Unicode characters eat up the allocations fast but on Bluesky it seems to by glyphs so what had to be trimmed for Twitter has lots of legroom on Bluesky.

#Tech #character #Unicode

The latest version 2.0.0 of the open-source application "Unicopedia Symbolica" (previously part of the "Unicopedia Plus" application) adds a new "Emoji Taxonomy" utility.

🔗 https://codeberg.org/tonton-pixel/unicopedia-symbolica

#Unicopedia #Symbolica #Unicode #Emoji #Taxonomy

Unicode support once was a great strength of the programming language go - two of the founders developed UTF-8. The current state of the unicode support is not looking good, bugs don't get fixed, almost no more commits in x/text/unicode. :-(

#golang #unicode

Today I learned that the 64 I Ching hexagram symbols are all included in Unicode.

I'm fairly sure I'll never need to use them so I'll add it to my ever growing list of probably useless bits of knowledge.

https://en.wikipedia.org/wiki/List_of_hexagrams_of_the_I_Ching

#unicode #hexagram #symbols