Brianna Townsend

As promised, a technical write-up about the knowledge-journey I went on to implement Japanese Unicode support in my software-rendered retro game engine.

There's a lot of juicy bits here about Unicode, string parsing, localization strategies, and more! Enjoy!

https://blog.brianna.town/japanese-font-support

#gamedev #indiedev #japanese #retrocomputing #unicode #gaming #pixelart

Japanese Font Support

Or: The Least Critical-Path Feature I Could Have Possibly Spent a Day On The General Plan Localization has been on my mind a bit recen...

The Ebonheim Chronicle
Oct 10, 2024 at 11:48pmWeb
Show threadgamedevjeffOct 11, 2024
@britown I had a poke at the ftp server, looks like they moved the files around: curl -O ftp://ftp.unicode.org/Public/MAPPINGS/OBSOLETE/EASTASIA/JIS/JIS0208.TXT
Show threadBrianna TownsendOct 11, 2024
@gamedevjeff thank you!! I'll at a note
Show threadgamedevjeffOct 11, 2024
@britown the rendering is pleasing result in the game btw, ty for the write-up!
Show threadgudenauOct 11, 2024
@britown This is one of my blindspots, I'll have to check it out!
Show threadgudenauOct 11, 2024
@britown Alright, I read through it and I have a couple of thoughts. The minor one is that writing a basic UTF-8 transcoder is a write of passage in text handling IMO. The second is have you thought about formatted strings at all/do you use formatted strings? That's one of the biggest things I'm worried I'll screw up because different languages have different rules for "you have %d item%s" sorts of strings. For me Unicode is mildly scary but not particularly intimidating because it's got a doc..
Show threadBrianna TownsendOct 11, 2024
@gudenau so right now I use a printf-style formatter that is %s %d etc but that is prone to an issue where the translated text might need to change the order of the arguments. A fix for that is to use a custom formatting scheme where you mark you arguments as like {0} {1} etc so the text can move them around but still be referencing the same argument list.
Show threadgudenauOct 11, 2024
@britown Yeah that's what I was afraid of, I really don't like using the braces for format strings but I guess I'll have to.
Show threadレイレイOct 11, 2024
@britown this is something i want to do for idtech3 some day, and i keep thinking of technical pitfalls (vram/tex cache, oob win9x support possibly needing unicows, so many hardcoded strings). I know I have the font i'd prefer to use for it all (Kosugi Maru / Motoya Maruberi)
Show threadEniko (moved ➡ gamedev.place)Oct 11, 2024
@britown is accept style parsing your technique? Cause I couldn't find anything about it when Googling it
Show threadBrianna TownsendOct 11, 2024
@eniko definitely not mine but I never know how to describe it, the wiki page for recursive descent parser is somewhat close, basically following an EBNF and then back tracking if a term fails, I can send you an example later if you'd like ☺️
Show threadEniko (moved ➡ gamedev.place)Oct 11, 2024
@britown yeah that'd be helpful!
Show threadBrianna TownsendOct 11, 2024

@eniko I put together a snippet for the code that parses my little scripting language that I showed off here: https://mastodon.gamedev.place/@britown/112736650843336944

Snippet: https://git.brianna.town/-/snippets/3

An accept function checks the current cursor for a condition, returns t/f, and if it is true, moves the cursor forward to the next character. Every accept function uses a collection of accepts defined before it.

So for an EBNF there's a 1:1 relationship between the grammar terms and accept() functions.

Brianna Townsend 👻 (@[email protected])

Attached: 1 image Sure ok let's hook up a code editor with custom syntax highlighting so we can write these scripts easily with error checking and auto-completion BalazsJako's ImGuiColorTextEdit is an older widget but it checks out #gamedev

Gamedev Mastodon
Show threadEniko (moved ➡ gamedev.place)Oct 11, 2024
@britown thanks!
Show threadNicola CocchiaroOct 13, 2024
@britown thanks for writing this, it was both enjoyable and illuminating!
Show threadBrianna TownsendOct 13, 2024
@wildbat yay I'm glad you enjoyed it!!!
Show threadMx. Paige AshlynnOct 22, 2024

@britown Just finished reading the post; very impressive! I hadn't realized that your in-game UI was auto-responsive like that!

I'll be interested to hear what you decide to do in the end regarding 8x8 vs larger glyphs

Show threadBrianna TownsendOct 22, 2024

@mxashlynn thank you! Yeah it'll be interesting, there's a middle option which is 9-high and 10-high which both I hear have great readability and would have fewer implications for the UI.

The UI closely resembled ImGui where the widgets are drawn sequentially every frame, moving a cursor forward based on the size of the elements so it's nice for things like this.

BUT there's things like iconography against 14x14 tiles that might not play well with larger text.