*DrawBot bug alert*:

While running out extensive proofing texts, I realized that GSUB and GPOS are not being applied to FormattedString() if it runs to more than three pages.

OTFeatures and, most importantly, *kerning* only get applied to the last few pages, and not the initial pages.

Issue filed and confirmed: https://github.com/typemytype/drawbot/issues/585

@justvanrossum identified the bug as part of CoreText.

FormattedString(openTypeFeatures) not applied consistently · Issue #585 · typemytype/drawbot

If I have a text that lays out on three pages or less, then the OT Features (ex. {liga}) get applied throughout. If the text is longer than three pages, then only the last three pages have features...

GitHub
@kentlew @justvanrossum Das ist ja voll toll!

@kupfers @justvanrossum Here I was thinking I was so clever now, leveraging DrawBot instead of my old InDesign CS6 templates.

Imagine me poring over proofs and second-guessing all the kerning I’d done ten years ago on the original fonts, thinking I had to tweak more than I’d been intending. 😬

Until: hey, wait a minute … 🤨

@kentlew @justvanrossum I know nothing about that stuff. I was just making a silly joke, sorry.
@kupfers @justvanrossum No worries. I figured you were being sarcastic. 😜
@kentlew Welcome to my morning. Spent a bunch of time making updates to MATH cut-in kerning data only to find that recompiling TeX documents doesn’t show any difference from previous version.
@TiroTypeworks Crazy-making! 🤪

@kentlew Today, I ended up making a diagnostic font to confirm what the heck I am seeing in the output.

Turns out a couple of the key test documents simply are not recompiling at all, so what I am seeing is always the old PDF with the embedded earlier version of the font. Grrrr.

@TiroTypeworks That sounds frustrating. Tools that you can’t be sure you can rely on are less-than-helpful tools. 🤨

(diagnostic font is good trick, but unfortunate to have to need.)

@kentlew Sorted now. A lot of TeX is still pretty opaque to me, so it is difficult for me to debug test docs for math layout. Thankfully, I have access to some people who can figure this out.
@kentlew @justvanrossum I love it when computers just get bored and give up.
@simoncozens @justvanrossum The weird thing is that otfeatures are only applied at the end. So it’s like it finally gets tired of *ignoring* and goes ahead and does what you want for the last few pages. Weird.
@kentlew @simoncozens the bug occurs when the total length of the text you feed to textBox exceeds a certain value. With each page you output, the remaining text gets shorter. So at some point you reach the short-enough length.
@justvanrossum @kentlew I saw something similar (but not the same) a few years back. Certainly CoreText does have a maximum number of rule applications that it's prepared to do. But when I saw it, it was stopping early.
@kentlew @justvanrossum Ohhh! I've been getting the same issue on some of my proofs and now I understand why changing the font size "fixed" it for me.
@kentlew @justvanrossum @jmsole Just FYI, Font Proofer has a workaround for this bug. I encountered it a few years ago when @arrowtype made some kerning proofs with content that overflowed to hundreds of pages. Kent, I just recreated the test from your GitHub issue, and can confirm it works.
@peter @kentlew @jmsole @arrowtype why didn’t you report it? And what is the workaround?
@peter @kentlew @jmsole @arrowtype I mean, you embed DrawBot (free, written and maintained by two unpaid volunteers) into your commercial app that you sell for money, you discover a problem with DrawBot and even develop a workaround, yet you share neither the fact there is a problem nor the workaround? That is … disappointing.
@justvanrossum @kentlew @jmsole @arrowtype Just, we’ve discussed how difficult it is to maintain and sustain complex tools. First, I had no idea if this was a bug w/ DrawBot, CoreText, or my own code—I just had to fix it. Second, I have posted GitHub issues in the past. I try to. And third, I've been very transparent that the money Font Proofer makes only covers about 10% of what I put in. It's not far from being unpaid. I'm just trying my best to make something useful. We’re on the same team.
@justvanrossum @kentlew @jmsole @arrowtype As Kent suggested on the Github issue, the workaround is to limit the size of a formatted string. My solution is simply to slice it into smaller pieces. I wasn't even sure what caused the issue, and I wasn't thrilled about creating a hacky workaround. But it seemed to work. I'm guessing it would be harder to implement this workaround w/in DrawBot—otherwise I would suggest it.
@peter Yet at some point you must have concluded the bug is either in DrawBot or CoreText yet you chose not to report it. I find that odd, given the value you get from DrawBot, regardless of how sustainable your tool is.
@justvanrossum No, I did not come to a definitive conclusion. I wasn't sure if it was an issue with my code—until I saw the bug report today and the issue sounded familiar. I do make an effort to report bugs to frameworks and to Apple, but it can also be very time consuming and often seems to make no difference. I’m surprised and a bit hurt by the insinuation that I'm exploiting DrawBot without giving back. I'm doing all I can to help the community and donate most of my nights and weekends.
@jmsole @kentlew why didn’t you report it?
@justvanrossum @kentlew I hadn’t gotten enough time to figure out if the issue was with my code or not. I only saw the issue a couple of weeks ago in a proof where I have a variable font with feature variations that switch diacritics, so I wasn’t sure if the issue was with VFs, feature variations, my code, Drawbot or what, and I didn’t feel I had enough info to report it. For me it also expresses itself differently, I use drawbotgrid and the issue only happened in the first column of the proof.
@justvanrossum @kentlew The first time I saw the issue I had just made some big changes to the code I was using, so I really mostly suspected my code, with the font I was using as second suspect. Then for some other reason I changed the font size on the proof and that eliminated the issue which was confusing. And I’ve only over seen it happen in the first column of my proofs using drawbotgrid, so I wasn’t sure if maybe the issue was with that library.
@justvanrossum @kentlew Actually, in my case, it seems like the issue happens inconsistently. Both images below are of proofs where I show a number of words that use each of the letters with diacritics in the font. The first one does this in a big size, and it's correct, where the diacritics for this Optical Size should be flat with vertical terminals; it runs for 8 pages. On the second image, that's where I see the issue, the first column shows more upright diacritics, second col. is correct.

@jmsole Can you give me an approximate character count of the input text (before it gets filled into the boxes) for both these cases? It can still be the same bug.

In case you’re comfortable with building DrawBot from the sources, you could try if this (merged) pr fixed it: https://github.com/typemytype/drawbot/pull/588

Use kCTTypesetterOptionAllowUnboundedLayout option for setting 'longer' strings, ensuring OT features are applied in all cases by justvanrossum · Pull Request #588 · typemytype/drawbot

This fixes #585.

GitHub
@justvanrossum Yeah, that fixes it 👍
I am still intrigued by the fact that with drawbotgrid it always only affected the first column. I know for sure there was a difference in string length, even if I can't actually predict what that might be. The way that part of the script works is by getting all characters with diacritics that are found in the font and then finding x number of words that contain that diacritic, so for any given run the number of entries and length in characters could change.
@justvanrossum And of course, thank you so much for looking into the issue @kentlew reported and fixing things so quickly. Next time I bump into an issue I'll do my best to be more proactive in exploring and reporting it.
@kentlew This needs to be said, too: thank you for reporting this so thoroughly, Kent!

@justvanrossum @jmsole Glad to help. In my case, it was easy to rule out font & code because I was reading in several text files and some runs featured ligs while others did not.

First I thought there might be some control char gremlin in affected text. Then I noticed inconsistency within a single text. I made up the repetitive control text, and noticed the break point kept shifting as I changed length. That’s how I narrowed down.

I was bouncing off @djrrb. He urged me to file the issue. 👍

@kentlew @jmsole @djrrb issues with data and code to reproduce are incredibly useful!

@kentlew @justvanrossum Could it be related to this: https://stackoverflow.com/questions/65293503/coretext-with-too-long-arabic-text

Try to add kCTTypesetterOptionAllowUnboundedLayout to a CoreText Typesetter to see if that works?

@Typeface @kentlew wow, this seems to be the solution, thank you so much! Brb with a PR.
Use kCTTypesetterOptionAllowUnboundedLayout option for setting 'longer' strings, ensuring OT features are applied in all cases by justvanrossum · Pull Request #588 · typemytype/drawbot

This fixes #585.

GitHub
@justvanrossum Cheers, happy to hear it fixed the issue!
@Typeface @justvanrossum Thanks for the quick fix. Awesome!