Kinda found a way to make this work. And it's ugly.
`<alias><family>Noto Color Emoji</family><prefer><family>Noto Sans</family></prefer></alias>`
Basically "If you're going to use Noto Color Emoji, try Noto Sans first, THEN try Emoji".
Good: Seems to stop the emoji numbers (because Noto Sans has those glyphs)
Bad: Overrides it for EVERYTHING (not just Firefox)
Also bad: Forces sans serif, even if the site uses a serif font in the list _after_ the emoji font.