i did not get a whole lot out of the active memory paper (it's complicated #alas) but it did have this useful insight:
"In a sequence-to-sequence model with attention, one retains the outputs of all steps of the encoder and concatenates them to a memory tensor."
i hadn't seen it put so succintly in other papers.
active memory compares favorably with single-headed attention mechanisms, but the paper kind of dismisses multi-head attention models, which are even more powerful.
i'm embracing my jet lag by scouring flights to europe for an extended (six week!) trip next spring, starting in stockholm to visit my daughter and her kids / my grandchildren
man i don't like connecting after an overnight flight i'm always exhausted and i get cranky but there are NO nonstops from detroit to anywhere in scandinavia #alas
i s'pose i could take a 4Β½ hour train to union station and a half hour cab to ORD and a finnair nonstop to helsinki and an overnight cruise to stockholm ... π€
I think I'd find Mastodon's IT engineers turned overnight πͺπΊ experts and their How to Trade War with Your Security Provider in the Middle of an Actual War diatribes less... dull if only they sprinkled it with some shitposting.
Where are my #YallQaeda π€ the Burkina tinpot dictator losing yet another province to jihadis ketchup memes??
peter honeyman
ABDELAZIZ
Veza85UE