Given an image (y) an inference network learns a distribution over context points positions (xM) and the CNPs are used as a generative network.
Latent variable (a) acts as an abstraction of the context points, providing control over different arrangements and pixel values(yM).
Interested in representation learning and Conditional Neural Processes (CNPs)?
Together with Victor Prokhorov, Siddharth N and Ivan Titov we propose Pixel Space Variational Autoencoder (PPS-VAE), an amortised variational framework that casts CNPs context points as latent variables.
Finally, we wondered what happens if we let the model choose its own order of compositions (basically define its own tree), so we modified StrAE so it can do just that!
This variant called Self-StrAE produces a tree and representations using a simple agglomerative clustering algorithm, no parser required!
To do this we introduce StrAE, a simple recursive autoencoder that learns representations going up a tree (encoder) and then, using the root as the bottleneck, learns another set of embeddings going back down (decoder).
StrAE is very simple, it doesn't have a cell state or skip-connections so every level in the tree forces a strict bottleneck. This means that StrAE has to conform to the compression order dictated by whatever structure it is given as input. We call this faithfulness.
[4/4]
We show that DooD generalises across varied data such as numbers, characters, and doodles, by performing effective zero-shot transfer from one dataset (e.g. MNIST) to other (e.g. QuickDraw, Omniglot, etc), clearly outperforming baselines.
DooD also generalises across task---evaluated on 3 of the Omniglot challenge tasks: unconditional generation, conditional generation, and 1-shot classification, showing that DooD typically matches or outperforms SotA methods.