@zoubin I find the cube iconic!

It also inspired me to try to better understand and systematize several other topics!

From #routing to #hierarchical multi-label classification and user #preference learning, SPLs outperform other baselines that relax constraints or use problem-specific architectures.

Even when they predict the wrong labels, they still form a valid configuration!

Join Kareem Ahmed, Stefano Teso, Kai-wei Chang, @guy
and me at
#NeurIPS2022 to talk about #SPLs and how to have #neural #nets to behave in the way we #expect them to do!

📜https://openreview.net/forum?id=o-mxIWAY1T8
🖥️https://github.com/KareemYousrii/SPL

6/6

How?

SPLs realize a #tractable product of #experts via 2 #circuits.

One encodes an #expressive distribution over the labels, the other compactly #compiles the #symbolic #constraint!

We can compute #exact #gradients because we can normalize them in one feedforward pass!

This can be of interest to many #probabilistic folks!

cc @nbranchini @avehtari @PhilippHennig

5/

Our #Semantic #Probabilistic #Layers #SPLs instead always guarantee 100% of the times that predictions satisfy the injected constraints!

They can be readily used in deep nets as they can be trained by #backprop and #maximum #likelihood #estimation.

4/

The usual recipe to inject constraints is to #relax them to make them #differentiable or to enforce them during training, by encoding them into auxiliary #losses...

This however does not guarantee that at test and #deployment time predictions will satisfy the constraints...

3/

We deal with #constraints over #structured outputs.

E.g., when predicting edges in a grid for #routing, they need to form #valid paths.

When predicting user #preferences, it shall output valid #rankings and in multi-label classification should respect the labels #hierarchy!

2/