Mathias Niepert

@[email protected]
384 Followers
138 Following
8 Posts
Professor @ University of Stuttgart and Scientific Advisor (Researcher) @ NEC Labs Europe. Geometric Deep Learning, NLP, and ML for science.
Mathias NiepertNov 15, 2022
Kevin K. Yang 楊凱筌Nov 15, 2022

Two-stage pretraining for chemicals:

1. Masked language model
2. Predict chemical properties

@omendezlucio Nicolaou, @bertonearnshaw

https://arxiv.org/abs/2211.0265

Interpolated polynomial multiple zeta values of fixed weight, depth, and height

We define the interpolated polynomial multiple zeta values as a generalization of all of multiple zeta values, multiple zeta-star values, interpolated multiple zeta values, symmetric multiple zeta values, and polynomial multiple zeta values. We then compute the generating function of the sum of interpolated polynomial multiple zeta values of fixed weight, depth, and height.

arXiv.org
Mathias NiepertNov 15, 2022
Phys.-based Sim. Group @ TUMNov 14, 2022

We're happy to announce that our paper "Towards high-accuracy deep learning inference of compressible turbulent flows over aerofoils" is published now in "Computers & Fluids" and can be enjoyed at: https://authors.elsevier.com/a/1g3shAQO4pqSu , congrats Liwei

The source code is also available at: https://github.com/tum-pbs/coord-trans-encoding

The trained neural network yields results that resolve all necessary structures such as shocks, and has an average error of less than 0.3% for turbulent transonic cases.

Mathias NiepertNov 15, 2022
Daniel LowdNov 15, 2022

Our paper on adversarially-robust regression was accepted to SaTML 2023 (https://satml.org) -- the first ever IEEE Conference on Secure and Trustworthy Machine Learning!

I'm really excited about this conference and hoping to see it take off. There's so much important work to do in this area.
#SaTML #AdversarialML

IEEE SaTML

IEEE Conference on Secure and Trustworthy Machine Learning

Mathias NiepertNov 15, 2022
Yisong YueNov 14, 2022

We recently put out a position paper titled "Neurosymbolic Programming for Science"
https://arxiv.org/abs/2210.05050

This position is informed by our experience collaborating with scientists: science is an iterative process of analyzing data, proposing hypotheses, and conducting experiments. Because scientists reason more readily in symbolic terms, it is important to develop frameworks that natively inherit the both the flexibility of neural networks and the rich semantics of symbolic models.

Mathias NiepertNov 11, 2022
Show threadDaniel LowdNov 11, 2022

Title: "Identifying a Training-Set Attack's Target Using Renormalized Influence Estimation"

Key ideas:
💡 Training attacks are *highly influential* to their targets
💡 Targets have *anomalous influence distributions*
💡 Attacks are the targets’ *top influences*

In other words: Stopping training set attacks is an influence estimation problem!

Show threadMathias NiepertNov 7, 2022
@callumtilbury @pminervini
Show threadMathias NiepertNov 7, 2022
@callumtilbury really enjoyed your blog post. I wasn’t familiar with that application of discrete gradient estimators. Now I am curious to see if a simple gradient estimator we came up with and which has shown improvements over ST Gumbel Softmax could also be used here. https://arxiv.org/abs/2210.01941 (Figure 3 and 4). Let me know if you are interested / have cycles to collaborate.
SIMPLE: A Gradient Estimator for $k$-Subset Sampling

$k$-subset sampling is ubiquitous in machine learning, enabling regularization and interpretability through sparsity. The challenge lies in rendering $k$-subset sampling amenable to end-to-end learning. This has typically involved relaxing the reparameterized samples to allow for backpropagation, with the risk of introducing high bias and high variance. In this work, we fall back to discrete $k$-subset sampling on the forward pass. This is coupled with using the gradient with respect to the exact marginals, computed efficiently, as a proxy for the true gradient. We show that our gradient estimator, SIMPLE, exhibits lower bias and variance compared to state-of-the-art estimators, including the straight-through Gumbel estimator when $k = 1$. Empirical results show improved performance on learning to explain and sparse linear regression. We provide an algorithm for computing the exact ELBO for the $k$-subset distribution, obtaining significantly lower loss compared to SOTA.

arXiv.org
Mathias NiepertNov 5, 2022

#Introduction I am hoping to join a welcoming and diverse community here. What I liked about Twitter was the convos about ML, the new papers, ideas, and feeling connected to other researchers across the globe. Looking forward to rebuilding this on an open platform without the volatile billionaire bit.

I’m a professor of CS and will occasionally post something about geometric (graph) deep learning, attempts to bridge discrete and continuous learning, and applications in the sciences.