The Effect of SGD Batch Size on Autoencoder Learning: Sparsity, Sharpness, and Feature Learning

UEBA в кибербезе: как профилирование поведения пользователей на основе Autoencoder помогает выявлять угрозы и аномалии

В современном мире количество атак растёт пропорционально количеству внедрений новых технологий, особенно когда технологии ещё недостаточно изучены. В последнее время атаки становятся всё более...

SciTech Chronicles. . . . . . . . .Mar 28th, 2025

  My best teacher is my last mistake. Vol II No 79 481 links Curated Fungus-like organism may be a completely unknown branch of life. https:...

Автоэнкодеры простыми словами

Автоэнкодеры являются базовой техникой машинного обучения и искусственного интеллекта, на основе которой строятся более сложные модели, например, в диффузионных моделях, таких как Stable Diffusion ....

Representation Learning via Manifold Flattening and Reconstruction

Untangling complexity: harnessing PCA for data dimensionality reduction

This tutorial explores the use of Principal Component Analysis (PCA), a powerful tool for reducing the complexity of high-dimensional data. By delving into both the theoretical underpinnings and practical Python applications, we illuminate how PCA can reveal hidden structures within data and make it more manageable for analysis.

Image denoising techniques: A comparison of PCA, kernel PCA, autoencoder, and CNN

In this post, we explore the performance of PCA, Kernel PCA, denoising autoencoder, and CNN for image denoising.

Compression of Structured Data with Autoencoders: Provable Benefit of Nonlinearities and Depth

Autoencoders are a prominent model in many empirical branches of machine learning and lossy data compression. However, basic theoretical questions remain unanswered even in a shallow two-layer setting. In particular, to what degree does a shallow autoencoder capture the structure of the underlying data distribution? For the prototypical case of the 1-bit compression of sparse Gaussian data, we prove that gradient descent converges to a solution that completely disregards the sparse structure of the input. Namely, the performance of the algorithm is the same as if it was compressing a Gaussian source - with no sparsity. For general data distributions, we give evidence of a phase transition phenomenon in the shape of the gradient descent minimizer, as a function of the data sparsity: below the critical sparsity level, the minimizer is a rotation taken uniformly at random (just like in the compression of non-sparse data); above the critical sparsity, the minimizer is the identity (up to a permutation). Finally, by exploiting a connection with approximate message passing algorithms, we show how to improve upon Gaussian performance for the compression of sparse data: adding a denoising function to a shallow architecture already reduces the loss provably, and a suitable multi-layer decoder leads to a further improvement. We validate our findings on image datasets, such as CIFAR-10 and MNIST.

A deep variational autoencoder for proteomics mass spectrometry data analysis

Jianwei Shuai's team and Jiahuai Han's team at Xiamen University have developed a deep autoencoder-based data-independent acquisition data analysis software for protein mass spectrometry, which realizes the analysis of relevant peptides and proteins from complex protein mass spectrometry data, and demonstrates the superiority and versatility of the method on different instruments and species samples. The study was published in Research as "Dear-DIAXMBD: deep autoencoder for data-independent acquisition proteomics".