"Extending our model to include #neuronal activity and #Hebbian plasticity, we find that clustering in the network also emerges naturally. We confirm these predictions in the #connectomes of several animals, suggesting that heavy-tailed and clustered #connectivity may arise from general principles of #network self-organization rather than mechanisms specific to individual species or systems"
https://www.nature.com/articles/s41567-023-02332-9
Heavy-tailed neuronal connectivity arises from Hebbian self-organization - Nature Physics
The strengths of connections in networks of neurons are heavy-tailed, with some neurons connected much more strongly than most. Now a simple network model can explain how this heavy-tailed connectivity emerges across four different species.
@charanranganath @NicoleCRust @cogneurophys @PessoaBrain
Indeed! It's also worth noting that more recent evidence suggests that the hippocampus does not work like a Hopfield network (e.g. behavioural timescale #plasticity in CA1 https://www.science.org/doi/full/10.1126/science.aan3846, and existing motifs in CA3 https://onlinelibrary.wiley.com/doi/abs/10.1002/hipo.23034).
Still lots to uncover, but the idea that the #hippocampus is just a #Hebbian #autoassociative #recurrent network is surely false.
#arxivfeed :
"Hebbian learning with gradients: Hebbian convolutional neural networks with modern deep learning frameworks"
https://arxiv.org/abs/2107.01729?context=cs
#neuralnetworks #deeplearning #hebbian #gradient
Hebbian learning with gradients: Hebbian convolutional neural networks with modern deep learning frameworks
Deep learning networks generally use non-biological learning methods. By
contrast, networks based on more biologically plausible learning, such as
Hebbian learning, show comparatively poor performance and difficulties of
implementation. Here we show that Hebbian learning in hierarchical,
convolutional neural networks can be implemented almost trivially with modern
deep learning frameworks, by using specific losses whose gradients produce
exactly the desired Hebbian updates. We provide expressions whose gradients
exactly implement a plain Hebbian rule (dw ~= xy), Grossberg's instar rule (dw
~= y(x-w)), and Oja's rule (dw ~= y(x-yw)). As an application, we build Hebbian
convolutional multi-layer networks for object recognition. We observe that
higher layers of such networks tend to learn large, simple features (Gabor-like
filters and blobs), explaining the previously reported decrease in decoding
performance over successive layers. To combat this tendency, we introduce
interventions (denser activations with sparse plasticity, pruning of
connections between layers) which result in sparser learned features, massively
increase performance, and allow information to increase over successive layers.
We hypothesize that more advanced techniques (dynamic stimuli, trace learning,
feedback connections, etc.), together with the massive computational boost
offered by modern deep learning frameworks, could greatly improve the
performance and biological relevance of multi-layer Hebbian networks.
katch wreck
Blake Richards
Dominic Boutet