Mastodawn

Bryan Sep 3, 2023

OMG, where was this article when I was first learning about tensor products?

https://www.math3ma.com/blog/the-tensor-product-demystified

In 10 minutes I learned more about tensor products than I did in grad school. In particular, I see more of the connection between the way math uses tensor products and how physicist types think of them, as higher-dimensional matrices.

This kind of thing really riles me up. It's the kind of thing that makes me want to go back to teaching, because it hits the exact personality/psychological trait that makes me like teaching: I learn about some cool math thing and I want to charge into a classroom and say "GUYS! I just figured this out! It's so cool and interesting! OMG tensor products are the best, let me show you why!"

It bothers me that I was taught about tensor products in such a bad way. I want students to get something better than I did.

(OTOH, maybe I'm just not very smart, or was a bad student. That's a real possibility. But how great would it be for even dim bulbs and lazy students to get *something* out of learning about a topic?)

#math #linearalgebra #mtbos #teaching

The Tensor Product, Demystified

Patrick Honner Sep 3, 2023

@ddrake Tai-Danae Bradley is amazing.

j_bertolotti Sep 3, 2023

@ddrake Took me (many) years after graduating before a random comment by @johncarlosbaez made me finally understand what a tensor actually is. All the explanations I got as a student were so needlessly convoluted that nothing sticked with me.

John Carlos Baez Sep 3, 2023

@j_bertolotti - thanks, Jacob! I agree that tensors are often explained badly. And I agree with @ddrake that Tai-Danae Bradley's blog explanation is delightfully clear.

I learned tensors because I wanted to understand general relativity and this was a bridge I *had* to cross. So I spent a lot of time studying them. Nowadays Google's open-source machine learning software "TensorFlow" may push other people into understanding tensors. But it would be nice if people got a tiny taste of them right around when learning matrices.

Like: "Hey! It's cool how you can multiply a vector by a 2-dimensional array of numbers and get a vector. But it doesn't stop there! In a viscous fluid the stress is a 2d array of numbers and so is the gradient of velocity, and you have to multiply the gradient of the velocity by a 4-dimensional array of numbers to get the stress!"

Well, that's probably too intense, but just some hint that after Tᵢⱼ thingies we'll want Tᵢⱼₖ thingies and Tᵢⱼₖₗ thingies and so on...

https://en.wikipedia.org/wiki/Viscosity#General_definition

Viscosity - Wikipedia

Nikita Lisitsa Sep 3, 2023

@johncarlosbaez @j_bertolotti @ddrake I always understood them from the algebraic perspective, as the most general way to multiply vectors. This explains everything else about tensors so nicely!

John Carlos Baez Sep 3, 2023

@lisyarus @j_bertolotti @ddrake - I probably started by understanding tensors from the physics perspective, where a rank-n tensor is a gadget that describes some numerical quantity that depends linearly on n different vectors. I might have seen them first in the Feynman Lectures.

Then we can refine this by distinguishing vectors from covectors. Then I learned about tensor products of vector spaces, and it became clear that the physicists' tensors are elements of V ⊗ ... ⊗ V ⊗ V* ⊗ ... ⊗ V*. This is what you learn in a math class, but also if you read a decent book on general relativity.

Aaron Sep 4, 2023

@johncarlosbaez @j_bertolotti @ddrake I came to an understanding through tensorflow. I wish I had had more of a mathematical exposure to them first, though.

Dan Drake 🦆Sep 4, 2023

@j_bertolotti I had the same thing happen with real analysis. I took it in undergrad, in grad school, and was always..."meh". But years later, after teaching basic stuff, and encountering little things about real analysis here and there, I came around to the "OMG, guys, this is amazing!" attitude.

Just like with the tensor product stuff that I started with here, I want to go back and teach real analysis, so that I can burst into a classroom and say, "guys! This amazing! The real numbers, and functions thereof, are SUPER WEIRD and amazing!"

ƧƿѦςɛ♏ѦਹѤʞ Sep 3, 2023

@ddrake
You may also enjoy going down the GA rabbit hole. :-)
https://www.youtube.com/watch?v=60z_hpEAtD8
#maths #geometricalgebra #vectors #physics

A Swift Introduction to Geometric Algebra

YouTube

Dan Drake 🦆Sep 4, 2023

@spacemagick I've done that! Although I wanted to go bit deeper. To drink a bit more of that kool-aid, so to speak. :)

ƧƿѦςɛ♏ѦਹѤʞ Sep 4, 2023

@ddrake
I thought you might have 🙂
May I recommend Alan Macdonald's books.
#maths #GeometricAlgebra
#SpreadingTheWord
( #math )

Jon Wilson Sep 3, 2023

@ddrake
I suspect that sometimes it is the second or third unique explanation that really makes something click. Not because it is a better explanation, but because it is the second or third. The previous understanding and experience paves the way for the effectiveness of the new explanation.

David Mankins Sep 3, 2023

This was my experience watching Gil Strang's lectures on linear algebra on YouTube. Every other lecture would contain a gem of clarity and exposition, it seemed like.

katch wreck Sep 3, 2023

@ddrake even simpler is to think of the vectors as functions f(x), g(y), ...,
h(z) whose arguments x, y, ..., z are the indices of the element(s) you want the value of, then the outer/tensor product of two (or more) vectors is just the regular product f(x) g(y) ... h(z) of their corresponding functions (as in good old multiplication)

Dan Drake 🦆Sep 4, 2023

@katchwreck ooh, I love that. Especially since my favorite way to motivate Fourier series and orthogonality of functions is the same analogy: vectors are orthogonal when the dot/inner product is zero. You get the inner product by multiplying corresponding entries and adding: for functions, the same idea is an integral: \(\int_{-\infty}^\infty f(t) g(t) dt\).

I like that because I understand the basic inner product well, and have good intuition for it, and can use that to understand the integrals and such. This goes the other way! If I have two functions, I feel very comfortable making a new function \( (x,y) \to f(x) g(y)\). And now likewise I can better understand having two vectors \(\vec{v}\) and \(\vec{w}\) and making a new vector/tensor \(\vec{v} \otimes \vec{w}\). Thanks!

Dan Drake 🦆Sep 4, 2023

@katchwreck so now, my question is: how does convolution of functions map back to vector space operations?

I've gotten better with my intuition for convolutions -- thanks in no small part to 3Blue1Brown: https://www.3blue1brown.com/lessons/convolutions

...so now I need to go think about what convolution is, in purely linear algebraic terms...

3Blue1Brown - Transformers, the tech behind LLMs | Deep Learning Chapter 5

A visual introduction to transformers. This chapter focuses on the overall structure, and word embeddings

katch wreck Sep 5, 2023

@ddrake my favorite way to connect convolutions to finite spaces is using shift operators. the geometric essence of the FT is that the unit circle is invariant with respect to rotation about its center, i.e. under the application of the shift operator (and powers of the shift operator). this carries over very nicely from continuous FTs to to matrices and DFTs: the shift operator is the circularly-permuted identity matrix. FT maps shifts to multiplications by phase, DFT diagonalizes shift matrix

katch wreck Sep 5, 2023

@ddrake these lecture notes define the discrete convolution in terms of shift operators:

https://ccrma.stanford.edu/~jos/mdft/Convolution.html

the continuous case works similarly

Convolution

Convolution

katch wreck Sep 5, 2023

@ddrake exactly! the fact that Hilbert spaces of functions are a lot like finite vector spaces is well appreciated, but (conversely, as you pointed out) finite vector spaces can also be considered as Hilbert spaces of functions, too :)

Joe Heafner Sep 3, 2023

@ddrake This is why classroom teaching is so important despite attitudes to the contrary. It’s not just you.

australopithecus Sep 3, 2023

@ddrake Amazing, thank you!
This sentence in particular brought it all together for me:
"The tensor product itself captures all ways that basic things can "interact" with each other!"
Because, yes, of course we solve quantum systems by combining all possible states

Bartosz Milewski Sep 3, 2023

@ddrake Compare this with:
For \(M\) a multicategory and \(A\) and \(B\) objects in \(M\), the tensor product \(A \otimes B\) is defined to be an object equipped with a universal multimorphism \(A,B\to A \otimes B\) in that any multimorphism \(A,B\to C\) factors uniquely through \(A,B\to A \otimes B\) via a (1-ary) morphism \(A \otimes B\to C\).

Aaron Sep 4, 2023

@BartoszMilewski @ddrake Is there a Mastodon add-on to render this? I am missing out

Bartosz Milewski Sep 4, 2023

@hosford42 @ddrake
Mathstodon renders LaTeX

Aaron Sep 4, 2023

@BartoszMilewski Only in the web interface, right?

I wish there were an android app that supported this. Most of my social networking happens on my phone.

Tobin Baker Sep 3, 2023

@ddrake My FIL did his dissertation in Lie algebras and admitted to me that he never really understood tensor products. So there's another data point for the hypothesis that your difficulties with tensor products might not be atypical. I got lucky because my first exposure to the concept was not via the abstract category-theoretic definition using the universal property, but via the concrete definition as a multilinear mapping from a cartesian product of vector spaces (it was actually from self-study of a fairly modern GR textbook). (It might have been even worse though if my first exposure had been to the old-school physics definition of a collection of components that transforms in some complicated apparently arbitrary way.)

cognitively accessible math Sep 3, 2023

@ddrake The whole elitism of being satisfied as long as the "good" students learned something is awful, especially since the lack of "goodness" of a student is often due to injustices.

Abdullah Khalid Sep 10, 2023

@ddrake How were you taught? The article follows the standard way tensor products are introduced to Physicists in Quantum Mechanics 1 course. Sometimes, a bit more abstractly with their properties as axioms, but then immediately jumping to this discussion so it's not confusing.