Paul Haddad 
The Mac Pro still has auxiliary power connectors, so you could theoretically install a GPU in it, just not the highest end ones. No drivers so you can't really do anything with them. I do suppose Nvidia could maybe do CUDA drivers for a card, but would Apple even allow it?
Mike@paul is Windows/PC still the way to go if you want raw GPU compute? I haven't kept up on the GPU side of things for the M2 Ultra benchmarks.