The Mac Pro still has auxiliary power connectors, so you could theoretically install a GPU in it, just not the highest end ones. No drivers so you can't really do anything with them. I do suppose Nvidia could maybe do CUDA drivers for a card, but would Apple even allow it?
@paul WTH, How many watts do GPUs take now?? I haven't bought a separate GPU for nearly 20 years…
@pixel @paul My new 4090 came with a power plant.