The Mac Pro still has auxiliary power connectors, so you could theoretically install a GPU in it, just not the highest end ones. No drivers so you can't really do anything with them. I do suppose Nvidia could maybe do CUDA drivers for a card, but would Apple even allow it?
@paul WTH, How many watts do GPUs take now?? I haven't bought a separate GPU for nearly 20 years…
@pixel @paul My new 4090 came with a power plant.
Some RTX 4090 Graphics Cards Recommend a 1200W Power Supply
That might be overkill, even for an RTX 4090
@pixel My falcon came with a 1000w PS for the 4090 and 13900ks
@steveriggins At some point (probably already) people will have to install a new circuit too.
Paul Haddad 
Eric Vitiello
Steve Riggins