@atpfm @siracusa Re: GPU on Apple Silicon and inference.

Now, Apple may have their own super-power access to the Neural Engine directly, but for anything else the only way to use it is CoreML, and CoreML decides on its own how to distribute across GPU, CPU, and ANE.

For example, in the CoreML enabled build of Whisper.cpp, most of the inference is done on the GPU. I think most inference would benefit from running on both the GPU and ANE.

So if Apple was building some big Apple Silicon based data center to do AI inference (with or without new chips)I think lots of GPU cores would help.