What a cool idea. I don't have a music setup capable of running this right now - perhaps in a couple of months - but if you were to post some sample recordings, I'd gladly listen to them.

How do you generate velocity values for the accompaniment notes?

Given that you already have a pitch tracker, it could be interesting to add key detection; just start playing, instead of telling the machine what key you're in, and it starts following along as soon as it catches on.