I also have frigate on proxmox with a Google coral but mine has been rock solid. The only difference is that I use an LXC instead of a VM. I recall there being more issues passing hardware to VMs in Proxmox since they don’t like to share.