I finally got my GPU power issues fixed on Thursday then spent Thursday night setting up PCIe passthrough in Proxmox. I looked at many blog posts and several YouTube videos about setting it up and Jim's Garage on YouTube worked out the best for me. Friday, Saturday, and most of Sunday was spent rebuilding my main KDE Neon VM because I needed EFI to support PCIe passthrough. Finally late tonight after dinner I got Ollama up and running along with John Capobianco's Document RAPTOR. I tried using Gemma2:27b but I don't think it was happy. I got a lot of warning messages about 'Max tokens to predict exceeds server configuration' and when I tried to query the model it never responded. In the web UI it just said thinking and in the console it said no response from server. As much as I want to keep going tonight until it's working, I need to get some sleep. I am only working Monday and Tuesday this week though so that leaves plenty of time for tinkering. 🤓

First place in the Dev Dash at the moment. #CiscoLive2023