It has to be something with either my network that I don't know how to figure out or my isp. My linux server is even failing to connect to update with is a different computer. Uugh someone please hire me full time.
Out of curiosity, I put my modem into it's default mode, and hooked it up to my network switch and things are back up to their faster speed. (though by faster I still mean ~3mb/s on a good day). I guess I have some bad settings in opensense now?