Mastodawn

Dec 23, 2022

Currently patching my Proxmox cluster to prep the experimental SDN functionality to enable me to do VXLAN across my cluster of nodes.

I want to test this so I can have virtual routers with devices on the same internal networks but spread across multiple physical nodes.

I'm familiar with doing this on VMware with dVSes and VLANs but trying to replicate it on Proxmox. If this doesn't work as expected I may end up trying some other options. Hope to solve this in software so I don't have to buy gear.

#homelab #proxmox #networking #sdn #vxlan #ovs #selfhosted #selfhosting

Show thread

Carlos Mogas da Silva Dec 23, 2022

@projectdp wait, it isn't possible now? all bridges are local (to the node) ?

I thought the bridges were local to the node, I don't think they are automatically spanned across your cluster is it?

Show thread

junq

@projectdp @r3pek bridges are local but then you could always add a tagged physical interface to every bridge and then have vlans that span through the cluster. But you'd have to configure your physical switch to know about every such vlan.

With VXLAN you can have a flat no-vlan network between the nodes, and then only use software to configure a complex network.

Another benefit - VXLAN connects across the nodes even in different data centers as long as you have any (routed, firewalled, wan) connectivity between your hypervisors.

Yeah the problem with that first paragraph for me is I don't currently have a VLAN capable switch behind my proxmox nodes, I could place one there but I'd have to physically re-network a bunch of stuff at home.

I do like those other aspects of VXLAN that you mentioned as well. Are you currently running VXLAN or VLANs with your setup?

Show thread

junq Dec 24, 2022

@projectdp @r3pek I'm running just 2 vlans in my small home proxmox cluster (int, ext) with openwrt based home router acting as a smart switch. But I've seen vxlans widely used with Docker and k8s for their flexibility and also to not depend on the underlying network setup. Glad this came to proxmox now.

Ah I see, I do want to redo much of my network with VLANs but I'll need to re-cable much of my switching gear and probably buy another switch.

I would really like to hear from some people who have done production k8s deployments who also have a good networking background to talk to so I can understand the range of possibilities of well-architected networks in these environments. When I look at docker networking where you have a host and containers, or a cluster of hosts with containers, the network abstraction seems to be a confusing mess.

Show thread

Carlos Mogas da Silva Dec 24, 2022

@projectdp @junq

Before I setup this homelab, I started with refacturing the home network. Bought 2 AX3600 routers and put OpenWRT on them for switching (and main router) on steroids. Just for the sake of it, I added a couple of Unifi ToughSwitches (they changed its name now) with OpenWRT too in them.