If you want to take it even further a fully virtualized Opnsense with Proxmox is amazing. Your router can float between cluster nodes and each VLAN becomes a virtual interface in the hypervisor. What still blows my mind is how I can migrate the instance to a second server and bring the original server down for maintenance without my users noticing a thing.
As long as your Proxmox Cluster is backed by shared storage (ceph, gluster etc.) and HA is configured for your opnSense VM you can just shutdown the Node. Works flawlessly with pfSense (PVE backed by ceph).
I use managed switches for this, L2 units are fine if you let opnsense do all the inter-VLAN routing. All network devices go into the switches which are connected to the servers. As CptKriechstrom mentioned my PON or modem is connected to a switch and is tagged into a VLAN, which enters opnsense through that specific virtual interface.
I use (EdgeMAX) managed switches, so that shouldn't be an issue, I understand the concept now of what I'd need to do but I'm going to have to find a tutorial or something as I wouldn't know how exactly to set that up.
Ok, It's slowly coming together. So I assume I'd then put the virtual WANs of each cluster node on the same VLAN and whichever is currently hosting the router would chat with the PON to give me a WAN link.
