What is the best way to force VLAN segregation accross multiple Virtual switches?

I have probably a rather complex setup going in the effort to get essentially 4 separate virtual switches.

The incoming line is a 10Gbe teamed connection in trunk mode which I then add to an external virtual switch and create 4 virtual nics for the management OS each set to access mode in the VLAN I want. Then I have four internal virtual switches created. Four being the number of VLANs I want to segregate out.

Now, I can set each NIC to access mode in the VLAN I want and then create on big bridge. Voila, all of the virtual servers can see the internet and can access the VLAN they need. But, there are a few problems. First, if I set any server to DHCP, then it only gets DHCP from the default vlan instead of whichever VLAN the switch should be assigned to. Second, I can actually assign an IP from any of the VLANs and the server will still communicate, no matter what the switch is supposed to do.

I understand the concepts of what is going on. Because everything is set in access mode it all sends untagged frames so once it finally gets to the bridge, it translates everything into the native (untagged) VLAN. My only problem is, I can't find a way to specify tagged frames. Or, better yet, to create 4 network bridges, one for each VLAN.

Any suggestions?


Solution 1:

1 switch per VLAN is needless overcomplication. Configure trunk port(s) on your TOR/access switch. Then create 1 vSwitch on your host. Connect the vSwitch NIC(s) to the trunk ports. When you create a VM, edit the vNIC settings and set the VLAN tag/ID/number.

This is both the simplest way to do it, and best practice. The vSwitch, as long as you use MSFT teaming (or follow instructions for unsupported 3rd party teaming), is secure from VLAN hopping.

The only real scenario where you need multiple teams is when you face something on the Internet. In that case you use a second NIC/team, not for security, but for DDOS.

Solution 2:

While I highly urge anyone to use Aidan's answer if at all possible. The final solution I found allowed me to preserve the original design of the server and still make it all work. Going off the aritcle provided here https://community.mellanox.com/docs/DOC-1845 it became apparent that I was able to add interfaces to my team.

I added a team interface for each VLAN and created an external switch for each team interface. I could swap between the new switches created and get DHCP perfectly.