ESXi 6.7 Multiple Subnets

I'm new to VM's as I've always setup physical servers in the past. I have installed VMWARE ESXi 6.7 on my HP server and I've setup 2 VM's. One is for a SCADA system and the other is for a Maintenance server. Both are running Windows 2016 Server.

I would like to have them on 2 different subnets... first on 192.168.2.x and the second is 192.168.1.x. I've used one of the 4 physical ports on the server to connect the Host to the .2 network and I can communicate fine to it. I've connected a second cable from a physical switch on my .1 network to a second port on the server. I've created a new vSwitch (vSwitch1) and linked it to the second port vmnic1. In the VM that I want to have on the .1 network, I've changed the hardware properties to have the network adapter point to Management Network 2 (A port group that uses the new vSwitch1).

Is this the right way to do this??? Although I've changed the Nic properties in Windows Server to a static IP on the .1 network, I can't communicate with this server... I think I'm missing something. Please advise and thank you for any help!!

Solution 1:

You are using almost all important terms... Let to do quick overview.

VMkernel NIC ... NIC which is used "internaly" for ESX for one or more management roles (also to access web management)
Physical NIC ... NIC physically available on the system which is used for uplink to outer network
Virtual Switch ... Logical interpretation of switch where is "interconnecting" Portgroups with physical NIC (there can be defined vSwitch without physical NIC)
Port group ... is used to connect VMs or VMkernel NIC

Portgroups has assigned VLAN ID so it is acting as logical switch in point of view of VLANs on physical switch. There is not problem to have more portgroups with the same VLAN ID on the same vSwitch. Once you are using 802.1q tagging on the physical switch (VLANs) you don't need to separate the traffic into 2 vSwitches. Two portgroups with defined VLAN ID on vSwitch will do the same stuff...

Once you will assigned VMkernel NIC to portgroup you cannot use the same portgroup for the VMs so even it would be in the same VLAN you have to define another port group on ESX even with the same VLAN ID on the same vSwitch. This second one can be then used for VMs and these two portgroups can then communicate together...

VLAN ID 0 on port group correspond to "native" VLAN (untagged) traffic on the connected physical NIC.

In networking (in general) the VLAN 1 is used for default VLAN and also used for e.g. spanning tree communication so I will try to avoid use VLAN 1 now to not cause misunderstanding...

Let assume

VLAN 11 for 192.168.1.0/24 (also used for mgmt of ESX in the following example)
VLAN 12 for 192.168.2.0/24

working option 1

    portgroup 1             vSwitch0               vmnic0
     VLAN11    | --------- |        | ----------- | 11T,12T
      VM1      |           |        |
                           |        |
                           |        |
    portgroup 2            |        |
     VLAN12    | --------- |        |
      VM2                  |        |
                           |        |
                           |        |
    portgroup mgmt         |        |
     vlan11    | --------- |        |
      vmk0     |           |        |

working option 2

    portgroup 1             vSwitch0               vmnic0
     VLAN0     | --------- |        | ----------- | 11,12T
      VM1      |           |        |
                           |        |
                           |        |
    portgroup 2            |        |
     VLAN12    | --------- |        |
      VM2                  |        |
                           |        |
                           |        |
    portgroup mgmt         |        |
      vlan0    | --------- |        |
      vmk0     |           |        |

not working option

The most probably this is your case...

    portgroup 1             vSwitch0               vmnic0
     VLAN0     | --------- |        | ----------- | 11
      VM1      |           |        |
                           |        |
                           |        |
    portgroup mgmt         |        |
     vlan0     | --------- |        |
      vmk0     |           |        |

    portgroup 2             vSwitch1               vmnic1
     VLAN12    | --------- |        | ----------- | 12
      VM2      |           |        |
                           |        |

In case I am right the issue would be that you are sending VLAN 12 as native ( not tagged ) traffic but portgroup expect specific VLAN which is not found there (as it is not tagged) so the communication is not working.

How to make it work

set VLAN on portgroup 2 to 0 so it will match native VLAN on vmnic1

portgroup 2             vSwitch1               vmnic1
 VLAN0     | --------- |        | ----------- | 12
  VM2      |           |        |
                       |        |

set VLAN on portgroup 2 to the same VLAN ID as what is used on external switch but on the switch send out the traffic as trunk with tagged VLAN (you can easily limit the allowed VLANs in the trunk just to this one VLAN)
```
portgroup 2             vSwitch1               vmnic1
 VLAN12    | --------- |        | ----------- | 12T
  VM2      |           |        |
                       |        |
```