I just got back from Cisco Live in Las Vegas and brought a nasty cold with me. With the extended time off work, I came back to a sizeable pile of “priorities”. One of the more interesting/challenging things I was working on was getting LACP going. We had some new rackmounts that were partially configured and had ESXi installed. They had two 10Gb ports which were handling all the traffic on standard virtual switches. The networking guy wanted to turn on LACP which is a best practice, but we couldn’t get it going at first for various reasons. It is one of those settings that has to be either on or off and both the host side and switch side have to be the same. Now that the project is nearing some deadlines, we decided to give it another go.
There are a couple key reasons you might want to setup Link Aggregation Control Protocol on uplink ports. 1. For faster failover in the event of a switch or port going offline. 2. Higher bandwidth for a single logical uplink.
VMware does a pretty nice job of handling failover without this turned on. VMs run on a single connection and jump over to the other connection if there are issues. So that might be reason enough not to go through the extra hassle of setting up LACP. Another reason is you might get yourself in a chicken/egg catch 22 scenario. If your vmkernal management ports have to run LACP, and you have a vCenter that runs on a host with only LACP available… you might have a hard time configuring your virtual distributed switch. You might be able to log into a remote console of the host and revert management network changes in 6.5, but I have not tested this. For this reason, I recommend using some different (perhaps a pair of onboard 1Gb ports) for your ESXi management network.
So step one is to get your host online with a VMware standard switch. Then you can deploy the vCenter 6.5 appliance to this host. You will need this to configure LACP. I would also recommend using the standard switch for vCenter and ESXi traffic. This can be done by editing the port group on the vCenter appliance VM.
On the physical switch side, you must setup a vPC. This is done by configuring a port channel on each switch port, then a virtual port channel that pairs the two ports.
Then in vCenter, you create a distributed virtual switch. Under the configuration tab there are LACP settings. First create a Link Aggregation Group. You will want to set this to active so the NIC will negotiate with the physical switch to aggregate the links. Create one LAG with the number of ports you will have for VM traffic in your entire cluster. This is one step that confused me. The documentation says to create one LAG per port channel ( https://docs.vmware.com/en/VMware-vSphere/6.0/com.vmware.vsphere.networking.doc/GUID-34A96848-5930-4417-9BEB-CEF487C6F8B6.html ), however, VMware handles creating a LAG for each host and you only need to create the overall LAG for the distributed switch. So basically I got one host setup pretty easily, but then when I went to setup my second host, I couldn’t add the second LAG into the uplinks options because two LAGs are not supported.
Once you create the LAG, you can now add hosts to the distributed switch and assign the physical NICs as uplinks with the LAG selected.
Lastly, create portgroups for each vlan. Then you can assign the LAG uplinks under teaming and failover for each port group.