It’s been a while since I wrote my last post. Many many things happened both in my life as well as work. Not an excuse for my lack of posting, but I do plan to get back to blogging more consistently in the year 2023.
What do we have here today?
In today’s edition, I bring you I script I wrote to move T1 gateways across Edge Clusters in NSX-T. It can programmatically move hundreds of T1s in a couple minutes.
Why would I need to move my T1 Gateways across Edge Clusters?
There are multiple scenarios that would trigger the need to move / evacuate T1 gateways to a different Edge Cluster. The most common are:
During a NSX-V to NSX-T migration, Migration Coordinator will, by default, put all T1 gateways in the same Edge Cluster that is being used for the T0s. In an architecture where there is a dedicated Edge Cluster for T0 ECMP / Uplink, and Edge Cluster/s used for T1 Stateful services (such as load balancing) this is not ideal.
A 10-node XL Edge Cluster can only host up to 400 Small load balancers. Going over this limit will require to build an additional Edge Cluster. vRA can only deploy Load Balancers to a single Edge Cluster at any given time per any given Network Profile. If we reach the limit, we either create the new cluster and change the network profile to the new cluster, or, we can migrate the current T1s to the new cluster, and keep using the one we previously had in the network profile
Rebalancing of T1 Gateways across Edge Cluster for maintain a similar number of T1s across all edge clusters.
How do I use this script?
In the initial comments of the script there is an explanation of the usage
<################################################
Move (T1s) across edge clusters
Author: @ldelorenzi - Jan 23
Usage:
moveT1s.ps1 -nsxUrl <NSX Manager URL (with HTTPS)) -sourceClusterName <Edge Cluster Name> -destinationClusterName <Edge Cluster Name> -execute <$true/$false> -count <count of load balancers to move>
Credentials will be asked at the beginning of the run
################################################>
To dive a little bit deeper into these parameters:
nsxUrl: The NSX-T manager we will be hitting with this script. Including HTTPS
sourceClusterName: The name of the Edge Cluster that hosts the T1 Gateways we want to move
destinationClusterName: The name of the Edge Cluster that will receive the T1 Gateways from the source cluster
execute: By default, the execute flag is set to false. This means that if no value is used in the script, it will default to false. If execute is false, the script will only show us what T1s were found in the source cluster and therefore will be moved to the destination cluster
count: If you don’t want to fully evacuate the cluster and you want to just move some T1s from source cluster to destination cluster, you can set a value for the count parameter and that will limit the amount of T1s that are moved.
Interesting things about the Script
If you look at the code you will see that I built my own wrapper for invoke-RestMethod called restCall – this function has logging included as well as retries. If you’re going to have a lot of REST API calls in your scripts, it could make sense to include something like this!
The ‘movement’ of T1 Gateways actually involves patching the T1 SR object with its new cluster ID. The script finds the Edge Cluster IDs using the names provided at the beginning of the run. This makes it friendlier for users / admins since just the name can be used instead of having to find the id.
Closing Note
I hope you enjoy this post and make use of this script in your environments. If you liked this, please share it!
I’m going to explain it bit by bit, mostly focusing on the logic and the problems I encountered while testing it.
The end result would be something like this:
Enter vCenter FQDN: vcsa-01a.corp.local
PowerShell credential request
Enter vCenter Credentials
User: administrator@vsphere.local
Password for user administrator@vsphere.local: ********
Enter NSX Manager FQDN: nsxapp-01a.corp.local
PowerShell credential request
Enter NSX Credentials
User: admin
Password for user admin: ****************
Enter NSX Overlay Transport Zone name: nsx-overlay-transportzone
Name Port User
---- ---- ----
vcsa-01a.corp.local 443 VSPHERE.LOCAL\Administrator
Querying data for rdsh-01a ...
Querying data for log-01a ...
Querying data for web-01a ...
Querying data for vm-01a ...
Querying data for app-01a ...
Querying data for web-02a ...
Querying data for edgenode-01a ...
############################################################
Found the following possible segments in your infrastructure
Portgroup PG-WEB with VLAN 100 gateway 172.16.10.1 and prefix length 24
Portgroup PG-VM with VLAN 200 gateway 172.16.101.1 and prefix length 24
Portgroup PG-APP with VLAN 300 gateway 172.16.20.1 and prefix length 24
Would you like to Create these segments on NSX-T?
( y / n ) : y
Yes, create segments
found transport zone id: 1b3a2f36-bfd1-443e-a0f6-4de01abc963e
Creating Segment PG-WEB-VLAN100-GW-172.16.10.1 on transport zone nsx-overlay-transportzone
Creating Segment PG-VM-VLAN200-GW-172.16.101.1 on transport zone nsx-overlay-transportzone
Creating Segment PG-APP-VLAN300-GW-172.16.20.1 on transport zone nsx-overlay-transportzone
Simple, right?
So let’s start breaking the script up in parts…
Part 1 – Getting FQDNs and Credentials
This is pretty self explanatory, we’re just getting the FQDNs and credentials and saving them into variables.
We’re iterating through every VM in the infrastructure, and getting the IP stack (which is part of the extensiondata.guest object, therefore, being read from VMware tools – This would be empty if this VM does not have VMware tools running)
We’re getting the device (virtual nic) that has the portgroup with the default gateway (this is needed in the scenario of multiple virtual nics and multiple portgroups) This is important because the VMware tools data is not available in vSphere, so we need to do the mapping ourselves.
We filter stuff out that we don’t need, such as:
Any network with less than 6 characters (ipv6 empty network)
Any network that does not have “.” on the address (so no ipv4)
Any prefix length that is 0 or 32 (useless in this scenario, this is not the gateway network)
Any network that starts with 224. or 169.254
After having filtered that, we’re going to have our gateway and network prefix, so what’s next?
Using the device, we get the portgroup that we’re going to use, and from that portgroup, we get the VLAN configuration and VLAN ID, and we discard it if it is a trunk portgroup
We will also discard the portgroup if it contains “vxw-dvs” because this will mean it is a NSX-V portgroup and won’t be VLAN backed
We create a new object that will contain:
Portgroup Name
Portgroup VLAN ID
Gateway
Network prefix
And we add this object to an array of objects
foreach ($vm in $vms) {
$networkObject = "" | Select Portgroups,Gateway,Prefix
$networkObject.Portgroups = ($vm | Get-NetworkAdapter | Get-VDPortgroup)
Write-Host Querying data for $vm...
if ($vm.extensiondata.guest.ipstack){
$device = ($vm.extensiondata.guest.ipstack[0].iprouteconfig.iproute | where {$_.network -eq "0.0.0.0"}).gateway.device
$networkObject.gateway = ($vm.extensiondata.guest.ipstack[0].iprouteconfig.iproute |
where {$_.network -eq "0.0.0.0"}).gateway.ipaddress
$networkObject.Prefix = ($vm.extensiondata.guest.ipstack[0].iprouteconfig.iproute |
where {$_.network.length -gt 6} | where {$_.network -like "*.*"} |
where {$_.prefixlength -ne 32} | where {$_.network.substring(0,4) -ne "224."} |
where {$_.prefixlength -ne 0} | where {$_.network.substring(0,8) -ne "169.254."} |
where {$_.gateway.device -eq $device}).prefixlength
if (($vm | Get-NetworkAdapter)[$device]){
$pg = ($vm | Get-NetworkAdapter)[$device] | get-vdportgroup
}
$PGObject = "" | Select Name, VLAN, Gateway, PrefixLength
$PGObject.Name = $pg.name
$PGObject.VLAN = $pg.VlanConfiguration.VlanId
$PGObject.Gateway = $networkObject.Gateway
$PGObject.PrefixLength = $networkObject.Prefix
#Skip Trunk vLAN
if ($pg.VlanConfiguration.vlantype -ne 'Trunk' -and $pg.name -notlike "*vxw-dvs*" -and $pg.name -ne $null){
$PossibleSegments += $PGObject
}
}
Part 3 – Parsing the data
We have an array of objects that have all the data we need, but this will likely have many repeated entries, since a lot of VMs are going to be using the same portgroup and same gateway. We could use a single entry per portgroup, but this will not be ideal. There is nothing stopping anyone from using multiple networks inside a same portgroup and VLAN, so the ‘uniqueness’ of the segment will be given by a combination of the portgroup as well as the gateway In that way, we will have all the data we need and won’t discard anything useful!
$UniqueSegments = $PossibleSegments | Where {$_.Gateway -ne $null} | sort-object -Property @{E="Name"; Descending=$True}, @{E="Gateway"; Descending=$True} -unique
Write-Host "############################################################"
Write-Host "Found the following possible segments in your infrastructure"
$uniqueSegments | % {
Write-Host Portgroup $_.name with VLAN $_.VLAN, gateway $_.gateway and prefix length $_.prefixlength
}
Part 4 – Pushing the data to NSX-T
So now that we have our array of segment objects fully sorted out and having unique entries, we need to push it to NSX-T
Remember at the beginning I asked for the NSX-T Overlay Transport zone? We’re going to need the transport zone ID
I hope you find this script useful – It should GREATLY improve times in fresh NSX-T deployments, not only by quickly creating all the segments automatically, but also since it does all the hard work of exporting all the gateway and prefix configuration, which can be super tedious!
If you have any questions regarding the script, please leave it on the comments below and I’ll address it
If you found any bugs or errors or better ways to accomplish the same thing, please also leave a comment! It will also be super helpful!