OTV terminology

Within OTV there are several terms that might be confusing. This is a list of all the terms I’ve encountered so far with an explanation.

  • OTV edge device: This is the switch that performs all the OTV operations. Layer 2 frames enter the switch and if they need to be transported over the overlay network they will be encapsulated here and sent out of the join interface. All the OTV configuration is done on these devices (with the possible exception of MTU configuration)
  • OTV internal interface: This is the interface on the edge device that points toward the datacenter network. It is a layer 2 interface and it needs to support all the vlans that need to be extended across the overlay network. To be able to do so it must be a (dot1q) trunk port.
  • OTV join interface: This is the interface on the edge device that points toward the routed network that carries the data toward the other datacenters.
  • OTV overlay interface: A logical interface on the edge device. A large part of the OTV configuration is done on this interface. It ensures the encapsulation of the layer 2 frames into layer 3 packets.
  • Transport network: This is the IP network that will carry the encapsulated layer 2 frames across to the other datacenter(s). The only requirement to this network is that it needs to be an IP network. It must also support jumboframes or you need to reduce the MTU in your network to ensure the additional overhead of OTV does not cause any problems (OTV does not support fragmentation)
  • Overlay network: The logical network that connects OTV devices.
  • Site: Most often a datacenter, but nothing prevents you from connecting a campus to a OTV network, so it might be a campus as well.
  • Site vlan: The vlan used by OTV edge devices within a site to communicate. This vlan must not be extended across the transport network. All OTV edge devices within a site need to agree on the site vlan. Because the vlan is not extended you could in theory use the same vlan on all sites.
  • Authoritative Edge Device: When using multiple OTV edge devices an AED is required. This is the device responsible for forwarding the data on a specific vlan. An AED is configured per vlan, so when using two OTV edge devices you can use both actively but for different vlans. (Only one Edge device is allowed to forward traffic for a specific vlan because of the removal of BPDU’s. If both devices were to forward traffic this might cause loops or MAC flapping.)
  • Site-ID: An identifier for the site that all Edge devices in that site must agree upon.
  • OTV Shim: OTV specific header which is added to the OTV encapsulated data to clarify among others the vlan from which the frame originated (and must be restored to)

I might encounter other OTV specific terms, and if I do I will list them here.

Overlay Transport Virtualization basics

I’ve had the privilege of working with OTV on several occasions the last couple of years and I’ve fallen in love with the technology. The simplicity of the technology is striking, even though the configuration can be complex in some cases.

OTV is designed solely as a datacenter interconnect (DCI) technology, and as such it is very efficient in it. Even though other solutions are available based on newer technologies like VXlan (for example ACI) OTV still has its merits (I believe).

The goal of OTV is to extend layer 2 segments across multiple datacenters. In early years this was a no-go area for many companies, limiting their (virtual) environments to a single datacenter. This severely limited their options for scalability and robustness. The reason many companies did not want to extend their layer 2 segments across datacenters is that it introduced many risks for catastrophic failure. With an extended vlan between datacenters you risked broadcast storms or spanning tree misconfigurations to bring down your entire environment. To be honest, I’ve seen this happen on several occasions.

What makes OTV so special is that it enables you to extend your layer 2 segments without any danger. It is built in such a way that it creates multiple failure domains. It does this by blocking BPDU’s at the datacenter edge and using a IP transport network. The layer 2 frames are encapsulated in a GRE like manner and transported to the other datacenter (only when needed). There they are decapsulated and sent on their way.

OTV has several smart constructs to limit the amount of traffic between two datacenters, for example ARP suppression, in which ARP’s are handled locally when possible. Only if the system that needs to respond to the ARP is in another datacenter and is not yet known to the OTV edge switches will the ARP request be forwarded to the other datacenter(s).

There is a lot to tell about OTV and probably I will create more posts about OTV in the future.

Cisco FabricPath

Cisco FabricPath is the answer to Spanning Tree. Whereas Spanning Tree offers no multipathing, inefficient routes, slow convergence, limited scalability and a high risk of a total network meltdown, FabricPath has none of these issues. FabricPath is scalable, supports multipathing, converges very fast, has efficient route selection and is easy to configure.

The reason we need a protocol like STP is because ethernet does not have any possibility to detect wether a frame should be dropped. There is no TTL like in IP. This causes frames to traverse the network endlessly. FabricPath does not need STP because it solves the ethernet challenges differently.

First of all, FabricPath does not forward frames in the same manner as a traditional network does. When a frame enters a FabricPath network it gets encapsulated in a FabricPath header. Within this header there is a different source and destination address. These addresses (the Outer mac Destination Address, or ODA and the Outer mac Source Address, or OSA) are based on the devices in the FabricPath network. When a frame enters the FabricPath network the FabricPath switch does a lookup on the destination address. If it is local it handles it locally, but if not it will find during the lookup the switchID of the destination switch the frame should be sent to. The switchID of this switch is put into the ODA. Now the FabricPath network knows where to send the frame.

But how does the FabricPath switch know which switch it should send the frame? It does this by MAC address learning. The same principle like normal switches, but with the exception that the addresses are only learned when there is a bi-directional traffic flow and not the outgoing port is recorded, but the destination switchID.

Bi-directional, or conversational learning of mac addresses works as follows:

  • For frames received on edge ports (the ports on which systems or ‘normal’ ethernet networks can be connected) the mac address is learned as usual.
  • Frames received on root ports (ports which are part of the FabricPath network) will only have their mac address learned if the destination mac address is already in the mac address table as a local address.
  • Broadcast frames will not trigger address learning, but will update existing entries
  • Multicast frames will trigger mac learning

An important part of FabricPath is the way switches can find each other. This is based on a implementation of the IS-IS routing protocol. This protocol is slightly changed to offer support for all the FabricPath features.

Configuring FabricPath is easy. This requires a Nexus switch with the correct licenses (enhanced L2) and the hardware support for FabricPath. If you have those available you can configure it with the following commands:

install feature-set fabricpath
feature-set fabricpath
vlan 
  mode fabricpath
interface 
  switchport mode fabricpath

And that’s it really.

Cisco Nexus Password recovery

Password recovery on a nexus is slightly different from recovery on catalyst switches, but the principle is the same. Physical access to a switch is required since a reboot is necessary. For a step by step description on the Cisco site go to Cisco NX-OS password recovery

When the switch is booting you need to press the escape sequence (CTRL+]) at the right time to get dropped into the boot prompt. Unfortunately this doesn’t work in a VIRL setup.

Executing Mod 1 2 SEEPROM Test….done
Mod 1 2 Post Completed Successfully
Mod 3 Post Completed Successfully
POST is completed
Checking all filesystems….r. done.
Ctrl-]
switch(boot)#

Once the boot prompt is shown it is possible to change the password. So use configure terminal and change the password with the admin-password command.

Exit the configure mode and use the load bootflash:nx-os.bin command. The switch now boots to normal operations and if required you can change the password or enable AAA after the boot process has been completed.

Another method is interrupting the boot process while the switch is still loading the kickstart image. This is done when the switch displays that it’s loading the kickstart image and can be done by pressing CTRL+C.

Autobooting bootflash:/n7000-s1-kickstart.x.x.x.bin bootflash:/n…
Filesystem type is ext2fs, partition type 0x83
Booting kickstart image: bootflash:/n7000-s1-kickstart.x.x.x.bin….(—-> Press Ctrl + C)
….Aborting Image Boot

After this you will be dropped into the loader prompt. Now you can load the kickstart image. This method will stop booting after the kickstart image while the normal bootprocess will also load the bin file and thus the full NX-OS. Do this by using the boot n7000-s1-kickstart.x.x.x.x.bin command. Notice the difference between the command used to load the bin file and the command used to boot the kickstart file. Once the kickstart file has been loaded you will be dropped into the boot prompt from which you can change the admin password in the same way I described earlier.

Let’s Encrypt public Beta

Let’s Encrypt has entered its public beta period. This means that everybody can request certificates from let’s encrypt. Another great plus is that the CA certificate is now trusted (at least in Chrome). The name of the CA has changed from “Happy Hacker Fake CA” to Let’s Encrypt Authority X1. Valid until October 2020.

There is still some work to do as the client often generates errors, but when following the instructions on the Let’s Encrypt website everybody can install a certificate this way.

1 2 3 4