Когда речь заходит о коммутаторах Cisco Nexus, один из первых вопросов, который мне задают: поддерживается ли на них стекирование? Услышав отрицательней ответ, следует логичное «Почему?».
Ответ – стек коммутаторов может служить единой точкой отказа. При этом Nexus позиционируется, как коммутатор в ЦОД, где отказоустойчивость стоит на одном из первых мест.
«Но вы ведь сами писали (часть 1, часть 2, VSS/IRF), что на базе стека можно построить отказоустойчивую инфраструктуру! Получается, обманывали?». Нет, ни в коем случае. Каждая технология уместна там, где её минусы не настолько критичны для работы сети, а плюсы дают ощутимые преимущества. Вот и со стеком ситуация аналогичная.
Стекирование обладает двумя основными преимуществами:
Поддержка MC-LAG (в терминах Cisco – Port channel) позволяет:
Общий control plane предоставляет ещё один приятный бонус — единую точку маршрутизации трафика в стеке. А значит необходимости настраивать протоколы обеспечения резервирования шлюза по умолчанию (First Hop Redundancy Protocols) нет.
Таким образом, стекирование предполагает общий control plane и management plane. Несмотря на все достоинства, это возможная точка отказа. И хоть аппаратно коммутаторы независимы, бывают сбои (не связанные с железом), при которых стек может перестать корректно функционировать. Например, если control plane на основном коммутаторе «зависнет» из-за утечки оперативной памяти. Последствия могут быть различными: потеря управления стеком, прекращение работы различных протоколов (например, LACP). При этом стек может продолжать передавать трафик. Ведь ASIC’и заполнены необходимыми данными и фактически не зависят от работы control plane. Но все динамические агрегации (MC-LAG) «развалятся», так как пакеты LACP перестанут отправляться на соседнее устройство.
Ещё одна возможная проблема – ситуация, когда сразу несколько коммутаторов решают, что они являются активными («split brain»). Так как конфигурация у них идентичная, имеем в сети два устройства с одинаковой адресацией. Происходит это из-за разрыва управляющего канала. Конечно же, существуют технологии, направленные на борьбу с таким явлением. В этом случае коммутаторы используют дополнительные механизмы отслеживания состояния соседей. Да и управляющий канал на некоторых типах стека сложно разорвать. Но не стоит сбрасывать со счетов такую ситуацию.
Таким образом, стек – хорошее решение для сетей, где его поломка не фатальна. Да, его нельзя назвать полностью отказоустойчивым решением. Но вероятность наступления критической ситуации не так велика. И с лихвой может окупиться теми преимуществами, которые он предоставляет.
Коммутаторы Nexus позиционируются как решение для сред (в первую очередь ЦОД’ов), где очень важна отказоустойчивость. Поэтому на этих устройствах стекирование вообще отсутствует. Замечу, область применения Nexus не ограничена только ЦОДами. Они могут использоваться, в том числе, при построении корпоративной сети.
Но стекирование имеет весомые плюсы. Поэтому Nexus’ы поддерживают ряд технологий, позволяющие получить их, не объединяя коммутаторы в стек.
Для реализации функций MC-LAG используется технология virtual Port-channel (vPC). Каждый Nexus имеет свой независимый control и management plane. При этом мы можем агрегировать каналы, распределённые между двумя коммутаторам. Безусловно мы не получаем полной независимости устройств. В процессе работы коммутаторы синхронизируют между собой информацию, необходимую для работы агрегации (MAC-адреса, ARP и IGMP записи, состояние портов). Но с точки зрения отказоустойчивости это всё же лучше, чем единый control и management plane. Эта схема более надёжна. Даже если и произойдёт сбой в работе vPC, он будет менее фатален для инфраструктуры.
Однако vPC привносит особые нюансы работы. Настроить его можно только между двумя коммутаторами Nexus, и они оба должны иметь набор идентичных настроек. Некоторые функции требуют небольших дополнительных настроек, которые при работе обычного стека не нужны. Например, корректная маршрутизация трафика между двумя vPC портами предполагает наличие команды «peer-gateway». Иначе можно споткнуться об механизм предотвращения петель при передаче трафика через vPC. Работа динамической маршрутизации через vPC требует «layer3 peer-router». Казалось бы, мелочь, а нервы попортить может. Не все технологии совместимы в своей работе с vPC. Причём это зависит достаточно сильно от модели Nexus’а. Стоит внимательно смотреть configuration guide. В общем, как обычно, у всего есть свои плюсы и минусы.
vPC в среде FabricPath носит название vPC+.
В случае классического vPC синхронизация происходит через выделенный канал peer link. В случае работы vPC в рамках ACI фабрики, канал peer link не требуется. Вся синхронизация происходит через фабрику.
В плане единой точки управления всеми коммутаторами отсутствие стекирования компенсируется следующими моментами:
Так как вся логика FEX’ов реализуется на родительском Nexus’е, FEX’ы и родительский коммутатор – единая точка отказа. На FEX’ах нет локальной коммутации. Пакеты между соседними портами передаются через родительское устройство. А значит имеем повышенную нагрузку на канал между ними.
Стоит помнить: чтобы решение было отказоустойчивым в нём не должно быть зависимых частей. Любые технологии, протоколы, обеспечивающие отказоустойчивость, могут также являться причиной отказов. Более того благодаря им, проблемы могут переходить с одного устройства на другое. Ни что не идеально. Поэтому если вопрос отказоустойчивости является определяющим, нужно стараться строить сеть таким образом, чтобы влияние устройств друг на друга было минимальным.
This chapter describes the best practices and operational procedures for the virtual port channel (vPC) feature on Cisco Nexus 5000 Series switches that run Cisco NX-OS Release 5.0(2)N2(1) and earlier releases.
This chapter includes the following sections:
Information About vPC Operations
A vPC allows links that are physically connected to two different Cisco Nexus 5000 Series switches to appear as a single port channel to a third switch. The third switch can be a Cisco Nexus 2000 Series Fabric Extender or a switch, server, or any other networking device. A vPC can provide Layer 2 multipath capability which allows you to create redundancy by increasing bandwidth, enabling multiple parallel paths between nodes, and load-balancing traffic where alternative paths exist.
For a quick overview of vPC configurations, see the Virtual PortChannel Quick Configuration Guide at the following URL: http://www.cisco.com/en/US/prod/collateral/switches/ps9441/ps9670/configuration_guide_c07-543563.html
vPC Consistency Checks
This section includes the following topics:
Type 1 and Type 2 Consistency Check Parameters
Before a Cisco Nexus 5000 Series switch brings up a vPC, the two Cisco Nexus 5000 Series switches in the same vPC domain exchange configuration information to verify if both switches have compatible configurations for a vPC topology. Depending on the severity of the impact of possible mismatched configurations, some configuration parameters are considered as Type 1 consistency check parameters while others are considered as Type 2.
When a mismatch in Type 1 parameters occur, the following applies:
NoteThe graceful consistency check is a new feature introduced in Cisco NX-OS Release 5.0(2)N2(1) and is enabled by default. For more details, see the “Graceful Consistency Check” section.
When Type 2 parameters exist, a configuration mismatch generates a warning syslog message. The vPC on the Cisco Nexus 5000 Series switch remains up and running. The global configuration, such as Spanning Tree Protocol (STP), and the interface-level configurations are included in the consistency check.
The show vpc consistency-parameters global command lists all global consistency check parameters. Beginning with Cisco NX-OS Release 5.0(2)N1(1), QoS parameters have been downgraded from Type 1 to Type 2.
This example shows how to display all global consistency check parameters:
Use the show vpc consistency-parameters interface port-channel number command to display the interface-level consistency parameters.
This example shows how to display the interface-level consistency parameters:
The Cisco Nexus 5000 Series switch conducts vPC consistency checks when it attempts to bring up a vPC or when you make a configuration change.
In the interface consistency parameters shown in the above output, all configurations except the Allowed VLANs are considered as Type 1 consistency check parameters. The Allowed VLAN (under the trunk interface) is considered as a Type 2 consistency check parameter. If the Allowed VLAN ranges are different on both VLANs that means that only common VLANs are active and trunked for the vPC while the remaining VLANs are suspended for this port channel.
Graceful Consistency Check
Beginning with Cisco NX-OS Release 5.0(2)N2(1) and later releases, when a Type 1 mismatch occurs, by default, the primary vPC links are not suspended. Instead, the vPC remains up on the primary switch and the Cisco Nexus 5000 Series switch performs Type 1 configurations without completely disrupting the traffic flow. The secondary switch brings down its vPC until the inconsistency is cleared.
However, in Cisco NX-OS Release 5.0(2)N2(1) and earlier releases, this feature is not enabled for dual-homed FEX ports. When Type-1 mismatches occur in this topology, the VLANs are suspended on both switches. The traffic is disrupted on these ports for the duration of the inconsistency.
To minimize disruption, we recommend that you use the configuration synchronization feature for making configuration changes on these ports.
To enable a graceful consistency check, use the graceful consistency-check command. Use the no form of this command to disable the feature. The graceful consistency check feature is enabled by default.
This example shows how to enable a graceful consistency check:
This example shows that the vPC ports are down on a secondary switch when an STP mode mismatch occurs:
This example shows that the vPC ports and the VLANs remain up on the primary switch when an STP mode mismatch occurs:
This example shows that the vPC ports are down on a secondary switch when an interface-level Type 1 inconsistency occurs:
This example shows that the vPC ports and the VLANs remain up on the primary switch when an interface-level Type 1 inconsistency occurs:
Configuring Per-VLAN Consistency Checks
Beginning with Cisco NX-OS Release 5.0(2)N2(1), the Cisco Nexus 5000 Series switch performs Type-1 consistency checks on a per-VLAN basis when you enable or disable STP on a VLAN. VLANs that do not pass this consistency check are brought down on the primary and secondary switches while other VLANs are not affected.
When you enter the no spanning-tree vlan number command on one peer switch, only the specified VLAN is suspended on both peer switches; the other VLANs remain up.
NotePer-VLAN consistency checks are not dependent on whether graceful consistency checks are enabled.
This example shows the active VLANs before suspending a specified VLAN:
This example shows that VLAN 5 is suspended but the remaining VLANs are up:
Identifying Inconsistent vPC Configurations
The show vpc command displays the vPC status and the vPC consistency check result for the global consistency check and the interface-specific consistency check.
This example shows the global vPC consistency check failed because of the mismatched Network QoS configuration:
You can use the show vpc consistency-parameters global command to identify the configuration difference between two vPC peer switches.
This example shows the global consistency check failed because the STP mode was configured differently on the two vPC switches:
You can use the show vpc command also shows the vPC consistency check result for each vPC and the reason for the consistency check failure.
This example shows how to display the vPC consistency check status:
If the consistency check fails, the consistency check is not performed on vPC member ports that are down.
If the consistency check has succeeded and the port is brought down, the consistency check shows that it was successful.
You can use the show vpc consistency-parameters interface ethernet slot/port command to identify the configuration difference that leads to a consistency check failure for a specified interface or port channel.
This example shows how to display configuration differences that lead to consistency check failures.
Bypassing a vPC Consistency Check When a Peer Link is Lost
The vPC consistency check message is sent by the vPC peer link. The vPC consistency check cannot be performed when the peer link is lost. When the vPC peer link is lost, the operational secondary switch suspends all of its vPC member ports while the vPC member ports remain on the operational primary switch. If the vPC member ports on the primary switch flaps afterwards (for example, when the switch or server that connects to the vPC primary switch is reloaded), the ports remain down due to the vPC consistency check and you cannot add or bring up more vPCs.
Beginning with Cisco NX-OS Release 5.0(2)N2(1), the auto-recovery feature brings up the vPC links when one peer is down. This feature performs two operations:
NoteThe auto-recovery feature in Cisco NX-OS Release 5.0(2)N2(1) and later releases replaces the reload restore feature in Cisco NX-OS Release 5.0(2)N1(1) and earlier releases.
The auto-recovery feature is disabled by default. To enable auto-recovery, enter the auto-recovery command in the vPC domain mode.
This example shows how to enable the auto-recovery feature and to set the reload delay period:
This chapter describes how to configure virtual port channels (vPCs) on Cisco Nexus 5000 Series switches. It contains the following sections:
Information About vPCs
vPC Overview
Figure 1. vPC Architecture
You configure the EtherChannels by using one of the following:
Note
You must enable the vPC feature before you can configure or run the vPC functionality.
To enable the vPC functionality, you must create a peer-keepalive link and a peer-link under the vPC domain for the two vPC peer switches to provide the vPC functionality.
To create a vPC peer link you configure an EtherChannel on one Cisco Nexus 5000 Series switch by using two or more Ethernet ports. On the other switch, you configure another EtherChannel again using two or more Ethernet ports. Connecting these two EtherChannels together creates a vPC peer link.
Note
We recommend that you configure the vPC peer-link EtherChannels as trunks.
The vPC domain includes both vPC peer devices, the vPC peer-keepalive link, the vPC peer link, and all of the EtherChannels in the vPC domain connected to the downstream device. You can have only one vPC domain ID on each vPC peer device.
Note
Always attach all vPC devices using EtherChannels to both vPC peer devices.
A vPC provides the following benefits:
vPC Terminology
The terminology used in vPCs is as follows:
vPC—The combined EtherChannel between the vPC peer devices and the downstream device.
vPC peer device—One of a pair of devices that are connected with the special EtherChannel known as the vPC peer link.
vPC peer link—The link used to synchronize states between the vPC peer devices.
vPC member port—Interfaces that belong to the vPCs.
Host vPC port— Fabric Extender host interfaces that belong to a vPC.
vPC domain—This domain includes both vPC peer devices, the vPC peer-keepalive link, and all of the port channels in the vPC connected to the downstream devices. It is also associated to the configuration mode that you must use to assign vPC global parameters. The vPC domain ID must be the same on both switches.
vPC peer-keepalive link—The peer-keepalive link monitors the vitality of a vPC peer Cisco Nexus 5000 Series device. The peer-keepalive link sends configurable, periodic keepalive messages between vPC peer devices.
No data or synchronization traffic moves over the vPC peer-keepalive link; the only traffic on this link is a message that indicates that the originating switch is operating and running vPCs.
Fabric Extender Terminology
The terminology used for the Cisco Nexus 2000 Series Fabric Extender is as follows:
Fabric interface—A 10-Gigabit Ethernet uplink port designated for connection from the Fabric Extender to its parent switch. A fabric interface cannot be used for any other purpose. It must be directly connected to the parent switch.
EtherChannel fabric interface—An EtherChannel uplink connection from the Fabric Extender to its parent switch. This connection consists of fabric interfaces bundled into a single logical channel.
Host interface—An Ethernet interface for server or host connectivity. These ports are 1-Gigabit Ethernet interfaces or 10-Gigabit Ethernet interfaces, depending on the fabric extender model.
EtherChannel host interface—An EtherChannel downlink connection from the Fabric Extender host interface to a server port.
Note
In Release 4.1(3)N1(1), an EtherChannel host interface consists of only one host interface and can be configured either as a Link Aggregation Control Protocol (LACP) or non-LACP EtherChannel.
Supported vPC Topologies
Cisco Nexus 5000 Series Switch vPC Topology
You can connect a pair of Cisco Nexus 5000 Series switches or a pair of Cisco Nexus 5500 Series switches in a vPC directly to another switch or to a server. vPC peer switches must be of the same type, for example, you can connect a pair of Nexus 5000 series switches or a pair of Nexus 5500 Series switches but you cannot connect a Nexus 5000 Series switch to a Nexus 5500 Series switch in a vPC topology. Up to 8 interfaces could be connected to each Cisco Nexus 5000 Series switch providing 16 interfaces bundled for the vPC pair. The topology that is shown in the following figure provides the vPC functionality to dual connected switches or servers with 10-Gigabit or 1-Gigabit Ethernet uplink interfaces.
Figure 2. Cisco Nexus 5000 Series Switch-to-Switch vPC Topology
The first 8 ports on the Cisco Nexus 5010 switch and the first 16 ports on the Cisco Nexus 5020 switch are switchable 1-Gigabit and 10-Gigabit ports. You can enable vPC functionality on these ports in 1-Gigabit mode.
The switch connected to the pair of Cisco Nexus 5000 Series switches can be any standards-based Ethernet switch. Common environments to use this configuration include Blade Chassis with dual switches connected to the pair of Cisco Nexus 5000 Series switches through vPC or Unified Computing Systems connected to the pair of Cisco Nexus 5000 Series switches.
Single Homed Fabric Extender vPC Topology
You can connect a server with dual or quad or more network adapters that are configured in a vPC to a pair of Cisco Nexus 2000 Series Fabric Extenders which are connected to the Cisco Nexus 5000 Series switches as depicted. Depending on the FEX model, you may be able to connect one or more network adapter interfaces to each fabric extender. As an example, Figure 10 refers to a topology built with the Cisco Nexus 2148T fabric extender, where a server has one link only to each fabric extender. A topology with Cisco Nexus 2248TP or with Cisco Nexus 2232PP fabric extender could consist of more links from the server to a single fabric extender.
. The topology that is shown in the following figure provides the vPC functionality to dual homed servers with 1-Gigabit Ethernet uplink interfaces.
Figure 3. Single Homed Fabric Extender vPC Topology
The Cisco Nexus 5000 Series switch can support up to 12 configured single homed Fabric Extender s (576 ports) with this topology however only 480 576 dual homed host servers can be configured in a vPCs with this configuration.
Note
The Cisco Nexus 2148T fabric extender does not support EtherChannels on its host interfaces. Therefore a maximum of two links can be configured in an EtherChannel from the server where each link is connected to a separate Fabric Extender.
Dual Homed Fabric Extender vPC Topology
You can connect the Cisco Nexus 2000 Series Fabric Extender to two upstream Cisco Nexus 5000 Series switches and downstream to a number of single homed servers. The topology shown in the following figure provides the vPC functionality to singly connected servers with 1-Gigabit Ethernet uplink interfaces.
Figure 4. Dual Homed Fabric Extender vPC Topology
The Cisco Nexus 5000 Series switch can support up to 12 configured dual homed Fabric Extender s with this topology. A maximum of 576 single homed servers can be connected to this configuration.
vPC Domain
To create a vPC domain, you must first create a vPC domain ID on each vPC peer switch using a number from 1 to 1000. This ID must be the same on a set of vPC peer devices.
You can configure the EtherChannels and vPC peer links by using LACP or no protocol. When possible, we recommend that you use LACP on the peer-link, because LACP provides configuration checks against a configuration mismatch on the etherchannel.
The vPC peer switches use the vPC domain ID that you configure to automatically assign a unique vPC system MAC address. Each vPC domain has a unique MAC address that is used as a unique identifier for the specific vPC-related operations, although the switches use the vPC system MAC addresses only for link-scope operations, such as LACP. We recommend that you create each vPC domain within the contiguous network with a unique domain ID. You can also configure a specific MAC address for the vPC domain, rather than having the Cisco NX-OS software assign the address.
The vPC peer switches use the vPC domain ID that you configure to automatically assign a unique vPC system MAC address. The switches use the vPC system MAC addresses only for link-scope operations, such as LACP or BPDUs. You can also configure a specific MAC address for the vPC domain.
After you create a vPC domain, the Cisco NX-OS software automatically creates a system priority for the vPC domain. You can also manually configure a specific system priority for the vPC domain.
Note
If you manually configure the system priority, you must ensure that you assign the same priority value on both vPC peer switches. If the vPC peer switches have different system priority values, the vPC will not come up.
Peer-Keepalive Link and Messages
The Cisco NX-OS software uses a peer-keepalive link between the vPC peers to transmit periodic, configurable keepalive messages. You must have Layer 3 connectivity between the peer switches to transmit these messages; the system cannot bring up the vPC peer link unless a peer-keepalive link is already up and running.
If one of the vPC peer switches fails, the vPC peer switch on the other side of the vPC peer link senses the failure when it does not receive any peer-keepalive messages. The default interval time for the vPC peer-keepalive message is 1 second. You can configure the interval between 400 milliseconds and 10 seconds. You can also configure a timeout value with a range of 3 to 20 seconds; the default timeout value is 5 seconds. The peer-keepalive status is checked only when the peer-link goes down.
The vPC peer-keepalive can be carried either in the management or default VRF on the Cisco Nexus 5000 Series switch. When you configure the switches to use the management VRF, the source and destination for the keepalive messages are the mgmt 0 interface IP addresses. When you configure the switches to use the default VRF, an SVI must be created to act as the source and destination addresses for the vPC peer-keepalive messages. Ensure that both the source and destination IP addresses used for the peer-keepalive messages are unique in your network and these IP addresses are reachable from the VRF associated with the vPC peer-keepalive link.
Note
We recommend that you configure the vPC peer-keepalive link on the Cisco Nexus 5000 Series switch to run in the management VRF using the mgmt 0 interfaces. If you configure the default VRF, ensure that the vPC peer link is not used to carry the vPC peer-keepalive messages.
Compatibility Parameters for vPC Peer Links
Many configuration and operational parameters must be identical on all interfaces in the vPC. After you enable the vPC feature and configure the peer link on both vPC peer switches, Cisco Fabric Services (CFS) messages provide a copy of the configuration on the local vPC peer switch configuration to the remote vPC peer switch. The system then determines whether any of the crucial configuration parameters differ on the two switches.
Enter the show vpc consistency-parameters command to display the configured values on all interfaces in the vPC. The displayed configurations are only those configurations that would limit the vPC peer link and vPC from coming up.
The compatibility check process for vPCs differs from the compatibility check for regular EtherChannels.
Configuration Parameters That Must Be Identical
The configuration parameters in this section must be configured identically on both switches at either end of the vPC peer link.
Note
You must ensure that all interfaces in the vPC have the identical operational and configuration parameters listed in this section.
Enter the show vpc consistency-parameters command to display the configured values on all interfaces in the vPC. The displayed configurations are only those configurations that would limit the vPC peer link and vPC from coming up.
The switch automatically check for compatibility of these parameters on the vPC interfaces. The per-interface parameters must be consistent per interface, and the global parameters must be consistent globally.
If any of these parameters are not enabled or defined on either switch, the vPC consistency check ignores those parameters.
Note
To ensure that none of the vPC interfaces are in the suspend mode, enter the show vpc brief and show vpc consistency-parameters commands and check the syslog messages.
Configuration Parameters That Should Be Identical
When any of the following parameters are not configured identically on both vPC peer switches, a misconfiguration may cause undesirable behavior in the traffic flow:
To ensure that all the configuration parameters are compatible, we recommend that you display the configurations for each vPC peer switch once you configure the vPC.
Graceful Type-1 Check
Beginning with Cisco NX—OS Release 5.0(2)N2(1), when a consistency check fails, vPCs are brought down only on the secondary vPC switch. The VLANs remain up on the primary switch and Type-1 configurations can be performed without traffic disruption. This feature is used both in the case of global as well as interface-specific Type-1 inconsistencies.
This feature is not enabled for dual-active FEX ports. When a Type-1 mismatch occurs, VLANs are suspended on these ports on both switches.
Per-VLAN Consistency Check
Beginning with Ciscon NX-OS Release 5.0(2)N2(1), some Type-1 consistency checks are performed on a per-VLAN basis for when spanning tree is enabled or disabled on a VLAN. VLANs that do not pass the consistency check are brought down on both the primary and secondary switches while other VLANs are not affected.
vPC Auto-Recovery
Beginning with Cisco NX-OS Release 5.0(2)N2(1), the vPC auto-recovery feature re-enables vPC links in the following scenarios:
When both vPC peer switches reload and only one switch reboots, auto-recovery allows that switch to assume the role of the primary switch and the vPC links will be allowed to come up after a predetermined period of time. The reload delay period in this scenario can range from 240-3600 seconds.
When vPCs are disabled on a secondary vPC switch due to a peer-link failure and then the primary vPC switch fails or is unable to forward traffic, the secondary switch re-enables the vPCs. In this scenario, the vPC waits for three consecutive keep-alive failures to recover the vPC links.
The vPC auto-recovery feature is disabled by default.
vPC Peer Links
A vPC peer link is the link that is used to synchronize the states between the vPC peer devices.
Note
You must configure the peer-keepalive link before you configure the vPC peer link or the peer link will not come up.
vPC Peer Link Overview
You can have only two switches as vPC peers; each switch can serve as a vPC peer to only one other vPC peer. The vPC peer switches can also have non-vPC links to other switches.
To make a valid configuration, you configure an EtherChannel on each switch and then configure the vPC domain. You assign the EtherChannel on each switch as a peer link. For redundancy, we recommend that you should configure at least two dedicated ports into the EtherChannel; if one of the interfaces in the vPC peer link fails, the switch automatically falls back to use another interface in the peer link.
Note
We recommend that you configure the EtherChannels in trunk mode.
Many operational parameters and configuration parameters must be the same in each switch connected by a vPC peer link. Because each switch is completely independent on the management plane, you must ensure that the switches are compatible on the critical parameters. vPC peer switches have separate control planes. After configuring the vPC peer link, you should display the configuration on each vPC peer switch to ensure that the configurations are compatible.
Note
You must ensure that the two switches connected by the vPC peer link have certain identical operational and configuration parameters.
When you configure the vPC peer link, the vPC peer switches negotiate that one of the connected switches is the primary switch and the other connected switch is the secondary switch. By default, the Cisco NX-OS software uses the lowest MAC address to elect the primary switch. The software takes different actions on each switch—that is, the primary and secondary—only in certain failover conditions. If the primary switch fails, the secondary switch becomes the operational primary switch when the system recovers, and the previously primary switch is now the secondary switch.
You can also configure which of the vPC switches is the primary switch. If you want to configure the role priority again to make one vPC switch the primary switch, configure the role priority on both the primary and secondary vPC switches with the appropriate values, shut down the EtherChannel that is the vPC peer link on both switches by entering the shutdown command, and reenable the EtherChannel on both switches by entering the no shutdown command.
MAC addresses that are learned over vPC links are also synchronized between the peers.
Configuration information flows across the vPC peer links using the Cisco Fabric Services over Ethernet (CFSoE) protocol. All MAC addresses for those VLANs configured on both switches are synchronized between vPC peer switches. The software uses CFSoE for this synchronization.
If the vPC peer link fails, the software checks the status of the remote vPC peer switch using the peer-keepalive link, which is a link between vPC peer switches, to ensure that both switches are up. If the vPC peer switch is up, the secondary vPC switch disables all vPC ports on its switch. The data then forwards down the remaining active links of the EtherChannel.
The software learns of a vPC peer switch failure when the keepalive messages are not returned over the peer-keepalive link.
Use a separate link (vPC peer-keepalive link) to send configurable keepalive messages between the vPC peer switches. The keepalive messages on the vPC peer-keepalive link determines whether a failure is on the vPC peer link only or on the vPC peer switch. The keepalive messages are used only when all the links in the peer link fail.
vPC Number
Once you have created the vPC domain ID and the vPC peer link, you can create EtherChannels to attach the downstream switch to each vPC peer switch. That is, you create one single EtherChannel on the downstream switch with half of the ports to the primary vPC peer switch and the other half of the ports to the secondary peer switch.
On each vPC peer switch, you assign the same vPC number to the EtherChannel that connects to the downstream switch. You will experience minimal traffic disruption when you are creating vPCs. To simplify the configuration, you can assign the vPC ID number for each EtherChannel to be the same as the EtherChannel itself (that is, vPC ID 10 for EtherChannel 10).
Note
The vPC number that you assign to the EtherChannel connecting to the downstream switch from the vPC peer switch must be identical on both vPC peer switches.
vPC Interactions with Other Features
vPC and LACP
The Link Aggregation Control Protocol (LACP) uses the system MAC address of the vPC domain to form the LACP Aggregation Group (LAG) ID for the vPC.
You can use LACP on all the vPC EtherChannels, including those channels from the downstream switch. We recommend that you configure LACP with active mode on the interfaces on each EtherChannel on the vPC peer switches. This configuration allows you to more easily detect compatibility between switches, unidirectional links, and multihop connections, and provides dynamic reaction to run-time changes and link failures.
The vPC peer link supports 16 EtherChannel interfaces.
Note
When manually configuring the system priority, you must ensure that you assign the same priority value on both vPC peer switches. If the vPC peer switches have different system priority values, vPC will not come up.
vPC Peer Links and STP
When you first bring up the vPC functionality, STP reconverges. STP treats the vPC peer link as a special link and always includes the vPC peer link in the STP active topology.
We recommend that you set all the vPC peer link interfaces to the STP network port type so that Bridge Assurance is automatically enabled on all vPC peer links. We also recommend that you do not enable any of the STP enhancement features on VPC peer links.
You must configure a list of parameters to be identical on the vPC peer switches on both sides of the vPC peer link.
STP is distributed; that is, the protocol continues running on both vPC peer switches. However, the configuration on the vPC peer switch elected as the primary switch controls the STP process for the vPC interfaces on the secondary vPC peer switch.
The primary vPC switch synchronizes the STP state on the vPC secondary peer switch using Cisco Fabric Services over Ethernet (CFSoE).
The vPC manager performs a proposal/handshake agreement between the vPC peer switches that sets the primary and secondary switches and coordinates the two switches for STP. The primary vPC peer switch then controls the STP protocol for vPC interfaces on both the primary and secondary switches.
The Bridge Protocol Data Units (BPDUs) use the MAC address set for the vPC for the STP bridge ID in the designated bridge ID field. The vPC primary switch sends these BPDUs on the vPC interfaces.
Note
Display the configuration on both sides of the vPC peer link to ensure that the settings are identical. Use the show spanning-tree command to display information about the vPC.
CFSoE
The Cisco Fabric Services over Ethernet (CFSoE) is a reliable state transport mechanism that you can use to synchronize the actions of the vPC peer devices. CFSoE carries messages and packets for many features linked with vPC, such as STP and IGMP. Information is carried in CFS/CFSoE protocol data units (PDUs).
When you enable the vPC feature, the device automatically enables CFSoE, and you do not have to configure anything. CFSoE distributions for vPCs do not need the capabilities to distribute over IP or the CFS regions. You do not need to configure anything for the CFSoE feature to work correctly on vPCs.
You can use the show mac address-table command to display the MAC addresses that CFSoE synchronizes for the vPC peer link.
Note
Do not enter the no cfs eth distribute or the no cfs distribute command. CFSoE must be enabled for vPC functionality. If you do enter either of these commands when vPC is enabled, the system displays an error message.
When you enter the show cfs application command, the output displays «Physical-eth,» which shows the applications that are using CFSoE.
vPC Guidelines and Limitations
vPC has the following configuration guidelines and limitations:
You must enable the vPC feature before you can configure vPC peer-link and vPC interfaces.
You must configure the peer-keepalive link before the system can form the vPC peer link.
You can connect a pair of Cisco Nexus 5000 Series switches or a pair of Cisco Nexus 5500 Series switches in a vPC directly to another switch or to a server. vPC peer switches must be of the same type, for example, you can connect a pair of Nexus 5000 series switches or a pair of Nexus 5500 Series switches but you cannot connect a Nexus 5000 Series switch to a Nexus 5500 Series switch in a vPC topology.
Only EtherChannels can be in vPCs. A vPC can be configured on a normal EtherChannel (switch-to-switch vPC topology), on an EtherChannel fabric interface (fabric extender vPC topology), and on an EtherChannel host interface (host interface vPC topology).
Note
Refer to the Cisco Nexus 2000 Series Fabric Extender Software Configuration Guide for information about Fabric Extender host and fabric interfaces.
A Fabric Extender can be a member of a Host Interface vPC topology or a Fabric Extender vPC topology but not both simultaneously.
You must configure both vPC peer switches; the configuration is not automatically synchronized between the vPC peer devices.
Check that the necessary configuration parameters are compatible on both sides of the vPC peer link.
You may experience minimal traffic disruption while configuring vPCs.
You should configure all the EtherChannels in the vPC using LACP with the interfaces in active mode.
The graceful consistency check is a new feature introduced in Cisco NX-OS Release 5.0(2)N2(1) and is enabled by default. For more details, see the “Graceful Consistency Check” section.
