一、VxLAN簡介
1.1、VxLAN簡要說明
- VxLAN是一種大二層網(wǎng)絡(luò)技術(shù),VxLAN也是SDN的基礎(chǔ);
- VxLAN是網(wǎng)絡(luò)Overlay技術(shù)的一種實(shí)現(xiàn);
- VxLAN具有良好的擴(kuò)展性,支持?jǐn)?shù)據(jù)中心多租戶環(huán)境,解決了傳統(tǒng)VLAN的局限性;
- VxLAN更適合云環(huán)境,解耦了物理網(wǎng)絡(luò)(Underlay)和虛擬網(wǎng)絡(luò)(Overlay),將物理網(wǎng)絡(luò)當(dāng)作網(wǎng)絡(luò)功能池,和服務(wù)器虛擬化資源池一樣,可實(shí)現(xiàn)虛擬網(wǎng)絡(luò)的按需交付,即由上層應(yīng)用驅(qū)動(dòng)虛擬網(wǎng)絡(luò);
- VxLAN由RFC7348定義。
1.2、 VxLAN名詞釋義
- VTEP(Virtual Tunnel End Point):作為VxLAN隧道的起始點(diǎn)或終結(jié)點(diǎn),負(fù)責(zé)VXLAN報(bào)文的封裝與解封裝,每個(gè)VTEP應(yīng)具備兩個(gè)接口:一個(gè)是本地橋接接口,負(fù)責(zé)原始以太幀接收和發(fā)送,另一個(gè)是IP接口,負(fù)責(zé)VXLAN數(shù)據(jù)幀接收和發(fā)送。VTEP可以是物理交換機(jī)或軟件交換機(jī);
- VxLAN Gateway:負(fù)責(zé)VXLAN和非VXLAN之間、不同VXLAN之間的網(wǎng)絡(luò)通訊;
- VNI(Virtual Network Identifier):虛擬網(wǎng)絡(luò)標(biāo)識(shí)符,即VxLAN ID,用于劃分不同的租戶,需要和租戶的VLAN ID進(jìn)行映射綁定。不同租戶通過不同的VNI實(shí)現(xiàn)隔離,每個(gè)租戶的設(shè)備可以位于同一數(shù)據(jù)中心,也可以位于不同的數(shù)據(jù)中心,VNI類似傳統(tǒng)網(wǎng)絡(luò)中的VLAN,但是VNI比傳統(tǒng)的VLAN容量要大;
- VTI(VxLAN Tunnel Interface):VxLAN隧道接口,思科叫NVE(Network Virtual Interface)接口,主要用于封裝/解封裝VxLAN的UDP頭,還用于建立VxLAN的隧道;
- VxLAN Segment:指VxLAN的網(wǎng)絡(luò),一個(gè)VNI就是一個(gè)VxLAN Segment,類似于傳統(tǒng)VLAN的概念,可以類比為一個(gè)廣播域,但它運(yùn)行在三層網(wǎng)絡(luò)之上。
二、VxLAN報(bào)文封裝


- 從以上兩張圖中可看出,VxLAN采用"MAC in UDP"的封裝方式;
- VxLAN在原始幀前添加了8個(gè)字節(jié)的VxLAN頭,添加了8個(gè)字節(jié)的UDP頭,加了20個(gè)字節(jié)的新IP頭,加了14字節(jié)的新MAC頭,共新加50字節(jié);
- VxLAN給原始幀新增了50字節(jié)的開銷,所以在項(xiàng)目中使用VxLAN時(shí)應(yīng)調(diào)整Underlay網(wǎng)絡(luò)的MTU以減少報(bào)文分片;
- VxLAN頭中的"VxLAN"字段第5位(上圖中紅色箭頭)如果為1,表示使用VxLAN;
- VxLAN頭中的"VNI"字段表示VxLAN ID,共24位(上圖中紅色箭頭),所以VxLAN ID容量為:2^24=16,777,216個(gè)。
三、VxLAN的實(shí)現(xiàn)和部署方式
實(shí)現(xiàn)方式:
- 單播:單播靜態(tài)方式、BGP EVPN方式;
- 組播:標(biāo)準(zhǔn)的組播方式。
部署方式:
- 單機(jī)方式:傳統(tǒng)網(wǎng)絡(luò)部署方式,按照網(wǎng)絡(luò)規(guī)劃登錄到每臺(tái)設(shè)備上進(jìn)行配置。云計(jì)算數(shù)據(jù)中心中,此方式無法協(xié)同云平臺(tái)實(shí)現(xiàn)網(wǎng)絡(luò)的自動(dòng)化部署;
- 控制器方式:在大二層網(wǎng)絡(luò)中,為了方便控制與部署引入了控制器??刂破魇墙y(tǒng)一的網(wǎng)絡(luò)控制平臺(tái),實(shí)現(xiàn)網(wǎng)絡(luò)資源統(tǒng)一協(xié)調(diào)及管理,協(xié)同云平臺(tái)可實(shí)現(xiàn)業(yè)務(wù)和網(wǎng)絡(luò)的自動(dòng)化部署。
四、靜態(tài)Ingress Replication實(shí)現(xiàn)VxLAN實(shí)驗(yàn)
4.1 實(shí)驗(yàn)環(huán)境
| 工具 | 版本 | 備注 |
|---|---|---|
| EVE-NG | 2.0.3-105 | 模擬器 |
| Cisco Nexus 9000v | 7.0.3.I7.8 | 支持VxLAN的交換機(jī),拓?fù)渲械腘9K1和N9K-2 |
| IOL | 15.7.3 | 用于模擬路由器,拓?fù)渲械腄C1、SP和DC2 |
| Wireshark | 3.2.2 | 抓包軟件 |
4.2 實(shí)驗(yàn)拓?fù)?/h2>
image.png
-
Underlay:N9K-1的E1/1口、N9K-2的E1/1口、DC1、SP和DC2運(yùn)行OSPF實(shí)現(xiàn)Underlay網(wǎng)絡(luò)的互聯(lián)互通;
-
Overlay:實(shí)現(xiàn)VPC6能ping通VPC7。
4.3 實(shí)驗(yàn)配置
4.3.1 配置Underlay
4.3.1.1 接口IP與VLAN配置

N9K-1配置:
vlan 10
interface Ethernet1/2
switchport access vlan 10
spanning-tree port type edge
interface Ethernet1/1
no switchport
ip address 20.1.1.1/30
no shutdown
interface loopback0
ip address 1.1.1.1/32
DC1配置:
interface Ethernet0/0
ip address 20.1.1.2 255.255.255.252
no shutdown
interface Ethernet0/1
ip address 40.1.1.1 255.255.255.252
no shutdown
SP配置:
interface Ethernet0/1
ip address 40.1.1.2 255.255.255.252
no shutdown
interface Ethernet0/2
ip address 50.1.1.1 255.255.255.252
no shutdown
DC2配置:
interface Ethernet0/2
ip address 50.1.1.2 255.255.255.252
no shutdown
interface Ethernet0/0
ip address 30.1.1.2 255.255.255.252
no shutdown
N9K-2配置:
vlan 10
interface Ethernet1/2
switchport access vlan 10
spanning-tree port type edge
interface Ethernet1/1
no switchport
ip address 30.1.1.1/30
no shutdown
interface loopback0
ip address 2.2.2.2/32
VPC6設(shè)置:
VPCS> ip 192.168.1.1/24
Checking for duplicate address...
PC1 : 192.168.1.1 255.255.255.0
VPC7設(shè)置:
VPCS> ip 192.168.1.2/24
Checking for duplicate address...
PC1 : 192.168.1.2 255.255.255.0
4.3.1.2 OSPF配置
N9K-1配置:
feature ospf
router ospf 1
router-id 1.1.1.1
interface loopback0
ip router ospf 1 area 0.0.0.0
interface Ethernet1/1
ip router ospf 1 area 0.0.0.0
DC1配置:
router ospf 1
network 20.1.1.0 0.0.0.3 area 0
network 40.1.1.0 0.0.0.3 area 0
SP配置:
router ospf 1
network 40.1.1.0 0.0.0.3 area 0
network 50.1.1.0 0.0.0.3 area 0
DC2配置:
router ospf 1
network 30.1.1.0 0.0.0.3 area 0
network 50.1.1.0 0.0.0.3 area 0
N9K-2配置:
feature ospf
router ospf 1
router-id 2.2.2.2
interface loopback0
ip router ospf 1 area 0.0.0.0
interface Ethernet1/1
ip router ospf 1 area 0.0.0.0
查看N9K-1 OSPF路由表:
N9K-1# show ip route ospf
IP Route Table for VRF "default"
'*' denotes best ucast next-hop
'**' denotes best mcast next-hop
'[x/y]' denotes [preference/metric]
'%<string>' in via output denotes VRF <string>
2.2.2.2/32, ubest/mbest: 1/0
*via 20.1.1.2, Eth1/1, [110/71], 00:02:15, ospf-1, intra
30.1.1.0/30, ubest/mbest: 1/0
*via 20.1.1.2, Eth1/1, [110/70], 00:02:15, ospf-1, intra
40.1.1.0/30, ubest/mbest: 1/0
*via 20.1.1.2, Eth1/1, [110/50], 00:04:59, ospf-1, intra
50.1.1.0/30, ubest/mbest: 1/0
*via 20.1.1.2, Eth1/1, [110/60], 00:04:59, ospf-1, intra
查看N9K-2 OSPF路由表:
N9K-2# show ip route ospf
IP Route Table for VRF "default"
'*' denotes best ucast next-hop
'**' denotes best mcast next-hop
'[x/y]' denotes [preference/metric]
'%<string>' in via output denotes VRF <string>
1.1.1.1/32, ubest/mbest: 1/0
*via 30.1.1.2, Eth1/1, [110/71], 00:03:10, ospf-1, intra
20.1.1.0/30, ubest/mbest: 1/0
*via 30.1.1.2, Eth1/1, [110/70], 00:03:10, ospf-1, intra
40.1.1.0/30, ubest/mbest: 1/0
*via 30.1.1.2, Eth1/1, [110/60], 00:03:10, ospf-1, intra
50.1.1.0/30, ubest/mbest: 1/0
*via 30.1.1.2, Eth1/1, [110/50], 00:03:10, ospf-1, intra
4.3.1.3 連通性測試
N9K-1# ping 2.2.2.2 source 1.1.1.1
PING 2.2.2.2 (2.2.2.2) from 1.1.1.1: 56 data bytes
64 bytes from 2.2.2.2: icmp_seq=0 ttl=251 time=11.635 ms
64 bytes from 2.2.2.2: icmp_seq=1 ttl=251 time=12.838 ms
64 bytes from 2.2.2.2: icmp_seq=2 ttl=251 time=15.182 ms
64 bytes from 2.2.2.2: icmp_seq=3 ttl=251 time=10.633 ms
64 bytes from 2.2.2.2: icmp_seq=4 ttl=251 time=10.743 ms
--- 2.2.2.2 ping statistics ---
5 packets transmitted, 5 packets received, 0.00% packet loss
round-trip min/avg/max = 10.633/12.206/15.182 ms
N9K-2# ping 1.1.1.1 source 2.2.2.2
PING 1.1.1.1 (1.1.1.1) from 2.2.2.2: 56 data bytes
64 bytes from 1.1.1.1: icmp_seq=0 ttl=251 time=45.525 ms
64 bytes from 1.1.1.1: icmp_seq=1 ttl=251 time=14.746 ms
64 bytes from 1.1.1.1: icmp_seq=2 ttl=251 time=11.07 ms
64 bytes from 1.1.1.1: icmp_seq=3 ttl=251 time=10.871 ms
64 bytes from 1.1.1.1: icmp_seq=4 ttl=251 time=11.149 ms
--- 1.1.1.1 ping statistics ---
5 packets transmitted, 5 packets received, 0.00% packet loss
round-trip min/avg/max = 10.871/18.672/45.525 ms
4.3.2 配置Overlay(static Ingress Replication VxLAN)
基于"Static Ingress Replication"實(shí)現(xiàn)的VxLAN,是流量驅(qū)動(dòng)式的MAC地址泛洪和學(xué)習(xí),沒有控制層面。
4.3.2.1 配置VTEP(N9K1和N9K-2)
N9K-1配置:
N9K-1(config)# feature nv overlay
N9K-1(config)# feature vn-segment-vlan-based
N9K-1(config)# vlan 10
N9K-1(config-vlan)# vn-segment 10010
N9K-1(config-if-nve)#interface nve1
no shutdown
source-interface loopback0
member vni 10010
ingress-replication protocol static
peer-ip 2.2.2.2
N9K-2配置:
N9K-2(config)# feature nv overlay
N9K-2(config)# feature vn-segment-vlan-based
N9K-2(config)# vlan 10
N9K-2(config-vlan)# vn-segment 10010
N9K-2(config-if-nve)#interface nve1
no shutdown
source-interface loopback0
member vni 10010
ingress-replication protocol static
peer-ip 1.1.1.1
連通性測試:
從VPC6 ping VPC7
VPCS> ping 192.168.1.2
84 bytes from 192.168.1.2 icmp_seq=1 ttl=64 time=52.581 ms
84 bytes from 192.168.1.2 icmp_seq=2 ttl=64 time=38.518 ms
84 bytes from 192.168.1.2 icmp_seq=3 ttl=64 time=34.455 ms
84 bytes from 192.168.1.2 icmp_seq=4 ttl=64 time=39.765 ms
84 bytes from 192.168.1.2 icmp_seq=5 ttl=64 time=35.878 ms
從VPC7 ping VPC6
VPCS> ping 192.168.1.1
84 bytes from 192.168.1.1 icmp_seq=1 ttl=64 time=36.590 ms
84 bytes from 192.168.1.1 icmp_seq=2 ttl=64 time=34.301 ms
84 bytes from 192.168.1.1 icmp_seq=3 ttl=64 time=40.432 ms
84 bytes from 192.168.1.1 icmp_seq=4 ttl=64 time=50.963 ms
84 bytes from 192.168.1.1 icmp_seq=5 ttl=64 time=34.739 ms
4.3.2.2 狀態(tài)查看
查看NVE接口狀態(tài)
N9K-1# show nve interface
Interface: nve1, State: Up, encapsulation: VXLAN
VPC Capability: VPC-VIP-Only [not-notified]
Local Router MAC: 5000.0001.0007
Host Learning Mode: Data-Plane
Source-Interface: loopback0 (primary: 1.1.1.1, secondary: 0.0.0.0)
查看NVE下的VNI狀態(tài),其中"DP"為數(shù)據(jù)層面
N9K-1# show nve vni
Codes: CP - Control Plane DP - Data Plane
UC - Unconfigured SA - Suppress ARP
SU - Suppress Unknown Unicast
Xconn - Crossconnect
MS-IR - Multisite Ingress Replication
Interface VNI Multicast-group State Mode Type [BD/VRF] Flags
--------- -------- ----------------- ----- ---- ------------------ -----
nve1 10010 UnicastStatic Up DP L2 [10]
查看NVE鄰居、NVE數(shù)據(jù)層面、NVE Ingress-replication信息
N9K-1# show nve peers
Interface Peer-IP State LearnType Uptime Router-Mac
--------- --------------- ----- --------- -------- -----------------
nve1 2.2.2.2 Up DP 00:17:33 n/a
N9K-1#
N9K-1# show nve vni data-plane
Codes: CP - Control Plane DP - Data Plane
UC - Unconfigured SA - Suppress ARP
SU - Suppress Unknown Unicast
Xconn - Crossconnect
MS-IR - Multisite Ingress Replication
Interface VNI Multicast-group State Mode Type [BD/VRF] Flags
--------- -------- ----------------- ----- ---- ------------------ -----
nve1 10010 UnicastStatic Up DP L2 [10]
N9K-1#
N9K-1# show nve vni ingress-replication
Interface VNI Replication List Source Up Time
--------- -------- ----------------- ------- -------
nve1 10010 2.2.2.2 CLI 00:17:52
查看N9K-1的VxLAN MAC地址表
N9K-1的VxLAN MAC地址表中,0050.7966.6806為VPC6的MAC地址,0050.7966.6807為VPC7的MAC地址。
N9K-1# show system internal l2fwder mac
Legend:
* - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MAC
age - seconds since last seen,+ - primary entry using vPC Peer-Link,
(T) - True, (F) - False, C - ControlPlane MAC
VLAN MAC Address Type age Secure NTFY Ports
---------+-----------------+--------+---------+------+----+------------------
* 10 0050.7966.6806 dynamic 00:00:20 F F Eth1/2
* 10 0050.7966.6807 dynamic 00:00:20 F F (0x47000001) nve-peer1
2.2.2.2
查看VPC6和VPC7的MAC地址
VPC6
VPCS> show ip
NAME : VPCS[1]
IP/MASK : 192.168.1.1/24
GATEWAY : 0.0.0.0
DNS :
MAC : 00:50:79:66:68:06
LPORT : 20000
RHOST:PORT : 127.0.0.1:30000
MTU : 1500
VPC7
VPCS> show ip
NAME : VPCS[1]
IP/MASK : 192.168.1.2/24
GATEWAY : 0.0.0.0
DNS :
MAC : 00:50:79:66:68:07
LPORT : 20000
RHOST:PORT : 127.0.0.1:30000
MTU : 1500
4.3.3 數(shù)據(jù)轉(zhuǎn)發(fā)流程
驗(yàn)證:從VPC6 ping VPC7,即從192.168.1.1 ping 192.168.1.2
- VPC6發(fā)出ARP請求報(bào)文,報(bào)文中源MAC為VPC6的MAC,目的MAC為廣播MAC即FF:FF:FF:FF:FF:FF,ARP協(xié)議中源IP為192.168.1.1,目的IP為192.168.1.2;
- VTEP N9K-1收到VPC6的ARP請求報(bào)文,除了轉(zhuǎn)發(fā)給本地的Trunk(已放行VLAN 10)和Access Vlan10接口外,還要對(duì)ARP請求進(jìn)行VxLAN的封裝,封裝后VxLAN頭部中VNI值為10010,外層新幀頭源MAC為N9K-1 E1/1口的MAC,目的MAC為DC1 E0/0口的MAC,新IP包頭即外層源IP為1.1.1.1,目的IP為2.2.2.2,被封裝的原始數(shù)據(jù)包中源目MAC和源目IP不變(BUM流量,即廣播、未知單播和組播流量會(huì)發(fā)給這個(gè)VNI下的所有peer);
- 通過Underlay網(wǎng)絡(luò)數(shù)據(jù)包抵達(dá)VTEP N9K-2,N9K-2收到后解封裝VxLAN報(bào)文,然后檢查報(bào)文中的VxLAN ID:10010是否有本地的VLAN與其匹配,N9K-2發(fā)現(xiàn)VxLAN ID:10010對(duì)應(yīng)本地的VLAN 10,這時(shí)N9K-2將原始ARP廣播報(bào)文發(fā)送至VPC7,N9K-2同時(shí)也會(huì)記錄報(bào)文中的VxLAN ID、原始報(bào)文的源MAC、外層的源IP信息,以構(gòu)建VxLAN MAC地址表;
- VPC7收到ARP請求報(bào)文后,開始以單播發(fā)送ARP回復(fù)報(bào)文;
- N9K-2收到來自VPC7的單播報(bào)文后,由于之前構(gòu)建的VxLAN MAC地址表中含有VPC6的MAC信息,N9K-2了解到去往VPC6的MAC下一跳為VTEP N9K-1即1.1.1.1,這時(shí)單播ARP回復(fù)報(bào)文被VxLAN封裝后通過Underlay網(wǎng)絡(luò)到達(dá)VTEP N9K-1(N9K-2發(fā)出報(bào)文時(shí):報(bào)文的外層源IP為2.2.2.2,目的IP為1.1.1.1,外層源MAC為N9K-2 E1/1口的MAC,目的MAC為DC2的E0/0口的MAC。內(nèi)層源IP為192.168.1.2,目的IP為192.168.1.1,內(nèi)層源MAC為VPC7的MAC,目的MAC為VPC6的MAC);
- VTEP N9K-1收到單播ARP回復(fù)報(bào)文進(jìn)行VxLAN解封裝,找到報(bào)文中VxLAN ID與本地VLAN對(duì)應(yīng)關(guān)系并轉(zhuǎn)發(fā)報(bào)文,N9K-1同時(shí)也會(huì)記錄報(bào)文中的VxLAN ID、原始報(bào)文的源MAC、外層的源IP信息,以構(gòu)建VxLAN MAC地址表;
- 此時(shí)VTEP N9K-1和VTEP N9K-2完成控制層面的VxLAN MAC地址表構(gòu)建,后續(xù)的ICMP報(bào)文均開始采用單播傳輸。
4.3.4 抓包分析
4.3.4.1 設(shè)備MAC信息表
下表中的MAC地址對(duì)應(yīng)報(bào)文中的MAC地址。
| 設(shè)備 | 接口 | MAC地址 |
|---|---|---|
| N9K-1 | E1/1 | 50:00:00:01:00:07 |
| DC1 | E0/0 | aa:bb:cc:0030:00 |
| N9K-2 | E1/1 | 50:00:00:02:00:07 |
| DC2 | E0/0 | aa:bb:cc:00:50:00 |
| VPC6 | Eth0 | 00:50:79:66:68:06 |
| VPC7 | Eth0 | 00:50:79:66:68:07 |
4.3.4.2 VPC6 Eth0口抓包
-
ARP請求報(bào)文
image.png -
ARP回復(fù)報(bào)文
image.png
4.3.4.3 N9K-1 E1/1口抓包
- 使用icmp ping
VPCS> ping 192.168.1.2
84 bytes from 192.168.1.2 icmp_seq=1 ttl=64 time=51.790 ms
84 bytes from 192.168.1.2 icmp_seq=2 ttl=64 time=33.325 ms
84 bytes from 192.168.1.2 icmp_seq=3 ttl=64 time=39.743 ms
84 bytes from 192.168.1.2 icmp_seq=4 ttl=64 time=34.220 ms
84 bytes from 192.168.1.2 icmp_seq=5 ttl=64 time=44.490 ms
報(bào)文分析,可看出VxLAN采用"MAC in UDP"封裝,UDP中源端口根據(jù)Hash隨機(jī),目的端口為4789。

- 使用TCP ping
VPCS> ping 192.168.1.2 -3
Connect 7@192.168.1.2 seq=1 ttl=64 time=36.241 ms
SendData 7@192.168.1.2 seq=1 ttl=64 time=44.041 ms
Close 7@192.168.1.2 seq=1 ttl=64 time=46.372 ms
Connect 7@192.168.1.2 seq=2 ttl=64 time=48.133 ms
SendData 7@192.168.1.2 seq=2 ttl=64 time=34.389 ms
Close 7@192.168.1.2 seq=2 ttl=64 time=58.858 ms
Connect 7@192.168.1.2 seq=3 ttl=64 time=49.761 ms
SendData 7@192.168.1.2 seq=3 ttl=64 time=35.541 ms
Close 7@192.168.1.2 seq=3 ttl=64 time=57.278 ms
Connect 7@192.168.1.2 seq=4 ttl=64 time=37.148 ms
SendData 7@192.168.1.2 seq=4 ttl=64 time=34.683 ms
Close 7@192.168.1.2 seq=4 ttl=64 time=50.096 ms
Connect 7@192.168.1.2 seq=5 ttl=64 time=36.775 ms
SendData 7@192.168.1.2 seq=5 ttl=64 time=37.062 ms
Close 7@192.168.1.2 seq=5 ttl=64 time=50.684 ms
報(bào)文分析,可看出VxLAN采用"MAC in UDP"封裝,UDP中源端口根據(jù)Hash隨機(jī),目的端口為4789。

- 使用UDP ping
VPCS> ping 192.168.1.2 -2
84 bytes from 192.168.1.2 udp_seq=1 ttl=64 time=48.823 ms
84 bytes from 192.168.1.2 udp_seq=2 ttl=64 time=56.103 ms
84 bytes from 192.168.1.2 udp_seq=3 ttl=64 time=34.814 ms
84 bytes from 192.168.1.2 udp_seq=4 ttl=64 time=37.074 ms
84 bytes from 192.168.1.2 udp_seq=5 ttl=64 time=34.839 ms
報(bào)文分析同上

4.3.4.4 SP E0/2口抓包
- 使用icmp ping
VPCS> ping 192.168.1.2
84 bytes from 192.168.1.2 icmp_seq=1 ttl=64 time=38.286 ms
84 bytes from 192.168.1.2 icmp_seq=2 ttl=64 time=34.641 ms
84 bytes from 192.168.1.2 icmp_seq=3 ttl=64 time=36.486 ms
84 bytes from 192.168.1.2 icmp_seq=4 ttl=64 time=35.365 ms
84 bytes from 192.168.1.2 icmp_seq=5 ttl=64 time=38.622 ms
報(bào)文抓取,分析同上

4.3.5 VxLAN網(wǎng)關(guān)
4.3.5.1 集中式網(wǎng)關(guān)配置說明
通過三層網(wǎng)關(guān)能實(shí)現(xiàn)不同子網(wǎng)之間的互通。
基于泛洪學(xué)習(xí)(非BGP EVPN)的VxLAN網(wǎng)關(guān)配置注意:

- NX-OS版本7.0(3)I1(2)或之前,集中式網(wǎng)關(guān)必須在外部路由設(shè)備上配置;
- NX-OS版本7.0(3)I2(1)或之后,為保障可靠性,集中式網(wǎng)關(guān)建議配置在一對(duì)運(yùn)行了vPC+FHRP(HSRP/VRRP)協(xié)議的設(shè)備上;
- NX-OS版本7.0(3)I2(1)或之后,僅支持集中式3層網(wǎng)關(guān),不支持任播網(wǎng)關(guān);
-
需配置"hardware access-list tcam region arp-ether size double-wide"命令以使保障正常的L3泛洪和學(xué)習(xí),使用此命令之前,必須釋放現(xiàn)有TCAM區(qū)域的空間占用;
a,通過"show hardware access-list tcam region"命令查看當(dāng)前TCAM的分配情況;
b,舉例:通過"hardware access-list tcam region racl 512"命令釋放"racl"占用的TCAM空間;
c,舉例:通過"hardware access-list tcam region arp-ether 256 double-wide"命令為"arp-ether"分配TCAM空間;
d,配置完畢后保存并重啟設(shè)備生效;
e,注:本實(shí)驗(yàn)未配置"hardware access-list tcam region arp-ether size double-wide"命令。 - 基于BGP EVPN實(shí)現(xiàn)的VxLAN,建議在所有VTEP上配置分布式任播網(wǎng)關(guān)。
參考鏈接:
4.3.5.2 在VTEP上配置集中式三層網(wǎng)關(guān)
由于本實(shí)驗(yàn)未規(guī)劃vPC環(huán)境,所以不在本篇展示vPC環(huán)境下的VxLAN配置,vPC環(huán)境下的VxLAN將在另外一篇中展示。
N9K-1變更后的配置:
vlan 20
vn-segment 10020
interface nve1
no shutdown
source-interface loopback0
member vni 10010
ingress-replication protocol static
peer-ip 2.2.2.2
member vni 10020
ingress-replication protocol static
peer-ip 2.2.2.2
feature interface-vlan
interface Vlan10
no shutdown
ip address 192.168.1.254/24
interface Vlan20
no shutdown
ip address 192.168.2.254/24
N9K-2變更后的配置:
vlan 20
vn-segment 10020
interface nve1
no shutdown
source-interface loopback0
member vni 10020
ingress-replication protocol static
peer-ip 1.1.1.1
interface Ethernet1/2
switchport access vlan 20
VPC7變更后的配置:
VPCS> show ip all
NAME IP/MASK GATEWAY MAC DNS
VPCS1 192.168.2.1/24 192.168.2.254 00:50:79:66:68:07
連通性測試:
VPC6 ping VPC7:
VPCS> ping 192.168.2.1
84 bytes from 192.168.2.1 icmp_seq=1 ttl=63 time=71.132 ms
84 bytes from 192.168.2.1 icmp_seq=2 ttl=63 time=41.996 ms
84 bytes from 192.168.2.1 icmp_seq=3 ttl=63 time=44.948 ms
84 bytes from 192.168.2.1 icmp_seq=4 ttl=63 time=113.173 ms
84 bytes from 192.168.2.1 icmp_seq=5 ttl=63 time=40.646 ms
VPC7 ping VPC6:
VPCS> ping 192.168.1.1
84 bytes from 192.168.1.1 icmp_seq=1 ttl=63 time=42.753 ms
84 bytes from 192.168.1.1 icmp_seq=2 ttl=63 time=45.416 ms
84 bytes from 192.168.1.1 icmp_seq=3 ttl=63 time=38.233 ms
84 bytes from 192.168.1.1 icmp_seq=4 ttl=63 time=45.141 ms
84 bytes from 192.168.1.1 icmp_seq=5 ttl=63 time=39.792 ms
數(shù)據(jù)流分析:
- 當(dāng)VPC7 ping VPC6時(shí),VPC7發(fā)現(xiàn)目的IP為非本地直連網(wǎng)段,所以會(huì)將流量通過VxLAN隧道交由網(wǎng)關(guān)即192.168.2.254處理;
- 由于VPC7并不知道網(wǎng)關(guān)的MAC地址,所以需發(fā)起ARP請求以獲取網(wǎng)關(guān)的MAC,在N9K-2的E1/1口抓包,可看到數(shù)據(jù)包內(nèi)層二層幀頭源MAC為00:50:79:66:68:07,目的MAC為ff:ff:ff:ff:ff:ff,此ARP請求(屬于BUM中的廣播)報(bào)文會(huì)被封裝進(jìn)VxLAN然后被泛洪到N9K-1;
- N9K-1收到VxLAN報(bào)文并解封裝后,N9K-1的SVI 20以單播回復(fù)來自VPC7的ARP請求,至此第一個(gè)ICMP包結(jié)束,由于ping第一個(gè)ICMP包ARP請求的時(shí)間超過了ping ICMP包連續(xù)發(fā)送的間隔時(shí)間,通常跨網(wǎng)段ping測試時(shí)第一個(gè)包會(huì)丟包;
- 第二個(gè)ICMP包,N9K-1解封裝后發(fā)現(xiàn)報(bào)文中目標(biāo)IP為192.168.1.1,查詢路由表后發(fā)現(xiàn)為本地直連路由,N9K-1將直接進(jìn)行ARP/MAC地址表查詢,然后將流量從E1/2口發(fā)出,此時(shí)流量到達(dá)VPC6;
- VPC6回復(fù)報(bào)文原理同上,此處不再贅述。
N9K-1 SVI 20的MAC地址:
N9K-1# show interface vlan 20
Vlan20 is up, line protocol is up, autostate enabled
Hardware is EtherSVI, address is 5000.0001.0007
Internet Address is 192.168.2.254/24
MTU 1500 bytes, BW 1000000 Kbit, DLY 10 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA, loopback not set
Keepalive not supported
ARP type: ARPA
Last clearing of "show interface" counters never
L3 in Switched:
ucast: 0 pkts, 0 bytes
N9K-2 E1/1口抓包:
ARP請求報(bào)文:

ARP回復(fù)報(bào)文:

4.3.5.3 在外部路由設(shè)備上配置集中式網(wǎng)關(guān)
本實(shí)驗(yàn)采用單臂路由方式,SW3作為外部路由設(shè)備。
N9K-1變更后的配置:
N9K-1(config)# no interface vlan 10
N9K-1(config)# no interface vlan 20
N9K-1(config)# no feature interface-vlan
interface nve1
no shutdown
source-interface loopback0
member vni 10010
ingress-replication protocol static
peer-ip 2.2.2.2
N9K-2變更后的配置:
interface nve1
no shutdown
source-interface loopback0
member vni 10010
ingress-replication protocol static
peer-ip 1.1.1.1
member vni 10020
ingress-replication protocol static
peer-ip 1.1.1.1
interface Ethernet1/3
switchport mode trunk
switchport trunk allowed vlan 10,20
SW1配置:
ip routing
interface Ethernet0/0
no ip address
interface Ethernet0/0.1
encapsulation dot1Q 10
ip address 192.168.1.254 255.255.255.0
interface Ethernet0/0.2
encapsulation dot1Q 20
ip address 192.168.2.254 255.255.255.0
連通性測試:
VPC6 ping VPC7:
VPCS> ping 192.168.2.1
84 bytes from 192.168.2.1 icmp_seq=1 ttl=63 time=59.828 ms
84 bytes from 192.168.2.1 icmp_seq=2 ttl=63 time=83.546 ms
84 bytes from 192.168.2.1 icmp_seq=3 ttl=63 time=60.527 ms
84 bytes from 192.168.2.1 icmp_seq=4 ttl=63 time=63.410 ms
84 bytes from 192.168.2.1 icmp_seq=5 ttl=63 time=61.804 ms
VPC7 ping VPC6:
VPCS> ping 192.168.1.1
84 bytes from 192.168.1.1 icmp_seq=1 ttl=63 time=131.247 ms
84 bytes from 192.168.1.1 icmp_seq=2 ttl=63 time=61.279 ms
84 bytes from 192.168.1.1 icmp_seq=3 ttl=63 time=102.038 ms
84 bytes from 192.168.1.1 icmp_seq=4 ttl=63 time=63.945 ms
84 bytes from 192.168.1.1 icmp_seq=5 ttl=63 time=65.245 ms
- 以上在VTEP上配置集中式網(wǎng)關(guān)和在外部路由設(shè)備上配置集中式網(wǎng)關(guān)時(shí),應(yīng)注意VTEP上NVE接口下VNI的映射關(guān)系;
- 在一個(gè)VTEP上配置集中式網(wǎng)關(guān)或外部路由設(shè)作為網(wǎng)關(guān)連接到這個(gè)VTEP,那么這個(gè)Ingress VTEP上應(yīng)有完整的VNI,否則流量無法被封裝進(jìn)VxLAN隧道!

