FortiGate_Transparent # diagnose sniffer packet any "arp or icmp" 4 0 l
interfaces=[any]
filters=[arp or icmp]
2019-07-03 18:05:12.481488 port1 in arp who-has 192.168.1.99 (aa:bb:cc:0:20:0) tell 192.168.1.1
2019-07-03 18:05:12.481508 port2 out arp who-has 192.168.1.99 (aa:bb:cc:0:20:0) tell 192.168.1.1
2019-07-03 18:05:12.486286 port2 in arp reply 192.168.1.99 is-at aa:bb:cc:0:20:0
2019-07-03 18:05:12.486295 port1 out arp reply 192.168.1.99 is-at aa:bb:cc:0:20:0
2019-07-03 18:05:15.442315 port1 in 192.168.1.1 -> 192.168.1.99: icmp: echo request
2019-07-03 18:05:17.444526 port1 in 192.168.1.1 -> 192.168.1.99: icmp: echo request
2019-07-03 18:05:19.444349 port1 in 192.168.1.1 -> 192.168.1.99: icmp: echo request
2019-07-03 18:05:21.444162 port1 in 192.168.1.1 -> 192.168.1.99: icmp: echo request
2019-07-03 18:05:23.444217 port1 in 192.168.1.1 -> 192.168.1.99: icmp: echo request
arp解析正常,但是icmp请求被防火墙丢弃,deubug flow分析不通的原因:
diagnose debug flow filter addr 192.168.1.1
diagnose debug flow filter proto 1
diagnose debug flow show console enable
diagnose debug flow show function-name enable
diagnose debug flow trace start 10
diagnose debug enable
Router1#show arp
Protocol Address Age (min) Hardware Addr Type Interface
Internet 192.168.1.1 - aabb.cc00.6000 ARPA Ethernet0/0
Internet 192.168.1.99 6 aabb.cc00.2000 ARPA Ethernet0/0
Internet 192.168.10.1 - aabb.cc00.6000 ARPA Ethernet0/0.10
Internet 192.168.10.99 0 Incomplete ARPA
Internet 192.168.20.1 - aabb.cc00.6000 ARPA Ethernet0/0.20
Internet 192.168.20.99 0 Incomplete ARPA
总结:
在默认的“set vlanforward disable”的情况下:
如果FGT本地存在VLAN-ID接口(此处为类似VLAN1的接口 port1)则数据会上送CPU处理,如果配置VLAN10的接口,则接收到了VLAN-TAG10的VLAN数据也会上送CPU处理,那么通不通就完全看安全策略是否放通了。
如果FGT本地不存在VLAN-ID的接口,收到了VLAN-TAG的数据,且本地并没有相应的VLAN接口,那么这样的数据将会被FGT底层当做非IP数据丢弃,上送CPU的机会都没有,直接丢弃,连arp都不处理,直接丢弃。
例如:我们在FGT的Port1和Port2上增进Inside_VLAN10和Outside_VLAN10的两个VLAN 10的接口,并加入到forward-domain 10里,再次测试携带了VLAN-TAG的业务:
注意:建议在不需要的情况下关闭接口配置下的“设备探测”(Device detection)功能,该功能用于MAC地址厂商设备信息识别及MAC地址过滤,会消耗较多的设备资源。
FortiGate_Transparent # config system interface
FortiGate_Transparent (interface) # edit Inside_VLAN10
FortiGate_Transparent (Inside_VLAN10) # set forward-domain 10
FortiGate_Transparent (Inside_VLAN10) # next
FortiGate_Transparent (interface) # edit Outside_VLAN10
FortiGate_Transparent (Outside_VLAN10) # set forward-domain 10
FortiGate_Transparent (Outside_VLAN10) # next
配置VLAN-ID的场景下则必须配置“forward-domain” 否则FGT的MAC地址表项会错乱翻转,业务不通,甚至存在广播风暴的风险。
假设不配置forward-domain,由于port2、port2.10(Outside_VLAN10)都是属于同一个广播域,那么192.168.10.1请求192.168.10.99的arp广播请求会被发送到port2、port2.10(Outside_VLAN10)甚至port1都会被广播出去,同时由于Router2的E0/0(192.168.1.99)和E0/0.10(192.168.10.99)的MAC地址一样都是:aa:bb:cc:00:20:00,由于在同一个广播域中的不同接口不能存在相同的MAC地址,此时port2也可以学习到aa:bb:cc:00:20:00,同时port2.10(Outside_VLAN10)也可以学习到aa:bb:cc:00:20:00,这样会造成MAC地址表冲突,同时MAC表中只能存在一个MAC表项,会操作MAC表翻转,如果配置全通策略,甚至可能引起环路的风险。因此配置VLAN-ID务必将forward-domain配置上,这是一个很必要的配置操作。
配置了forward-domain的fgt MAC表和Router1的arp表:
FortiGate_Transparent # diag netlink brctl name host root.b
show bridge control interface root.b host.
fdb: size=2048, used=6, num=10, depth=2
Bridge root.b host table
port no device devname mac addr ttl attributes
6 14 Outside_VLAN10 50:00:00:03:00:01 0 Local Static
2 4 port2 50:00:00:03:00:01 0 Local Static
5 13 Inside_VLAN10 50:00:00:03:00:00 0 Local Static
1 3 port1 50:00:00:03:00:00 0 Local Static
6 14 Outside_VLAN10 aa:bb:cc:00:20:00 5 Hit(5)
2 4 port2 aa:bb:cc:00:20:00 0 Hit(0)
4 6 port4 50:00:00:03:00:03 0 Local Static
5 13 Inside_VLAN10 aa:bb:cc:00:60:00 5 Hit(5)
1 3 port1 aa:bb:cc:00:60:00 2 Hit(2)
3 5 port3 50:00:00:03:00:02 0 Local Static
Router1#show arp
Protocol Address Age (min) Hardware Addr Type Interface
Internet 192.168.1.1 - aabb.cc00.6000 ARPA Ethernet0/0
Internet 192.168.1.99 0 aabb.cc00.2000 ARPA Ethernet0/0
Internet 192.168.10.1 - aabb.cc00.6000 ARPA Ethernet0/0.10
Internet 192.168.10.99 0 aabb.cc00.2000 ARPA Ethernet0/0.10
Internet 192.168.20.1 - aabb.cc00.6000 ARPA Ethernet0/0.20
此时再看VLAN10的业务,在Router1上192.168.10.1 ping 192.168.10.99:
Router1#ping 192.168.10.99
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.10.99, timeout is 2 seconds:
.....
Success rate is 0 percent (0/5)
虽然还是不通,但是情况已经不一样了,明显VLAN10的数据全部上送到了CPU进行处理,arp可正常学习,只是ping不通,原因是防火墙没有配置策略,所以匹配了默认的drop策略。
FortiGate_Transparent # diagnose sniffer packet any "arp or icmp" 4 0 l
interfaces=[any]
filters=[arp or icmp]
2019-07-03 18:24:49.697190 Inside_VLAN10 in arp who-has 192.168.10.99 tell 192.168.10.1
2019-07-03 18:24:49.697218 Outside_VLAN10 out arp who-has 192.168.10.99 tell 192.168.10.1
2019-07-03 18:24:49.698089 Outside_VLAN10 in arp reply 192.168.10.99 is-at aa:bb:cc:0:20:0
2019-07-03 18:24:49.698102 Inside_VLAN10 out arp reply 192.168.10.99 is-at aa:bb:cc:0:20:
2019-07-03 18:24:51.700405 Inside_VLAN10 in 192.168.10.1 -> 192.168.10.99: icmp: echo request
2019-07-03 18:24:53.700372 Inside_VLAN10 in 192.168.10.1 -> 192.168.10.99: icmp: echo request
2019-07-03 18:24:55.704350 Inside_VLAN10 in 192.168.10.1 -> 192.168.10.99: icmp: echo request
2019-07-03 18:24:57.704314 Inside_VLAN10 in 192.168.10.1 -> 192.168.10.99: icmp: echo request
Router1#show arp
Protocol Address Age (min) Hardware Addr Type Interface
Internet 192.168.1.1 - aabb.cc00.6000 ARPA Ethernet0/0
Internet 192.168.1.99 5 aabb.cc00.2000 ARPA Ethernet0/0
Internet 192.168.10.1 - aabb.cc00.6000 ARPA Ethernet0/0.10
Internet 192.168.10.99 0 aabb.cc00.2000 ARPA Ethernet0/0.10 // arp可以学习到
Internet 192.168.20.1 - aabb.cc00.6000 ARPA Ethernet0/0.20
diagnose debug flow filter addr 192.168.10.1
diagnose debug flow filter proto 1
diagnose debug flow show console enable
diagnose debug flow show function-name enable
diagnose debug flow trace start 10
diagnose debug enable
id=20085 trace_id=2 func=print_pkt_detail line=5428 msg="vd-root:0 received a packet(proto=1, 192.168.10.1:17->192.168.10.99:2048) from Inside_VLAN10. type=8, code=0, id=17, seq=0."
id=20085 trace_id=2 func=init_ip_session_common line=5593 msg="allocate a new session-00000244"
id=20085 trace_id=2 func=br_fw_forward_handler line=565 msg="Denied by forward policy check"
如果此时配置安全策略,VLAN10的数据即可通信了:
Router1#ping 192.168.10.99 repeat 100
Type escape sequence to abort.
Sending 100, 100-byte ICMP Echos to 192.168.10.99, timeout is 2 seconds:
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Success rate is 100 percent (100/100), round-trip min/avg/max = 1/1/2 ms
Router1#
FortiGate_Transparent # diagnose sniffer packet any "icmp or arp" 4 0 l
interfaces=[any]
filters=[icmp or arp]
2019-07-03 18:57:39.090299 Inside_VLAN10 in 192.168.10.1 -> 192.168.10.99: icmp: echo request
2019-07-03 18:57:39.090376 Outside_VLAN10 out 192.168.10.1 -> 192.168.10.99: icmp: echo request
2019-07-03 18:57:39.090379 port2 out 192.168.10.1 -> 192.168.10.99: icmp: echo request
2019-07-03 18:57:39.091154 Outside_VLAN10 in 192.168.10.99 -> 192.168.10.1: icmp: echo reply
2019-07-03 18:57:39.091166 Inside_VLAN10 out 192.168.10.99 -> 192.168.10.1: icmp: echo reply
2019-07-03 18:57:39.091168 port1 out 192.168.10.99 -> 192.168.10.1: icmp: echo reply
其实这个就回到了上一个主题"Forward-Domain(有VLAN的场景建议配上)",VLAN穿透的问题,正常就应该这样配置。
打开“vlanforward enable”
OK,我们来看一个不正常的,在port1和port2上开启vlanforward:
FortiGate_Transparent # config system interface
FortiGate_Transparent (interface) # edit port1
FortiGate_Transparent (port1) # set vlanforward enable
FortiGate_Transparent (port1) # show
config system interface
edit "port1"
set vdom "root"
set allowaccess ping https ssh http
set vlanforward enable
set type physical
set alias "LAN_Inside"
set snmp-index 1
next
end
FortiGate_Transparent (port1) # next
FortiGate_Transparent (interface) # edit port2
FortiGate_Transparent (port2) # set vlanforward enable
FortiGate_Transparent (port2) # show
config system interface
edit "port2"
set vdom "root"
set allowaccess ping https ssh http
set vlanforward enable
set type physical
set alias "WAN_Outside"
set snmp-index 2
next
end
FortiGate_Transparent (port2) # end
此时FGT没有创建VLAN20的相关接口:
我们再来测试一下Router1到Router2的VLAN20的业务:
Router1#ping 192.168.20.99
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.20.99, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/2 ms
Router1#
---你会发现vlan20的数据是通的!!!
FortiGate_Transparent # diagnose sniffer packet any "icmp or arp" 4 0 l
interfaces=[any]
filters=[icmp or arp]
---但是在FGT上抓取不到任何的VLAN20相关的数据,数据没有上送到CPU处理,直接被bypass掉了
总结来看:如果FGT上没有创建VLAN20的这个接口,同时接口开启了vlanforward,那么相当于这些未知的携带了VLAN-TAG的数据,将会统统bypass掉,不上CPU处理直接转发走。
如果不熟悉vlanforward和forward-domain的工作原理,这样的bypass流量风险比较大,因为那些bypass的流量FGT完全不可控,也不可见,一旦出问题完全无法掌控,因此vlanforward不建议开启,保持默认disable即可。
我的建议:任何时候都不要开启vlan-forward,这功能没有什么很大的好处,但是一旦没用好就会环路风暴引起断网,断网是非常严重的故障,因此建议永远不要将vlan-forward enable!!!