Lesson Contents
As a firewall, the Cisco ASA drops packets. That’s great until it drops packets that you want to permit, and you have no idea what is going on. Fortunately, the ASA supports different tools to show you why and what packets it drops.
In this lesson, we’ll cover the following tools:
- Connection State
- Interface Drops
- Syslog
- ASP Drops
- Packet Capture
- Packet Tracer
- Service Policy
- Conclusion
Here is the topology I use:
We use a simple topology:
- Two “host” devices:
- These are routers with ip routing disabled, and they use ASA1 as their default gateway.
- An HTTP server is enabled on both devices.
- ASA1 runs ASAv Version 9.9(2) and has two interfaces:
- INSIDE: security level 100
- OUTSIDE: security level 0
In this topology, H1 will be able to initiate a connection to H2. H2 won’t be able to initiate a connection to H1 because we go from a low-security level (0) to a high-security level (100).
Configurations
Want to take a look for yourself? Here you will find the startup configuration of each device.
ASA1
hostname ASA1
!
interface GigabitEthernet0/0
nameif INSIDE
security-level 100
ip address 192.168.1.254 255.255.255.0
!
interface GigabitEthernet0/1
nameif OUTSIDE
security-level 0
ip address 192.168.2.254 255.255.255.0
!
class-map inspection_default
match default-inspection-traffic
!
policy-map global_policy
class inspection_default
inspect ip-options
inspect netbios
inspect rtsp
inspect sunrpc
inspect tftp
inspect xdmcp
inspect dns preset_dns_map
inspect ftp
inspect h323 h225
inspect h323 ras
inspect rsh
inspect esmtp
inspect sqlnet
inspect sip
inspect skinny
!
service-policy global_policy global
!
: end
H1
hostname H1
!
no ip routing
!
interface GigabitEthernet0/1
ip address 192.168.1.1 255.255.255.0
!
ip default-gateway 192.168.1.254
!
ip http server
no ip http secure-server
!
end
H2
hostname H2
!
no ip routing
!
interface GigabitEthernet0/1
ip address 192.168.2.2 255.255.255.0
!
ip default-gateway 192.168.2.254
!
ip http server
no ip http secure-server
!
end
Let’s get started!
Connection State
The ASA keeps track of all existing connections and their state. Let’s see what this looks like. First, we connect from H1 to H2:
H1#telnet 192.168.2.2 80
Trying 192.168.2.2, 80 ... Open
Now look at the connections with the show conn
command:
ASA1# show conn
1 in use, 1 most used
TCP OUTSIDE 192.168.2.2:80 INSIDE 192.168.1.1:50195, idle 0:00:00, bytes 0, flags U
You can see the flags if you add the detail
parameter:
ASA1# show conn detail
0 in use, 1 most used
Flags: A - awaiting inside ACK to SYN, a - awaiting outside ACK to SYN,
B - initial SYN from outside, b - TCP state-bypass or nailed,
C - CTIQBE media, c - cluster centralized,
D - DNS, d - dump, E - outside back connection, e - semi-distributed,
F - outside FIN, f - inside FIN,
G - group, g - MGCP, H - H.323, h - H.225.0, I - inbound data,
i - incomplete, J - GTP, j - GTP data, K - GTP t3-response
k - Skinny media, L - LISP triggered flow owner mobility
l - local director/backup stub flow
M - SMTP data, m - SIP media, n - GUP
N - inspected by Snort
O - outbound data, o - offloaded,
P - inside back connection,
Q - Diameter, q - SQL*Net data,
R - outside acknowledged FIN,
R - UDP SUNRPC, r - inside acknowledged FIN, S - awaiting inside SYN,
s - awaiting outside SYN, T - SIP, t - SIP transient, U - up, u - STUN,
V - VPN orphan, v - M3UA W - WAAS,
w - secondary domain backup,
X - inspected by service module,
x - per session, Y - director stub flow, y - backup stub flow,
Z - Scansafe redirection, z - forwarding stub flow
Let’s try to connect from H2 to H1:
H2#telnet 192.168.1.1 80
Trying 192.168.1.1, 80 ...
% Connection timed out; remote host not responding
This connection fails, so it doesn’t show up in the connection overview:
ASA1# show conn
0 in use, 1 most used
At least we know something is going on.
Interface drops
The ASA keeps track of drops on the interface. Here’s where you find this:
ASA1# show interface GigabitEthernet 0/1 | include packets dropped
10 packets dropped
We see the ASA drops packets on the interface, but we have no idea what. You can use clear interface
to reset this counter.
Syslog
Syslog is one of the best tools to figure out what is going on with your packet drops. The ASA has over 2000 unique syslog messages. You can send syslog messages to different locations. For debugging, the console works fine. Let’s see how we can use this.
First, you need to enable logging up to the debug level:
ASA1(config)# logging enable
ASA1(config)# logging buffered debugging
Optionally, you can increase the buffer size so you can store more syslog messages. For this example, I’m disabling the timestamps so that the messages are easier to look at:
ASA1(config)# logging buffer-size 1000000
ASA1(config)# no logging timestamp
Let’s try to connect from H2 to H1 on TCP port 80:
H2#telnet 192.168.1.1 80
Trying 192.168.1.1, 80 ...
% Connection timed out; remote host not responding
Now check out your log with the show logging
command:
ASA1(config)# show logging
Syslog logging: enabled
Facility: 20
Timestamp logging: disabled
Hide Username logging: enabled
Standby logging: disabled
Debug-trace logging: disabled
Console logging: disabled
Monitor logging: disabled
Buffer logging: level debugging, 11 messages logged
Trap logging: disabled
Permit-hostdown logging: disabled
History logging: disabled
Device ID: disabled
Mail logging: disabled
ASDM logging: disabled
%ASA-5-111008: User 'enable_15' executed the 'logging buffer-size 1000000' command.
%ASA-5-111010: User 'enable_15', running 'CLI' from IP 0.0.0.0, executed 'logging buffer-size 1000000'
%ASA-5-111008: User 'enable_15' executed the 'no logging timestamp' command.
%ASA-5-111010: User 'enable_15', running 'CLI' from IP 0.0.0.0, executed 'no logging timestamp'
%ASA-7-111009: User 'enable_15' executed cmd: show logging
%ASA-2-106001: Inbound TCP connection denied from 192.168.2.2/13279 to 192.168.1.1/80 flags SYN on interface OUTSIDE
%ASA-2-106001: Inbound TCP connection denied from 192.168.2.2/13279 to 192.168.1.1/80 flags SYN on interface OUTSIDE
%ASA-2-106001: Inbound TCP connection denied from 192.168.2.2/13279 to 192.168.1.1/80 flags SYN on interface OUTSIDE
%ASA-2-106001: Inbound TCP connection denied from 192.168.2.2/13279 to 192.168.1.1/80 flags SYN on interface OUTSIDE
Above, we see the syslog messages and the code. If you want to know what each message means, you have to check the Cisco ASA Series Syslog Messages. In this case, we see message code 106001. Here is the explanation for this message, taken directly from Cisco.com:
Explanation An attempt was made to connect to an inside address is denied by the security policy that is defined for the specified traffic type. The IP address displayed is the real IP address instead of the IP address that appears through NAT. Possible tcp_flags values correspond to the flags in the TCP header that were present when the connection was denied. For example, a TCP packet arrived for which no connection state exists in the ASA, and it was dropped. The tcp_flags in this packet are FIN and ACK.
When there is much traffic going on, you’ll need to filter these messages. You can either use include
to filter the message:
ASA1(config)# show logging | include 106001
%ASA-2-106001: Inbound TCP connection denied from 192.168.2.2/13279 to 192.168.1.1/80 flags SYN on interface OUTSIDE
%ASA-2-106001: Inbound TCP connection denied from 192.168.2.2/13279 to 192.168.1.1/80 flags SYN on interface OUTSIDE
%ASA-2-106001: Inbound TCP connection denied from 192.168.2.2/13279 to 192.168.1.1/80 flags SYN on interface OUTSIDE
%ASA-2-106001: Inbound TCP connection denied from 192.168.2.2/13279 to 192.168.1.1/80 flags SYN on interface OUTSIDE
Or get rid of what you don’t need with exclude
:
ASA1(config)# show logging | exclude 111008|111009|111010|302010
Syslog logging: enabled
Facility: 20
Timestamp logging: disabled
Hide Username logging: enabled
Standby logging: disabled
Debug-trace logging: disabled
Console logging: disabled
Monitor logging: disabled
Buffer logging: level debugging, 19 messages logged
Trap logging: disabled
Permit-hostdown logging: disabled
History logging: disabled
Device ID: disabled
Mail logging: disabled
ASDM logging: disabled
%ASA-2-106001: Inbound TCP connection denied from 192.168.2.2/13279 to 192.168.1.1/80 flags SYN on interface OUTSIDE
%ASA-2-106001: Inbound TCP connection denied from 192.168.2.2/13279 to 192.168.1.1/80 flags SYN on interface OUTSIDE
%ASA-2-106001: Inbound TCP connection denied from 192.168.2.2/13279 to 192.168.1.1/80 flags SYN on interface OUTSIDE
%ASA-2-106001: Inbound TCP connection denied from 192.168.2.2/13279 to 192.168.1.1/80 flags SYN on interface OUTSIDE
These syslog messages are handy to figure out what is going on.
ASP drops
Another useful tool is to check the Accelerated Security Path (ASP) drops with the show asp drop
command. This command gives an overview of packets that the ASA drops with a reason. Let’s have a look:
ASA1# show asp drop
Frame drop:
Flow is denied by configured rule (acl-drop) 3
Last clearing: 12:12:46 UTC Apr 24 2020 by enable_15
Flow drop:
Last clearing: 12:12:46 UTC Apr 24 2020 by enable_15
Above, we see 3 hits because of “acl-drop”. You can reset this counter with the clear asp drop
command if needed. There is a huge list of reasons, which you can find on the Cisco “show asp drop” command reference page.
Packet Capture
We can also capture packets to take a closer look. There are two options:
- Capture ASP dropped packets
- Capture any packets you want.
Let’s check both options.
ASP Drops Capture
The show asp drop
command tells us why something is dropped with a counter, but that’s it. It doesn’t tell us exactly what is dropped. Let’s capture some packets so we can see them. We do this with the capture
command:
ASA1(config)# capture ASP_DROPS type asp-drop acl-drop
The command above supports some extra parameters. For example, you could capture only specific protocol numbers (AH, ESP, GRE, etc.) or add an access-list. If you have a lot of traffic, you probably want a specific capture, but in my case, this is fine.
Let’s generate some more traffic from H2 to H1:
H2#telnet 192.168.1.1 80
Trying 192.168.1.1, 80 ...
% Connection timed out; remote host not responding
Here are the packets we captured:
ASA1(config)# show capture ASP_DROPS
4 packets captured
1: 12:16:18.018919 192.168.2.2.30833 > 192.168.1.1.80: S 178607441:178607441(0) win 4128 Drop-reason: (acl-drop) Flow is denied by configured rule
2: 12:16:20.021666 192.168.2.2.30833 > 192.168.1.1.80: S 178607441:178607441(0) win 4128 Drop-reason: (acl-drop) Flow is denied by configured rule
3: 12:16:24.028456 192.168.2.2.30833 > 192.168.1.1.80: S 178607441:178607441(0) win 4128 Drop-reason: (acl-drop) Flow is denied by configured rule
4: 12:16:32.034986 192.168.2.2.30833 > 192.168.1.1.80: S 178607441:178607441(0) win 4128 Drop-reason: (acl-drop) Flow is denied by configured rule
4 packets shown
The above output is interesting. This output tells us which packets get dropped with the reason. When you are finished, don’t forget to get rid of the capture:
Good morning Everyone,
I couldn’t find a similar one to the topic I am starting so I apologize if a repetition occur.
I am trying to join an MS Win 10 from my DMZ to an AD- server (MS Server 2016) into my LAN.
I did a research, and the port listed into the attachment are the ones that supposed to be opened but I think, I am missing something because I am still not able to join the domain. I also opened port 53 tcp/udp.
Can you, please give a hand with this issue ?
Best,
Asen
https://cdn-forum.networklessons.com/uploads/default/original/2X/4/4f815398c8daa57f3cad427fcdea6b2d46e74706.png
Hello Asen
It’s great that you mention this at this time because Rene has just completed a lesson that will help you identify specific packet drop events on an ASA. Take a look at this lesson and see if it helps you in your troubleshooting process.
https://networklessons.com/cisco/asa-firewall/cisco-asa-packet-drop-troubleshooting
Take a look and let us know how you get along.
I hope this has been helpful!
Laz
Hi,
logging buffered debugging ,I Want to send to the syslog server .
In that case what type of logging I should chose .
Why did you exclude 111008|111009|111010|302010
Thanks
Hello Sims
The type of logging you choose depends on the detail that you want included. There are eight levels of severity:
By choosing the severity, you choose what kind of events you want to be logged regardless of whether you are using a syslog server or not. You can find out mor
... Continue reading in our forumHi Rene,
Can we have two more details explanation on
Thanks in Advance
Manami