Tuesday, 22 November 2016

MyFortiGate cheat sheet

CLI Commands for Troubleshooting FortiGate Firewalls


This blog post is a list of common troubleshooting commands I find usefull in my daily work. It is not complete nor very detailled, but provides the basic commands for troubleshooting network related issues that are not resolvable via the GUI. I am not focused on too many memory, process, kernel, etc. details. These must only be used if there are really specific problems. I am more focused on the general troubleshooting stuff. I am using it personally as a cheat sheet / quick reference and will update it from time to time.

# fnsysctl


# MV

usage: mv [-f] src dst


# GREP

Usage: grep [-ilHhnqvscABC] PATTERN [FILE...]

Options:

-i Ignore case distinctions

-l List names of files that match

-H Prefix output lines with filename where match was found

-h Suppress the prefixing filename on output

-n Print line number with output lines

-q Quiet

-v Select non-matching lines

-s Suppress file open/read error messages

-c Only print count of matching lines

-A Print NUM lines of trailing context

-B Print NUM lines of leading context

-C Print NUM lines of output context


# KILL

# KILLALL

# LS

# CAT

usage: cat [file ...]


# IFCONFIG

# PWD

# DU

usage: du

Summarize disk space used for each FILE and/or directory


-a Show file sizes too

-L Follow all symlinks

-H Follow symlinks on command line

-d N Limit output to directories (and files with -a) of depth < N

-c Show grand total

-l Count sizes many times if hard linked

-s Display only a total for each argument

-x Skip directories on different filesystems

-i Show number of inodes


# PS

# DATE

# SH

# DF




In Cisco, everything is “show”. With Fortinet, you have the choice between show | get | diagnose | execute
Likewise the sys | system keyword. It is always “diagnose sys” but “execute system”


Fortigate Cisco
show full-configuration show run
execute factory-reset write erase
show system interface show run interface brief
diagnose hardware deviceinfo nic show interface
get system status show version
get system arp | diagnose ip arp list show arp
get router info routing-table all show ip route
diagnose system session list show ip nat translation
diagnose system session clear clear ip nat translation
get router info ospf neighbor show ip ospf neighbor
get router info bgp neighbor show ip bgp neighbor
get router info bgp summary show ip bgp summary

Entering the correct vdom/gobal Config

Remember to enter the correct vdom or global configuration tree before configuring anything:

config global
config vdom
edit <vdom-name>

To show the running configuration (such as “show run”), simply type in:

show

To show the entire running configuration with default values, use:

show full-configuration
To omit the “–More–” stops when displaying many lines, you can set the terminal output to the following, which will display all lines at once. This is similar to “terminal length 0” from Cisco. Be careful with it, because this command is persistent. Set it to default after usage!
config system console
  set output standard
end
To find a CLI command within the configuration, you can use the pipe sign “|” with “grep” (similar to “include” on Cisco devices). Note the “-f” flag to show the whole config tree in which the keywords was found, e.g.:
show | grep -f ipv6
show full-configuration | grep -f ipv6

General Information



get system interface physical       #overview of hardware interfaces
get hardware nic <nic-name>         #details of a single network interface, same as: diagnose hardware deviceinfo nic <nic-name>
fnsysctl ifconfig <nic-name>        #kind of hidden command to see more interface stats such as errors
get system status                   #show version
get system performance status       #CPU and network usage
diagnose sys top                    #top with all forked processed
diagnose sys top-summary            #top easier, incl. CPU and mem bars. Forks are displayed by [x13] or whatever
execute dhcp lease-list
diagnose ip arp list
diagnose ipv6 address list
diagnose ipv6 neighbor-cache list
diagnose sys ntp status
diagnose autoupdate versions        #lists the attack definition versions, last update, etc.
diagnose log test                   #generated all possibe log entries
diagnose test application dnsproxy 6    #shows the IP addresses of FQDN objects
diagnose debug crashlog read       #shows crashlog, a status of 0 indicates a normal close of a process!

After rebooting a fresh device which is already licensed, it takes some time until it is “green” at the dashboard. The following commands can troubleshoot and start the “get license” process. Use the first three to enable debugging and start the process, while the last one disables the debugging again:
diag debug app update -1
diag debug enable
exec update-now
diag debug disable

General Network Troubleshooting



execute ping-options ?
execute ping-options source <ip-address-of-the-interface>
execute ping <hostname|ip>
execute ping6-options ?
execute ping6 <hostname|ip>
execute traceroute <hostname|ip>
execute tracert6 <hostname|ip>
get router info routing-table all   #routing table
get router info6 routing-table      #IPv6 without the "all" keyword
get router info kernel              #Forwarding Information Base
get router info6 kernel
get router <routing-protocol>       #basic information about the enabled routing protocol
diagnose firewall proute list       #policy-based routing
diagnose firewall proute6 list
diagnose ip rtcache list            #route cache = current sessions w/ routing information

HA

diagnose sys ha status
execute ha manage ?                 #switch to the CLI of a secondary unit
execute ha manage <device-index>
diagnose sys ha showcsum            #verify the checksum of all synchronized peers
get system session list             #rough view with NAT, only IPv4

Session Table

diagnose sys session filter clear
diagnose sys session filter ?
diagnose sys session filter dst 8.8.8.8
diagnose sys session filter dport 53
diagnose sys session list           #show the session table with the filter just set
View per-packet operations:

1#diagnose debug disable
2#diagnose debug flow trace stop
3#diagnose debug flow filter clear
4#diagnose debug reset

5#diagnose debug flow filter addr x.x.x.x
6#diagnose debug flow show console enable
7#diagnose debug flow show function-name enable
8#diagnose debug console timestamp enable
9#diagnose debug flow trace start 100
10#diagnose debug enable

View specific traffic session with:
#diag sys session list
or
#diag sys session filter  sport:dport:dst:src:proto:policy

View per-ethernet frame operations:
#diag sniffer packet

View object dependencies:
# diagnose sys checkused firewall.address:name 'SSLVPN_TUNNEL_ADDR1'
# diagnose sys checkused firewall.address:name 'iPhoneVPN_range'
# diagnose sys checkused firewall.address:name 'Local LAN'
# diagnose sys checkused system.interface:name  <interface name>



Debug

Sniffer - URL https://gateIP/ng/page/p/firewall/sniffer/

Sniff packets like tcpdump does. This can be used for investigating connection problems between two hosts. There are no details of the firewall policy decisions. Use the debug flow (next paragraph) for analysis about firewall policies, etc.
with:
verbose:
1: print header of packets
2: print header and data from ip of packets
3: print header and data from ethernet of packets (if available)
4: print header of packets with interface name
5: print header and data from ip of packets with interface name
6: print header and data from ethernet of packets (if available) with intf name

count: number of packets
time-format:
a: UTC time
l: local time
Examples: (Thanks to the comment from Ulrich for the IPv6 example)

Here are two more examples on how to show LLDP or CDP packets in order to reveal the connected layer 2 ports from switches. Kudos to Joachim Schwierzeck.
For more information on how to capture and then convert the date to pcap file please follow this link:
https://kb.fortinet.com/kb/documentLink.do?externalID=11186&languageId

Flow

If you want to see the FortiGate details about a connection, use this kind of debug. E.g., it shows the routing decision and the policy, which allowed the connection.
Example:

 

VPN

To show details about IKE/IPsec connections, use these commands:
To debug IKE/IPsec sessions, use the VPN debug:
To reset a certain VPN connection, use :

Log

For investigating the log entries (similar to the GUI), use the following filters, etc.:


Defaults

Just a reminder for myself:
  • IP: 192.168.1.99
  • Login: admin
  • Password: <blank>
To change the IP address of the mgmt interface (or any other) via the CLI, these commands can be used:
#diag debug <option>  <level>

Options:
application - Application
cli - debug cli
console
crashlog - crash log info
disable
flow - trace packet flow
haproxy
info - show active debug level settings
report - report for tech support
reset - reset all debug level to default

Capture TCP packets with SYN flag set on
# diagnose sniffer packet  any 'port 80 and tcp[13]&2==2' 4






// Health and Status

show [enter]               //Note that output is only non-default values.

show full-configuration   // Show all configurations on the device.

show system interface wan1 | grep -A2 ip // Show WAN and interface information.

get system info admin status // Show logged in users

get system status            // Show system hardware/software update versions

get hardware status          // Detailed hardware model information

get system performance status

get system performance top

show system interface // Interface Configuration



DEBUG





diagnose hardware deviceinfo nic // Interface Statistics/Settings

diagnose hardware sysinfo memory

diag debug crashlog read

diag hardware sysinfo shm     // Device should be in 0, if (>0) then conserve mode

get system global | grep -i timer    // Show tcp and udp timers for half open and idle

get system session-ttl     // System default tcp-idle session timeout

execute ha manage <devid>    // send heartbeat across management link.

get hardware nic

diagnose ip address list

get system interface physical // ARP

diagnose ip arp list // Track and Troubleshoot

get system session status      // Connection count for ingress/egress

get system session-info full-stat    // Displays session status with breakdown by state

get system session list       // Session list, protocol, expire, src nat, dst nat

diag sys session              // Basic output with no filters of diag sys session

diag sys session filter <option> <value>      // Capture filter based on src, dst, duraction, policy id, vd // Packet capture

diag debug info     // Displays active debug

diag debug enable     // Enable debug

#diagnose debug flow filter (shows what filters are configured)

#diagnose debug flow filter clear (clear all filter)

#diagnose debug flow filter <options> <value> (configures the filter)

#diagnose debug flow show con enable <show output on console>

#diagnose debug flow show fun enable <show functions>

#diagnose debug flow trace start <number of lines> (to start the trace)

#diagnose debug flow trace stop (to stop the trace)

Example:

diagnose debug reset

diagnose debug enable

diagnose debug flow filter clear

diagnose debug flow filter saddr 192.168.10.1

diagnose debug flow filter dport 80

diagnose debug flow show con enable

diagnose debug flow show fun enable

diagnose debug flow trace start 20

diagnose sniffer packet <interface or ANY> ‘<arguments>’ <level 1-6>

example:

diagnose sniffer packet ANY ‘net 192.168.10.0/24 and not host 192.168.10.1 and port 80 and TCP’ 6

Syn packets only:

diag sniffer packet internal ‘tcp[13] == 2′



 Enable packet capture in GUI - System -> Config -> Advanced

Setup packet capture filter, Check box to start, Uncheck to stop.

Download Debug Log

// Show identified devices

diag user device list



Routing

diagnose ip rtcache list // rt cache table





Interface Up -> Multiple: Select lowest distance -> Dynamic: If same distance choose lowest metric -> Dynamic: If multiple have same distance/metric, depends on protocol -> All “Best Routes” places in table. Match goes to most exact subnet -> Policy routing applied before table lookups.

Route lookups are only for the first packet of each session.

All packets will use same path.

After topology change, routes are flushed and sessions relearned.

get system arp          // ARP Table

get router info routing-table all       // All routing table entries



get router info routing-table details <ip>         // Shows if custom static or dynamic routes exist for dest.

get router info kernel             // Raw kernel routing table

show router static    // Display static routes

show router policy //



Restore image

execute restore image <firmware_file_name> <TFTP server_ipaddress>    // Restore an image from TFTP



Provisioning

config system settings        // Configure for layer-3

set opmode nat

end

config system settings       // Configure transparent

set opmode transparent

end

config system global         // Set port for admin if VPN is sharing

set admin-sport 8443

set sslvpn-sport 443

end

config system global        // Enable SCP

set admin-scp enable

cofng system ntp            // Setup NTP

config ntpserver

edit 1

set server 10.0.0.0

end

edit 2

set server 10.0.0.1

end

set ntpsync enable

end

execute time

config system dns          // Setup DNS

set primary  0.0.0.0

set secondary 0.0.00

config log syslogd(2|3) setting     // Enable syslog

set status enable

set server <IP address>

set port 514

set facility user

end

diagnose log test     // Test logging

config system interface          // Setup IP Address

edit wan1

set mode static

set ip 172.16.0.0 255.255.255.0

set vlan id 50

end

config system interface          // LACP port aggregation

edit aggr1

set member “port8″ “port9″

end

config system zone             // Add interfaces to zone

edit outside

set interface internal1 internal 2

enable intrazone traffic

set intrazone allow

end

config router static       // Add default route

edit 1

set gateway 172.16.0.0

end

config router static       // Static route

edit2

set device port1

set dst 10.0.0.0 255.0.0.0

set gateway 10.0.1.1

// Vendor Notes

http://docs.fortinet.com/fgt.html

http://docs.fortinet.com/fgt/handbook/40mr3/fortigate-cli-40-mr3.pdf

http://docs.fortinet.com/fgt50.html