NetOps Toolkit

Operations

Troubleshooting Guide

Structured methodology, common scenarios, and diagnostic commands

Structured Methodology

  1. Identify the Problem — Gather info, question users, check logs, try to replicate
  2. Establish a Theory — Most probable cause first (layer-by-layer, bottom-up)
  3. Test the Theory — If confirmed, proceed. If not, re-theorize
  4. Establish a Plan — Plan the fix, assess impact, schedule if needed
  5. Implement the Solution — Make the change, escalate if necessary
  6. Verify Full Functionality — Confirm fix and check for new issues
  7. Document Findings — Root cause, steps taken, lessons learned

Can't Communicate Across VLANs

  1. Verify VLAN config (show vlan brief)
  2. Check trunk allowed VLANs (show interfaces trunk)
  3. Confirm IP addresses and subnets match
  4. Verify inter-VLAN routing is configured
  5. Check ACLs blocking traffic (show access-lists)

Interface Down / Flapping

  1. Check physical cables and connections
  2. show interface <intf> — look for errors, CRC, resets
  3. Try shutdown / no shutdown
  4. Check speed/duplex mismatch (both sides)
  5. Verify SFP/transceiver compatibility
  6. Check for port-security violations
  7. Default the port to reset all config: default interface <intf> — then reconfigure from scratch

No Internet / WAN Down

  1. Ping default gateway, then upstream
  2. Check NAT translations (show ip nat translations)
  3. Verify default route exists (show ip route)
  4. Check ISP-facing interface status
  5. Verify DNS resolution (nslookup)

DHCP Not Working

  1. Verify DHCP pool config (show run | section dhcp)
  2. Check excluded addresses aren't overlapping the pool
  3. Verify ip helper-address if DHCP server is remote
  4. Confirm client interface is up and in correct VLAN
  5. Check DHCP bindings (show ip dhcp binding)

Slow Network / High Latency

  1. Check interface utilization (show interfaces)
  2. Look for CRC errors, collisions, drops
  3. Verify duplex settings (full duplex expected)
  4. Check for broadcast storms (show interfaces counters)
  5. Monitor bandwidth with SNMP/NetFlow
  6. Verify QoS policies if VoIP affected
  7. Check service-policy drops: show policy-map interface <intf>

OSPF Neighbor Not Forming

  1. Verify interfaces are in correct area (show ip ospf interface)
  2. Check hello/dead timers match on both sides
  3. Verify subnet masks match
  4. Check area type matches (stub, NSSA)
  5. Verify authentication matches
  6. Check MTU (must match for adjacency)

QoS / Service Policy Issues

  1. Confirm policy applied: show run interface <intf> | include service-policy
  2. View live counters and drops per class: show policy-map interface <intf>
  3. Check for queue tail-drops or WRED drops in the output direction
  4. Verify class-map match conditions are correct: show class-map
  5. Confirm correct policy-map is attached: show run | section service-policy
  6. Check DSCP/CoS markings on ingress traffic: show mls qos interface <intf>
  7. Reset counters: remove service-policy, re-apply, observe fresh counts
  8. Verify DSCP-to-queue mappings: show mls qos maps dscp-output-q