LEARN LINUX NETWORKING TOOLS
Learn Linux Networking Tools: From Zero to Network Wizard
Goal: Deeply understand Linux networking from the packet level up—mastering the tools that let you see, diagnose, control, and secure network traffic. You’ll learn not just command syntax, but why networks behave as they do, how packets flow through the kernel, and how to troubleshoot any connectivity issue like a seasoned systems administrator.
Why Linux Networking Tools Matter
In 1969, ARPANET connected four computers. Today, billions of devices communicate using protocols built on those foundations. Linux powers the majority of servers, routers, and network appliances on the internet. Understanding its networking tools isn’t just useful—it’s essential for anyone who wants to:
- Debug connectivity issues that cost companies millions in downtime
- Secure systems against increasingly sophisticated attacks
- Optimize performance for applications serving millions of users
- Build distributed systems that actually work under real-world conditions
These tools are the “X-ray vision” of networking. While others guess why things are broken, you’ll see exactly what’s happening at every layer of the stack.
The Reality of Modern Network Administration
┌─────────────────────────────────────────────────────────────────────┐
│ Your Application │
├─────────────────────────────────────────────────────────────────────┤
│ curl, wget (Application Layer - L7) │
├─────────────────────────────────────────────────────────────────────┤
│ ss, netstat, lsof (Transport Layer - L4) │
├─────────────────────────────────────────────────────────────────────┤
│ ping, traceroute, mtr, ip route (Network Layer - L3) │
├─────────────────────────────────────────────────────────────────────┤
│ ip link, ethtool, arp, ip neigh (Data Link Layer - L2) │
├─────────────────────────────────────────────────────────────────────┤
│ tcpdump, iptables, nft, tc (Kernel Netfilter/Traffic) │
├─────────────────────────────────────────────────────────────────────┤
│ dmesg, journalctl (System Logs) │
└─────────────────────────────────────────────────────────────────────┘
Every tool you’ll learn maps to a specific layer or function. By the end, you’ll know exactly which tool to reach for in any situation.
Core Concept Analysis
1. The Linux Network Stack: How Packets Flow
Understanding how a packet travels through Linux is fundamental to using these tools effectively.
USERSPACE
┌─────────────────────────────────────────────────────────┐
│ Application (curl, nginx, ssh...) │
│ │ ▲ │
│ ▼ │ │
│ Socket API (send/recv) Socket API │
└──────────────┼────────────────────┼──────────────────────┘
│ │
═══════════════════╪════════════════════╪═══════════════════════
│ KERNEL │
┌──────────────▼────────────────────┼──────────────────────┐
│ Transport Layer (TCP/UDP) │
│ ┌─────────────────────────────────────────────┐ │
│ │ ss, netstat see connections HERE │ │
│ └─────────────────────────────────────────────┘ │
├──────────────────────────────────────────────────────────┤
│ Network Layer (IP) │
│ ┌─────────────────────────────────────────────┐ │
│ │ ip route, routing decisions HERE │ │
│ └─────────────────────────────────────────────┘ │
├──────────────────────────────────────────────────────────┤
│ NETFILTER HOOKS │
│ ┌─────────────────────────────────────────────┐ │
│ │ iptables/nft filter packets HERE │ │
│ │ │ │
│ │ PREROUTING → INPUT → FORWARD → OUTPUT → │ │
│ │ POSTROUTING │ │
│ └─────────────────────────────────────────────┘ │
├──────────────────────────────────────────────────────────┤
│ Traffic Control (tc) │
│ ┌─────────────────────────────────────────────┐ │
│ │ Queueing disciplines shape traffic HERE │ │
│ └─────────────────────────────────────────────┘ │
├──────────────────────────────────────────────────────────┤
│ Data Link Layer │
│ ┌─────────────────────────────────────────────┐ │
│ │ ARP, MAC addresses, ethtool HERE │ │
│ └─────────────────────────────────────────────┘ │
└──────────────────────────────────────────────────────────┘
│ ▲
▼ │
┌──────────────────────────────────────────────────────────┐
│ Network Interface (NIC) │
│ ┌─────────────────────────────────────────────┐ │
│ │ Physical hardware / tcpdump captures HERE │ │
│ └─────────────────────────────────────────────┘ │
└──────────────────────────────────────────────────────────┘
│ ▲
▼ │
═══════════════════════════════════
Physical Network
Key insight: Different tools operate at different points in this flow. tcpdump sees raw packets at the NIC level. iptables intercepts at Netfilter hooks. ss sees established connections at the transport layer.
2. The Tool Categories: A Mental Map
┌─────────────────────────────────────────────────────────────────────┐
│ LINUX NETWORKING TOOLS │
├─────────────────────────────────────────────────────────────────────┤
│ │
│ ┌─────────────────────┐ ┌─────────────────────┐ │
│ │ CONFIGURATION │ │ DIAGNOSTICS │ │
│ │ │ │ │ │
│ │ ip (addresses, │ │ ping (reachability)│ │
│ │ routes, links) │ │ traceroute (path) │ │
│ │ nmcli (NetworkMgr) │ │ tracepath (PMTUD) │ │
│ │ ethtool (NIC) │ │ mtr (combined) │ │
│ │ ifconfig (legacy) │ │ dig (DNS queries) │ │
│ │ route (legacy) │ │ nslookup (DNS) │ │
│ │ ip netns (namespcs)│ │ resolvectl (systemd│ │
│ └─────────────────────┘ │ DNS) │ │
│ └─────────────────────┘ │
│ ┌─────────────────────┐ ┌─────────────────────┐ │
│ │ CONNECTION STATE │ │ PACKET ANALYSIS │ │
│ │ │ │ │ │
│ │ ss (sockets, │ │ tcpdump (capture) │ │
│ │ modern) │ │ │ │
│ │ netstat (legacy) │ │ │ │
│ │ lsof (files/socks) │ │ │ │
│ └─────────────────────┘ └─────────────────────┘ │
│ │
│ ┌─────────────────────┐ ┌─────────────────────┐ │
│ │ FIREWALLING │ │ TRAFFIC CONTROL │ │
│ │ │ │ │ │
│ │ iptables (legacy) │ │ tc (qdisc/class) │ │
│ │ nft (nftables) │ │ iperf3 (benchmark) │ │
│ │ ufw (Ubuntu front) │ │ nload (monitor) │ │
│ │ firewall-cmd │ │ iftop (per-conn) │ │
│ │ (firewalld) │ │ │ │
│ └─────────────────────┘ └─────────────────────┘ │
│ │
│ ┌─────────────────────┐ ┌─────────────────────┐ │
│ │ LAYER 2 / ARP │ │ DATA TRANSFER │ │
│ │ │ │ │ │
│ │ ip neigh (ARP tbl) │ │ curl (HTTP client) │ │
│ │ arp (legacy) │ │ wget (downloader) │ │
│ └─────────────────────┘ └─────────────────────┘ │
│ │
│ ┌─────────────────────┐ │
│ │ SYSTEM LOGS │ │
│ │ │ │
│ │ dmesg (kernel) │ │
│ │ journalctl (systemd│ │
│ └─────────────────────┘ │
│ │
└─────────────────────────────────────────────────────────────────────┘
3. The Netfilter Framework: Where Firewalls Live
INCOMING PACKET
│
▼
┌──────────────────┐
│ PREROUTING │ ← DNAT happens here
│ (raw, mangle, │
│ nat tables) │
└────────┬─────────┘
│
▼
┌──────────────────┐
│ Routing Decision │
└────────┬─────────┘
│
┌───────────────────┴───────────────────┐
│ │
▼ ▼
┌──────────────────┐ ┌──────────────────┐
│ INPUT │ │ FORWARD │
│ (filter, mangle)│ │ (filter, mangle)│
│ │ │ │
│ Packets FOR │ │ Packets THROUGH │
│ this host │ │ this host │
└────────┬─────────┘ └────────┬─────────┘
│ │
▼ │
┌──────────────────┐ │
│ Local Process │ │
│ (your app) │ │
└────────┬─────────┘ │
│ │
▼ │
┌──────────────────┐ │
│ OUTPUT │ │
│ (raw, mangle, │ │
│ nat, filter) │ │
└────────┬─────────┘ │
│ │
└───────────────────┬─────────────────┘
│
▼
┌──────────────────┐
│ POSTROUTING │ ← SNAT/Masquerade here
│ (mangle, nat) │
└────────┬─────────┘
│
▼
OUTGOING PACKET
Key insight:
iptablesandnftadd rules to these chainsufwandfirewall-cmdare frontends that generate iptables/nft rules- Understanding the packet flow helps you place rules correctly
4. Socket States: What ss/netstat Show You
TCP Connection State Machine (Simplified)
Client Server
│ │
│ ┌─────────┐ │
│ │ LISTEN │◄────────────┤ Server waiting
│ └────┬────┘ │
│ │ │
┌──────┴──────┐ │ │
│ SYN_SENT │────────┼──── SYN ────────►│
└──────┬──────┘ │ │
│ │ ┌──────┴──────┐
│◄──────────────┼── SYN+ACK─│ SYN_RCVD │
│ │ └──────┬──────┘
┌──────┴──────┐ │ │
│ ESTABLISHED │────────┼──── ACK ────────►│
└──────┬──────┘ │ ┌──────┴──────┐
│ │ │ ESTABLISHED │
│ │ └──────┬──────┘
│ │ │
│◄══════════════╪═══ DATA ════════►│
│ │ │
┌──────┴──────┐ │ │
│ FIN_WAIT_1 │────────┼──── FIN ────────►│
└──────┬──────┘ │ ┌──────┴──────┐
│ │ │ CLOSE_WAIT │
│◄──────────────┼── ACK ────│ │
┌──────┴──────┐ │ └──────┬──────┘
│ FIN_WAIT_2 │ │ │
└──────┬──────┘ │ ┌──────┴──────┐
│ │ │ LAST_ACK │
│◄──────────────┼── FIN ────│ │
┌──────┴──────┐ │ └─────────────┘
│ TIME_WAIT │────────┼──── ACK ────────►│
└──────┬──────┘ │ │
│ │ │
(2MSL wait) │ ┌─────┴─────┐
│ │ │ CLOSED │
┌──────┴──────┐ │ └───────────┘
│ CLOSED │ │
└─────────────┘ │
States you'll see with ss/netstat:
- LISTEN: Server waiting for connections
- ESTABLISHED: Active data transfer
- TIME_WAIT: Connection closed, waiting for stray packets
- CLOSE_WAIT: Remote side closed, waiting for local close
5. DNS Resolution: What dig/nslookup Query
Your Computer
│
│ 1. "What's the IP of www.example.com?"
▼
┌─────────────┐
│ Stub │ /etc/resolv.conf points here
│ Resolver │ (resolvectl shows this)
└──────┬──────┘
│
│ 2. Query to recursive resolver
▼
┌─────────────┐
│ Recursive │ (Usually your ISP or 8.8.8.8)
│ Resolver │
└──────┬──────┘
│
│ 3. If not cached, ask root servers
▼
┌─────────────┐
│ Root │ "I don't know, but .com is over there"
│ Servers │ (13 root server clusters globally)
└──────┬──────┘
│
│ 4. Ask .com TLD servers
▼
┌─────────────┐
│ .com │ "example.com's NS is ns1.example.com"
│ TLD Servers │
└──────┬──────┘
│
│ 5. Ask authoritative server
▼
┌─────────────┐
│ Authoritat. │ "www.example.com = 93.184.216.34"
│ Server │
└─────────────┘
dig +trace www.example.com shows this entire journey!
6. Network Namespaces: Isolated Network Stacks
┌─────────────────────────────────────────────────────────────────┐
│ HOST SYSTEM │
│ │
│ ┌─────────────────────┐ ┌─────────────────────┐ │
│ │ Namespace: ns1 │ │ Namespace: ns2 │ │
│ │ │ │ │ │
│ │ ┌───────────────┐ │ │ ┌───────────────┐ │ │
│ │ │ Own interfaces│ │ │ │ Own interfaces│ │ │
│ │ │ Own routes │ │ │ │ Own routes │ │ │
│ │ │ Own iptables │ │ │ │ Own iptables │ │ │
│ │ │ Own /proc/net │ │ │ │ Own /proc/net │ │ │
│ │ └───────┬───────┘ │ │ └───────┬───────┘ │ │
│ │ │ │ │ │ │ │
│ │ veth-ns1 │ │ veth-ns2 │ │
│ └──────────┼──────────┘ └──────────┼──────────┘ │
│ │ │ │
│ │ veth pair │ veth pair │
│ │ (virtual cable) │ (virtual cable) │
│ │ │ │
│ ┌──────────┴──────────────────────────────┴──────────┐ │
│ │ br0 (bridge) │ │
│ │ Default Network Namespace │ │
│ └─────────────────────────┬───────────────────────────┘ │
│ │ │
│ eth0 │ (physical) │
└────────────────────────────┼─────────────────────────────────────┘
│
Physical Network
ip netns creates these isolated stacks!
Docker/Kubernetes use this heavily.
7. Traffic Control: Queuing Disciplines
Outgoing Packets
│
▼
┌───────────────────────┐
│ Root Qdisc │
│ (e.g., htb) │
└───────────┬───────────┘
│
┌───────────────┼───────────────┐
│ │ │
▼ ▼ ▼
┌───────────┐ ┌───────────┐ ┌───────────┐
│ Class │ │ Class │ │ Class │
│ 1:10 │ │ 1:20 │ │ 1:30 │
│ (High) │ │ (Normal) │ │ (Low) │
│ rate 5mb │ │ rate 2mb │ │ rate 1mb │
└─────┬─────┘ └─────┬─────┘ └─────┬─────┘
│ │ │
▼ ▼ ▼
┌───────────┐ ┌───────────┐ ┌───────────┐
│ Filter │ │ Filter │ │ Filter │
│ port 22 │ │ port 80 │ │ default │
└───────────┘ └───────────┘ └───────────┘
│ │ │
└───────────────┴───────────────┘
│
▼
Network Interface
tc qdisc / tc class / tc filter control this!
Concept Summary Table
| Concept Cluster | What You Need to Internalize |
|---|---|
| Packet Flow through Kernel | Packets traverse PREROUTING → routing decision → INPUT/FORWARD → OUTPUT → POSTROUTING. Know where each tool intercepts. |
| Socket States | TCP connections move through states (LISTEN, ESTABLISHED, TIME_WAIT). ss/netstat show where connections are stuck. |
| IP Addressing & Routing | Every packet needs a destination. The kernel consults the routing table (ip route) to decide which interface/gateway to use. |
| DNS Resolution | Names → IPs via hierarchical queries. Stub resolver → Recursive → Root → TLD → Authoritative. Cache at every level. |
| Netfilter Chains | Firewall rules are organized in chains (INPUT, OUTPUT, FORWARD) within tables (filter, nat, mangle). Order matters. |
| Network Namespaces | Linux can have multiple isolated network stacks. Each namespace has its own interfaces, routes, and firewall rules. |
| Traffic Control (QoS) | The kernel queues outgoing packets. Qdiscs shape/limit/prioritize traffic. HTB, TBF, netem are common disciplines. |
| Layer 2 vs Layer 3 | ARP maps IP addresses to MAC addresses on local network. Routing happens at L3; switching at L2. |
| Modern vs Legacy Tools | ip replaces ifconfig/route. ss replaces netstat. nft replaces iptables. Know both for compatibility. |
Deep Dive Reading by Concept
This section maps each concept to specific book chapters for deeper understanding. Read these before or alongside the projects to build strong mental models.
Network Fundamentals
| Concept | Book & Chapter |
|---|---|
| TCP/IP Protocol Stack | “TCP/IP Illustrated, Volume 1” by W. Richard Stevens — Ch. 1-3 |
| How Linux Implements Networking | “The Linux Programming Interface” by Michael Kerrisk — Ch. 56-61 |
| Sockets Programming | “The Sockets Networking API (UNIX Network Programming Vol 1)” by W. Richard Stevens — Ch. 1-8 |
| Understanding Packets | “Computer Networks” by Tanenbaum — Ch. 1-5 |
Linux Network Configuration
| Concept | Book & Chapter |
|---|---|
The ip Command Suite |
“How Linux Works, 3rd Edition” by Brian Ward — Ch. 9 |
| Network Manager & nmcli | “Linux for Networking Professionals” (O’Reilly) — Ch. 3-4 |
| Interface Configuration | “Linux System Programming” by Robert Love — Ch. 10 |
Firewalling & Security
| Concept | Book & Chapter |
|---|---|
| iptables Deep Dive | “Linux Firewalls” by Michael Rash — Ch. 1-6 |
| nftables Transition | “Linux for Networking Professionals” (O’Reilly) — Ch. 9 |
| Netfilter Internals | “Understanding Linux Network Internals” by Christian Benvenuti — Ch. 18-21 |
DNS & Name Resolution
| Concept | Book & Chapter |
|---|---|
| DNS Protocol | “TCP/IP Illustrated, Volume 1” by W. Richard Stevens — Ch. 14 |
| DNS Operations | “DNS and BIND” by Cricket Liu — Ch. 1-4 |
| Troubleshooting DNS | “Linux for Networking Professionals” (O’Reilly) — Ch. 6 |
Packet Analysis
| Concept | Book & Chapter |
|---|---|
| tcpdump Mastery | “The Practice of Network Security Monitoring” by Richard Bejtlich — Ch. 5 |
| Packet Capture Techniques | “Practical Packet Analysis” by Chris Sanders — Ch. 1-6 |
Traffic Control & Performance
| Concept | Book & Chapter |
|---|---|
| Linux QoS with tc | “Linux Advanced Routing & Traffic Control HOWTO” (lartc.org) — Full document |
| Network Performance | “Systems Performance” by Brendan Gregg — Ch. 10 |
Essential Reading Order
For maximum comprehension, read in this order:
- Foundation (Week 1-2):
- “How Linux Works” Ch. 9 (Networking basics)
- “TCP/IP Illustrated Vol 1” Ch. 1-3 (Protocol fundamentals)
- Deep Protocol Understanding (Week 3-4):
- “TCP/IP Illustrated Vol 1” Ch. 4-8 (IP, ICMP, UDP, TCP)
- “The Linux Programming Interface” Ch. 56-58 (Sockets)
- Tools Mastery (Week 5-6):
- “Linux for Networking Professionals” (Full book)
- “Linux Firewalls” Ch. 1-6 (iptables/psad)
- Advanced Topics (Week 7-8):
- “Understanding Linux Network Internals” (Selected chapters)
- LARTC HOWTO (Traffic control)
Project List
Projects are ordered from fundamental understanding to advanced implementations. Each project teaches specific tools through hands-on building with real, observable outcomes.
Project 1: Network Interface Inspector
- File: LEARN_LINUX_NETWORKING_TOOLS.md
- Main Programming Language: Bash (shell scripting)
- Alternative Programming Languages: Python, Go, Rust
- Coolness Level: Level 2: Practical but Forgettable
- Business Potential: 1. The “Resume Gold”
- Difficulty: Level 1: Beginner
- Knowledge Area: Network Configuration / Interface Management
- Software or Tool: ip, ifconfig, ethtool, nmcli
- Main Book: “How Linux Works, 3rd Edition” by Brian Ward
What you’ll build: A comprehensive network interface auditing tool that displays all interfaces, their states, IP addresses, MAC addresses, driver information, link speeds, and connection status in a clean, organized format.
Why it teaches these tools: You can’t diagnose network problems if you don’t know the current state. This project forces you to understand what each tool reveals about your hardware and configuration—and when to use which tool.
Core challenges you’ll face:
- Parsing
ipcommand output → maps to understanding modern iproute2 tools - Extracting hardware info with
ethtool→ maps to NIC driver/hardware layer - Comparing
ifconfigvsip→ maps to legacy vs modern tool knowledge - Understanding NetworkManager with
nmcli→ maps to desktop/server network management
Key Concepts:
- Interface naming conventions: “How Linux Works” Ch. 9 - Brian Ward
- Link layer addresses (MAC): “TCP/IP Illustrated Vol 1” Ch. 3 - W. Richard Stevens
- Network namespaces basics: “The Linux Programming Interface” Ch. 58 - Michael Kerrisk
Difficulty: Beginner Time estimate: Weekend Prerequisites: Basic Linux command line familiarity; understanding of IP addresses and what a network interface is
Real World Outcome
You’ll have a script that produces output like this:
$ sudo ./netaudit.sh
╔══════════════════════════════════════════════════════════════════════╗
║ NETWORK INTERFACE AUDIT ║
║ Host: devserver01 ║
║ Date: 2024-12-22 14:32:01 UTC ║
╠══════════════════════════════════════════════════════════════════════╣
┌─────────────────────────────────────────────────────────────────────┐
│ Interface: eth0 [UP/RUNNING]│
├─────────────────────────────────────────────────────────────────────┤
│ MAC Address : 52:54:00:12:34:56 │
│ IPv4 Address : 192.168.1.100/24 │
│ IPv6 Address : fe80::5054:ff:fe12:3456/64 │
│ Gateway : 192.168.1.1 │
│ DNS Servers : 8.8.8.8, 8.8.4.4 │
├─────────────────────────────────────────────────────────────────────┤
│ Driver : virtio_net │
│ Speed : 1000Mb/s (Full Duplex) │
│ Link Detected : yes │
│ TX Packets : 1,234,567 (523.4 MB) │
│ RX Packets : 2,345,678 (1.2 GB) │
│ Errors : TX: 0 RX: 0 Dropped: 12 │
└─────────────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────────┐
│ Interface: lo [UP/RUNNING]│
├─────────────────────────────────────────────────────────────────────┤
│ IPv4 Address : 127.0.0.1/8 │
│ IPv6 Address : ::1/128 │
│ Type : Loopback │
└─────────────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────────┐
│ Interface: docker0 [UP/NO-CARRIER]│
├─────────────────────────────────────────────────────────────────────┤
│ MAC Address : 02:42:ac:11:00:01 │
│ IPv4 Address : 172.17.0.1/16 │
│ Type : Bridge │
│ Attached : veth123abc, veth456def │
└─────────────────────────────────────────────────────────────────────┘
SUMMARY:
Total interfaces: 5 (3 UP, 2 DOWN)
Total IPv4 addresses: 4
Interfaces with errors: 0
NetworkManager Status: running (connected)
The Core Question You’re Answering
“What is the current state of my network, and how do I see EVERYTHING about each interface?”
Before you write any code, sit with this question. Most people run ifconfig and call it done. But there’s so much more: Is the link actually connected? What speed is it running at? What driver is the kernel using? Is NetworkManager managing it or is it static? This project forces you to extract every piece of information the kernel knows about your network hardware.
Concepts You Must Understand First
Stop and research these before coding:
- Network Interface Types
- What’s the difference between eth0, ens33, and enp0s3 naming?
- What is a loopback interface and why is it special?
- What are virtual interfaces (veth, bridge, tun/tap)?
- Book Reference: “How Linux Works” Ch. 9 - Brian Ward
- IP Address Assignment
- What’s CIDR notation (e.g., /24)?
- How does DHCP differ from static assignment?
- What’s the difference between primary and secondary addresses?
- Book Reference: “TCP/IP Illustrated Vol 1” Ch. 3-4 - W. Richard Stevens
- Link Layer Concepts
- What is a MAC address and how is it structured?
- What does “link speed” mean and how is it negotiated?
- What’s Full Duplex vs Half Duplex?
- Book Reference: “Computer Networks” Ch. 4 - Tanenbaum
Questions to Guide Your Design
Before implementing, think through these:
- Data Collection
- Which command gives interface list most reliably?
- How do you get IPv4 AND IPv6 addresses?
- Where does gateway information come from (it’s not per-interface)?
- How do you detect if NetworkManager is managing an interface?
- Tool Selection
- When would
ip addrfail whereifconfigworks (or vice versa)? - What information does
ethtoolprovide thatipdoesn’t? - How do you get DNS server information?
- When would
- Output Design
- How do you handle interfaces with multiple IPs?
- Should you show DOWN interfaces? How do you indicate state?
- How do you format large numbers (bytes) readably?
Thinking Exercise
Trace the Information Sources
Before coding, for each piece of data, write down which command(s) provide it:
Interface name: ip link | ??? | ???
IPv4 address: ip addr | ifconfig | ???
MAC address: ip link | ifconfig | ???
Link speed: ??? | ???
Driver name: ??? | ???
Gateway: ip route | ???
DNS servers: ??? | resolvectl | ???
TX/RX statistics: ip -s link | ???
NetworkManager state: nmcli ???
Questions while tracing:
- Which tools require root and which don’t?
- Which tools work on minimal systems without NetworkManager?
- What’s in /sys/class/net/*/ that might help?
The Interview Questions They’ll Ask
Prepare to answer these:
- “What’s the difference between
ipandifconfig, and which would you use?” - “How would you check if a network cable is plugged in?”
- “An interface shows UP but has no IP address. What could cause this?”
- “How do you find out what driver a network interface is using?”
- “What’s the difference between eth0, ens33, and enp0s3 naming schemes?”
- “How do you add a secondary IP address to an interface?”
- “How would you check the current MTU and why might you change it?”
Hints in Layers
Hint 1: Starting Point
Start with ip link show to get all interfaces, then enrich each with additional commands.
Hint 2: Parsing Strategy
The ip -json flag outputs JSON, which is much easier to parse than text. Consider using jq or Python.
Hint 3: ethtool Requires Interface Name
Run ethtool <interface> for speed/duplex and ethtool -i <interface> for driver info. Not all interfaces support ethtool (virtual ones don’t).
Hint 4: Gateway and DNS Are System-Wide
ip route show default gives the default gateway. DNS servers are in /etc/resolv.conf or via resolvectl status on systemd systems.
Books That Will Help
| Topic | Book | Chapter |
|---|---|---|
| Interface management | “How Linux Works, 3rd Edition” by Brian Ward | Ch. 9 |
| IP addressing | “TCP/IP Illustrated, Vol 1” by W. Richard Stevens | Ch. 3-4 |
| Linux network internals | “The Linux Programming Interface” by Michael Kerrisk | Ch. 56 |
| NetworkManager | “Linux for Networking Professionals” (O’Reilly) | Ch. 3 |
Implementation Hints
The key insight is that no single tool gives you everything. You need to combine:
ip link show: Interface names, MAC addresses, state (UP/DOWN), MTUip addr show: IP addresses (v4 and v6) per interfaceip route show default: Default gatewayethtool <iface>: Link speed, duplex, link detectionethtool -i <iface>: Driver name, firmware version/etc/resolv.conforresolvectl status: DNS serversnmcli device show: NetworkManager’s view (if running)ip -s link show: TX/RX packet statistics
The challenge is merging all this into a coherent per-interface view. Consider building a data structure per interface, then populating it from each source.
Learning milestones:
- You can list all interfaces → You understand
ip link - You can distinguish interface types → You understand virtual vs physical
- You can get hardware details → You understand
ethtooland /sys - Your tool works without NetworkManager → You understand the layers
Project 2: Connectivity Diagnostic Suite
- File: LEARN_LINUX_NETWORKING_TOOLS.md
- Main Programming Language: Bash (shell scripting)
- Alternative Programming Languages: Python, Go, Rust
- Coolness Level: Level 3: Genuinely Clever
- Business Potential: 2. The “Micro-SaaS / Pro Tool”
- Difficulty: Level 2: Intermediate
- Knowledge Area: Network Diagnostics / Troubleshooting
- Software or Tool: ping, traceroute, tracepath, mtr
- Main Book: “TCP/IP Illustrated, Volume 1” by W. Richard Stevens
What you’ll build: An intelligent network diagnostic tool that, given a target (hostname or IP), runs a battery of tests (ICMP ping, TCP ping, traceroute variants, MTR) and produces a comprehensive report identifying where connectivity breaks and why.
Why it teaches these tools: These are the fundamental troubleshooting tools every network engineer uses. By building a tool that runs them intelligently and interprets their output, you’ll understand what each reveals about the path packets take and where problems occur.
Core challenges you’ll face:
- Understanding ICMP vs TCP connectivity tests → maps to protocol differences
- Interpreting traceroute output (asterisks, asymmetric routing) → maps to real-world routing complexity
- Detecting MTU/PMTUD issues with tracepath → maps to fragmentation problems
- Combining mtr’s real-time statistics → maps to identifying intermittent issues
Key Concepts:
- ICMP Protocol: “TCP/IP Illustrated Vol 1” Ch. 6-7 - W. Richard Stevens
- IP Routing: “TCP/IP Illustrated Vol 1” Ch. 8-9 - W. Richard Stevens
- Path MTU Discovery: “TCP/IP Illustrated Vol 1” Ch. 20 - W. Richard Stevens
Difficulty: Intermediate Time estimate: 1 week Prerequisites: Project 1 completed; understanding of IP routing basics; familiarity with TCP/IP layers
Real World Outcome
$ ./netdiag.sh google.com
╔══════════════════════════════════════════════════════════════════════╗
║ NETWORK CONNECTIVITY DIAGNOSTIC REPORT ║
║ Target: google.com (142.250.80.46) ║
║ From: 192.168.1.100 (eth0) ║
╠══════════════════════════════════════════════════════════════════════╣
┌─────────────────────────────────────────────────────────────────────┐
│ DNS RESOLUTION [✓ PASS] │
├─────────────────────────────────────────────────────────────────────┤
│ Resolved: google.com → 142.250.80.46 (23ms) │
│ Nameserver: 8.8.8.8 │
│ IPv6: 2607:f8b0:4004:800::200e │
└─────────────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────────┐
│ ICMP CONNECTIVITY [✓ PASS] │
├─────────────────────────────────────────────────────────────────────┤
│ 5 packets transmitted, 5 received, 0% packet loss │
│ RTT min/avg/max/mdev = 12.3/14.1/18.2/2.1 ms │
└─────────────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────────┐
│ TCP CONNECTIVITY (Port 443) [✓ PASS] │
├─────────────────────────────────────────────────────────────────────┤
│ Connection established in 15ms │
│ TLS handshake successful │
└─────────────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────────┐
│ PATH ANALYSIS (traceroute) │
├─────────────────────────────────────────────────────────────────────┤
│ Hop IP Address Hostname RTT Loss │
├─────────────────────────────────────────────────────────────────────┤
│ 1 192.168.1.1 router.local 1ms 0% │
│ 2 10.0.0.1 isp-gateway.net 12ms 0% │
│ 3 72.14.215.85 * 15ms 0% │
│ 4 142.250.80.46 lax17s01-in-f14... 14ms 0% │
└─────────────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────────┐
│ PATH MTU DISCOVERY [✓ PASS] │
├─────────────────────────────────────────────────────────────────────┤
│ Path MTU: 1500 bytes (no fragmentation needed) │
└─────────────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────────┐
│ MTR ANALYSIS (30 second sample) │
├─────────────────────────────────────────────────────────────────────┤
│ Jitter detected at hop 3: avg 15ms, stddev 8ms │
│ Recommendation: ISP link may be congested during peak hours │
└─────────────────────────────────────────────────────────────────────┘
DIAGNOSIS: ✓ Full connectivity to target
Network path is stable with 4 hops
No packet loss detected
Minor jitter at ISP handoff (normal)
The Core Question You’re Answering
“My connection to X seems slow/broken. WHERE exactly is the problem?”
This is THE question network engineers face daily. Is it DNS? The local network? The ISP? A router in between? The destination server? By combining multiple diagnostic tools, you’ll learn to pinpoint exactly where in the path the problem lies.
Concepts You Must Understand First
Stop and research these before coding:
- ICMP Protocol
- What’s the difference between ICMP Echo Request/Reply (ping) and other ICMP types?
- Why might ping work but web access fail?
- What is ICMP rate limiting and why do routers do it?
- Book Reference: “TCP/IP Illustrated Vol 1” Ch. 6-7 - W. Richard Stevens
- How Traceroute Works
- What is TTL (Time To Live) and how does traceroute abuse it?
- Why do you see asterisks (*) in traceroute output?
- What’s the difference between ICMP, UDP, and TCP traceroute modes?
- Book Reference: “TCP/IP Illustrated Vol 1” Ch. 8 - W. Richard Stevens
- Path MTU Discovery
- What is fragmentation and why is it bad for performance?
- How does PMTUD work?
- What’s an “ICMP Fragmentation Needed” message?
- Book Reference: “TCP/IP Illustrated Vol 1” Ch. 20 - W. Richard Stevens
Questions to Guide Your Design
Before implementing, think through these:
- Test Ordering
- In what order should you run tests for fastest diagnosis?
- Should DNS resolution happen first or in parallel?
- When should you skip certain tests?
- Failure Interpretation
- If ICMP ping fails but TCP succeeds, what does that mean?
- If traceroute shows asterisks mid-path, is the host unreachable?
- How do you distinguish a firewall block from a routing problem?
- Output Design
- How do you make the output useful for both experts and novices?
- What actionable recommendations can you provide?
Thinking Exercise
Interpret This Output
Before coding, analyze this real traceroute output:
traceroute to 8.8.8.8 (8.8.8.8), 30 hops max, 60 byte packets
1 192.168.1.1 (192.168.1.1) 1.234 ms 1.102 ms 1.089 ms
2 10.0.0.1 (10.0.0.1) 12.456 ms 12.234 ms 12.102 ms
3 * * *
4 * * *
5 72.14.215.85 (72.14.215.85) 18.234 ms 18.102 ms 18.089 ms
6 8.8.8.8 (8.8.8.8) 15.456 ms 15.234 ms 15.102 ms
Questions to answer:
- Can we reach 8.8.8.8? How do you know?
- Why do hops 3 and 4 show asterisks?
- Why is hop 6 (the destination) faster than hop 5?
- Is there a problem with this path?
The Interview Questions They’ll Ask
Prepare to answer these:
- “Ping works but I can’t access the website. What do you check?”
- “What does it mean when traceroute shows asterisks?”
- “How would you diagnose intermittent packet loss?”
- “What’s the difference between traceroute and tracepath?”
- “A user says the network is ‘slow’. How do you diagnose this?”
- “How would you test if a specific port is reachable?”
- “What is MTR and why is it better than traceroute for some diagnostics?”
Hints in Layers
Hint 1: Starting Point
Start with ping -c 5 to establish basic reachability, then expand to traceroute variants.
Hint 2: TCP Connectivity
Use nc -zv host port or curl --connect-timeout 5 to test TCP connections when ICMP might be blocked.
Hint 3: mtr Output
mtr --report-wide -c 30 host gives you 30 samples in report mode—parse this for statistics.
Hint 4: Path MTU
tracepath automatically does PMTUD. Look for “asymm” (asymmetric) and “pmtu” values in output.
Books That Will Help
| Topic | Book | Chapter |
|---|---|---|
| ICMP and Ping | “TCP/IP Illustrated, Vol 1” by W. Richard Stevens | Ch. 6-7 |
| Traceroute | “TCP/IP Illustrated, Vol 1” by W. Richard Stevens | Ch. 8 |
| Path MTU | “TCP/IP Illustrated, Vol 1” by W. Richard Stevens | Ch. 20 |
| Network troubleshooting | “Linux for Networking Professionals” (O’Reilly) | Ch. 5 |
Implementation Hints
The diagnostic flow should be:
- DNS Resolution:
dig +shortorhost- confirms name resolution works - ICMP Ping:
ping -c 5- basic reachability - TCP Ping:
nc -zvorcurl- application-level reachability - Traceroute:
tracerouteortraceroute -T(TCP mode) - path discovery - Path MTU:
tracepath- discovers MTU issues - MTR Analysis:
mtr --report -c 30- statistical path analysis
Interpret failures intelligently:
- ICMP blocked but TCP works → firewall filtering ICMP (common)
- High RTT at one hop but low at final → that hop is slow responding to ICMP (normal)
- Packet loss at intermediate hop only → likely ICMP rate limiting (normal)
- Packet loss at ALL subsequent hops → real problem at that point
Learning milestones:
- You can run all diagnostic tools → Basic command familiarity
- You understand what each tool reveals → Protocol understanding
- You can interpret asterisks and asymmetry → Real-world routing knowledge
- You can diagnose where a problem is → Network troubleshooting skill
Project 3: DNS Deep Dive Tool
- File: LEARN_LINUX_NETWORKING_TOOLS.md
- Main Programming Language: Bash (shell scripting)
- Alternative Programming Languages: Python, Go, Rust
- Coolness Level: Level 3: Genuinely Clever
- Business Potential: 2. The “Micro-SaaS / Pro Tool”
- Difficulty: Level 2: Intermediate
- Knowledge Area: DNS Protocol / Name Resolution
- Software or Tool: dig, nslookup, resolvectl
- Main Book: “DNS and BIND” by Cricket Liu
What you’ll build: A comprehensive DNS analysis tool that performs full resolution traces, queries all record types, compares answers across multiple resolvers, measures resolution times, and detects DNS misconfigurations or anomalies.
Why it teaches these tools: DNS is the foundation of everything on the internet. When it breaks, everything breaks. By building a tool that deeply interrogates DNS, you’ll understand the hierarchy, caching, record types, and common failure modes.
Core challenges you’ll face:
- Understanding DNS record types → maps to A, AAAA, MX, CNAME, NS, TXT, SOA, PTR
- Tracing resolution hierarchy → maps to recursive vs authoritative resolution
- Comparing resolver responses → maps to caching, TTL, and DNS consistency
- Understanding systemd-resolved → maps to modern Linux DNS configuration
Key Concepts:
- DNS Record Types: “DNS and BIND” Ch. 4 - Cricket Liu
- Resolution Process: “TCP/IP Illustrated Vol 1” Ch. 14 - W. Richard Stevens
- systemd-resolved: “How Linux Works” Ch. 9 - Brian Ward
Difficulty: Intermediate Time estimate: 1 week Prerequisites: Project 1 completed; basic understanding of client-server model; familiarity with what DNS does conceptually
Real World Outcome
$ ./dnsdeep.sh example.com
╔══════════════════════════════════════════════════════════════════════╗
║ DNS DEEP DIVE ANALYSIS ║
║ Target: example.com ║
╠══════════════════════════════════════════════════════════════════════╣
┌─────────────────────────────────────────────────────────────────────┐
│ SYSTEM DNS CONFIGURATION │
├─────────────────────────────────────────────────────────────────────┤
│ Resolver: systemd-resolved (127.0.0.53) │
│ Upstream DNS: 8.8.8.8, 8.8.4.4 │
│ DNS over TLS: enabled │
│ DNSSEC: yes (validated) │
└─────────────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────────┐
│ RESOLUTION TRACE (+trace) │
├─────────────────────────────────────────────────────────────────────┤
│ . (root) → a.root-servers.net (2ms) │
│ com. → a.gtld-servers.net (15ms) │
│ example.com. → a.iana-servers.net (23ms) │
│ Total resolution time: 40ms │
└─────────────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────────┐
│ RECORD TYPES │
├─────────────────────────────────────────────────────────────────────┤
│ A : 93.184.216.34 TTL: 86400 │
│ AAAA : 2606:2800:220:1:248:1893:25c8:1946 TTL: 86400 │
│ NS : a.iana-servers.net, b.iana-servers.net TTL: 86400 │
│ MX : (none) │
│ TXT : "v=spf1 -all" TTL: 86400 │
│ SOA : ns.icann.org hostmaster.icann.org TTL: 3600 │
│ Serial: 2024122001 │
└─────────────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────────┐
│ RESOLVER COMPARISON │
├─────────────────────────────────────────────────────────────────────┤
│ Resolver │ A Record │ Time │ DNSSEC │
├────────────────────┼──────────────────┼───────┼───────────────────┤
│ System (localhost) │ 93.184.216.34 │ 2ms │ ✓ validated │
│ Google (8.8.8.8) │ 93.184.216.34 │ 15ms │ ✓ validated │
│ Cloudflare (1.1.1.1)│ 93.184.216.34 │ 12ms │ ✓ validated │
│ Quad9 (9.9.9.9) │ 93.184.216.34 │ 18ms │ ✓ validated │
└─────────────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────────┐
│ REVERSE DNS (PTR) │
├─────────────────────────────────────────────────────────────────────┤
│ 93.184.216.34 → 93.184.216.34 (no PTR record) │
└─────────────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────────┐
│ HEALTH CHECK │
├─────────────────────────────────────────────────────────────────────┤
│ ✓ All resolvers return consistent A record │
│ ✓ DNSSEC validation passing │
│ ✓ Authoritative servers responding │
│ ⚠ No MX record (cannot receive email at this domain) │
│ ✓ SPF record present (email security configured) │
└─────────────────────────────────────────────────────────────────────┘
The Core Question You’re Answering
“How does a name become an IP address, and what can go wrong in that process?”
DNS seems simple—type a name, get an IP. But there’s a global distributed database with hierarchical delegation, caching at multiple levels, multiple record types, and security extensions. Understanding DNS deeply means you can troubleshoot issues others call “random network problems.”
Concepts You Must Understand First
Stop and research these before coding:
- DNS Hierarchy
- What are root servers and how many are there?
- What is a TLD (Top Level Domain) and who operates them?
- What’s the difference between a registrar and DNS hosting?
- Book Reference: “DNS and BIND” Ch. 1-2 - Cricket Liu
- Record Types
- What’s the difference between A, AAAA, CNAME, and ALIAS?
- What are MX records and how does email use them?
- What are TXT records used for (SPF, DKIM, DMARC)?
- Book Reference: “DNS and BIND” Ch. 4 - Cricket Liu
- Resolution Process
- What’s the difference between recursive and iterative queries?
- How does caching work and what is TTL?
- What is negative caching?
- Book Reference: “TCP/IP Illustrated Vol 1” Ch. 14 - W. Richard Stevens
Questions to Guide Your Design
Before implementing, think through these:
- Query Strategy
- How do you query specific DNS servers vs using system resolver?
- How do you get ALL record types for a domain?
- How do you trace the full resolution path?
- Output Interpretation
- What does a CNAME chain look like and how do you follow it?
- How do you detect if DNSSEC is enabled and validated?
- What TTL values are suspiciously short or long?
- Common Issues to Detect
- Missing MX records for domains that should have email
- Mismatched A records across resolvers
- Very short TTLs that might indicate DDoS protection
- Missing reverse DNS (PTR) records
Thinking Exercise
Parse This dig Output
Before coding, analyze this real dig output:
$ dig +noall +answer mx gmail.com
gmail.com. 3542 IN MX 5 gmail-smtp-in.l.google.com.
gmail.com. 3542 IN MX 10 alt1.gmail-smtp-in.l.google.com.
gmail.com. 3542 IN MX 20 alt2.gmail-smtp-in.l.google.com.
gmail.com. 3542 IN MX 30 alt3.gmail-smtp-in.l.google.com.
gmail.com. 3542 IN MX 40 alt4.gmail-smtp-in.l.google.com.
Questions to answer:
- What do the numbers 5, 10, 20, 30, 40 mean?
- What is 3542?
- If gmail-smtp-in.l.google.com is down, what happens?
- How would you find the IP of gmail-smtp-in.l.google.com?
The Interview Questions They’ll Ask
Prepare to answer these:
- “DNS isn’t resolving for users but works on the server. What do you check?”
- “What’s the difference between dig and nslookup?”
- “How would you find all the mail servers for a domain?”
- “A website works for some users but not others. Could DNS be the issue?”
- “What is DNS cache poisoning and how does DNSSEC prevent it?”
- “How would you change DNS servers on a Linux system?”
- “What is the purpose of the SOA record?”
Hints in Layers
Hint 1: Starting Point
dig +trace example.com shows the full resolution path from root to answer.
Hint 2: Multiple Record Types
dig any example.com used to get all records, but many servers block ANY queries. Instead, query each type individually: A, AAAA, MX, TXT, NS, SOA.
Hint 3: Specific Resolver
dig @8.8.8.8 example.com queries Google’s DNS specifically. Compare responses across resolvers.
Hint 4: systemd-resolved Status
resolvectl status shows the current DNS configuration, including per-link DNS servers and DNSSEC status.
Books That Will Help
| Topic | Book | Chapter |
|---|---|---|
| DNS fundamentals | “DNS and BIND” by Cricket Liu | Ch. 1-4 |
| dig command mastery | “DNS and BIND” by Cricket Liu | Ch. 12 |
| DNS protocol | “TCP/IP Illustrated, Vol 1” by W. Richard Stevens | Ch. 14 |
| systemd-resolved | “How Linux Works” by Brian Ward | Ch. 9 |
Implementation Hints
Your tool should perform these queries:
- System Configuration:
resolvectl status- shows current resolver setup - Resolution Trace:
dig +trace domain- full hierarchy - Individual Records:
dig domain A,dig domain AAAA,dig domain MX, etc. - Reverse Lookup:
dig -x IP- PTR records - Cross-Resolver Comparison: Query @8.8.8.8, @1.1.1.1, @9.9.9.9
- DNSSEC Check:
dig +dnssec domain- check AD flag in response
Parse dig output with these fields:
- Answer section: the actual records
- AUTHORITY section: NS servers for the zone
- ADDITIONAL section: glue records
- Flags: QR, AA (authoritative), RD, RA, AD (DNSSEC validated)
Learning milestones:
- You can query any record type → Basic dig usage
- You can trace resolution hierarchy → Understand DNS delegation
- You can interpret TTL and caching → Understand DNS performance
- You can detect DNSSEC status → Understand DNS security
Project 4: Socket State Analyzer
- File: LEARN_LINUX_NETWORKING_TOOLS.md
- Main Programming Language: Bash (shell scripting)
- Alternative Programming Languages: Python, Go, Rust
- Coolness Level: Level 3: Genuinely Clever
- Business Potential: 3. The “Service & Support” Model
- Difficulty: Level 2: Intermediate
- Knowledge Area: Socket Programming / Connection State
- Software or Tool: ss, netstat, lsof
- Main Book: “The Linux Programming Interface” by Michael Kerrisk
What you’ll build: A comprehensive socket analysis tool that displays all listening services, established connections, connection states, process bindings, and can detect anomalies like too many TIME_WAIT connections or orphaned sockets.
Why it teaches these tools: Every network service is ultimately a socket. Understanding socket states is essential for debugging connection issues, identifying resource leaks, and understanding how your applications communicate. ss is the modern tool; netstat is legacy but still widely used.
Core challenges you’ll face:
- Interpreting TCP states → maps to understanding the TCP state machine
- Correlating sockets to processes → maps to understanding PID/program binding
- Distinguishing listen vs established → maps to server vs client connections
- Identifying socket issues → maps to TIME_WAIT accumulation, port exhaustion
Key Concepts:
- TCP State Machine: “TCP/IP Illustrated Vol 1” Ch. 18-19 - W. Richard Stevens
- Socket API: “The Linux Programming Interface” Ch. 56-57 - Michael Kerrisk
- Linux /proc filesystem: “The Linux Programming Interface” Ch. 12 - Michael Kerrisk
Difficulty: Intermediate Time estimate: 1 week Prerequisites: Projects 1-2 completed; understanding of TCP/UDP differences; basic knowledge of client-server architecture
Real World Outcome
$ sudo ./sockstat.sh
╔══════════════════════════════════════════════════════════════════════╗
║ SOCKET STATE ANALYSIS ║
║ Host: webserver01 ║
╠══════════════════════════════════════════════════════════════════════╣
┌─────────────────────────────────────────────────────────────────────┐
│ LISTENING SERVICES │
├─────────────────────────────────────────────────────────────────────┤
│ Proto Local Address Port PID/Program State │
├─────────────────────────────────────────────────────────────────────┤
│ tcp 0.0.0.0 22 1234/sshd LISTEN │
│ tcp 0.0.0.0 80 5678/nginx LISTEN │
│ tcp 127.0.0.1 5432 9012/postgres LISTEN │
│ tcp ::: 443 5678/nginx LISTEN │
│ udp 0.0.0.0 53 3456/dnsmasq - │
│ unix /var/run/docker.sock - 7890/dockerd LISTEN │
└─────────────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────────┐
│ ESTABLISHED CONNECTIONS │
├─────────────────────────────────────────────────────────────────────┤
│ Proto Local Remote PID/Program │
├─────────────────────────────────────────────────────────────────────┤
│ tcp 192.168.1.10:443 ← 203.0.113.45:52341 nginx │
│ tcp 192.168.1.10:443 ← 203.0.113.45:52342 nginx │
│ tcp 192.168.1.10:443 ← 198.51.100.23:48291 nginx │
│ tcp 192.168.1.10:5432 ← 127.0.0.1:43210 postgres │
│ tcp 192.168.1.10:22 ← 10.0.0.5:55432 sshd │
└─────────────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────────┐
│ CONNECTION STATE SUMMARY │
├─────────────────────────────────────────────────────────────────────┤
│ State Count Percentage │
├─────────────────────────────────────────────────────────────────────┤
│ ESTABLISHED 156 ████████████████░░░░░░░░ 52% │
│ TIME_WAIT 89 ████████████░░░░░░░░░░░░ 30% │
│ CLOSE_WAIT 12 ████░░░░░░░░░░░░░░░░░░░░ 4% │
│ LISTEN 15 █████░░░░░░░░░░░░░░░░░░░ 5% │
│ SYN_RECV 8 ███░░░░░░░░░░░░░░░░░░░░░ 3% │
│ Other 18 ██████░░░░░░░░░░░░░░░░░░ 6% │
│ TOTAL 298 │
└─────────────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────────┐
│ TOP CONNECTIONS BY REMOTE IP │
├─────────────────────────────────────────────────────────────────────┤
│ Remote IP Connections State Distribution │
├─────────────────────────────────────────────────────────────────────┤
│ 203.0.113.45 42 ESTABLISHED: 38, TIME_WAIT: 4 │
│ 198.51.100.23 28 ESTABLISHED: 25, CLOSE_WAIT: 3 │
│ 10.0.0.5 15 ESTABLISHED: 15 │
└─────────────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────────┐
│ POTENTIAL ISSUES DETECTED │
├─────────────────────────────────────────────────────────────────────┤
│ ⚠ High TIME_WAIT count (89) - consider adjusting tcp_tw_reuse │
│ ⚠ CLOSE_WAIT sockets (12) - application may not be closing sockets │
│ ✓ No SYN flood detected │
│ ✓ Port exhaustion unlikely (298/65535 ephemeral ports used) │
└─────────────────────────────────────────────────────────────────────┘
The Core Question You’re Answering
“What network connections does my system have, what state are they in, and which processes own them?”
Every network issue eventually comes down to sockets. Is the server listening? Is the connection established? Why are there so many TIME_WAIT connections? Which process is holding port 8080? This project teaches you to see the network from the application’s perspective.
Concepts You Must Understand First
Stop and research these before coding:
- TCP State Machine
- What are all the TCP states (LISTEN, SYN_SENT, SYN_RECV, ESTABLISHED, FIN_WAIT_1, FIN_WAIT_2, CLOSE_WAIT, TIME_WAIT, LAST_ACK, CLOSED)?
- Why does TIME_WAIT exist and how long does it last?
- What does CLOSE_WAIT indicate about an application?
- Book Reference: “TCP/IP Illustrated Vol 1” Ch. 18 - W. Richard Stevens
- Socket Types
- What’s the difference between TCP, UDP, and UNIX domain sockets?
- What does “listening on 0.0.0.0” vs “127.0.0.1” mean?
- What are ephemeral ports and why do they matter?
- Book Reference: “The Linux Programming Interface” Ch. 56-57 - Michael Kerrisk
- Process/Socket Binding
- How does the kernel know which process owns a socket?
- What’s in /proc/net/tcp and how is it structured?
- Why do some sockets show no process (orphaned)?
- Book Reference: “The Linux Programming Interface” Ch. 12 - Michael Kerrisk
Questions to Guide Your Design
Before implementing, think through these:
- Data Collection
- When do you need root access to see all information?
- How do you get both IPv4 and IPv6 sockets?
- How do you include UNIX domain sockets in the analysis?
- Interpretation
- What count of TIME_WAIT is “too many”?
- How do you identify potential SYN flood attacks?
- What does a high CLOSE_WAIT count indicate?
- Correlation
- How do you match a socket to its process name (not just PID)?
- How do you aggregate connections by remote IP?
- How do you show the relationship between listening and connected sockets?
Thinking Exercise
Interpret This ss Output
Before coding, analyze this real ss output:
$ ss -tanp
State Recv-Q Send-Q Local Address:Port Peer Address:Port
LISTEN 0 128 127.0.0.1:5432 0.0.0.0:* users:(("postgres",pid=1234,fd=5))
LISTEN 0 128 0.0.0.0:22 0.0.0.0:* users:(("sshd",pid=567,fd=3))
ESTAB 0 0 192.168.1.10:22 10.0.0.5:54321 users:(("sshd",pid=890,fd=3))
TIME-WAIT 0 0 192.168.1.10:80 203.0.113.1:45678
CLOSE-WAIT 0 0 192.168.1.10:8080 198.51.100.2:33456 users:(("app",pid=2345,fd=12))
Questions to answer:
- Why is postgres only listening on 127.0.0.1 but sshd on 0.0.0.0?
- What does the ESTAB line tell you about who’s SSH’d in?
- Why does TIME-WAIT have no process listed?
- What should you investigate about the CLOSE-WAIT connection?
- What is Recv-Q/Send-Q and when would non-zero values be concerning?
The Interview Questions They’ll Ask
Prepare to answer these:
- “What’s the difference between ss and netstat?”
- “How would you find what process is listening on port 8080?”
- “We’re seeing a lot of TIME_WAIT connections. Is this a problem?”
- “What does CLOSE_WAIT mean and how do you fix it?”
- “How would you check if a remote host has a port open?”
- “We’re running out of ephemeral ports. How do you diagnose this?”
- “How do you find all connections to a specific remote IP?”
Hints in Layers
Hint 1: Starting Point
ss -tunapln gives you TCP and UDP, with numeric addresses, all sockets, process info, and listening sockets.
Hint 2: State Filtering
ss state established or ss state time-wait filters by TCP state.
Hint 3: Process Names
ss -p requires root to show all process info. Without root, you only see your own processes.
Hint 4: lsof for Detail
lsof -i :8080 shows all processes using port 8080. lsof -i -P -n shows all network connections.
Books That Will Help
| Topic | Book | Chapter |
|---|---|---|
| TCP states | “TCP/IP Illustrated, Vol 1” by W. Richard Stevens | Ch. 18-19 |
| Socket programming | “The Linux Programming Interface” by Michael Kerrisk | Ch. 56-61 |
| ss command | “Linux for Networking Professionals” (O’Reilly) | Ch. 5 |
| /proc filesystem | “The Linux Programming Interface” by Michael Kerrisk | Ch. 12 |
Implementation Hints
Key commands to use:
- ss -tunapln: All TCP/UDP sockets with process info
- ss -s: Summary statistics (quick overview)
- ss state established: Filter by state
- ss dst IP: Filter by destination
- netstat -anp: Legacy equivalent (slower but compatible)
- lsof -i: Alternative view, great for specific ports
Parse ss output:
- State column: TCP state (LISTEN, ESTAB, TIME-WAIT, etc.)
- Recv-Q/Send-Q: Bytes in receive/send buffers (high values = slow consumer/producer)
- Local/Peer Address: Endpoints with ports
- Process info: PID and program name
Anomaly detection:
- TIME_WAIT > 10000: May need tcp_tw_reuse
- CLOSE_WAIT accumulating: Application not closing sockets
- SYN_RECV > 128: Possible SYN flood
- Same remote IP with hundreds of connections: Possible abuse or connection leak
Learning milestones:
- You can list all listening services → Basic ss/netstat usage
- You can identify connection states → TCP state machine understanding
- You can correlate sockets to processes → System-level understanding
- You can detect socket-related issues → Troubleshooting skills
Project 5: Live Packet Capture Dashboard
- File: LEARN_LINUX_NETWORKING_TOOLS.md
- Main Programming Language: Bash + Python
- Alternative Programming Languages: Go, Rust, C
- Coolness Level: Level 4: Hardcore Tech Flex
- Business Potential: 3. The “Service & Support” Model
- Difficulty: Level 3: Advanced
- Knowledge Area: Packet Analysis / Protocol Decoding
- Software or Tool: tcpdump
- Main Book: “Practical Packet Analysis” by Chris Sanders
What you’ll build: A real-time network traffic analyzer that captures packets, decodes protocols, displays statistics (packets/second, bytes/second, protocol distribution), and can filter/highlight specific traffic patterns—all in a terminal UI.
Why it teaches tcpdump: tcpdump is the most powerful packet capture tool on Linux. By building a wrapper that parses its output in real-time, you’ll understand packet structure at every layer, BPF filter syntax, and how to extract meaning from raw network traffic.
Core challenges you’ll face:
- Understanding BPF filter syntax → maps to efficient packet selection
- Parsing packet headers at each layer → maps to protocol understanding
- Real-time stream processing → maps to handling high-volume captures
- Interpreting packet contents → maps to application-layer protocol knowledge
Key Concepts:
- Ethernet/IP/TCP Headers: “TCP/IP Illustrated Vol 1” Ch. 2-4, 17-20 - W. Richard Stevens
- BPF Filters: “The Practice of Network Security Monitoring” Ch. 5 - Richard Bejtlich
- Packet Capture: “Practical Packet Analysis” Ch. 3-4 - Chris Sanders
Difficulty: Advanced Time estimate: 2 weeks Prerequisites: Projects 1-4 completed; solid understanding of TCP/IP layers; comfort with parsing structured data
Real World Outcome
$ sudo ./packetdash.sh -i eth0
╔══════════════════════════════════════════════════════════════════════╗
║ LIVE PACKET CAPTURE - eth0 [Press q to quit] ║
║ Running for: 00:05:32 Filter: (none) ║
╠══════════════════════════════════════════════════════════════════════╣
┌─ TRAFFIC RATE ──────────────────────────────────────────────────────┐
│ Packets/sec: 1,247 Bytes/sec: 892.3 KB │
│ ████████████████████░░░░░░░░░░░░░░░░░░░░ 52% of peak │
└─────────────────────────────────────────────────────────────────────┘
┌─ PROTOCOL DISTRIBUTION ─────────────────────────────────────────────┐
│ TCP: 78% ████████████████████████████████░░░░░░░░ │
│ UDP: 18% ████████░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░ │
│ ICMP: 2% █░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░ │
│ Other: 2% █░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░ │
└─────────────────────────────────────────────────────────────────────┘
┌─ TOP TALKERS ───────────────────────────────────────────────────────┐
│ Source IP Packets Bytes Top Port │
│ ─────────────────────────────────────────────────── │
│ 192.168.1.100 12,456 8.2 MB 443 (HTTPS) │
│ 10.0.0.5 8,234 2.1 MB 22 (SSH) │
│ 203.0.113.45 5,678 1.8 MB 80 (HTTP) │
│ │
│ Dest IP Packets Bytes Top Port │
│ ─────────────────────────────────────────────────── │
│ 142.250.80.46 10,234 6.5 MB 443 (HTTPS) │
│ 93.184.216.34 4,567 892 KB 80 (HTTP) │
└─────────────────────────────────────────────────────────────────────┘
┌─ LIVE PACKET STREAM (latest 10) ────────────────────────────────────┐
│ 14:32:01.234 TCP 192.168.1.100:52341 → 142.250.80.46:443 [SYN] │
│ 14:32:01.256 TCP 142.250.80.46:443 → 192.168.1.100:52341 [SYN,ACK] │
│ 14:32:01.257 TCP 192.168.1.100:52341 → 142.250.80.46:443 [ACK] │
│ 14:32:01.258 TCP 192.168.1.100:52341 → 142.250.80.46:443 [PSH,ACK] │
│ 14:32:01.312 TCP 142.250.80.46:443 → 192.168.1.100:52341 [ACK] │
│ 14:32:01.415 UDP 192.168.1.100:52342 → 8.8.8.8:53 (DNS Query) │
│ 14:32:01.438 UDP 8.8.8.8:53 → 192.168.1.100:52342 (DNS Response) │
│ 14:32:01.523 ICMP 192.168.1.1 → 192.168.1.100 (Echo Reply) │
│ 14:32:01.678 TCP 10.0.0.5:55432 → 192.168.1.100:22 [PSH,ACK] │
│ 14:32:01.679 TCP 192.168.1.100:22 → 10.0.0.5:55432 [ACK] │
└─────────────────────────────────────────────────────────────────────┘
┌─ FLAGS/ALERTS ──────────────────────────────────────────────────────┐
│ ⚠ Port scan detected: 203.0.113.99 probing 20 ports in 5 seconds │
│ ✓ Normal traffic patterns │
└─────────────────────────────────────────────────────────────────────┘
The Core Question You’re Answering
“What data is actually flowing on my network, and can I see every packet?”
This is the ultimate visibility into your network. While other tools show you state and statistics, tcpdump shows you the actual bits on the wire. You’ll see TCP handshakes happen, DNS queries and responses, HTTP requests, and everything else that crosses your network interface.
Concepts You Must Understand First
Stop and research these before coding:
- Packet Structure
- What are the Ethernet, IP, TCP, and UDP header formats?
- What are TCP flags (SYN, ACK, FIN, RST, PSH)?
- How are packets encapsulated (Ethernet wraps IP wraps TCP wraps payload)?
- Book Reference: “TCP/IP Illustrated Vol 1” Ch. 2-4, 17-20 - W. Richard Stevens
- BPF Filter Language
- What does
host 192.168.1.1filter? - What does
port 80 and tcpmean? - How do you filter by TCP flags (
tcp[tcpflags])? - Book Reference: “Practical Packet Analysis” Ch. 4 - Chris Sanders
- What does
- Capture Mechanics
- What’s promiscuous mode and when do you need it?
- What’s the difference between libpcap and raw sockets?
- Why do you need root for packet capture?
- Book Reference: “The Practice of Network Security Monitoring” Ch. 5 - Richard Bejtlich
Questions to Guide Your Design
Before implementing, think through these:
- Capture Strategy
- How do you handle high packet rates without dropping packets?
- Should you capture to a buffer and process separately?
- How do you limit memory usage for statistics?
- Parsing Approach
- tcpdump’s
-l(line-buffered) vs processing pcap directly? - How do you parse the text output efficiently?
- When do you use
-X(hex dump) vs-A(ASCII)?
- tcpdump’s
- Real-time Display
- How do you update terminal display efficiently?
- How do you handle scrolling packet stream?
- How do you calculate rolling statistics?
Thinking Exercise
Parse This tcpdump Output
Before coding, analyze these real tcpdump lines:
14:32:01.234567 IP 192.168.1.100.52341 > 142.250.80.46.443: Flags [S], seq 1234567890, win 65535, options [mss 1460,sackOK,TS val 123456 ecr 0,nop,wscale 7], length 0
14:32:01.256789 IP 142.250.80.46.443 > 192.168.1.100.52341: Flags [S.], seq 987654321, ack 1234567891, win 65535, options [mss 1430,sackOK,TS val 789012 ecr 123456,nop,wscale 8], length 0
14:32:01.257012 IP 192.168.1.100.52341 > 142.250.80.46.443: Flags [.], ack 1, win 512, options [nop,nop,TS val 123457 ecr 789012], length 0
Questions to answer:
- What protocol and ports are involved?
- What do [S], [S.], and [.] flags mean?
- This is a TCP handshake—identify the three steps
- What is “mss 1460” telling you?
- What is the sequence/ack number relationship?
The Interview Questions They’ll Ask
Prepare to answer these:
- “How would you capture all HTTP traffic on a server?”
- “What’s the BPF filter to see only SYN packets?”
- “How would you capture traffic between two specific hosts?”
- “We suspect DNS exfiltration. How would you investigate?”
- “How do you capture packets without affecting performance?”
- “What’s the difference between tcpdump and Wireshark?”
- “How would you find all connections to a specific port?”
Hints in Layers
Hint 1: Starting Point
tcpdump -i eth0 -n -l gives you line-buffered output with numeric addresses—easy to parse.
Hint 2: Common Filters
- All traffic to/from IP:
host 192.168.1.1 - Specific port:
port 443 - TCP SYN packets only:
'tcp[tcpflags] & (tcp-syn) != 0' - DNS queries:
port 53
Hint 3: Output Parsing
tcpdump -tt -n -l gives Unix timestamps, numeric IPs, line-buffered. Parse with awk/sed or pipe to Python.
Hint 4: Statistics in Memory Use dictionaries keyed by IP address. Clear old entries periodically to prevent memory growth.
Books That Will Help
| Topic | Book | Chapter |
|---|---|---|
| Packet structure | “TCP/IP Illustrated, Vol 1” by W. Richard Stevens | Ch. 2-4, 17-20 |
| tcpdump usage | “Practical Packet Analysis” by Chris Sanders | Ch. 4-6 |
| BPF filters | “The Practice of Network Security Monitoring” by Richard Bejtlich | Ch. 5 |
| Traffic analysis | “Network Flow Analysis” by Michael Lucas | Full book |
Implementation Hints
Key tcpdump options:
tcpdump -i eth0 # Interface
-n # Numeric (don't resolve names)
-nn # Numeric ports too
-l # Line-buffered (for piping)
-tt # Unix timestamp
-q # Quiet (less protocol detail)
-c 1000 # Capture only 1000 packets
-w file.pcap # Write to file
-r file.pcap # Read from file
Parsing strategy (pseudo-code):
while read line from tcpdump -l:
extract timestamp, src_ip, src_port, dst_ip, dst_port, flags
update packet_count[src_ip]
update byte_count[src_ip] (if length available)
update protocol_count[proto]
add to rolling_window for rate calculation
detect patterns (port scan = many dst_ports from one src)
update display
BPF filter examples:
- HTTP:
tcp port 80 - HTTPS:
tcp port 443 - DNS:
udp port 53 - SSH:
tcp port 22 - SYN only:
'tcp[tcpflags] == tcp-syn' - Not broadcast:
not broadcast
Learning milestones:
- You can capture specific traffic → BPF filter mastery
- You can read packet headers → Protocol understanding
- You can identify TCP handshakes → Connection lifecycle
- You can detect anomalies → Security awareness
Project 6: Firewall Rule Auditor
- File: LEARN_LINUX_NETWORKING_TOOLS.md
- Main Programming Language: Bash (shell scripting)
- Alternative Programming Languages: Python, Go
- Coolness Level: Level 3: Genuinely Clever
- Business Potential: 3. The “Service & Support” Model
- Difficulty: Level 3: Advanced
- Knowledge Area: Network Security / Packet Filtering
- Software or Tool: iptables, nft, ufw, firewall-cmd
- Main Book: “Linux Firewalls” by Michael Rash
What you’ll build: A firewall analysis and auditing tool that displays all active rules across different firewall systems (iptables, nftables, ufw, firewalld), visualizes packet flow through chains, detects common misconfigurations, and can simulate how a packet would be processed.
Why it teaches these tools: Linux has multiple firewall frontends, but they all ultimately configure Netfilter. Understanding iptables/nftables chains, tables, and rule ordering is essential for security. By building an auditor, you’ll learn to read, interpret, and debug firewall configurations.
Core challenges you’ll face:
- Understanding Netfilter chains and tables → maps to packet flow architecture
- Parsing rule syntax across different tools → maps to iptables vs nft vs frontends
- Detecting rule ordering issues → maps to understanding first-match semantics
- Simulating packet matching → maps to deep rule understanding
Key Concepts:
- Netfilter Architecture: “Linux Firewalls” Ch. 1-3 - Michael Rash
- iptables Syntax: “Linux iptables Pocket Reference” - Gregor Purdy
- nftables Transition: “Linux for Networking Professionals” Ch. 9 - O’Reilly
Difficulty: Advanced Time estimate: 2 weeks Prerequisites: Projects 1-5 completed; understanding of TCP/IP; familiarity with basic firewall concepts (allow/deny)
Real World Outcome
$ sudo ./fwaudit.sh
╔══════════════════════════════════════════════════════════════════════╗
║ FIREWALL AUDIT REPORT ║
║ Host: webserver01 ║
║ Firewall Backend: nftables (via iptables-nft) ║
╠══════════════════════════════════════════════════════════════════════╣
┌─────────────────────────────────────────────────────────────────────┐
│ DETECTED FIREWALL SYSTEMS │
├─────────────────────────────────────────────────────────────────────┤
│ ✓ nftables: active (kernel backend) │
│ ✓ iptables-nft: active (iptables frontend to nftables) │
│ ✗ firewalld: not running │
│ ✗ ufw: disabled │
└─────────────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────────┐
│ PACKET FLOW VISUALIZATION │
├─────────────────────────────────────────────────────────────────────┤
│ │
│ INCOMING → [PREROUTING] → [routing] → [INPUT] → Local Process │
│ ↓ │
│ [FORWARD] → [POSTROUTING] → OUTGOING │
│ ↑ │
│ Local Process → [OUTPUT] → [routing] → [POSTROUTING] → OUTGOING │
│ │
└─────────────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────────┐
│ FILTER TABLE - INPUT CHAIN (12 rules) │
├─────────────────────────────────────────────────────────────────────┤
│ # │ Target │ Proto │ Source │ Dest │ Port │ Pkts │
├────┼──────────┼───────┼─────────────────┼─────────┼───────┼────────┤
│ 1 │ ACCEPT │ all │ 0.0.0.0/0 │ lo │ - │ 45.2K │
│ 2 │ ACCEPT │ all │ state RELATED,ESTABLISHED │ 1.2M │
│ 3 │ ACCEPT │ tcp │ 0.0.0.0/0 │ any │ 22 │ 892 │
│ 4 │ ACCEPT │ tcp │ 0.0.0.0/0 │ any │ 80 │ 234K │
│ 5 │ ACCEPT │ tcp │ 0.0.0.0/0 │ any │ 443 │ 1.8M │
│ 6 │ ACCEPT │ tcp │ 10.0.0.0/8 │ any │ 5432 │ 12.3K │
│ 7 │ DROP │ tcp │ 0.0.0.0/0 │ any │ 5432 │ 342 │
│ 8 │ ACCEPT │ icmp │ 0.0.0.0/0 │ any │ - │ 5.6K │
│ 9 │ LOG │ all │ 0.0.0.0/0 │ any │ - │ 1.2K │
│ 10 │ DROP │ all │ 0.0.0.0/0 │ any │ - │ 1.2K │
└─────────────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────────┐
│ NAT TABLE - POSTROUTING CHAIN (2 rules) │
├─────────────────────────────────────────────────────────────────────┤
│ 1 │ MASQUERADE│ all │ 172.17.0.0/16 │ !docker0│ │ 45K │
│ 2 │ MASQUERADE│ all │ 192.168.100.0/24│ eth0 │ │ 12K │
└─────────────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────────┐
│ SECURITY AUDIT │
├─────────────────────────────────────────────────────────────────────┤
│ ✓ Default INPUT policy: DROP (good) │
│ ✓ Default FORWARD policy: DROP (good) │
│ ✓ SSH (22) is rate-limited │
│ ⚠ SSH (22) open to 0.0.0.0/0 - consider IP restriction │
│ ✓ PostgreSQL (5432) restricted to 10.0.0.0/8 │
│ ✓ Logging enabled before final DROP │
│ ⚠ No IPv6 rules detected - IPv6 may be unfiltered! │
│ ✗ Rule 4 (HTTP 80) could be merged with rule 5 (HTTPS 443) │
└─────────────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────────┐
│ PACKET SIMULATION: TCP 203.0.113.50:54321 → 192.168.1.10:5432 │
├─────────────────────────────────────────────────────────────────────┤
│ Chain: INPUT │
│ Rule 1 (lo accept): NO MATCH (interface mismatch) │
│ Rule 2 (RELATED,ESTABLISHED): NO MATCH (new connection) │
│ Rule 3 (SSH 22): NO MATCH (port mismatch) │
│ Rule 4 (HTTP 80): NO MATCH (port mismatch) │
│ Rule 5 (HTTPS 443): NO MATCH (port mismatch) │
│ Rule 6 (Postgres from 10.0.0.0/8): NO MATCH (source mismatch) │
│ Rule 7 (Postgres DROP): ✗ MATCH → DROP │
│ │
│ RESULT: Packet would be DROPPED at rule 7 │
└─────────────────────────────────────────────────────────────────────┘
The Core Question You’re Answering
“What traffic can enter/exit my system, and is my firewall configured securely?”
Firewalls are the first line of defense. Misconfigurations are common and often invisible until exploited. By building a tool that audits rules and simulates packet processing, you’ll deeply understand how Linux firewalls work and catch issues before attackers do.
Concepts You Must Understand First
Stop and research these before coding:
- Netfilter Architecture
- What are the five Netfilter hooks (PREROUTING, INPUT, FORWARD, OUTPUT, POSTROUTING)?
- What is the difference between the filter, nat, mangle, and raw tables?
- In what order are tables and chains processed?
- Book Reference: “Linux Firewalls” Ch. 1-2 - Michael Rash
- iptables vs nftables
- What is iptables-nft vs iptables-legacy?
- How does nftables syntax differ from iptables?
- Why is nftables the future and what advantages does it offer?
- Book Reference: “Linux for Networking Professionals” Ch. 9 - O’Reilly
- Firewall Frontends
- What is ufw and how does it generate iptables rules?
- What is firewalld and what are zones?
- How do frontends and backends coexist (or conflict)?
- Book Reference: “How Linux Works” Ch. 9 - Brian Ward
Questions to Guide Your Design
Before implementing, think through these:
- Detection Strategy
- How do you detect which firewall system is active?
- How do you handle systems with multiple firewall tools?
- How do you read rules from each system?
- Rule Analysis
- How do you detect rules that will never match (shadowed rules)?
- How do you identify overly permissive rules?
- How do you check for IPv4 vs IPv6 coverage?
- Simulation Logic
- How do you match a packet against rules in order?
- How do you handle connection tracking (ESTABLISHED, RELATED)?
- How do you visualize the decision path?
Thinking Exercise
Trace This Packet
Given these iptables rules, trace what happens to a packet:
iptables -P INPUT DROP
iptables -A INPUT -i lo -j ACCEPT
iptables -A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
iptables -A INPUT -p tcp --dport 22 -j ACCEPT
iptables -A INPUT -p tcp --dport 80 -j DROP
iptables -A INPUT -p tcp --dport 80 -s 10.0.0.0/8 -j ACCEPT
Scenario: A new TCP SYN packet arrives for port 80 from IP 10.0.0.5
Questions to answer:
- Will the packet be accepted or dropped?
- Which rule is the problem here?
- What’s wrong with the rule order?
- How would you fix it?
The Interview Questions They’ll Ask
Prepare to answer these:
- “What’s the difference between iptables and nftables?”
- “Explain the difference between the INPUT and FORWARD chains.”
- “How would you allow SSH only from a specific IP range?”
- “What is connection tracking and why is it important?”
- “How would you debug why a connection is being blocked?”
- “What’s the difference between DROP and REJECT?”
- “How do you make iptables rules persistent across reboots?”
Hints in Layers
Hint 1: Starting Point
iptables -L -n -v --line-numbers shows all filter rules with packet counts.
Hint 2: All Tables
iptables-save dumps all rules in all tables. nft list ruleset does the same for nftables.
Hint 3: Frontend Detection
Check for ufw with ufw status. Check for firewalld with firewall-cmd --state.
Hint 4: Packet Simulation Parse rules into data structures, then iterate checking each field against the simulated packet.
Books That Will Help
| Topic | Book | Chapter |
|---|---|---|
| Netfilter internals | “Linux Firewalls” by Michael Rash | Ch. 1-6 |
| iptables syntax | “Linux iptables Pocket Reference” by Gregor Purdy | Full book |
| nftables | “Linux for Networking Professionals” (O’Reilly) | Ch. 9 |
| Security hardening | “Linux Firewalls” by Michael Rash | Ch. 7-10 |
Implementation Hints
Commands to extract rules:
# iptables
iptables -L -n -v --line-numbers # filter table
iptables -t nat -L -n -v # nat table
iptables-save # all rules, parseable
# nftables
nft list ruleset # all rules
nft list tables # just table names
# ufw
ufw status verbose # ufw rules
ufw show raw # underlying iptables
# firewalld
firewall-cmd --list-all # current zone
firewall-cmd --list-all-zones # all zones
Rule parsing strategy:
- Chain policies (default action)
- Per-rule: target, protocol, source, destination, port, interface, state
- Packet counters (useful for identifying dead rules)
Audit checks:
- Default policies should be DROP for INPUT and FORWARD
- SSH should have rate limiting (
-m limit) - ESTABLISHED,RELATED rule should be near top (performance)
- No rules after unconditional DROP/REJECT (dead rules)
- IPv6 should mirror IPv4 protection
Learning milestones:
- You can list rules from any firewall tool → Basic command usage
- You understand chain and table flow → Netfilter architecture
- You can identify rule ordering issues → Security understanding
- You can simulate packet matching → Deep rule comprehension
Project 7: Routing Table Explorer
- File: LEARN_LINUX_NETWORKING_TOOLS.md
- Main Programming Language: Bash (shell scripting)
- Alternative Programming Languages: Python, Go, Rust
- Coolness Level: Level 3: Genuinely Clever
- Business Potential: 2. The “Micro-SaaS / Pro Tool”
- Difficulty: Level 2: Intermediate
- Knowledge Area: IP Routing / Layer 3 Networking
- Software or Tool: ip route, ip neigh, arp, route
- Main Book: “TCP/IP Illustrated, Volume 1” by W. Richard Stevens
What you’ll build: A routing visualization and debugging tool that displays the complete routing table, shows which route a destination will use, visualizes the ARP/neighbor cache, and can trace the decision-making process for any destination IP.
Why it teaches these tools: Routing is fundamental—every packet leaving your system needs to know where to go. By building a routing explorer, you’ll understand default gateways, route selection, metrics, and how ARP bridges L3 to L2.
Core challenges you’ll face:
- Understanding route selection (longest prefix match) → maps to how routing decisions work
- Distinguishing direct vs gateway routes → maps to local vs remote destinations
- Understanding the ARP cache → maps to IP-to-MAC resolution
- Handling policy routing → maps to advanced routing tables
Key Concepts:
- IP Routing Fundamentals: “TCP/IP Illustrated Vol 1” Ch. 8-9 - W. Richard Stevens
- ARP Protocol: “TCP/IP Illustrated Vol 1” Ch. 4 - W. Richard Stevens
- Linux Routing: “Linux for Networking Professionals” Ch. 4 - O’Reilly
Difficulty: Intermediate Time estimate: 1 week Prerequisites: Projects 1-3 completed; understanding of IP addresses and subnets; basic CIDR notation knowledge
Real World Outcome
$ ./routeexplore.sh
╔══════════════════════════════════════════════════════════════════════╗
║ ROUTING TABLE EXPLORER ║
║ Host: server01 ║
╠══════════════════════════════════════════════════════════════════════╣
┌─────────────────────────────────────────────────────────────────────┐
│ MAIN ROUTING TABLE │
├─────────────────────────────────────────────────────────────────────┤
│ Destination │ Gateway │ Interface │ Metric │ Type │
├──────────────────┼─────────────────┼───────────┼────────┼──────────┤
│ default │ 192.168.1.1 │ eth0 │ 100 │ gateway │
│ 10.0.0.0/8 │ 10.255.0.1 │ tun0 │ 50 │ gateway │
│ 172.17.0.0/16 │ - │ docker0 │ 0 │ direct │
│ 192.168.1.0/24 │ - │ eth0 │ 100 │ direct │
│ 192.168.100.0/24 │ - │ eth1 │ 100 │ direct │
└─────────────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────────┐
│ ROUTE LOOKUP: Where does traffic go? │
├─────────────────────────────────────────────────────────────────────┤
│ Destination: 8.8.8.8 │
│ Route: default via 192.168.1.1 dev eth0 │
│ Next hop: 192.168.1.1 (gateway) │
│ Source IP: 192.168.1.100 │
│ │
│ Destination: 10.50.25.100 │
│ Route: 10.0.0.0/8 via 10.255.0.1 dev tun0 │
│ Next hop: 10.255.0.1 (VPN gateway) │
│ Source IP: 10.255.0.50 │
│ │
│ Destination: 192.168.1.50 │
│ Route: 192.168.1.0/24 dev eth0 (direct) │
│ Next hop: 192.168.1.50 (direct, no gateway needed) │
│ Source IP: 192.168.1.100 │
└─────────────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────────┐
│ ARP/NEIGHBOR CACHE │
├─────────────────────────────────────────────────────────────────────┤
│ IP Address │ MAC Address │ Interface │ State │ Age │
├──────────────────┼───────────────────┼───────────┼──────────┼──────┤
│ 192.168.1.1 │ 00:1a:2b:3c:4d:5e │ eth0 │ REACHABLE│ 45s │
│ 192.168.1.50 │ 52:54:00:ab:cd:ef │ eth0 │ STALE │ 5m │
│ 192.168.1.75 │ (incomplete) │ eth0 │ FAILED │ - │
│ 172.17.0.2 │ 02:42:ac:11:00:02 │ docker0 │ REACHABLE│ 10s │
└─────────────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────────┐
│ ROUTING DECISION TRACE: 1.1.1.1 │
├─────────────────────────────────────────────────────────────────────┤
│ Step 1: Check routing table for matching prefixes │
│ ├─ 10.0.0.0/8: NO MATCH (1.1.1.1 not in 10.0.0.0/8) │
│ ├─ 172.17.0.0/16: NO MATCH │
│ ├─ 192.168.1.0/24: NO MATCH │
│ ├─ 192.168.100.0/24: NO MATCH │
│ └─ default (0.0.0.0/0): MATCH ✓ │
│ │
│ Step 2: Selected route: default via 192.168.1.1 dev eth0 │
│ │
│ Step 3: Next hop is 192.168.1.1 (gateway) │
│ Need ARP lookup for 192.168.1.1 on eth0 │
│ │
│ Step 4: ARP cache lookup │
│ 192.168.1.1 → 00:1a:2b:3c:4d:5e (cached, REACHABLE) │
│ │
│ Step 5: Packet ready to send │
│ Src MAC: 52:54:00:12:34:56 (eth0) │
│ Dst MAC: 00:1a:2b:3c:4d:5e (gateway) │
│ Src IP: 192.168.1.100 │
│ Dst IP: 1.1.1.1 │
└─────────────────────────────────────────────────────────────────────┘
The Core Question You’re Answering
“When I send a packet to address X, how does the kernel decide where to send it?”
Every packet needs a path. The routing table is the map. Understanding longest-prefix match, default routes, and the relationship between routing and ARP is essential for debugging connectivity issues.
Concepts You Must Understand First
Stop and research these before coding:
- Longest Prefix Match
- Why does a /24 route take precedence over a /16 route for the same destination?
- What is the default route and when is it used?
- What happens when multiple routes have the same prefix length?
- Book Reference: “TCP/IP Illustrated Vol 1” Ch. 8 - W. Richard Stevens
- Direct vs Gateway Routes
- When does a packet go directly to the destination?
- When is a gateway needed?
- What determines the source IP for outgoing packets?
- Book Reference: “TCP/IP Illustrated Vol 1” Ch. 8-9 - W. Richard Stevens
- ARP and Neighbor Discovery
- How does ARP resolve IP addresses to MAC addresses?
- What are the ARP cache states (REACHABLE, STALE, FAILED)?
- What is neighbor discovery in IPv6?
- Book Reference: “TCP/IP Illustrated Vol 1” Ch. 4 - W. Richard Stevens
Questions to Guide Your Design
Before implementing, think through these:
- Route Display
- How do you show all routes, not just main table?
- How do you indicate whether a route is direct or via gateway?
- How do you show the source address selection?
- Route Lookup
- How do you find which route matches a specific destination?
- How do you handle policy routing (multiple tables)?
- How do you show the complete decision process?
- ARP Integration
- How do you correlate ARP entries with routes?
- How do you detect ARP issues (incomplete entries)?
- How do you show the relationship between L3 and L2?
Thinking Exercise
Predict the Route
Given this routing table:
default via 192.168.1.1 dev eth0 metric 100
10.0.0.0/8 via 10.255.0.1 dev tun0 metric 50
10.50.0.0/16 via 192.168.1.254 dev eth0 metric 100
192.168.1.0/24 dev eth0 scope link
For each destination, predict which route will be used and why:
- Destination: 8.8.8.8
- Destination: 10.100.50.25
- Destination: 10.50.25.100
- Destination: 192.168.1.50
- Destination: 10.50.0.1
Questions to consider:
- Which has the longest matching prefix?
- When are metrics relevant?
- Which destinations need ARP lookup for the final hop?
The Interview Questions They’ll Ask
Prepare to answer these:
- “A server can’t reach the internet. How do you check the routing table?”
- “What is the default route and why is it important?”
- “What’s the difference between
routeandip route?” - “How does longest prefix match work?”
- “What is ARP and when does it happen?”
- “What does an incomplete ARP entry mean?”
- “How would you add a static route to send 10.0.0.0/8 through a specific gateway?”
Hints in Layers
Hint 1: Starting Point
ip route show displays the main routing table. ip route get <dest> shows which route would be used.
Hint 2: Source Address
ip route get 8.8.8.8 also shows the source IP that would be used.
Hint 3: ARP Cache
ip neigh show or arp -n shows the neighbor/ARP cache.
Hint 4: All Tables
ip route show table all shows routes from all routing tables (for policy routing).
Books That Will Help
| Topic | Book | Chapter |
|---|---|---|
| IP Routing | “TCP/IP Illustrated, Vol 1” by W. Richard Stevens | Ch. 8-9 |
| ARP | “TCP/IP Illustrated, Vol 1” by W. Richard Stevens | Ch. 4 |
| Linux routing | “Linux for Networking Professionals” (O’Reilly) | Ch. 4 |
| ip command | “How Linux Works” by Brian Ward | Ch. 9 |
Implementation Hints
Key commands:
# Routing table
ip route show # main table
ip route show table all # all tables
ip route get 8.8.8.8 # lookup specific destination
ip route show match 10.0.0.0 # routes matching prefix
# Legacy
route -n # old-style routing table
# Neighbor/ARP cache
ip neigh show # neighbor cache
arp -n # legacy ARP table
# Policy routing tables
ip rule show # routing policy rules
cat /etc/iproute2/rt_tables # table name mapping
Route parsing:
- Destination network and prefix length
- Gateway (via) or direct (scope link)
- Interface (dev)
- Metric (priority when multiple routes match)
- Source preference (src)
Longest prefix match algorithm:
- Find all routes that match the destination
- Select the one with the longest prefix
- If tie, use lowest metric
- If still tie, prefer more specific source
Learning milestones:
- You can read the routing table → Basic
ip routeusage - You can predict route selection → Longest prefix match understanding
- You understand ARP’s role → L2/L3 relationship
- You can trace packet forwarding decisions → Complete routing knowledge
Project 8: Bandwidth Monitor & Performance Tester
- File: LEARN_LINUX_NETWORKING_TOOLS.md
- Main Programming Language: Bash + Python
- Alternative Programming Languages: Go, Rust
- Coolness Level: Level 3: Genuinely Clever
- Business Potential: 2. The “Micro-SaaS / Pro Tool”
- Difficulty: Level 2: Intermediate
- Knowledge Area: Network Performance / Bandwidth Testing
- Software or Tool: iperf3, nload, iftop
- Main Book: “Systems Performance” by Brendan Gregg
What you’ll build: A comprehensive network performance toolkit that monitors real-time bandwidth per interface, shows per-connection throughput, runs bandwidth tests between hosts, and generates performance reports with historical comparison.
Why it teaches these tools: Understanding network performance is crucial for troubleshooting slow applications and capacity planning. By building a monitoring dashboard, you’ll learn to measure, interpret, and troubleshoot network throughput and latency.
Core challenges you’ll face:
- Reading real-time bandwidth from /proc → maps to kernel statistics
- Running iperf3 tests programmatically → maps to throughput measurement
- Correlating bandwidth with connections → maps to per-flow analysis
- Visualizing traffic in terminal → maps to effective monitoring
Key Concepts:
- Network Performance Metrics: “Systems Performance” Ch. 10 - Brendan Gregg
- TCP Throughput: “TCP/IP Illustrated Vol 1” Ch. 20 - W. Richard Stevens
- Bandwidth vs Latency: “Systems Performance” Ch. 10 - Brendan Gregg
Difficulty: Intermediate Time estimate: 1 week Prerequisites: Projects 1-4 completed; understanding of bits vs bytes; basic statistics knowledge
Real World Outcome
$ ./bandwidth-mon.sh
╔══════════════════════════════════════════════════════════════════════╗
║ NETWORK PERFORMANCE MONITOR ║
║ Host: server01 ║
║ Sampling: 1 second intervals ║
╠══════════════════════════════════════════════════════════════════════╣
┌─ INTERFACE THROUGHPUT ──────────────────────────────────────────────┐
│ Interface │ RX Rate │ TX Rate │ Total Rate │ Capacity │
├───────────┼──────────────┼──────────────┼──────────────┼───────────┤
│ eth0 │ 245.3 Mbps │ 12.4 Mbps │ 257.7 Mbps │ 1 Gbps │
│ │ ████████████████████████░░░░░░░░ 26% │
├───────────┼──────────────┼──────────────┼──────────────┼───────────┤
│ eth1 │ 0.2 Mbps │ 0.1 Mbps │ 0.3 Mbps │ 1 Gbps │
│ │ ░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░ 0% │
├───────────┼──────────────┼──────────────┼──────────────┼───────────┤
│ docker0 │ 5.6 Mbps │ 3.2 Mbps │ 8.8 Mbps │ 10 Gbps │
│ │ █░░░░░░░░░░░░░░░░░░░░░░░░░░░░░ 0% │
└─────────────────────────────────────────────────────────────────────┘
┌─ TOP CONNECTIONS BY BANDWIDTH ──────────────────────────────────────┐
│ Source Destination Protocol Rate │
├─────────────────────────────────────────────────────────────────────┤
│ 192.168.1.100:443 203.0.113.45:52341 TCP 125.3 Mbps │
│ 192.168.1.100:443 198.51.100.23:48291 TCP 82.1 Mbps │
│ 192.168.1.100:443 10.0.0.5:55432 TCP 23.4 Mbps │
│ 192.168.1.100:22 10.0.0.25:49201 TCP 2.1 Mbps │
│ 192.168.1.100:53 * UDP 0.5 Mbps │
└─────────────────────────────────────────────────────────────────────┘
┌─ HISTORICAL TREND (last hour) ──────────────────────────────────────┐
│ │
│ 300 Mbps │ ∙∙ │
│ │ ∙∙∙∙∙∙ ∙∙∙∙ │
│ 200 Mbps │ ∙∙∙∙ ∙∙∙∙ │
│ │ ∙∙∙∙ ∙∙∙∙∙∙ │
│ 100 Mbps │ ∙∙∙∙∙ ∙∙∙∙∙∙∙∙ │
│ │ ∙∙∙ ∙∙∙∙ │
│ 0 Mbps └──────────────────────────────────────────────────────── │
│ 13:00 13:30 14:00 14:30 now │
└─────────────────────────────────────────────────────────────────────┘
Press [t] to run iperf3 test, [q] to quit
$ ./bandwidth-mon.sh test 192.168.1.200
┌─ IPERF3 THROUGHPUT TEST ────────────────────────────────────────────┐
│ Target: 192.168.1.200:5201 │
│ │
│ TCP Bandwidth Test (10 seconds): │
│ Direction │ Throughput │ Retransmits │ Avg RTT │
│ ──────────────────────────────────────────────────── │
│ Upload │ 941 Mbps │ 0 │ 0.3 ms │
│ Download │ 938 Mbps │ 0 │ 0.3 ms │
│ │
│ UDP Bandwidth Test (1 Gbps target): │
│ Direction │ Throughput │ Jitter │ Packet Loss │
│ ──────────────────────────────────────────────────── │
│ Upload │ 956 Mbps │ 0.023 ms │ 0.01% │
│ │
│ Assessment: Link performing at ~94% of theoretical 1 Gbps capacity │
│ No significant packet loss or retransmits detected │
└─────────────────────────────────────────────────────────────────────┘
The Core Question You’re Answering
“How much bandwidth is my network actually using, and how fast can it go?”
Performance problems often come down to bandwidth. Is the link saturated? Is one connection hogging everything? What’s the real throughput capacity? This project gives you the tools to answer these questions.
Concepts You Must Understand First
Stop and research these before coding:
- Bandwidth vs Throughput vs Latency
- What’s the difference between link capacity and actual throughput?
- How does latency affect perceived speed?
- What is the bandwidth-delay product?
- Book Reference: “Systems Performance” Ch. 10 - Brendan Gregg
- TCP Throughput Dynamics
- How does TCP window size affect throughput?
- What causes TCP retransmissions?
- Why might you get less than wire speed?
- Book Reference: “TCP/IP Illustrated Vol 1” Ch. 20 - W. Richard Stevens
- Measurement Techniques
- What’s in /proc/net/dev and how often is it updated?
- How does iperf3 measure throughput?
- What’s the difference between TCP and UDP testing?
- Book Reference: “Systems Performance” Ch. 10 - Brendan Gregg
Questions to Guide Your Design
Before implementing, think through these:
- Data Collection
- How do you calculate rate from cumulative byte counters?
- How often should you sample?
- How do you get per-connection bandwidth?
- Visualization
- How do you display rates in appropriate units (Kbps, Mbps, Gbps)?
- How do you show historical trends?
- How do you identify bandwidth hogs?
- Testing
- How do you integrate iperf3 for active testing?
- How do you interpret UDP vs TCP results?
- How do you calculate capacity utilization?
Thinking Exercise
Interpret iperf3 Output
Analyze this iperf3 result:
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 112 MBytes 941 Mbits/sec 0 378 KBytes
[ 5] 1.00-2.00 sec 111 MBytes 935 Mbits/sec 0 378 KBytes
[ 5] 2.00-3.00 sec 112 MBytes 940 Mbits/sec 0 378 KBytes
[ 5] 3.00-4.00 sec 105 MBytes 878 Mbits/sec 12 312 KBytes
[ 5] 4.00-5.00 sec 98 MBytes 823 Mbits/sec 45 245 KBytes
[ 5] 5.00-6.00 sec 112 MBytes 937 Mbits/sec 0 378 KBytes
Questions to answer:
- What happened during intervals 3-5?
- What does “Retr” mean and why did it spike?
- What does Cwnd tell you?
- Is this link healthy overall?
The Interview Questions They’ll Ask
Prepare to answer these:
- “How would you check if a network link is saturated?”
- “What’s the difference between iperf3 TCP and UDP tests?”
- “A user says the network is slow. How do you measure actual throughput?”
- “What causes TCP retransmissions?”
- “How do you identify which connection is using the most bandwidth?”
- “What is jitter and why does it matter?”
- “How do you interpret the output of iftop?”
Hints in Layers
Hint 1: Starting Point
/proc/net/dev contains cumulative byte counts per interface. Read it twice with a delay and calculate the difference.
Hint 2: Real-time Bandwidth
nload shows nice ASCII graphs. iftop shows per-connection bandwidth.
Hint 3: iperf3 Testing
Run iperf3 -s on one end, iperf3 -c <server> on the other. Add -u for UDP, -R for reverse direction.
Hint 4: Per-connection Bandwidth
Combine ss -ti (TCP info including bytes) with periodic sampling to calculate per-socket throughput.
Books That Will Help
| Topic | Book | Chapter |
|---|---|---|
| Network performance | “Systems Performance” by Brendan Gregg | Ch. 10 |
| TCP throughput | “TCP/IP Illustrated, Vol 1” by W. Richard Stevens | Ch. 20 |
| Linux networking | “Linux for Networking Professionals” (O’Reilly) | Ch. 5 |
| iperf3 usage | “High Performance Browser Networking” by Ilya Grigorik | Ch. 2 |
Implementation Hints
Data sources:
# Interface statistics
cat /proc/net/dev # cumulative RX/TX bytes per interface
ip -s link show # same data, friendlier format
# Real-time monitors
nload # per-interface bandwidth graph
iftop -n # per-connection bandwidth
vnstat # historical bandwidth (if installed)
# Active testing
iperf3 -s # server mode
iperf3 -c host # TCP test
iperf3 -c host -u -b 1G # UDP test at 1 Gbps
iperf3 -c host -R # reverse (download test)
iperf3 -c host --json # machine-readable output
# TCP connection info
ss -ti # TCP internal info (bytes, RTT, etc.)
Rate calculation:
rate_bps = (bytes_now - bytes_prev) * 8 / interval_seconds
rate_mbps = rate_bps / 1_000_000
iperf3 JSON output fields:
bits_per_second: throughputretransmits: TCP retransmissionsjitter_ms: UDP jitterlost_percent: UDP packet loss
Learning milestones:
- You can read /proc/net/dev → Kernel statistics understanding
- You can calculate throughput → Basic measurement skills
- You can run iperf3 tests → Active performance testing
- You can identify bandwidth issues → Troubleshooting ability
Project 9: HTTP/API Testing Suite
- File: LEARN_LINUX_NETWORKING_TOOLS.md
- Main Programming Language: Bash (shell scripting)
- Alternative Programming Languages: Python, Go
- Coolness Level: Level 2: Practical but Forgettable
- Business Potential: 2. The “Micro-SaaS / Pro Tool”
- Difficulty: Level 2: Intermediate
- Knowledge Area: HTTP Protocol / API Testing
- Software or Tool: curl, wget
- Main Book: “HTTP: The Definitive Guide” by David Gourley
What you’ll build: A comprehensive HTTP testing toolkit that can probe endpoints, analyze response headers, measure timing breakdowns (DNS, connect, TLS, transfer), follow redirects with analysis, test authentication, and benchmark API performance.
Why it teaches these tools: curl and wget are the Swiss Army knives of HTTP. By building a testing framework around them, you’ll understand HTTP deeply—headers, methods, authentication, TLS, redirects, cookies, and performance characteristics.
Core challenges you’ll face:
- Understanding HTTP request/response cycle → maps to protocol knowledge
- Parsing timing information → maps to performance debugging
- Handling authentication methods → maps to security concepts
- Following redirect chains → maps to HTTP semantics
Key Concepts:
- HTTP Protocol: “HTTP: The Definitive Guide” Ch. 1-5 - David Gourley
- TLS/SSL: “Serious Cryptography” Ch. 14 - Jean-Philippe Aumasson
- REST APIs: “Design and Build Great Web APIs” - Mike Amundsen
Difficulty: Intermediate Time estimate: 1 week Prerequisites: Projects 1-3 completed; basic understanding of HTTP (GET, POST); familiarity with JSON
Real World Outcome
$ ./httptest.sh https://api.github.com/users/octocat
╔══════════════════════════════════════════════════════════════════════╗
║ HTTP ENDPOINT ANALYSIS ║
║ Target: https://api.github.com/users/octocat ║
╠══════════════════════════════════════════════════════════════════════╣
┌─────────────────────────────────────────────────────────────────────┐
│ REQUEST │
├─────────────────────────────────────────────────────────────────────┤
│ Method: GET │
│ URL: https://api.github.com/users/octocat │
│ Headers Sent: │
│ User-Agent: httptest/1.0 │
│ Accept: */* │
│ Host: api.github.com │
└─────────────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────────┐
│ RESPONSE │
├─────────────────────────────────────────────────────────────────────┤
│ Status: 200 OK │
│ Headers: │
│ Content-Type: application/json; charset=utf-8 │
│ Cache-Control: public, max-age=60, s-maxage=60 │
│ ETag: "abc123..." │
│ X-RateLimit-Limit: 60 │
│ X-RateLimit-Remaining: 58 │
│ X-RateLimit-Reset: 1703264400 │
│ │
│ Body (preview): │
│ { │
│ "login": "octocat", │
│ "id": 583231, │
│ "avatar_url": "https://avatars...", │
│ ... │
│ } │
│ Body size: 1,423 bytes │
└─────────────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────────┐
│ TIMING BREAKDOWN │
├─────────────────────────────────────────────────────────────────────┤
│ Phase │ Duration │ Cumulative │ Visual │
├────────────────────┼───────────┼────────────┼──────────────────────┤
│ DNS Lookup │ 23 ms │ 23 ms │ ███░░░░░░░ │
│ TCP Connect │ 45 ms │ 68 ms │ █████░░░░░ │
│ TLS Handshake │ 89 ms │ 157 ms │ █████████░ │
│ Time to First Byte │ 52 ms │ 209 ms │ ██████░░░░ │
│ Content Transfer │ 12 ms │ 221 ms │ ██░░░░░░░░ │
├────────────────────┼───────────┼────────────┼──────────────────────┤
│ TOTAL │ │ 221 ms │ ██████████████████ │
└─────────────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────────┐
│ TLS/SSL ANALYSIS │
├─────────────────────────────────────────────────────────────────────┤
│ Protocol: TLSv1.3 │
│ Cipher: TLS_AES_128_GCM_SHA256 │
│ Certificate: │
│ Subject: CN=*.github.com │
│ Issuer: DigiCert │
│ Valid: 2024-01-15 to 2025-01-15 (✓ valid) │
│ Alt Names: github.com, *.github.com │
└─────────────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────────┐
│ PERFORMANCE BENCHMARK (10 requests) │
├─────────────────────────────────────────────────────────────────────┤
│ Metric │ Min │ Avg │ Max │ P95 │
├─────────────────────┼──────────┼──────────┼──────────┼─────────────┤
│ Total Time │ 198 ms │ 215 ms │ 287 ms │ 252 ms │
│ Time to First Byte │ 45 ms │ 52 ms │ 78 ms │ 68 ms │
│ DNS (cached) │ 0 ms │ 2 ms │ 23 ms │ 3 ms │
│ │
│ Success Rate: 100% (10/10) │
│ Errors: 0 │
└─────────────────────────────────────────────────────────────────────┘
The Core Question You’re Answering
“Is this HTTP endpoint working correctly, and what’s happening at each step of the request?”
HTTP is everywhere—APIs, websites, microservices. When something’s slow or broken, you need to see exactly what’s happening: headers, timing, TLS, redirects. curl gives you this visibility; wget gives you download capabilities.
Concepts You Must Understand First
Stop and research these before coding:
- HTTP Request/Response
- What are HTTP methods (GET, POST, PUT, DELETE)?
- What are the important HTTP headers?
- What do status codes mean (2xx, 3xx, 4xx, 5xx)?
- Book Reference: “HTTP: The Definitive Guide” Ch. 1-3 - David Gourley
- HTTPS and TLS
- How does TLS handshake work?
- What’s SNI and why does it matter?
- How do you verify certificates?
- Book Reference: “Serious Cryptography” Ch. 14 - Jean-Philippe Aumasson
- Performance Timing
- What are the phases of an HTTP request?
- What is Time to First Byte (TTFB)?
- Why does DNS/TLS add latency?
- Book Reference: “High Performance Browser Networking” Ch. 4 - Ilya Grigorik
Questions to Guide Your Design
Before implementing, think through these:
- Request Configuration
- How do you set custom headers?
- How do you send different HTTP methods with body data?
- How do you handle authentication (Basic, Bearer, OAuth)?
- Response Analysis
- How do you extract and display headers?
- How do you follow redirects and trace the chain?
- How do you handle different content types (JSON, XML, HTML)?
- Timing and Performance
- How do you get timing breakdown from curl?
- How do you run multiple requests for benchmarking?
- How do you calculate percentiles?
Thinking Exercise
Decode This curl Command
Analyze this curl command and predict what it does:
curl -X POST \
-H "Content-Type: application/json" \
-H "Authorization: Bearer eyJhbGc..." \
-d '{"name": "test", "value": 42}' \
-w "\n%{http_code} %{time_total}s\n" \
-o /dev/null \
-s \
https://api.example.com/resources
Questions to answer:
- What HTTP method is used?
- What headers are sent?
- What’s in the request body?
- What output will you see?
- What does
-o /dev/nulldo?
The Interview Questions They’ll Ask
Prepare to answer these:
- “How would you test if an API endpoint is working?”
- “What’s the difference between curl and wget?”
- “How do you send a POST request with JSON data?”
- “An API call is slow. How do you find where the time is spent?”
- “How do you handle authentication in curl?”
- “What is TTFB and why does it matter?”
- “How would you download a file and resume if it fails?”
Hints in Layers
Hint 1: Starting Point
curl -v URL shows request and response headers in verbose mode.
Hint 2: Timing Breakdown
curl -w "@format.txt" URL with a format file containing timing variables like %{time_namelookup}, %{time_connect}, etc.
Hint 3: JSON Pretty Print
Pipe to jq for JSON: curl -s URL | jq .
Hint 4: Certificate Info
curl -vI https://example.com 2>&1 | grep -A5 "Server certificate"
Books That Will Help
| Topic | Book | Chapter |
|---|---|---|
| HTTP protocol | “HTTP: The Definitive Guide” by David Gourley | Ch. 1-5 |
| TLS/HTTPS | “Serious Cryptography” by Jean-Philippe Aumasson | Ch. 14 |
| Web performance | “High Performance Browser Networking” by Ilya Grigorik | Ch. 4 |
| API design | “Design and Build Great Web APIs” by Mike Amundsen | Full book |
Implementation Hints
curl timing variables for -w:
time_namelookup: DNS lookup time
time_connect: TCP connect time (cumulative)
time_appconnect: TLS handshake complete (cumulative)
time_starttransfer: TTFB (cumulative)
time_total: Total time
Example format file:
DNS: %{time_namelookup}s
Connect: %{time_connect}s
TLS: %{time_appconnect}s
TTFB: %{time_starttransfer}s
Total: %{time_total}s
Status: %{http_code}
Size: %{size_download} bytes
Key curl options:
-X POST # HTTP method
-H "Header: value" # Custom header
-d "data" # Request body
-d @file.json # Body from file
-u user:pass # Basic auth
-H "Authorization: Bearer TOKEN" # Bearer auth
-L # Follow redirects
-I # HEAD request only
-v # Verbose (debug)
-s # Silent (no progress)
-o file # Output to file
-w "format" # Custom output format
--connect-timeout 5 # Connection timeout
-k # Skip TLS verification (dangerous!)
wget for downloads:
wget URL # Download file
wget -c URL # Resume download
wget -r URL # Recursive download
wget --mirror URL # Mirror website
wget -q -O - URL # Output to stdout (like curl)
Learning milestones:
- You can make any HTTP request → curl basics
- You understand timing breakdown → Performance debugging
- You can analyze TLS certificates → Security awareness
- You can benchmark APIs → Performance testing skills
Project 10: Network Log Analyzer
- File: LEARN_LINUX_NETWORKING_TOOLS.md
- Main Programming Language: Bash + awk
- Alternative Programming Languages: Python, Go
- Coolness Level: Level 2: Practical but Forgettable
- Business Potential: 3. The “Service & Support” Model
- Difficulty: Level 2: Intermediate
- Knowledge Area: System Logging / Network Troubleshooting
- Software or Tool: dmesg, journalctl
- Main Book: “How Linux Works, 3rd Edition” by Brian Ward
What you’ll build: A network-focused log analysis tool that extracts network-related events from kernel logs (dmesg) and system logs (journalctl), identifies patterns (link up/down, driver errors, firewall blocks), and correlates events across time for troubleshooting.
Why it teaches these tools: When network issues happen, the answers are often in the logs. The kernel logs driver issues, link state changes, and firewall blocks. By building an analyzer, you’ll learn where to find network events and how to interpret them.
Core challenges you’ll face:
- Extracting network events from noisy logs → maps to log filtering
- Understanding kernel network messages → maps to driver/hardware issues
- Correlating events across time → maps to root cause analysis
- Identifying firewall log entries → maps to security monitoring
Key Concepts:
- Kernel Ring Buffer (dmesg): “How Linux Works” Ch. 4 - Brian Ward
- systemd Journal: “How Linux Works” Ch. 6 - Brian Ward
- Network Driver Messages: “Linux Kernel Development” Ch. 17 - Robert Love
Difficulty: Intermediate Time estimate: 1 week Prerequisites: Projects 1-4 completed; basic understanding of Linux logging; familiarity with grep/awk
Real World Outcome
$ sudo ./netlog-analyzer.sh --last 24h
╔══════════════════════════════════════════════════════════════════════╗
║ NETWORK LOG ANALYSIS ║
║ Period: Last 24 hours ║
║ Host: server01 ║
╠══════════════════════════════════════════════════════════════════════╣
┌─────────────────────────────────────────────────────────────────────┐
│ LINK STATE CHANGES │
├─────────────────────────────────────────────────────────────────────┤
│ Time │ Interface │ Event │ Details │
├──────────────────────┼───────────┼──────────┼──────────────────────┤
│ 2024-12-22 03:45:12 │ eth0 │ LINK DOWN│ carrier lost │
│ 2024-12-22 03:45:15 │ eth0 │ LINK UP │ 1000Mbps Full │
│ 2024-12-22 08:12:33 │ eth1 │ LINK DOWN│ NIC reset │
│ 2024-12-22 08:12:38 │ eth1 │ LINK UP │ 1000Mbps Full │
│ │
│ Summary: 4 link state changes (2 DOWN events) - investigate eth0/1 │
└─────────────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────────┐
│ NETWORK DRIVER EVENTS │
├─────────────────────────────────────────────────────────────────────┤
│ Time │ Interface │ Driver │ Event │
├──────────────────────┼───────────┼──────────┼──────────────────────┤
│ 2024-12-22 03:45:12 │ eth0 │ igb │ Reset adapter │
│ 2024-12-22 08:12:33 │ eth1 │ igb │ Detected Tx Unit Hang│
│ 2024-12-22 08:12:35 │ eth1 │ igb │ Reset adapter │
│ │
│ ⚠ Warning: Tx Unit Hang may indicate hardware or driver issue │
└─────────────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────────┐
│ DHCP/IP EVENTS │
├─────────────────────────────────────────────────────────────────────┤
│ Time │ Interface │ Event │
├──────────────────────┼───────────┼─────────────────────────────────┤
│ 2024-12-22 03:45:20 │ eth0 │ DHCP: Bound 192.168.1.100 │
│ 2024-12-22 15:30:00 │ eth0 │ DHCP: Renewed lease │
│ │
│ DHCP working normally │
└─────────────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────────┐
│ FIREWALL BLOCKS (Top Sources) │
├─────────────────────────────────────────────────────────────────────┤
│ Source IP │ Count │ Ports Attempted │
├────────────────────┼───────┼───────────────────────────────────────┤
│ 45.227.253.98 │ 1,247 │ 22, 23, 3389 │
│ 185.220.101.45 │ 892 │ 22, 2222 │
│ 103.251.167.20 │ 456 │ 5432, 3306 │
│ │
│ Summary: 2,595 blocked connection attempts from 23 unique IPs │
│ Pattern: SSH brute force and database scanning detected │
└─────────────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────────┐
│ DNS/NETWORKING SERVICES │
├─────────────────────────────────────────────────────────────────────┤
│ Service │ Events │ Status │
├─────────────────────┼────────┼─────────────────────────────────────┤
│ NetworkManager │ 12 │ ✓ Running, no errors │
│ systemd-resolved │ 5 │ ✓ Running, upstream: 8.8.8.8 │
│ systemd-networkd │ 0 │ Not active (NM in use) │
│ sshd │ 234 │ ✓ 12 successful, 222 failed auth │
└─────────────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────────┐
│ TIMELINE CORRELATION │
├─────────────────────────────────────────────────────────────────────┤
│ 03:45:12 ──┬── eth0 link down (carrier lost) │
│ │ └── Likely: Cable unplugged or switch issue │
│ 03:45:15 ──┼── eth0 link up │
│ 03:45:20 ──┴── DHCP renewed (IP: 192.168.1.100) │
│ Duration: 8 seconds downtime │
│ │
│ 08:12:33 ──┬── eth1 Tx Unit Hang detected │
│ │ └── Driver detected stuck transmit queue │
│ 08:12:35 ──┼── eth1 adapter reset │
│ 08:12:38 ──┴── eth1 link up │
│ Duration: 5 seconds downtime │
│ ⚠ Recommendation: Check for driver updates │
└─────────────────────────────────────────────────────────────────────┘
The Core Question You’re Answering
“What network events happened on my system, and what do they mean?”
When something breaks, the evidence is in the logs. But logs are noisy. By building a tool that extracts and correlates network events, you’ll learn to find the signal in the noise and understand what the kernel is telling you.
Concepts You Must Understand First
Stop and research these before coding:
- Kernel Ring Buffer
- What is dmesg and where does it come from?
- How is the ring buffer different from persistent logs?
- What timestamp format does dmesg use?
- Book Reference: “How Linux Works” Ch. 4 - Brian Ward
- systemd Journal
- What is journalctl and how does it organize logs?
- How do you filter by time, unit, or priority?
- What’s the difference between volatile and persistent journals?
- Book Reference: “How Linux Works” Ch. 6 - Brian Ward
- Network Driver Messages
- What messages do network drivers log?
- What does “link up/down” mean at the kernel level?
- What are common error patterns (tx timeout, watchdog)?
- Book Reference: “Linux Kernel Development” Ch. 17 - Robert Love
Questions to Guide Your Design
Before implementing, think through these:
- Event Extraction
- What patterns indicate network events in dmesg?
- What journalctl filters give you network-related entries?
- How do you handle different log formats?
- Correlation
- How do you correlate events that are seconds apart?
- How do you identify cause-and-effect relationships?
- How do you detect repeated patterns?
- Presentation
- How do you summarize thousands of log entries?
- How do you highlight anomalies?
- How do you show timeline of related events?
Thinking Exercise
Interpret These Log Entries
Analyze this sequence of kernel messages:
[89532.123] igb 0000:04:00.0 eth1: Reset adapter
[89532.456] igb 0000:04:00.0 eth1: igb_reset_task: failed to remove HW
[89533.789] igb 0000:04:00.0 eth1: NIC Link is Down
[89535.012] igb 0000:04:00.0 eth1: NIC Link is Up 1000 Mbps Full Duplex
[89535.345] IPv6: ADDRCONF(NETDEV_CHANGE): eth1: link becomes ready
Questions to answer:
- What happened to the network interface?
- Was there data loss during this event?
- How long was the link down?
- What might have caused this?
- Is this a hardware or software issue?
The Interview Questions They’ll Ask
Prepare to answer these:
- “Where do you look when a network interface keeps going up and down?”
- “What’s the difference between dmesg and journalctl?”
- “How do you find when a network interface got its IP address?”
- “How would you check if the firewall is blocking connections?”
- “What does ‘Tx timeout’ mean in network driver logs?”
- “How do you correlate log events across different services?”
- “How do you make logs persist across reboots?”
Hints in Layers
Hint 1: Starting Point
dmesg --time-format=iso | grep -i 'eth\|link\|network' for kernel network events.
Hint 2: journalctl Filtering
journalctl -u NetworkManager -u systemd-networkd --since "1 hour ago" for network service logs.
Hint 3: Firewall Logs
journalctl -k | grep -i 'iptables\|nftables\|DROP\|REJECT' for firewall blocks.
Hint 4: Driver Events
dmesg | grep -E '(igb|e1000|ixgbe|mlx|virtio)' for NIC driver events (adjust for your driver).
Books That Will Help
| Topic | Book | Chapter |
|---|---|---|
| Linux logging | “How Linux Works” by Brian Ward | Ch. 4, 6 |
| systemd journal | “How Linux Works” by Brian Ward | Ch. 6 |
| Kernel messages | “Linux Kernel Development” by Robert Love | Ch. 17 |
| Log analysis | “The Practice of System Administration” by Limoncelli | Ch. 21 |
Implementation Hints
Key commands:
# Kernel ring buffer
dmesg # all kernel messages
dmesg -T # human-readable timestamps
dmesg --time-format=iso # ISO timestamps (parseable)
dmesg -l err,warn # only errors and warnings
dmesg -w # follow (like tail -f)
# systemd journal
journalctl -k # kernel messages (like dmesg)
journalctl -u NetworkManager # specific service
journalctl --since "1 hour ago" # time filter
journalctl -p err # priority filter
journalctl -f # follow
journalctl -o json # JSON output (parseable)
journalctl -b # current boot only
journalctl -b -1 # previous boot
# Network-specific filters
journalctl -u NetworkManager -u systemd-networkd -u systemd-resolved
journalctl | grep -i 'dhcp\|dns\|network\|link'
Patterns to match:
# Link state
"NIC Link is Down"
"NIC Link is Up"
"link becomes ready"
"carrier lost"
# Driver issues
"Reset adapter"
"Tx timeout"
"watchdog"
"hardware error"
# DHCP
"DHCPDISCOVER"
"DHCPOFFER"
"DHCPACK"
"bound to"
# Firewall (if logging enabled)
"IN=eth0 OUT= SRC="
"DPT=22"
"DROP"
Learning milestones:
- You can find network events in dmesg → Kernel log familiarity
- You can filter journalctl effectively → systemd journal mastery
- You can correlate events → Root cause analysis skill
- You can identify patterns → Anomaly detection
Project 11: Network Namespace Laboratory
- File: LEARN_LINUX_NETWORKING_TOOLS.md
- Main Programming Language: Bash (shell scripting)
- Alternative Programming Languages: Python, Go
- Coolness Level: Level 4: Hardcore Tech Flex
- Business Potential: 1. The “Resume Gold”
- Difficulty: Level 3: Advanced
- Knowledge Area: Container Networking / Network Isolation
- Software or Tool: ip netns
- Main Book: “The Linux Programming Interface” by Michael Kerrisk
What you’ll build: A virtual network laboratory that creates isolated network environments using namespaces, connects them with virtual ethernet pairs, sets up routing between them, and demonstrates the networking foundations that Docker and Kubernetes use.
Why it teaches ip netns: Network namespaces are the foundation of container networking. By building your own mini-network in a box, you’ll understand how containers get isolated networks, how they communicate, and what happens under the hood when you run Docker.
Core challenges you’ll face:
- Creating and managing namespaces → maps to container isolation concepts
- Creating veth pairs and connecting namespaces → maps to container-to-host networking
- Setting up routing between namespaces → maps to inter-container communication
- Understanding namespace isolation → maps to security boundaries
Key Concepts:
- Linux Namespaces: “The Linux Programming Interface” Ch. 58 - Michael Kerrisk
- Virtual Ethernet Devices: “Linux for Networking Professionals” Ch. 7 - O’Reilly
- Container Networking: “Docker Deep Dive” Ch. 11 - Nigel Poulton
Difficulty: Advanced Time estimate: 2 weeks Prerequisites: Projects 1-7 completed; understanding of routing, firewalls, and bridges; basic Docker knowledge helpful
Real World Outcome
$ sudo ./netns-lab.sh create three-tier-network
╔══════════════════════════════════════════════════════════════════════╗
║ NETWORK NAMESPACE LABORATORY ║
║ Creating: three-tier-network ║
╠══════════════════════════════════════════════════════════════════════╣
Creating network topology:
┌─────────────────────────────────────┐
│ Host (Default NS) │
│ │
│ br0 (bridge) │
│ 192.168.100.1/24 │
└───────────┬─────────────────────────┘
│
┌───────────────────────┼───────────────────────┐
│ │ │
▼ ▼ ▼
┌───────────────┐ ┌───────────────┐ ┌───────────────┐
│ NS: web │ │ NS: app │ │ NS: db │
│ │ │ │ │ │
│ eth0 │ │ eth0 │ │ eth0 │
│ 192.168.100.10 │ 192.168.100.20 │ 192.168.100.30
│ │ │ │ │ │
│ 🌐 Port 80 │ ───► │ 📱 Port 8080 │ ───► │ 🗄 Port 5432 │
│ (nginx) │ │ (app) │ │ (postgres) │
└───────────────┘ └───────────────┘ └───────────────┘
✓ Created namespace: web
✓ Created namespace: app
✓ Created namespace: db
✓ Created bridge: br0
✓ Created veth pairs and attached to namespaces
✓ Configured IP addresses
✓ Enabled routing in all namespaces
TESTING CONNECTIVITY:
┌─────────────────────────────────────────────────────────────────────┐
│ Connectivity Matrix │
├─────────────────────────────────────────────────────────────────────┤
│ From │ To web │ To app │ To db │ Host │
├───────────┼───────────────┼───────────────┼───────────────┼───────┤
│ web │ - │ ✓ 0.1ms │ ✓ 0.1ms │ ✓ │
│ app │ ✓ 0.1ms │ - │ ✓ 0.1ms │ ✓ │
│ db │ ✓ 0.1ms │ ✓ 0.1ms │ - │ ✓ │
│ Host │ ✓ 0.1ms │ ✓ 0.1ms │ ✓ 0.1ms │ - │
└─────────────────────────────────────────────────────────────────────┘
FIREWALL RULES (applied):
- web: Accept 80/tcp from any
- app: Accept 8080/tcp from web only
- db: Accept 5432/tcp from app only
$ sudo ./netns-lab.sh exec web curl -s http://192.168.100.20:8080
OK - Response from app namespace
$ sudo ./netns-lab.sh exec db curl -s http://192.168.100.10:80
BLOCKED - Firewall prevents db→web direct access (security!)
COMMANDS TO INTERACT:
ip netns exec web bash # Shell in web namespace
ip netns exec web ip addr # Show web namespace IPs
ip netns exec app ping 192.168.100.30 # Ping db from app
./netns-lab.sh destroy three-tier-network # Clean up
The Core Question You’re Answering
“How do containers get their own isolated network, and how do they communicate?”
Every time you run a Docker container, Linux creates a network namespace. Understanding how to create and connect namespaces manually demystifies container networking and prepares you for debugging complex Kubernetes network issues.
Concepts You Must Understand First
Stop and research these before coding:
- Linux Namespaces
- What types of namespaces exist (pid, net, mnt, user, etc.)?
- How does the network namespace isolate networking?
- What gets its own copy in a network namespace?
- Book Reference: “The Linux Programming Interface” Ch. 58 - Michael Kerrisk
- Virtual Ethernet Pairs (veth)
- What is a veth pair and how does it work?
- How do you connect a namespace to the host?
- What happens when you put one end in a namespace?
- Book Reference: “Linux for Networking Professionals” Ch. 7 - O’Reilly
- Linux Bridges
- What is a software bridge?
- How does it differ from a switch?
- How do you attach veth interfaces to a bridge?
- Book Reference: “How Linux Works” Ch. 9 - Brian Ward
Questions to Guide Your Design
Before implementing, think through these:
- Namespace Management
- How do you run commands inside a namespace?
- How do you check what’s in each namespace?
- What happens to a namespace when the last process exits?
- Network Topology
- How do you connect multiple namespaces?
- Bridge vs point-to-point veth connections?
- How do you enable inter-namespace routing?
- Cleanup
- How do you cleanly destroy the lab?
- What resources need explicit cleanup?
- How do you handle partial failures during creation?
Thinking Exercise
Design a Docker Network
Before coding, diagram what Docker does when you run:
docker network create my-net
docker run --network my-net --name web nginx
docker run --network my-net --name db postgres
Questions to answer:
- How many namespaces are created?
- What virtual interfaces exist and where?
- How does ‘web’ reach ‘db’ by name?
- What’s the equivalent using just
ipcommands?
The Interview Questions They’ll Ask
Prepare to answer these:
- “How do containers get network isolation?”
- “What is a veth pair and how is it used in Docker?”
- “How does a container communicate with the host?”
- “What’s the difference between bridge and host networking in Docker?”
- “How would you debug networking between two containers?”
- “What happens to network connections when a container restarts?”
- “How does Kubernetes pod networking differ from Docker?”
Hints in Layers
Hint 1: Starting Point
ip netns add myns creates a namespace. ip netns exec myns bash gets a shell inside.
Hint 2: veth Creation
ip link add veth0 type veth peer name veth1 creates a pair. Move one end with ip link set veth1 netns myns.
Hint 3: Bridge Connection
Create bridge with ip link add br0 type bridge. Attach veth with ip link set veth0 master br0.
Hint 4: Enable Communication
Don’t forget: ip link set dev X up for every interface, and configure IPs with ip addr add.
Books That Will Help
| Topic | Book | Chapter |
|---|---|---|
| Namespaces | “The Linux Programming Interface” by Michael Kerrisk | Ch. 58 |
| Container networking | “Docker Deep Dive” by Nigel Poulton | Ch. 11 |
| Virtual devices | “Linux for Networking Professionals” (O’Reilly) | Ch. 7 |
| Advanced routing | LARTC HOWTO at lartc.org | Full document |
Implementation Hints
Creating a complete namespace network:
# Create namespace
ip netns add web
# Create veth pair
ip link add veth-web type veth peer name veth-web-br
# Move one end to namespace
ip link set veth-web netns web
# Configure inside namespace
ip netns exec web ip addr add 192.168.100.10/24 dev veth-web
ip netns exec web ip link set veth-web up
ip netns exec web ip link set lo up
ip netns exec web ip route add default via 192.168.100.1
# Configure host end (attach to bridge or use directly)
ip addr add 192.168.100.1/24 dev veth-web-br
ip link set veth-web-br up
# Enable forwarding for routing
sysctl -w net.ipv4.ip_forward=1
Key commands for namespace work:
ip netns list # List namespaces
ip netns exec ns1 CMD # Run command in namespace
ip netns exec ns1 ip addr # Show addresses in ns1
ip -n ns1 addr # Shorthand for above
nsenter --net=/var/run/netns/ns1 bash # Enter namespace (alternative)
Cleanup:
ip netns delete web # Delete namespace (kills veth too)
ip link delete br0 # Delete bridge
Learning milestones:
- You can create and enter namespaces → Basic namespace operations
- You can connect namespaces with veth → Virtual networking
- You can set up bridges → Multi-namespace connectivity
- You understand container networking → Real-world application
Project 12: Traffic Control Simulator
- File: LEARN_LINUX_NETWORKING_TOOLS.md
- Main Programming Language: Bash (shell scripting)
- Alternative Programming Languages: Python, Go
- Coolness Level: Level 4: Hardcore Tech Flex
- Business Potential: 3. The “Service & Support” Model
- Difficulty: Level 4: Expert
- Knowledge Area: QoS / Traffic Shaping / Network Simulation
- Software or Tool: tc
- Main Book: “Linux Advanced Routing & Traffic Control HOWTO” (lartc.org)
What you’ll build: A network condition simulator that uses tc to add latency, packet loss, bandwidth limits, and jitter to network interfaces—letting you test how applications behave under poor network conditions or implement QoS policies.
Why it teaches tc: Traffic control is one of the most powerful and least understood Linux networking features. By building a simulator, you’ll understand queuing disciplines, how to shape traffic, and how to test applications under realistic network conditions.
Core challenges you’ll face:
- Understanding qdisc/class/filter hierarchy → maps to tc architecture
- Implementing bandwidth limits with TBF/HTB → maps to rate limiting
- Simulating network conditions with netem → maps to testing/development
- Classifying traffic for different treatment → maps to QoS policies
Key Concepts:
- Queuing Disciplines: LARTC HOWTO - lartc.org
- HTB (Hierarchical Token Bucket): LARTC HOWTO - lartc.org
- netem (Network Emulator):
man tc-netem
Difficulty: Expert Time estimate: 2 weeks Prerequisites: Projects 1-8 completed; solid understanding of networking; comfort with complex command syntax
Real World Outcome
$ sudo ./tc-sim.sh
╔══════════════════════════════════════════════════════════════════════╗
║ TRAFFIC CONTROL SIMULATOR ║
║ Interface: eth0 ║
╠══════════════════════════════════════════════════════════════════════╣
PROFILES AVAILABLE:
┌─────────────────────────────────────────────────────────────────────┐
│ Profile │ Bandwidth │ Latency │ Loss │ Jitter │
├────────────────────┼───────────┼─────────┼────────┼────────────────┤
│ 1. lan │ 1 Gbps │ 0.5 ms │ 0% │ 0 ms │
│ 2. fiber │ 100 Mbps │ 10 ms │ 0% │ 2 ms │
│ 3. cable │ 50 Mbps │ 25 ms │ 0.1% │ 5 ms │
│ 4. dsl │ 10 Mbps │ 50 ms │ 0.5% │ 10 ms │
│ 5. 4g-good │ 20 Mbps │ 50 ms │ 0.1% │ 20 ms │
│ 6. 4g-poor │ 5 Mbps │ 100 ms │ 2% │ 50 ms │
│ 7. 3g │ 1 Mbps │ 200 ms │ 1% │ 100 ms │
│ 8. satellite │ 10 Mbps │ 600 ms │ 0.5% │ 50 ms │
│ 9. lossy │ ∞ │ 0 ms │ 10% │ 0 ms │
│ 10. custom │ (configure your own) │
└─────────────────────────────────────────────────────────────────────┘
$ sudo ./tc-sim.sh apply 4g-poor eth0
Applied profile: 4g-poor to eth0
- Bandwidth limit: 5 Mbps (using HTB qdisc)
- Latency: 100ms (using netem)
- Packet loss: 2%
- Jitter: ±50ms (normal distribution)
CURRENT TC RULES:
┌─────────────────────────────────────────────────────────────────────┐
│ eth0 (egress): │
│ qdisc htb 1: root default 10 │
│ class htb 1:1 rate 5mbit ceil 5mbit │
│ class htb 1:10 parent 1:1 rate 5mbit ceil 5mbit │
│ qdisc netem 10: parent 1:10 delay 100ms 50ms loss 2% │
├─────────────────────────────────────────────────────────────────────┤
│ ifb0 (ingress via redirect): │
│ qdisc htb 1: root default 10 │
│ class htb 1:10 rate 5mbit ceil 5mbit │
└─────────────────────────────────────────────────────────────────────┘
LIVE TESTING:
$ ping 8.8.8.8
PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.
64 bytes from 8.8.8.8: icmp_seq=1 ttl=117 time=112.3 ms
64 bytes from 8.8.8.8: icmp_seq=2 ttl=117 time=143.7 ms
64 bytes from 8.8.8.8: icmp_seq=3 ttl=117 time=98.2 ms
64 bytes from 8.8.8.8: icmp_seq=5 ttl=117 time=152.1 ms ← seq 4 lost!
$ curl -w "Time: %{time_total}s\n" -o /dev/null -s https://example.com
Time: 0.892s ← Much slower due to latency + limited bandwidth
$ sudo ./tc-sim.sh clear eth0
Removed all tc rules from eth0
The Core Question You’re Answering
“How do I shape, limit, or simulate network traffic on Linux?”
Whether you’re testing mobile app performance, implementing QoS, or debugging latency-sensitive applications, tc is the tool. It’s notoriously complex, but this project makes it approachable.
Concepts You Must Understand First
Stop and research these before coding:
- Queuing Disciplines (qdiscs)
- What is a qdisc and where does it sit in the network stack?
- What’s the difference between classless and classful qdiscs?
- What does “root” qdisc mean?
- Book Reference: LARTC HOWTO at lartc.org
- Classes and Filters
- How do classes divide bandwidth?
- How do filters direct packets to classes?
- What are handles like 1:10?
- Book Reference: LARTC HOWTO at lartc.org
- Common Qdiscs
- TBF (Token Bucket Filter): Simple rate limiting
- HTB (Hierarchical Token Bucket): Complex bandwidth sharing
- netem: Network emulation (delay, loss, jitter)
- Book Reference: LARTC HOWTO and man pages
Questions to Guide Your Design
Before implementing, think through these:
- Egress vs Ingress
- tc primarily controls egress (outgoing) traffic
- How do you limit incoming traffic? (IFB device)
- Why is ingress shaping harder?
- Profile Design
- What network conditions are useful to simulate?
- How do you combine rate limiting with latency/loss?
- What’s a realistic latency distribution?
- Safety
- How do you ensure cleanup happens?
- What if rules break network access?
- How do you make rules persistent (or explicitly not)?
Thinking Exercise
Decode This tc Configuration
Analyze these tc commands:
tc qdisc add dev eth0 root handle 1: htb default 20
tc class add dev eth0 parent 1: classid 1:1 htb rate 100mbit ceil 100mbit
tc class add dev eth0 parent 1:1 classid 1:10 htb rate 80mbit ceil 100mbit prio 1
tc class add dev eth0 parent 1:1 classid 1:20 htb rate 20mbit ceil 100mbit prio 2
tc filter add dev eth0 protocol ip parent 1:0 prio 1 u32 match ip dport 22 0xffff flowid 1:10
tc filter add dev eth0 protocol ip parent 1:0 prio 2 u32 match ip dport 80 0xffff flowid 1:10
Questions to answer:
- What gets 80mbit guaranteed? What gets 20mbit?
- What happens when 1:10 isn’t using its bandwidth?
- Which ports are high priority?
- What traffic uses class 1:20?
The Interview Questions They’ll Ask
Prepare to answer these:
- “How would you rate limit a specific application or IP?”
- “What is a qdisc and why does Linux have them?”
- “How would you simulate network latency for testing?”
- “What’s the difference between TBF and HTB?”
- “How do you limit incoming (ingress) traffic?”
- “How would you prioritize SSH over web traffic?”
- “What is netem and when would you use it?”
Hints in Layers
Hint 1: Starting Point
tc qdisc show dev eth0 shows current configuration. tc qdisc del dev eth0 root clears everything.
Hint 2: Simple Rate Limit
tc qdisc add dev eth0 root tbf rate 10mbit burst 32kbit latency 400ms - simple bandwidth limit.
Hint 3: netem for Simulation
tc qdisc add dev eth0 root netem delay 100ms 20ms loss 1% - adds delay, jitter, and loss.
Hint 4: Ingress with IFB
modprobe ifb
ip link set dev ifb0 up
tc qdisc add dev eth0 ingress
tc filter add dev eth0 parent ffff: protocol ip u32 match u32 0 0 flowid 1:1 action mirred egress redirect dev ifb0
# Now apply rules to ifb0 for ingress limiting
Books That Will Help
| Topic | Book | Chapter |
|---|---|---|
| tc fundamentals | LARTC HOWTO at lartc.org | Full document |
| HTB | HTB user guide (online) | Full document |
| netem | man tc-netem | Man page |
| Traffic shaping | “Linux for Networking Professionals” (O’Reilly) | Ch. 8 |
Implementation Hints
netem combinations for realistic profiles:
# Simple latency
tc qdisc add dev eth0 root netem delay 100ms
# Latency with jitter (normal distribution)
tc qdisc add dev eth0 root netem delay 100ms 20ms distribution normal
# Packet loss
tc qdisc add dev eth0 root netem loss 5%
# Corrupt/duplicate/reorder
tc qdisc add dev eth0 root netem corrupt 1% duplicate 1% reorder 25% 50%
# Combined with rate limiting (netem as child of HTB)
tc qdisc add dev eth0 root handle 1: htb default 10
tc class add dev eth0 parent 1: classid 1:10 htb rate 5mbit
tc qdisc add dev eth0 parent 1:10 handle 10: netem delay 100ms 50ms loss 2%
Useful commands:
tc qdisc show dev eth0 # Show qdiscs
tc class show dev eth0 # Show classes
tc filter show dev eth0 # Show filters
tc -s qdisc show dev eth0 # With statistics
tc qdisc del dev eth0 root # Clear all
Learning milestones:
- You can add/remove qdiscs → Basic tc operations
- You can limit bandwidth → TBF/HTB understanding
- You can simulate conditions → netem mastery
- You can classify traffic → Full tc understanding
Project 13: Network Troubleshooting Wizard
- File: LEARN_LINUX_NETWORKING_TOOLS.md
- Main Programming Language: Bash + Python
- Alternative Programming Languages: Go
- Coolness Level: Level 3: Genuinely Clever
- Business Potential: 3. The “Service & Support” Model
- Difficulty: Level 3: Advanced
- Knowledge Area: Network Diagnostics / Troubleshooting
- Software or Tool: All tools combined
- Main Book: “TCP/IP Illustrated, Volume 1” by W. Richard Stevens
What you’ll build: An intelligent troubleshooting system that, given a symptom (“can’t reach website”, “connection slow”, “SSH keeps dropping”), runs a systematic diagnostic workflow using all the tools you’ve learned and produces a diagnosis with recommended fixes.
Why it’s the integration project: This project ties everything together. It’s not about one tool—it’s about knowing WHEN to use WHICH tool and HOW to interpret results in context. This is how experts actually troubleshoot.
Core challenges you’ll face:
- Building a decision tree → maps to systematic troubleshooting methodology
- Interpreting tool outputs programmatically → maps to automation
- Correlating findings across tools → maps to root cause analysis
- Providing actionable recommendations → maps to practical expertise
Key Concepts:
- Network Layer Isolation: Troubleshooting layer by layer
- Common Failure Patterns: What breaks and why
- Tool Selection: Which tool for which problem
Difficulty: Advanced Time estimate: 2 weeks Prerequisites: All previous projects completed; broad familiarity with all tools
Real World Outcome
$ sudo ./netwizard.sh "Can't access https://example.com"
╔══════════════════════════════════════════════════════════════════════╗
║ NETWORK TROUBLESHOOTING WIZARD ║
║ Symptom: Can't access https://example.com ║
╠══════════════════════════════════════════════════════════════════════╣
PHASE 1: LOCAL NETWORK CHECK
─────────────────────────────────────────────────────────────────────
[✓] Interface eth0 is UP with IP 192.168.1.100/24
[✓] Default gateway 192.168.1.1 is reachable (ping: 1ms)
[✓] DNS servers configured: 8.8.8.8, 8.8.4.4
PHASE 2: DNS RESOLUTION
─────────────────────────────────────────────────────────────────────
[✓] example.com resolves to 93.184.216.34 (23ms via 8.8.8.8)
PHASE 3: CONNECTIVITY TEST
─────────────────────────────────────────────────────────────────────
[✓] ICMP ping to 93.184.216.34: 4/5 packets received, 12ms avg
[✗] TCP connect to 93.184.216.34:443: TIMEOUT after 5s
PHASE 4: PATH ANALYSIS
─────────────────────────────────────────────────────────────────────
$ traceroute 93.184.216.34
1 192.168.1.1 1ms
2 10.0.0.1 12ms
3 * * *
4 * * *
5 * * *
[!] Path appears blocked after hop 2
PHASE 5: FIREWALL CHECK
─────────────────────────────────────────────────────────────────────
[!] Local iptables OUTPUT chain has rule:
-A OUTPUT -p tcp --dport 443 -j DROP
THIS IS BLOCKING YOUR CONNECTION!
DIAGNOSIS:
═══════════════════════════════════════════════════════════════════════
Root Cause: LOCAL FIREWALL BLOCKING HTTPS
The iptables OUTPUT chain contains a rule that drops all
TCP traffic to port 443 (HTTPS).
Evidence:
- DNS works (name resolution succeeds)
- ICMP ping works (basic IP connectivity OK)
- TCP to port 443 times out (blocked at local firewall)
- iptables shows explicit DROP rule for port 443
═══════════════════════════════════════════════════════════════════════
RECOMMENDED FIX:
─────────────────────────────────────────────────────────────────────
Remove the blocking rule:
sudo iptables -D OUTPUT -p tcp --dport 443 -j DROP
Or insert an allow rule before it:
sudo iptables -I OUTPUT 1 -p tcp --dport 443 -j ACCEPT
After fixing, verify with:
curl -v https://example.com
─────────────────────────────────────────────────────────────────────
The Core Question You’re Answering
“Given a network symptom, how do I systematically identify the root cause?”
This is what separates experts from novices. Experts have a mental model and process. They know to check layer by layer, tool by tool, and can correlate findings. This project codifies that expertise.
Concepts You Must Understand First
Stop and research these before coding:
- Layer-by-Layer Troubleshooting
- Physical → Data Link → Network → Transport → Application
- What breaks at each layer?
- How do you test each layer?
- Common Failure Patterns
- DNS failures: Works sometimes, fails others
- Firewall blocks: ICMP works, TCP doesn’t
- Routing issues: Some destinations work, others don’t
- MTU problems: Small packets work, large don’t
- Tool Selection Matrix
- Can’t ping → check route, interface, ARP
- DNS fails → check resolv.conf, dig, resolvectl
- Connection slow → check latency, bandwidth, packet loss
Questions to Guide Your Design
Before implementing, think through these:
- Workflow Design
- What order should tests run?
- How do you skip tests based on earlier results?
- When do you have enough evidence for diagnosis?
- Output Interpretation
- How do you programmatically detect failures in tool output?
- How do you correlate findings across tools?
- How do you rank possible causes?
- Recommendations
- How do you generate actionable fix commands?
- How do you handle multiple possible causes?
- How do you verify the fix worked?
Thinking Exercise
Design the Diagnostic Tree
For the symptom “Can’t SSH to remote server”, design the diagnostic workflow:
Start
│
├─ Is local network up?
│ ├─ No → Check interface, cable, DHCP
│ └─ Yes ↓
│
├─ Does DNS resolve for host?
│ ├─ No → Check resolv.conf, dig
│ └─ Yes ↓
│
├─ Can we ping the host?
│ ├─ No → traceroute, routing, firewall
│ └─ Yes ↓
│
├─ Can we connect to port 22?
│ ├─ No → [What checks here?]
│ └─ Yes ↓
│
└─ SSH connects but fails auth
└─ [What checks here?]
Fill in the question marks with specific checks and tools
The Interview Questions They’ll Ask
Prepare to answer these:
- “Walk me through how you’d troubleshoot a user who can’t reach a website.”
- “What’s your systematic approach to network troubleshooting?”
- “Ping works but HTTP doesn’t. What could cause this?”
- “How do you distinguish between DNS issues and connectivity issues?”
- “The network is slow. How do you identify the bottleneck?”
- “How do you troubleshoot intermittent connectivity?”
- “What tools do you reach for first when debugging network issues?”
Hints in Layers
Hint 1: Starting Point
Build the check functions first: check_interface(), check_dns(), check_connectivity(), etc.
Hint 2: Output Parsing
Use command return codes plus grep patterns. ping -c 1 host && echo "reachable" or check specific output patterns.
Hint 3: Decision Logic Simple cascading: if earlier checks fail, later checks may be meaningless. Skip them with clear explanation.
Hint 4: Common Patterns
- ICMP works, TCP doesn’t → firewall
- Some sites work, others don’t → DNS or specific routing
- Works locally, fails remotely → NAT, external firewall
- Slow but works → congestion, packet loss, high latency
Implementation Hints
Decision tree structure (pseudo-code):
def diagnose(symptom):
# Phase 1: Local
if not check_interface_up():
return "Interface down"
if not check_default_route():
return "No default route"
if not check_gateway_reachable():
return "Gateway unreachable"
# Phase 2: DNS
target_ip = resolve_dns(symptom.target)
if not target_ip:
return diagnose_dns_failure()
# Phase 3: Connectivity
if not ping_works(target_ip):
return diagnose_icmp_failure(target_ip)
if not tcp_works(target_ip, symptom.port):
return diagnose_tcp_failure(target_ip, symptom.port)
# Phase 4: Application
return diagnose_application_layer()
Key detection patterns:
# Interface up?
ip link show eth0 | grep -q "state UP"
# Route exists?
ip route get 8.8.8.8 | grep -q "via"
# DNS works?
dig +short example.com | grep -E '^[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+$'
# TCP port open?
timeout 5 bash -c "echo >/dev/tcp/93.184.216.34/443" 2>/dev/null
# Firewall blocking?
iptables -L OUTPUT -n | grep -q "DROP.*443"
Learning milestones:
- You have a systematic approach → Troubleshooting methodology
- You can automate diagnostics → Tool integration
- You can correlate findings → Root cause analysis
- You provide actionable fixes → Complete solutions
Project 14: Legacy to Modern Tool Migration Guide
- File: LEARN_LINUX_NETWORKING_TOOLS.md
- Main Programming Language: Bash (shell scripting)
- Alternative Programming Languages: N/A
- Coolness Level: Level 2: Practical but Forgettable
- Business Potential: 2. The “Micro-SaaS / Pro Tool”
- Difficulty: Level 1: Beginner
- Knowledge Area: Tool Migration / Compatibility
- Software or Tool: ifconfig, route, netstat → ip, ss
- Main Book: “How Linux Works, 3rd Edition” by Brian Ward
What you’ll build: A tool that shows side-by-side comparisons of legacy commands (ifconfig, route, netstat, arp) with their modern equivalents (ip, ss), explains the differences, and can translate scripts from old to new syntax.
Why it matters: You’ll encounter legacy systems and scripts using old tools. Understanding both ensures you can work anywhere and migrate systems to modern tooling.
Core challenges you’ll face:
- Understanding what deprecated tools did → maps to historical knowledge
- Mapping to modern equivalents → maps to iproute2 suite
- Handling edge cases in translation → maps to subtle differences
- Updating legacy scripts → maps to practical migration
Key Concepts:
- net-tools vs iproute2: “How Linux Works” Ch. 9 - Brian Ward
- Command equivalences: Official iproute2 documentation
Difficulty: Beginner Time estimate: Weekend Prerequisites: Projects 1-4 completed; familiarity with both tool sets
Real World Outcome
$ ./legacy-modern.sh translate "ifconfig eth0 192.168.1.100 netmask 255.255.255.0 up"
╔══════════════════════════════════════════════════════════════════════╗
║ LEGACY → MODERN TRANSLATION ║
╠══════════════════════════════════════════════════════════════════════╣
Legacy Command:
ifconfig eth0 192.168.1.100 netmask 255.255.255.0 up
Modern Equivalent:
ip addr add 192.168.1.100/24 dev eth0
ip link set eth0 up
Explanation:
- ifconfig sets IP and brings interface up in one command
- Modern 'ip' separates address assignment (ip addr) from link state (ip link)
- netmask 255.255.255.0 converts to CIDR /24
$ ./legacy-modern.sh reference
┌─────────────────────────────────────────────────────────────────────┐
│ COMMAND REFERENCE: net-tools → iproute2 │
├─────────────────────────────────────────────────────────────────────┤
│ Legacy (net-tools) │ Modern (iproute2) │
├────────────────────────────┼───────────────────────────────────────┤
│ ifconfig │ ip addr, ip link │
│ ifconfig eth0 │ ip addr show dev eth0 │
│ ifconfig eth0 up │ ip link set eth0 up │
│ ifconfig eth0 192.168.1.1 │ ip addr add 192.168.1.1/24 dev eth0 │
├────────────────────────────┼───────────────────────────────────────┤
│ route │ ip route │
│ route -n │ ip route show │
│ route add default gw X │ ip route add default via X │
│ route add -net 10.0.0.0/8 │ ip route add 10.0.0.0/8 via X │
│ gw X │ │
├────────────────────────────┼───────────────────────────────────────┤
│ netstat │ ss │
│ netstat -tuln │ ss -tuln │
│ netstat -anp │ ss -anp │
│ netstat -r │ ip route │
│ netstat -i │ ip -s link │
├────────────────────────────┼───────────────────────────────────────┤
│ arp │ ip neigh │
│ arp -n │ ip neigh show │
│ arp -d 192.168.1.1 │ ip neigh del 192.168.1.1 dev eth0 │
└─────────────────────────────────────────────────────────────────────┘
KEY DIFFERENCES:
- ip commands are more consistent and scriptable
- ip supports JSON output (-json flag)
- ss is faster than netstat (reads kernel data directly)
- ip commands modify, legacy commands often just display
- ifconfig doesn't show all addresses (missing secondary IPs)
Implementation Hints
Common translations:
# Show all interfaces
ifconfig → ip addr show
ifconfig -a → ip link show
# Show specific interface
ifconfig eth0 → ip addr show dev eth0
# Set IP address
ifconfig eth0 192.168.1.1 netmask 255.255.255.0
→ ip addr add 192.168.1.1/24 dev eth0
# Bring interface up/down
ifconfig eth0 up → ip link set eth0 up
ifconfig eth0 down → ip link set eth0 down
# Show routing table
route -n → ip route show
# Add route
route add default gw 192.168.1.1
→ ip route add default via 192.168.1.1
route add -net 10.0.0.0/8 gw 192.168.1.254
→ ip route add 10.0.0.0/8 via 192.168.1.254
# Delete route
route del default → ip route del default
# Show sockets
netstat -tuln → ss -tuln
netstat -anp → ss -anp
# Show ARP cache
arp -n → ip neigh show
Learning milestones:
- You know both tool sets → Full compatibility
- You can translate commands → Migration ability
- You understand the differences → Deeper knowledge
- You can update scripts → Practical value
Project 15: Real-Time Network Security Monitor
- File: LEARN_LINUX_NETWORKING_TOOLS.md
- Main Programming Language: Bash + Python
- Alternative Programming Languages: Go, Rust
- Coolness Level: Level 4: Hardcore Tech Flex
- Business Potential: 3. The “Service & Support” Model
- Difficulty: Level 3: Advanced
- Knowledge Area: Network Security / Intrusion Detection
- Software or Tool: tcpdump, ss, iptables, journalctl
- Main Book: “The Practice of Network Security Monitoring” by Richard Bejtlich
What you’ll build: A real-time security monitoring dashboard that detects suspicious network activity: port scans, brute force attempts, unusual outbound connections, and firewall violations—combining packet capture, connection monitoring, and log analysis.
Why it integrates security tools: Network security requires correlating data from multiple sources. This project brings together packet analysis, connection state, firewall logs, and system logs to detect and alert on potential threats.
Core challenges you’ll face:
- Detecting port scans from packet data → maps to pattern recognition
- Identifying brute force attempts → maps to rate analysis
- Correlating across data sources → maps to security analysis
- Alerting without false positives → maps to tuning
Difficulty: Advanced Time estimate: 2 weeks Prerequisites: Projects 5, 6, and 10 completed; security awareness
Real World Outcome
$ sudo ./netsec-monitor.sh
╔══════════════════════════════════════════════════════════════════════╗
║ NETWORK SECURITY MONITOR ║
║ Started: 2024-12-22 14:00:00 ║
╠══════════════════════════════════════════════════════════════════════╣
┌─ CURRENT CONNECTIONS ───────────────────────────────────────────────┐
│ Listening services: 12 │
│ Established connections: 47 │
│ External connections: 23 │
│ Connections from unusual geo: 3 ⚠ │
└─────────────────────────────────────────────────────────────────────┘
┌─ LIVE ALERTS ───────────────────────────────────────────────────────┐
│ │
│ 14:32:15 [HIGH] PORT SCAN DETECTED │
│ Source: 45.227.253.98 │
│ Ports probed: 22, 23, 25, 80, 443, 3389 (6 in 10 seconds) │
│ Action: Added to blocklist │
│ │
│ 14:28:43 [MED] SSH BRUTE FORCE ATTEMPT │
│ Source: 185.220.101.45 │
│ Failed attempts: 47 in last 5 minutes │
│ Action: Blocked via iptables │
│ │
│ 14:15:22 [LOW] UNUSUAL OUTBOUND CONNECTION │
│ Process: curl (PID 12345, user: www-data) │
│ Destination: 103.25.67.89:4444 (Unknown, Eastern Europe) │
│ Action: Logged for review │
│ │
└─────────────────────────────────────────────────────────────────────┘
┌─ STATISTICS (last hour) ────────────────────────────────────────────┐
│ Firewall blocks: 2,847 │
│ Port scan attempts: 12 (from 8 unique IPs) │
│ SSH brute force: 342 attempts (from 5 IPs, all blocked) │
│ New outbound conns: 156 (23 to unusual destinations) │
│ Bytes in: 2.3 GB │
│ Bytes out: 156 MB │
└─────────────────────────────────────────────────────────────────────┘
┌─ TOP BLOCKED IPs ───────────────────────────────────────────────────┐
│ IP Address │ Blocks │ First Seen │ Reason │
├──────────────────┼────────┼────────────┼───────────────────────────┤
│ 45.227.253.98 │ 1,247 │ 14:32:15 │ Port scan │
│ 185.220.101.45 │ 892 │ 14:28:43 │ SSH brute force │
│ 103.251.167.20 │ 456 │ 13:45:22 │ Database scan │
│ 91.199.212.52 │ 252 │ 14:02:11 │ Web scanner │
└─────────────────────────────────────────────────────────────────────┘
Implementation Hints
Detection techniques:
# Port scan detection (many ports from one IP in short time)
tcpdump -i eth0 -n 'tcp[tcpflags] & tcp-syn != 0' | \
awk '{print $3}' | cut -d. -f1-4 | uniq -c | \
awk '$1 > 10 {print "SCAN:", $2}'
# Brute force detection (many failed auth from one IP)
journalctl -u sshd --since "5 minutes ago" | \
grep "Failed password" | \
awk '{print $NF}' | sort | uniq -c | sort -rn | \
awk '$1 > 10 {print "BRUTE:", $2}'
# Unusual outbound connections
ss -tnp state established '( dport != :80 and dport != :443 )' | \
grep -v "127.0.0.1"
# Firewall blocks count
journalctl -k --since "1 hour ago" | \
grep -c "DROP\|REJECT"
Automated blocking:
# Block IP with iptables
block_ip() {
iptables -I INPUT 1 -s "$1" -j DROP
echo "$(date) Blocked $1" >> /var/log/netsec-blocks.log
}
Learning milestones:
- You can detect port scans → Pattern recognition
- You can identify brute force → Rate analysis
- You can correlate events → Security analysis
- You can automate response → Active defense
Project Comparison Table
| # | Project Name | Main Tools | Difficulty | Time | Depth of Understanding | Fun Factor |
|---|---|---|---|---|---|---|
| 1 | Network Interface Inspector | ip, ifconfig, ethtool, nmcli | Beginner | Weekend | ⭐⭐⭐ Foundation | ⭐⭐⭐ |
| 2 | Connectivity Diagnostic Suite | ping, traceroute, mtr | Beginner | Weekend | ⭐⭐⭐⭐ Core diagnostics | ⭐⭐⭐⭐ |
| 3 | DNS Deep Dive Tool | dig, nslookup, resolvectl | Beginner | 1 week | ⭐⭐⭐⭐ DNS mastery | ⭐⭐⭐ |
| 4 | Socket State Analyzer | ss, netstat, lsof | Intermediate | 1 week | ⭐⭐⭐⭐⭐ Critical | ⭐⭐⭐ |
| 5 | Live Packet Capture Dashboard | tcpdump | Intermediate | 1-2 weeks | ⭐⭐⭐⭐⭐ Deep insight | ⭐⭐⭐⭐⭐ |
| 6 | Firewall Rule Auditor | iptables, nft, ufw | Intermediate | 1-2 weeks | ⭐⭐⭐⭐⭐ Security | ⭐⭐⭐⭐ |
| 7 | Routing Table Explorer | ip route, ip neigh, arp | Intermediate | 1 week | ⭐⭐⭐⭐ Routing knowledge | ⭐⭐⭐ |
| 8 | Bandwidth Monitor | iperf3, nload, iftop | Beginner | Weekend | ⭐⭐⭐ Performance | ⭐⭐⭐⭐⭐ |
| 9 | HTTP/API Testing Suite | curl, wget | Beginner | Weekend | ⭐⭐⭐⭐ Web/API | ⭐⭐⭐⭐ |
| 10 | Network Log Analyzer | dmesg, journalctl | Intermediate | 1 week | ⭐⭐⭐⭐ Troubleshooting | ⭐⭐⭐ |
| 11 | Network Namespace Laboratory | ip netns | Advanced | 2 weeks | ⭐⭐⭐⭐⭐ Container networking | ⭐⭐⭐⭐⭐ |
| 12 | Traffic Control Simulator | tc | Advanced | 2 weeks | ⭐⭐⭐⭐⭐ QoS/shaping | ⭐⭐⭐⭐ |
| 13 | Network Troubleshooting Wizard | All tools | Advanced | 3-4 weeks | ⭐⭐⭐⭐⭐ Integration | ⭐⭐⭐⭐⭐ |
| 14 | Legacy to Modern Migration | ifconfig, route, netstat | Beginner | Weekend | ⭐⭐⭐ Compatibility | ⭐⭐ |
| 15 | Real-Time Security Monitor | tcpdump, ss, iptables | Advanced | 2 weeks | ⭐⭐⭐⭐⭐ Security | ⭐⭐⭐⭐⭐ |
Recommendation
For Beginners
Start with Projects 1, 2, and 8 in this order:
- Project 1 (Network Interface Inspector) - Foundation. You need to understand interfaces before anything else.
- Project 2 (Connectivity Diagnostic Suite) - Core troubleshooting skills you’ll use every day.
- Project 8 (Bandwidth Monitor) - Visually rewarding, builds confidence.
Then proceed to Projects 3, 9, and 14.
For Intermediate Users
Start with Projects 4, 5, and 6:
- Project 4 (Socket State Analyzer) - Understanding sockets unlocks everything.
- Project 5 (Packet Capture Dashboard) - tcpdump is the ultimate diagnostic tool.
- Project 6 (Firewall Rule Auditor) - Security knowledge that’s immediately applicable.
Then tackle Projects 7, 10, and 13.
For Advanced Users
Start with Projects 11, 12, and 15:
- Project 11 (Network Namespace Laboratory) - Critical for container/cloud work.
- Project 12 (Traffic Control Simulator) - Deep Linux networking internals.
- Project 15 (Security Monitor) - Combines everything for security ops.
Then complete Project 13 (Troubleshooting Wizard) as the capstone.
Final Overall Project: Enterprise Network Operations Center
- File: LEARN_LINUX_NETWORKING_TOOLS.md
- Main Programming Language: Python + Bash
- Alternative Programming Languages: Go, Rust
- Coolness Level: Level 5: Pure Magic (Super Cool)
- Business Potential: 4. The “Open Core” Infrastructure
- Difficulty: Level 5: Master
- Knowledge Area: Full Network Stack / Operations
- Software or Tool: All 30+ tools integrated
- Main Book: “The Practice of System and Network Administration” by Limoncelli, Hogan, and Chalup
What you’ll build: A comprehensive Network Operations Center (NOC) that combines ALL the tools you’ve learned into a single, unified dashboard. Real-time monitoring of interfaces, connections, packets, firewall status, bandwidth, DNS health, routing, namespaces, and security—with automated alerting, historical trends, and incident response playbooks.
Why this is the ultimate capstone: This project forces you to integrate everything. You’re not just using tools—you’re orchestrating them into a coherent operational view. This is what production network monitoring looks like.
Core challenges you’ll face:
- Data aggregation from 30+ sources → maps to integration architecture
- Real-time processing without overload → maps to performance engineering
- Alert correlation to reduce noise → maps to operational intelligence
- Historical trending and anomaly detection → maps to data analysis
- Runbook automation for common issues → maps to operational maturity
- Multi-host/multi-namespace monitoring → maps to scale challenges
Difficulty: Master Time estimate: 1-2 months Prerequisites: All previous projects completed; Python proficiency; understanding of distributed systems
Real World Outcome
$ sudo ./noc-dashboard.sh
╔══════════════════════════════════════════════════════════════════════════════╗
║ NETWORK OPERATIONS CENTER - FULL STACK VIEW ║
║ Host: production-web-01.corp.local ║
║ Updated: 2024-12-22 14:35:22 UTC ║
╚══════════════════════════════════════════════════════════════════════════════╝
┌─ INFRASTRUCTURE HEALTH ──────────────────────────────────────────────────────┐
│ │
│ ┌─ INTERFACES ─┐ ┌─ CONNECTIVITY ─┐ ┌─ DNS ─────────┐ ┌─ FIREWALL ──┐ │
│ │ eth0: UP ✓ │ │ Gateway: OK ✓ │ │ Primary: OK ✓│ │ Rules: 127 │ │
│ │ eth1: UP ✓ │ │ Internet: OK ✓ │ │ Backup: OK ✓ │ │ Drops: 2.8k │ │
│ │ docker0: UP │ │ CDN: OK ✓ │ │ Latency: 3ms │ │ Blocks: 423 │ │
│ │ veth*: 12 │ │ DB: OK ✓ │ │ Cache: 94% │ │ NAT: Active │ │
│ └──────────────┘ └────────────────┘ └──────────────┘ └─────────────┘ │
│ │
└──────────────────────────────────────────────────────────────────────────────┘
┌─ LIVE TRAFFIC ───────────────────────────────────────────────────────────────┐
│ │
│ IN: ████████████████████░░░░░░░░░░ 847 Mbps (84% of 1 Gbps) │
│ OUT: ████████░░░░░░░░░░░░░░░░░░░░░░ 312 Mbps (31% of 1 Gbps) │
│ │
│ Top Talkers (In): Top Talkers (Out): │
│ 142.250.80.46 (google): 234 Mbps 10.0.1.50 (db-master): 156 Mbps │
│ 151.101.1.69 (cdn): 189 Mbps 10.0.1.51 (db-slave): 89 Mbps │
│ 13.107.42.12 (azure): 123 Mbps 54.231.0.1 (s3): 45 Mbps │
│ │
└──────────────────────────────────────────────────────────────────────────────┘
┌─ CONNECTION STATE ───────────────────────────────────────────────────────────┐
│ │
│ LISTEN: 28 ESTABLISHED: 1,247 TIME_WAIT: 89 CLOSE_WAIT: 3 │
│ SYN_SENT: 2 SYN_RECV: 0 FIN_WAIT: 12 CLOSING: 0 │
│ │
│ ┌─ Top Ports ────────────────────┐ ┌─ By Process ────────────────────────┐ │
│ │ :443 → 892 connections │ │ nginx → 723 conns (58%) │ │
│ │ :80 → 234 connections │ │ postgres → 289 conns (23%) │ │
│ │ :5432 → 89 connections │ │ redis → 156 conns (13%) │ │
│ │ :6379 → 32 connections │ │ other → 79 conns (6%) │ │
│ └────────────────────────────────┘ └─────────────────────────────────────┘ │
│ │
└──────────────────────────────────────────────────────────────────────────────┘
┌─ NETWORK NAMESPACES ─────────────────────────────────────────────────────────┐
│ │
│ Container Network Status: │
│ ├─ ns-web-frontend (3 containers): ✓ 47 connections, 12 Mbps │
│ ├─ ns-api-backend (5 containers): ✓ 234 connections, 89 Mbps │
│ ├─ ns-database (2 containers): ✓ 89 connections, 23 Mbps │
│ └─ ns-monitoring (4 containers): ✓ 12 connections, 2 Mbps │
│ │
│ Inter-namespace traffic: 156 Mbps across 567 flows │
│ │
└──────────────────────────────────────────────────────────────────────────────┘
┌─ ROUTING & ARP ──────────────────────────────────────────────────────────────┐
│ │
│ Default Gateway: 10.0.0.1 via eth0 (latency: 0.3ms) ✓ │
│ Routes: 47 (12 static, 35 container-injected) │
│ ARP Cache: 234 entries (STALE: 12, REACHABLE: 222) │
│ │
│ ⚠ New ARP entries (last 5 min): 3 - verify if expected │
│ │
└──────────────────────────────────────────────────────────────────────────────┘
┌─ SECURITY STATUS ────────────────────────────────────────────────────────────┐
│ │
│ Firewall: ACTIVE Port Scans (1h): 23 blocked │
│ Fail2ban: ACTIVE (12 jails) Brute Force (1h): 847 blocked │
│ Unusual Outbound: 0 ✓ Geo-blocked: 156 attempts │
│ │
│ Recent Blocks: │
│ 14:32:15 45.227.253.98 Port scan (22,80,443,3389) → BLOCKED │
│ 14:28:43 185.220.101.45 SSH brute (47 attempts) → BLOCKED │
│ 14:15:22 103.251.167.20 DB scan (3306,5432) → BLOCKED │
│ │
└──────────────────────────────────────────────────────────────────────────────┘
┌─ ACTIVE ALERTS ──────────────────────────────────────────────────────────────┐
│ │
│ ● [WARN] High connection count on :443 (892 > 800 threshold) │
│ Duration: 12 minutes | Runbook: high-conn-nginx │
│ │
│ ○ [INFO] New container namespace detected: ns-canary-deploy │
│ Auto-added to monitoring | 2 containers | 0 connections │
│ │
└──────────────────────────────────────────────────────────────────────────────┘
┌─ HISTORICAL TRENDS (24h) ────────────────────────────────────────────────────┐
│ │
│ Connections: ▂▂▃▃▄▅▅▆▇▇█████████████▇▇▆▅▄▃▂▂ Peak: 14:00 (1,892) │
│ Bandwidth In: ▁▁▂▂▃▄▄▅▆▇▇████████████▇▆▅▄▃▂▁▁ Peak: 15:30 (987 Mbps) │
│ Errors: ▁▁▁▁▁▁▁▂▁▁▁▁▁▁▁▃▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁ Spike: 16:45 (timeout) │
│ │
└──────────────────────────────────────────────────────────────────────────────┘
Commands: [r]efresh | [d]rill-down | [a]lerts | [h]istory | [t]roubleshoot | [q]uit
The Core Question You’re Answering
“How do all these tools work together to give complete visibility into network operations?”
This is the question that separates tool users from network operators. Individual tools give fragments of information. This project teaches you to synthesize them into operational intelligence.
Concepts You Must Understand First
Stop and research these before coding:
- Tool Orchestration
- How do you run multiple monitoring processes concurrently?
- How do you aggregate data from different formats?
- What’s the refresh rate vs. accuracy tradeoff?
- Reference: “Unix and Linux System Administration Handbook” Ch. 15
- Data Correlation
- How do you match a packet (tcpdump) to a socket (ss) to a process (lsof)?
- How do you track flows across namespaces?
- Reference: “The Practice of Network Security Monitoring” Ch. 8
- Alerting Philosophy
- What makes a good alert vs. noise?
- How do you set thresholds that adapt to patterns?
- Reference: “Site Reliability Engineering” Ch. 29
- Operational Maturity
- What’s a runbook and why does it matter?
- How do you build institutional knowledge into automation?
- Reference: “The Practice of System and Network Administration” Ch. 8
Questions to Guide Your Design
Before implementing, think through these:
- Data Collection
- Which tools need to run continuously vs. on-demand?
- How do you avoid overwhelming the system with monitoring overhead?
- What’s your sampling strategy for high-volume data?
- Display Architecture
- How do you fit 30+ data sources into one coherent view?
- What’s the information hierarchy? What’s most important?
- How do you handle drill-down without losing context?
- Alert Design
- What’s the difference between WARNING, CRITICAL, and INFO?
- How do you avoid alert fatigue?
- When should alerts auto-resolve?
- Integration Depth
- Can you click on a connection and see its packet trace?
- Can you click on an IP and see all related events?
- How do you provide context for every data point?
Thinking Exercise
Design Your Dashboard Layout
Before coding, sketch answers to these:
-
Information Priority: List the 5 most important things a network operator needs to know at a glance.
-
Tool Mapping: For each dashboard panel, which tools provide the data?
┌─────────────────────────────────────────────────────────────┐
│ Dashboard Panel │ Tools Used │
├─────────────────────────┼────────────────────────────────────┤
│ Interface Status │ ? │
│ Connectivity Health │ ? │
│ Active Connections │ ? │
│ Traffic Volume │ ? │
│ Security Events │ ? │
│ Namespace Overview │ ? │
└─────────────────────────┴────────────────────────────────────┘
- Alert Thresholds: For 3 metrics, define what triggers WARNING vs CRITICAL.
The Interview Questions They’ll Ask
Prepare to answer these:
- “Describe how you would monitor a production network. What tools would you use and why?”
- “Walk me through troubleshooting a user complaint about ‘the network is slow.’”
- “How would you detect a compromised server exfiltrating data?”
- “Explain how you’d set up monitoring for a containerized microservices environment.”
- “What metrics matter most for network health? How would you collect them?”
- “How do you balance monitoring comprehensiveness with system overhead?”
- “Describe your approach to reducing alert fatigue.”
Hints in Layers
Hint 1: Start with the Shell
Create a single terminal view that refreshes. Use tput for cursor positioning and colors.
Hint 2: Background Collectors
Run tools like tcpdump, ss, and journalctl as background processes writing to temp files. Your main loop reads and aggregates these.
Hint 3: Data Structures Create associative arrays (bash) or dictionaries (Python) keyed by IP address. This lets you correlate: “IP 1.2.3.4 has 50 connections AND triggered 3 firewall blocks AND appears in SSH logs.”
Hint 4: Modular Design
Create functions for each panel. show_interfaces(), show_connections(), show_security(). This lets you test and develop independently.
Hint 5: Configuration
Define thresholds in a config file. This makes tuning easy: CONN_WARN=800, CONN_CRIT=1000, SCAN_THRESHOLD=10.
Books That Will Help
| Topic | Book | Chapter |
|---|---|---|
| Monitoring philosophy | “The Practice of System and Network Administration” | Ch. 22 |
| Dashboard design | “Information Dashboard Design” by Stephen Few | Entire book |
| Alert engineering | “Site Reliability Engineering” (Google) | Ch. 29 |
| Security monitoring | “The Practice of Network Security Monitoring” | Ch. 7-9 |
| Linux integration | “How Linux Works” by Brian Ward | Ch. 9-10 |
| Network architecture | “TCP/IP Illustrated, Vol. 1” | Reference as needed |
Implementation Hints
Architecture approach:
┌────────────────────────────────────────────────────────────┐
│ NOC DASHBOARD │
├────────────────────────────────────────────────────────────┤
│ │
│ ┌─────────────┐ ┌─────────────┐ ┌──────────────┐ │
│ │ Interface │ │ Connection │ │ Packet │ │
│ │ Collector │ │ Collector │ │ Collector │ │
│ │ │ │ │ │ │ │
│ │ ip addr │ │ ss -tunaep │ │ tcpdump │ │
│ │ ethtool │ │ lsof -i │ │ (sampled) │ │
│ │ nmcli │ │ │ │ │ │
│ └──────┬──────┘ └──────┬──────┘ └──────┬───────┘ │
│ │ │ │ │
│ └────────┬─────────┴──────────────────┘ │
│ │ │
│ ┌─────▼─────┐ │
│ │ Aggregator│ │
│ │ & State │ │
│ └─────┬─────┘ │
│ │ │
│ ┌───────────────┴───────────────────┐ │
│ │ │ │
│ ▼ ▼ │
│ ┌──────────┐ ┌──────────┐ ┌──────────┐ │
│ │ Display │ │ Alerter │ │ History │ │
│ │ Engine │ │ │ │ Logger │ │
│ └──────────┘ └──────────┘ └──────────┘ │
│ │
└────────────────────────────────────────────────────────────┘
Data collection patterns:
# Background collectors writing to temp files
collect_interfaces() {
while true; do
ip -j addr > /tmp/noc/interfaces.json
ip -j link > /tmp/noc/links.json
sleep 5
done
}
collect_connections() {
while true; do
ss -tunaep | tail -n +2 > /tmp/noc/connections.txt
sleep 2
done
}
collect_security() {
journalctl -k -f --grep="DROP\|REJECT" >> /tmp/noc/security.log &
journalctl -u sshd -f --grep="Failed" >> /tmp/noc/ssh.log &
}
# Start all collectors as background processes
start_collectors() {
collect_interfaces &
collect_connections &
collect_security &
}
Alert correlation example:
def correlate_ip(ip_address):
"""Get all data about a single IP across all sources."""
return {
'connections': get_connections_for_ip(ip_address),
'packets': get_packet_counts(ip_address),
'firewall_blocks': get_blocks_for_ip(ip_address),
'auth_attempts': get_auth_logs(ip_address),
'geo_info': lookup_geo(ip_address),
'reputation': check_reputation(ip_address)
}
Learning milestones:
- You can monitor one host → Basic integration
- You correlate across tools → Operational intelligence
- You alert meaningfully → Production readiness
- You troubleshoot from the dashboard → Full mastery
- You extend to multiple hosts → Scale understanding
Summary
This learning path covers Linux networking tools through 16 hands-on projects (15 focused projects + 1 capstone). Here’s the complete list:
| # | Project Name | Main Tools | Main Language | Difficulty | Time Estimate |
|---|---|---|---|---|---|
| 1 | Network Interface Inspector | ip, ifconfig, ethtool, nmcli | Bash | Beginner | Weekend |
| 2 | Connectivity Diagnostic Suite | ping, traceroute, tracepath, mtr | Bash | Beginner | Weekend |
| 3 | DNS Deep Dive Tool | dig, nslookup, resolvectl | Bash + Python | Beginner | 1 week |
| 4 | Socket State Analyzer | ss, netstat, lsof | Python | Intermediate | 1 week |
| 5 | Live Packet Capture Dashboard | tcpdump | Bash + Python | Intermediate | 1-2 weeks |
| 6 | Firewall Rule Auditor | iptables, nft, ufw, firewall-cmd | Bash + Python | Intermediate | 1-2 weeks |
| 7 | Routing Table Explorer | ip route, ip neigh, arp, route | Bash | Intermediate | 1 week |
| 8 | Bandwidth Monitor & Performance Tester | iperf3, nload, iftop | Bash | Beginner | Weekend |
| 9 | HTTP/API Testing Suite | curl, wget | Bash | Beginner | Weekend |
| 10 | Network Log Analyzer | dmesg, journalctl | Bash + Python | Intermediate | 1 week |
| 11 | Network Namespace Laboratory | ip netns | Bash | Advanced | 2 weeks |
| 12 | Traffic Control Simulator | tc | Bash + Python | Advanced | 2 weeks |
| 13 | Network Troubleshooting Wizard | All tools combined | Bash + Python | Advanced | 3-4 weeks |
| 14 | Legacy to Modern Migration Guide | ifconfig, route, netstat | Bash | Beginner | Weekend |
| 15 | Real-Time Network Security Monitor | tcpdump, ss, iptables, journalctl | Bash + Python | Advanced | 2 weeks |
| 16 | Enterprise NOC (Capstone) | All 30+ tools | Python + Bash | Master | 1-2 months |
Recommended Learning Path
For beginners: Start with projects #1, #2, #8, #9, #14
- Builds fundamental understanding of interfaces, connectivity, bandwidth, HTTP, and tool evolution
- Estimated time: 2-3 weeks
For intermediate: Jump to projects #4, #5, #6, #7, #10
- Focuses on sockets, packets, firewalls, routing, and logs
- Estimated time: 5-6 weeks
For advanced: Focus on projects #11, #12, #13, #15, #16
- Covers namespaces, traffic control, full integration, security monitoring, and the capstone
- Estimated time: 2-3 months
Tools Covered
This learning path covers 31 Linux networking tools:
| Category | Tools |
|---|---|
| Interface Management | ip, ifconfig, ethtool, nmcli |
| Connectivity Testing | ping, traceroute, tracepath, mtr |
| DNS | dig, nslookup, resolvectl |
| Socket/Connection Analysis | ss, netstat, lsof |
| Packet Capture | tcpdump |
| Firewall | iptables, nft, ufw, firewall-cmd |
| Routing & ARP | ip route, ip neigh, arp, route |
| Bandwidth & Performance | iperf3, nload, iftop |
| HTTP/Downloads | curl, wget |
| Logging | dmesg, journalctl |
| Advanced | ip netns, tc |
Expected Outcomes
After completing these projects, you will:
- Diagnose any network issue using the appropriate tools
- Understand packet flow through the Linux network stack from NIC to application
- Configure and audit firewalls across multiple firewall frameworks
- Analyze connections and sockets to find resource leaks and security issues
- Capture and filter packets with tcpdump using complex BPF expressions
- Understand container networking through network namespaces and veth pairs
- Shape and control traffic using Linux traffic control (tc)
- Correlate logs with network events for comprehensive troubleshooting
- Migrate between legacy and modern tools (net-tools vs. iproute2)
- Build operational monitoring that integrates all tools into cohesive views
You’ll have built 16 working projects that demonstrate deep, practical mastery of Linux networking tools from first principles—exactly the skills that distinguish senior engineers and make you invaluable for DevOps, SRE, and network engineering roles.
Total Estimated Learning Time: 4-6 months (doing all projects thoroughly)
Key Books to Acquire:
- “TCP/IP Illustrated, Volume 1” by W. Richard Stevens
- “How Linux Works” by Brian Ward
- “The Linux Programming Interface” by Michael Kerrisk
- “The Practice of Network Security Monitoring” by Richard Bejtlich
Happy networking! Remember: Every packet has a story. Learn to read them.