Project 1: Build Your Own Network Reconnaissance Toolkit
Project 1: Build Your Own Network Reconnaissance Toolkit
Project Overview
| Attribute | Value |
|---|---|
| Difficulty | Beginner-Intermediate |
| Time Estimate | 1-2 weeks |
| Programming Language | Python |
| Primary Tool | Scapy / Raw Sockets |
| Main Book | “Black Hat Python” by Justin Seitz |
| Knowledge Area | Networking / Security |
Learning Objectives
By completing this project, you will:
- Understand TCP/IP at the packet level - Not just conceptually, but what actual bytes travel over the wire
- Implement port scanning from scratch - Learn why Nmap works the way it does by building your own
- Master DNS enumeration techniques - Discover subdomains, mail servers, and zone transfer vulnerabilities
- Develop service fingerprinting skills - Identify what’s running on open ports by analyzing banners
- Build production-quality security tools - Handle threading, timeouts, and error conditions like professional tools
The Core Question
“How do attackers discover what’s running on a network before they even attempt exploitation?”
Reconnaissance is the foundation of every penetration test. Before you can hack anything, you need to know:
- What hosts are alive on the network?
- What ports are open on each host?
- What services are running behind those ports?
- What versions of software are exposed?
By building these tools yourself, you’ll understand the mechanics of network discovery at a fundamental level—knowledge that makes you effective with tools like Nmap and helps you understand what defenders see.
Deep Theoretical Foundation
The TCP/IP Stack: Where Scanning Lives
Before writing a single line of code, you must understand how data travels across networks. The TCP/IP model has four layers, and your reconnaissance tools will operate primarily at Layer 3 (Network) and Layer 4 (Transport):
┌─────────────────────────────────────────────────────────────────────┐
│ APPLICATION LAYER │
│ HTTP, HTTPS, SSH, FTP, DNS, SMTP, MySQL │
│ "What humans interact with" │
│ │
│ Example: GET /index.html HTTP/1.1 │
└─────────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────┐
│ TRANSPORT LAYER │
│ TCP (connection-oriented) / UDP (connectionless) │
│ "How data is segmented and guaranteed delivery" │
│ │
│ TCP Header Structure: │
│ ┌────────────────┬────────────────┐ │
│ │ Source Port │ Dest Port │ (16 bits each) │
│ ├────────────────┴────────────────┤ │
│ │ Sequence Number │ (32 bits) │
│ ├──────────────────────────────────┤ │
│ │ Acknowledgment Number │ (32 bits) │
│ ├────────────────┬─────────────────┤ │
│ │ Data Offset, │ Flags (URG,ACK, │ │
│ │ Reserved │ PSH,RST,SYN,FIN)│ ◄── Port scanning uses these!│
│ ├────────────────┴─────────────────┤ │
│ │ Window Size │ (16 bits) │
│ ├──────────────────────────────────┤ │
│ │ Checksum │ Urgent Pointer │ │
│ └────────────────┴──────────────────┘ │
└─────────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────┐
│ NETWORK LAYER │
│ IP (IPv4/IPv6), ICMP, Routing │
│ "How packets find their way across networks" │
│ │
│ IPv4 Header: │
│ ┌────────────────┬────────────────┐ │
│ │ Version (4) │ IHL │ (4 bits each) │
│ ├────────────────┼────────────────┤ │
│ │ DSCP │ ECN │ │
│ ├────────────────┴────────────────┤ │
│ │ Total Length │ (16 bits) │
│ ├──────────────────────────────────┤ │
│ │ Identification │ (16 bits) │
│ ├────────────────┬─────────────────┤ │
│ │ Flags │ Fragment Offset │ │
│ ├────────────────┴─────────────────┤ │
│ │ TTL │ Protocol │ ◄── 6=TCP, 17=UDP, 1=ICMP │
│ ├────────────────┴─────────────────┤ │
│ │ Header Checksum │ │
│ ├──────────────────────────────────┤ │
│ │ Source IP Address │ (32 bits) │
│ ├──────────────────────────────────┤ │
│ │ Destination IP Address │ (32 bits) │
│ └──────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────┐
│ LINK LAYER │
│ Ethernet, WiFi, ARP │
│ "Physical transmission on local network" │
└─────────────────────────────────────────────────────────────────────┘
The TCP Three-Way Handshake: Why Port Scanning Works
TCP is a connection-oriented protocol. Before any data transfer, client and server must establish a connection using a “three-way handshake”:
Normal TCP Connection Establishment
═══════════════════════════════════
Client Server
│ │
│ SYN (Synchronize) │
│ seq=100, ack=0, flags=SYN │
├─────────────────────────────────────────────────►│
│ "I want to connect. My sequence starts at 100" │
│ │
│ SYN-ACK │
│ seq=300, ack=101, flags=SYN+ACK │
│◄─────────────────────────────────────────────────┤
│ "OK. My sequence starts at 300. │
│ I expect your next byte to be 101" │
│ │
│ ACK (Acknowledge) │
│ seq=101, ack=301, flags=ACK │
├─────────────────────────────────────────────────►│
│ "Got it. Connection established." │
│ │
│ ═══════ CONNECTION ESTABLISHED ═══════ │
│ │
Why this matters for scanning: By sending specific combinations of TCP flags, we can probe a port’s state without completing a full connection:
Port States and What They Mean
OPEN PORT (Service Listening)
─────────────────────────────
Scanner Target
│ │
│ SYN │
├─────────────────────────────────────────────────►│
│ │
│ SYN-ACK │
│◄─────────────────────────────────────────────────┤
│ "Yes, I'm listening on this port!" │
│ │
│ RST (or complete handshake) │
├─────────────────────────────────────────────────►│
│ │
Result: PORT IS OPEN
─────────────────────
CLOSED PORT (Nothing Listening)
───────────────────────────────
Scanner Target
│ │
│ SYN │
├─────────────────────────────────────────────────►│
│ │
│ RST │
│◄─────────────────────────────────────────────────┤
│ "Nothing running here. Go away!" │
│ │
Result: PORT IS CLOSED
──────────────────────
FILTERED PORT (Firewall Blocking)
─────────────────────────────────
Scanner Target
│ │
│ SYN │
├─────────────────────────────────────────────────►│
│ [Firewall]
│ ... silence ... X
│ (timeout after 1-2 seconds) │
│ │
Result: PORT IS FILTERED (probably firewalled)
──────────────────────────────────────────────
ALTERNATIVE: ICMP Unreachable
─────────────────────────────
Scanner Target
│ │
│ SYN │
├─────────────────────────────────────────────────►│
│ [Firewall]
│ ICMP Type 3 │
│◄─────────────────────────────────────────────────┤
│ "Destination port unreachable" │
│ │
Result: PORT IS FILTERED (administratively blocked)
───────────────────────────────────────────────────
Types of Port Scans
Your toolkit will implement multiple scan types, each with different tradeoffs:
| Scan Type | How It Works | Advantages | Disadvantages |
|---|---|---|---|
| TCP Connect | Complete 3-way handshake | No root required, reliable | Logged by target, slow |
| SYN Scan | Send SYN, analyze response, send RST | Faster, stealthier (no logging) | Requires root/raw sockets |
| UDP Scan | Send UDP packets, wait for response | Finds UDP services (DNS, SNMP) | Very slow, unreliable |
| ACK Scan | Send ACK, analyze RST response | Detects firewall rules | Can’t determine if port is open |
DNS: The Reconnaissance Goldmine
DNS (Domain Name System) is often overlooked as an attack surface, but it contains a wealth of information:
DNS RECORD TYPES FOR RECONNAISSANCE
═══════════════════════════════════
A Record: Maps domain to IPv4 address
example.com → 93.184.216.34
Use: Discover IP addresses of hosts
AAAA Record: Maps domain to IPv6 address
example.com → 2606:2800:220:1:248:1893:25c8:1946
Use: Often overlooked, may have different security
MX Record: Mail server for domain
example.com → mail.example.com (priority 10)
Use: Identify mail servers (common phishing targets)
NS Record: Authoritative nameservers
example.com → ns1.example.com, ns2.example.com
Use: Identify DNS infrastructure
TXT Record: Arbitrary text (SPF, DKIM, verification)
example.com → "v=spf1 include:_spf.google.com ~all"
Use: Reveals email infrastructure, third-party services
CNAME Record: Alias pointing to another domain
www.example.com → cdn.cloudflare.net
Use: Identifies CDNs, cloud providers
PTR Record: Reverse DNS (IP to hostname)
93.184.216.34 → example.com
Use: Discover internal naming conventions
Zone Transfers: When DNS Gets Really Chatty
A zone transfer (AXFR) is meant for DNS server replication, but misconfigured servers allow anyone to request the entire zone:
ZONE TRANSFER ATTACK
════════════════════
Normal Query (one record at a time):
┌─────────────────────────────────────────────────────┐
│ Attacker: "What's the IP for www.target.com?" │
│ DNS: "93.184.216.34" │
│ │
│ Attacker: "What's the IP for mail.target.com?" │
│ DNS: "93.184.216.35" │
│ │
│ (Must guess each subdomain individually) │
└─────────────────────────────────────────────────────┘
Zone Transfer (get everything at once):
┌─────────────────────────────────────────────────────┐
│ Attacker: "Give me all records for target.com" │
│ DNS: "Here's 500 records: │
│ - www.target.com → 93.184.216.34 │
│ - mail.target.com → 93.184.216.35 │
│ - admin.target.com → 93.184.216.36 │
│ - dev-internal.target.com → 10.0.0.50 │ ← Internal!
│ - jenkins.target.com → 10.0.0.51 │ ← CI/CD server!
│ - vpn.target.com → 203.0.113.10 │
│ - ..." │
└─────────────────────────────────────────────────────┘
Why this is devastating:
- Reveals ALL subdomains (including internal systems)
- Exposes infrastructure layout
- Identifies high-value targets (admin panels, CI/CD)
- No rate limiting or detection
Service Fingerprinting: What’s Actually Running?
Knowing a port is open isn’t enough—you need to know what service is running and what version:
BANNER GRABBING
═══════════════
Connect to port 22:
┌─────────────────────────────────────────────────────┐
│ Server Response: │
│ SSH-2.0-OpenSSH_8.2p1 Ubuntu-4ubuntu0.5 │
│ │
│ Information Revealed: │
│ - Protocol: SSH version 2 │
│ - Software: OpenSSH 8.2p1 │
│ - OS: Ubuntu (specific patch level) │
│ │
│ Now you can search: "OpenSSH 8.2p1 vulnerabilities"│
└─────────────────────────────────────────────────────┘
Connect to port 80:
┌─────────────────────────────────────────────────────┐
│ Send: "GET / HTTP/1.0\r\n\r\n" │
│ │
│ Response Headers: │
│ HTTP/1.1 200 OK │
│ Server: Apache/2.4.41 (Ubuntu) │
│ X-Powered-By: PHP/7.4.3 │
│ │
│ Information Revealed: │
│ - Web Server: Apache 2.4.41 │
│ - Language: PHP 7.4.3 │
│ - OS: Ubuntu │
│ │
│ Attack vector: Look for Apache 2.4.41 CVEs │
└─────────────────────────────────────────────────────┘
Project Specification
What You’re Building
A command-line toolkit with three main components:
network-recon-toolkit/
├── scanner.py # Port scanner (TCP connect + SYN scan)
├── dns_enum.py # DNS enumeration and zone transfer
├── fingerprint.py # Service fingerprinting and banner grabbing
├── output/ # Scan results (JSON, CSV, HTML)
├── wordlists/ # Subdomain wordlists
│ └── subdomains.txt
├── requirements.txt # Dependencies
└── README.md # Usage documentation
Functional Requirements
1. Port Scanner (scanner.py)
Must implement:
- TCP Connect scan (no root required)
- Scan single port, port range, or common ports list
- Configurable timeout per connection
- Multi-threaded scanning (100+ connections concurrent)
- Output: JSON with host, port, state, response time
Should implement:
- SYN scan (requires root/raw sockets)
- UDP scan for common ports (DNS, SNMP, NTP)
- Service version detection via banner grabbing
- Rate limiting to avoid detection
Example usage:
# Scan single host, common ports
python scanner.py 192.168.1.1
# Scan port range with increased threads
python scanner.py 192.168.1.1 -p 1-1000 --threads 200
# Scan network range
python scanner.py 192.168.1.0/24 -p 22,80,443
# SYN scan (requires sudo)
sudo python scanner.py 192.168.1.1 --syn
2. DNS Enumerator (dns_enum.py)
Must implement:
- Query A, AAAA, MX, NS, TXT, CNAME records
- Subdomain enumeration via wordlist
- Attempt zone transfer (AXFR)
- Reverse DNS lookups
Should implement:
- Multi-threaded subdomain bruteforcing
- Wildcard detection (*.domain.com)
- Integration with Certificate Transparency logs
- Output to structured format
Example usage:
# Basic enumeration
python dns_enum.py example.com
# Subdomain bruteforce
python dns_enum.py example.com --wordlist wordlists/subdomains.txt
# Attempt zone transfer
python dns_enum.py example.com --zone-transfer
3. Service Fingerprinter (fingerprint.py)
Must implement:
- HTTP/HTTPS banner grabbing
- SSH version detection
- FTP banner grabbing
- Generic TCP banner grabbing
- TLS certificate extraction
Should implement:
- Service-specific probes (MySQL, PostgreSQL, Redis)
- Custom fingerprint database
- Version-to-CVE mapping suggestions
Example usage:
# Fingerprint all services on host
python fingerprint.py 192.168.1.1
# Fingerprint specific ports
python fingerprint.py 192.168.1.1 -p 22,80,443
# Deep fingerprinting (all probes)
python fingerprint.py 192.168.1.1 --deep
Non-Functional Requirements
- Performance: Scan 1000 ports in under 30 seconds
- Reliability: Handle network errors gracefully, never crash
- Accuracy: Match Nmap results for same targets
- Documentation: Clear help messages, README with examples
- Ethics: Include legal disclaimer, respect robots.txt
Solution Architecture
Component Design
┌─────────────────────────────────────────────────────────────────────┐
│ RECONNAISSANCE TOOLKIT │
└─────────────────────────────────────────────────────────────────────┘
│
┌─────────────────────┼─────────────────────┐
│ │ │
▼ ▼ ▼
┌───────────────┐ ┌───────────────┐ ┌───────────────┐
│ Port Scanner │ │ DNS Enumer. │ │ Fingerprinter │
├───────────────┤ ├───────────────┤ ├───────────────┤
│ - TCP Connect │ │ - Record │ │ - Banner Grab │
│ - SYN Scan │ │ Queries │ │ - TLS Extract │
│ - UDP Scan │ │ - Subdomain │ │ - Service │
│ - Threading │ │ Bruteforce │ │ Probes │
└───────┬───────┘ │ - Zone AXFR │ └───────┬───────┘
│ └───────┬───────┘ │
│ │ │
└────────────────────┼────────────────────┘
│
▼
┌───────────────┐
│ Output Engine │
├───────────────┤
│ - JSON Export │
│ - CSV Export │
│ - HTML Report │
│ - Console │
└───────────────┘
Key Data Structures
# Port scan result
@dataclass
class PortResult:
ip: str
port: int
state: Literal["open", "closed", "filtered"]
service: Optional[str] = None
banner: Optional[str] = None
response_time_ms: Optional[float] = None
# DNS enumeration result
@dataclass
class DNSResult:
domain: str
record_type: str # A, AAAA, MX, NS, TXT, CNAME
value: str
ttl: int
# Service fingerprint
@dataclass
class ServiceInfo:
ip: str
port: int
protocol: str # tcp, udp
service_name: str # ssh, http, mysql
version: Optional[str] = None
os_guess: Optional[str] = None
cves: List[str] = field(default_factory=list)
Threading Model for Port Scanning
PRODUCER-CONSUMER PATTERN
═════════════════════════
┌──────────────┐
│ Target Queue │
│ (ports to │
│ scan) │
└──────┬───────┘
│
┌──────────────────┼──────────────────┐
│ │ │
▼ ▼ ▼
┌───────────────┐ ┌───────────────┐ ┌───────────────┐
│ Worker 1 │ │ Worker 2 │ │ Worker N │
│ │ │ │ │ │
│ - Get port │ │ - Get port │ │ - Get port │
│ - Attempt │ │ - Attempt │ │ - Attempt │
│ connect │ │ connect │ │ connect │
│ - Report │ │ - Report │ │ - Report │
│ result │ │ result │ │ result │
└───────┬───────┘ └───────┬───────┘ └───────┬───────┘
│ │ │
└──────────────────┼──────────────────┘
│
▼
┌──────────────┐
│ Result Queue │
│ (completed │
│ scans) │
└──────────────┘
Phased Implementation Guide
Phase 1: Basic TCP Connect Scanner (Days 1-2)
Goal: Scan a single port on a single host
Implementation steps:
- Create a function that attempts to connect to a TCP port:
def scan_port(ip: str, port: int, timeout: float = 1.0) -> PortResult: """ Attempt TCP connection to ip:port Returns: PortResult with state='open' if connection succeeds PortResult with state='closed' if connection refused PortResult with state='filtered' if timeout """ # Use socket.socket() with SO_REUSEADDR # Set timeout with socket.settimeout() # Try connect_ex() for non-blocking connect # Map return codes to states: # 0 = open # 111 (ECONNREFUSED) = closed # 110 (ETIMEDOUT) = filtered - Add CLI argument parsing:
- Target IP/hostname
- Port number or range
- Timeout value
- Test against your local machine:
- Start a local HTTP server:
python -m http.server 8080 - Scan port 8080 (should be open)
- Scan port 8081 (should be closed)
- Start a local HTTP server:
Verification: Can you correctly identify an open port on localhost?
Phase 2: Multi-Threaded Scanning (Days 2-3)
Goal: Scan 1000 ports in under 30 seconds
Implementation steps:
- Use
concurrent.futures.ThreadPoolExecutor: ```python from concurrent.futures import ThreadPoolExecutor, as_completed
def scan_ports(ip: str, ports: List[int], threads: int = 100) -> List[PortResult]: results = [] with ThreadPoolExecutor(max_workers=threads) as executor: futures = { executor.submit(scan_port, ip, port): port for port in ports } for future in as_completed(futures): results.append(future.result()) return results
2. Add progress indicator (tqdm or simple counter)
3. Implement rate limiting to avoid overwhelming target
**Verification**: Scan ports 1-1000 on a target in under 30 seconds
### Phase 3: DNS Enumeration (Days 3-4)
**Goal**: Enumerate all DNS record types and attempt zone transfer
**Implementation steps**:
1. Use the `dnspython` library for queries:
```python
import dns.resolver
import dns.zone
import dns.query
def query_records(domain: str, record_type: str) -> List[DNSResult]:
"""Query specific DNS record type"""
resolver = dns.resolver.Resolver()
try:
answers = resolver.resolve(domain, record_type)
return [DNSResult(domain, record_type, str(rdata), answers.ttl)
for rdata in answers]
except dns.resolver.NXDOMAIN:
return [] # Domain doesn't exist
except dns.resolver.NoAnswer:
return [] # No records of this type
- Implement subdomain enumeration:
- Read subdomain wordlist
- Query A record for each:
subdomain.domain.com - Track which ones resolve
- Implement zone transfer attempt:
def attempt_zone_transfer(domain: str) -> Optional[List[DNSResult]]: """Attempt AXFR zone transfer""" # Get NS records for domain # For each NS, attempt: dns.query.xfr(ns, domain) # If successful, extract all records
Verification: Enumerate google.com and see A, MX, NS, TXT records
Phase 4: Service Fingerprinting (Days 4-5)
Goal: Identify services running on open ports
Implementation steps:
- Implement banner grabbing:
def grab_banner(ip: str, port: int, timeout: float = 2.0) -> Optional[str]: """Connect and read initial response""" sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) sock.settimeout(timeout) try: sock.connect((ip, port)) # Some services send banner immediately (SSH, FTP) # Others need us to send something first (HTTP) banner = sock.recv(1024) return banner.decode('utf-8', errors='ignore') except: return None finally: sock.close() - Add service-specific probes:
- HTTP: Send
GET / HTTP/1.0\r\n\r\n - MySQL: Parse greeting packet
- TLS: Extract certificate info
- HTTP: Send
- Build a simple service detection database:
SERVICE_SIGNATURES = { "ssh": re.compile(r"SSH-[\d.]+-"), "http": re.compile(r"HTTP/[\d.]+"), "ftp": re.compile(r"220.*FTP"), "smtp": re.compile(r"220.*SMTP"), }
Verification: Correctly identify SSH, HTTP, and MySQL on test servers
Phase 5: SYN Scanning with Scapy (Days 5-6)
Goal: Implement stealthy SYN scanning (requires root)
Implementation steps:
-
Install Scapy:
pip install scapy -
Implement SYN scan: ```python from scapy.all import IP, TCP, sr1, conf
def syn_scan(ip: str, port: int) -> str: “”” SYN scan using raw packets Requires root privileges “”” conf.verb = 0 # Disable Scapy output
# Craft SYN packet
syn = IP(dst=ip)/TCP(dport=port, flags='S')
# Send and wait for response
response = sr1(syn, timeout=1)
if response is None:
return "filtered"
elif response.haslayer(TCP):
if response[TCP].flags == 0x12: # SYN-ACK
# Send RST to close (stealth)
rst = IP(dst=ip)/TCP(dport=port, flags='R')
send(rst, verbose=0)
return "open"
elif response[TCP].flags == 0x14: # RST
return "closed"
return "filtered" ```
- Add sudo detection and privilege check
Verification: Compare SYN scan results to TCP connect results
Phase 6: Output and Reporting (Days 6-7)
Goal: Generate professional reports
Implementation steps:
- JSON output:
def export_json(results: List[PortResult], filepath: str): data = [asdict(r) for r in results] with open(filepath, 'w') as f: json.dump(data, f, indent=2) -
CSV output for spreadsheet analysis
- HTML report with styling: ```html
Network Reconnaissance Report
Target: {target}
Scan Time: {timestamp}
Open Ports
| Port | Service | Banner |
|---|
4. Compare output to Nmap format
**Verification**: Generate all three output formats for a scan
---
## Testing Strategy
### Unit Tests
```python
# test_scanner.py
import pytest
from scanner import scan_port, parse_port_range
def test_parse_port_range():
assert parse_port_range("80") == [80]
assert parse_port_range("1-5") == [1, 2, 3, 4, 5]
assert parse_port_range("22,80,443") == [22, 80, 443]
def test_scan_localhost_open_port():
# Start a server on port 9999 before test
result = scan_port("127.0.0.1", 9999)
assert result.state == "open"
def test_scan_localhost_closed_port():
result = scan_port("127.0.0.1", 9998) # Nothing running
assert result.state == "closed"
Integration Tests
- Against known targets:
- Scan
scanme.nmap.org(Nmap provides this for testing) - Compare results to Nmap output
- Verify same open ports detected
- Scan
- DNS tests:
- Enumerate
google.com(should find A, MX, NS records) - Test zone transfer against intentionally vulnerable server
- Enumerate
- Fingerprinting tests:
- Correctly identify Apache vs Nginx
- Correctly identify OpenSSH version
Performance Tests
import time
def test_scan_performance():
start = time.time()
results = scan_ports("192.168.1.1", range(1, 1001), threads=100)
elapsed = time.time() - start
assert elapsed < 30, f"Scan took {elapsed}s, should be under 30s"
Common Pitfalls and Debugging
1. “My scanner is too slow”
Problem: Scanning takes minutes instead of seconds
Debug steps:
- Check thread count (increase to 100-200)
- Check timeout (decrease to 0.5-1s)
- Look for blocking operations in main thread
- Use
cProfileto find bottleneck
Solution: Use ThreadPoolExecutor properly, don’t wait sequentially
2. “I get ‘Permission denied’ for SYN scan”
Problem: Raw socket access requires root
Debug steps:
- Run with
sudo - Check if Scapy is installed correctly
- Verify raw socket capability
Solution: Either use sudo or fall back to TCP connect scan
3. “DNS zone transfer always fails”
Problem: Most servers have zone transfers disabled
This is expected! Zone transfers are a security misconfiguration. For testing:
- Use intentionally vulnerable DNS servers (set up your own)
- Zonetransfer.me is a test server that allows AXFR
4. “Banner grabbing returns empty”
Problem: Some services need a prompt before responding
Debug steps:
- Check if connection succeeds
- Try sending a probe (like HTTP GET)
- Increase timeout
- Handle binary protocols (MySQL handshake)
Solution: Implement service-specific probes
5. “Results don’t match Nmap”
Problem: Different scan techniques yield different results
Debug steps:
- Compare exact scan types (Nmap -sT for TCP connect)
- Check timing/timeout differences
- Verify target hasn’t changed between scans
- Consider rate limiting differences
Extensions and Challenges
Beginner Extensions
- Add color output: Green for open, red for closed
- Progress bar: Show scan progress with tqdm
- Save/resume: Allow resuming interrupted scans
Intermediate Extensions
- OS Detection: Analyze TCP/IP stack quirks to guess OS
- Service probes: Add MySQL, PostgreSQL, Redis detection
- Rate limiting: Avoid triggering IDS/IPS
Advanced Extensions
- Evasion techniques: Fragmented packets, timing variations
- IPv6 support: Scan IPv6 addresses
- Web interface: Flask dashboard for results
- Parallel host scanning: Scan entire subnets efficiently
Real-World Connections
How Nmap Does It
Your scanner is a simplified Nmap. The real Nmap:
- Uses a sophisticated timing engine
- Has 600+ service probes
- Includes 600+ NSE scripts
- Does OS fingerprinting via TCP/IP stack analysis
After this project, read the Nmap source code to see professional implementation.
Shodan and Censys
Commercial services like Shodan scan the entire internet continuously. They use:
- Distributed scanning nodes
- Massive parallelization
- Historical data tracking
Your skills translate to understanding how these services work.
Bug Bounty Applications
Many bug bounties start with reconnaissance:
- Subdomain enumeration reveals forgotten assets
- Open ports expose services you didn’t know existed
- Banner grabbing identifies vulnerable versions
Self-Assessment Checklist
Before considering this project complete, verify:
Core Functionality
- TCP connect scan correctly identifies open/closed/filtered ports
- Multi-threaded scanning completes 1000 ports in < 30 seconds
- DNS enumeration returns A, MX, NS, TXT records
- Zone transfer attempt works (even if server refuses)
- Banner grabbing returns version strings for SSH, HTTP
Code Quality
- CLI has –help with clear usage
- Errors are handled gracefully (no crashes)
- Output is structured (JSON export works)
- Code is documented with docstrings
Understanding
- Can explain TCP three-way handshake whiteboard-style
- Can explain difference between SYN and connect scans
- Understand why zone transfers are security issues
- Know which scan types require root and why
Comparison
- Results match Nmap for same target/options
- Performance is comparable for connect scans
- Output is parseable by other tools
Resources
Primary Reading
- “TCP/IP Illustrated, Volume 1” by W. Richard Stevens - Chapters 17-18 (TCP Connection)
- “Black Hat Python” by Justin Seitz - Chapter 3 (Network Scanning)
- “The Linux Programming Interface” by Michael Kerrisk - Chapters 56-61 (Sockets)
Online Resources
- Nmap Reference Guide - Understand the reference implementation
- Scapy Documentation - Raw packet manipulation
- DNS RFC 1035 - DNS protocol specification
Practice Targets
scanme.nmap.org- Nmap provides this for testing- Your own VMs - Set up services to scan
- TryHackMe/HTB - Use recon tools on CTF challenges
This project is part of the Ethical Hacking & Penetration Testing learning path.