LEARN WEB FRAMEWORK FROM SCRATCH

Build a Web Framework from Scratch: Deep Learning Journey

Goal: Understand how modern web frameworks work by building one piece at a time, from raw TCP sockets to a fully-featured application framework.

Why Build a Web Framework?

Every time you use Flask, Django, Express, or FastAPI, hundreds of design decisions are invisible to you. Building a framework teaches you:

How HTTP really works beyond just “GET” and “POST”
Request lifecycle from bytes on a socket to a response
Routing engines and how to map URLs to handlers
Middleware systems and request/response pipelines
Template engines and how they parse and render HTML
Asynchronous handling and why it matters
Error handling and recovery strategies
Testing strategies for framework code
Performance optimization and caching

After these projects, you won’t just use frameworks—you’ll understand why they’re designed the way they are.

Core Concept Analysis

The Web Request Lifecycle

Every HTTP request follows this pipeline:

Raw TCP Connection
    ↓
Parse HTTP Headers
    ↓
Route to Handler
    ↓
Run Middleware (Before)
    ↓
Execute Business Logic
    ↓
Run Middleware (After)
    ↓
Generate Response
    ↓
Send Bytes Back to Client
    ↓
Close/Keep-Alive Connection

Fundamental Concepts You’ll Master

TCP/IP and Sockets - The foundation of everything
- Raw socket programming
- Listen/accept/send/receive
- Non-blocking I/O
- Connection lifecycle
HTTP Protocol - The language of the web
- Request/response format (RFC 7230-7235)
- Headers and their meaning
- Status codes and semantics
- Persistent connections (Keep-Alive)
URL Routing - Mapping paths to handlers
- URL pattern matching
- Parameter extraction
- Route precedence
- Wildcard and regex patterns
Middleware Pipeline - Cross-cutting concerns
- Request preprocessing
- Response postprocessing
- Error handling chains
- Ordered execution
Templating - Dynamic HTML generation
- Template parsing and compilation
- Variable substitution
- Control flow (loops, conditionals)
- Escaping and safety
Request/Response Objects - Abstractions
- Headers parsing
- Query string handling
- Form data parsing (URL-encoded, multipart)
- Cookie handling
Asynchronous I/O - Non-blocking patterns
- Event loops
- Callbacks/Promises/Async-Await
- Handling many concurrent connections
- Backpressure handling
Session Management - Stateful interactions
- Session IDs and tokens
- Storage backends
- Security considerations
- Cookie attributes
Error Handling - Graceful failure
- Exception catching
- HTTP error responses
- Logging
- Recovery strategies
Database Integration - Data persistence
- Query builders
- ORM concepts
- Connection pooling
- Transaction management

Project Sequence

Projects are ordered from foundational to advanced. Each builds on previous concepts.

Project 1: TCP Socket Server (Raw Foundation)

File: LEARN_WEB_FRAMEWORK_FROM_SCRATCH.md
Main Programming Language: Python
Alternative Programming Languages: C, Rust, Go
Coolness Level: Level 2: Practical but Forgettable
Business Potential: 1. The “Resume Gold”
Difficulty: Level 1: Beginner
Knowledge Area: Socket Programming / Network I/O
Software or Tool: netcat, Wireshark
Main Book: “The Linux Programming Interface” by Michael Kerrisk

What you’ll build: A basic TCP server that accepts connections on port 8000, reads bytes from clients, and sends back simple responses. You can connect with telnet localhost 8000 and see it work.

Why it teaches web frameworks: A web framework is built on top of TCP sockets. You need to understand the raw mechanics before you can abstract them. This project shows you that “accepting HTTP requests” is just reading bytes from a socket.

Core challenges you’ll face:

Creating a listening socket (maps to understanding socket(), bind(), listen())
Accepting multiple clients (maps to the accept() loop)
Reading variable amounts of data (maps to buffering and protocol framing)
Keeping connections alive (maps to understanding connection lifecycle)

Key Concepts:

Socket creation and binding: “The Linux Programming Interface” by Michael Kerrisk - Chapter 55
TCP/IP basics: “TCP/IP Illustrated, Volume 1” by W. Richard Stevens (Chapter 3)
Non-blocking I/O: “The Linux Programming Interface” - Chapter 63
Connection management: “Advanced Programming in the UNIX Environment” by Stevens & Rago (Chapter 16)

Difficulty: Beginner Time estimate: Weekend Prerequisites: Basic understanding of networking (TCP, IP, ports), comfort with your chosen language

Real world outcome:

$ python tcp_server.py &
Listening on port 8000...

$ telnet localhost 8000
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
Hello from server!

Implementation Hints:

A TCP server requires these steps:

Create a socket with socket() (address family AF_INET, type SOCK_STREAM)
Bind it to an address and port with bind()
Mark it as listening with listen()
Loop accepting connections with accept()
For each connection, read data with recv() and send with send()
Handle connection closure gracefully

Think about:

What happens when a client connects before you’re ready to handle them? (backlog queue)
What happens if you try to bind to a port that’s already in use?
What does accept() return and how do you use it?
How do you cleanly shutdown the server?

Learning milestones:

Server accepts one connection and sends data → You understand socket basics
Server handles multiple sequential clients → You understand the accept loop
Server keeps connections alive → You understand connection lifecycle
Server gracefully handles client disconnections → You understand edge cases

Project 2: HTTP Request Parser (Understanding the Wire Format)

Main Programming Language: Python
Alternative Programming Languages: C, Rust, Go, JavaScript
Coolness Level: Level 3: Genuinely Clever
Business Potential: 1. The “Resume Gold”
Difficulty: Level 1: Beginner
Knowledge Area: HTTP Protocol / String Parsing
Software or Tool: curl, Postman
Main Book: “HTTP: The Definitive Guide” by David Gourley and Brian Totty

What you’ll build: A parser that reads raw HTTP request bytes and extracts: method (GET/POST), path, query string, headers, and body. You can feed it real HTTP requests from curl and print out the parsed components.

Why it teaches web frameworks: Before routing or responding, a framework must parse HTTP requests. Understanding the request format deeply is critical. This teaches you that HTTP is just text with conventions.

Core challenges you’ll face:

Parsing the request line (maps to method, path, HTTP version)
Handling headers (maps to key:value parsing, case-insensitivity)
Detecting request body start (maps to empty line delimiter)
Handling malformed requests (maps to error handling)
Understanding Content-Length and chunking (maps to body termination strategies)

Key Concepts:

HTTP Request Format: RFC 7230 Section 3 - Official RFC
Status codes and semantics: RFC 7231 Section 6
Headers: “HTTP: The Definitive Guide” Chapter 5 by Gourley & Totty
Request body handling: RFC 7230 Section 3.3

Difficulty: Beginner Time estimate: Weekend Prerequisites: Basic Python string manipulation, understanding of HTTP basics (methods, headers)

Real world outcome:

$ curl -X POST http://localhost:8000/api/users -H "Content-Type: application/json" -d '{"name": "Alice"}'

$ python http_parser.py
Parsed Request:
  Method: POST
  Path: /api/users
  Query: (none)
  HTTP Version: 1.1
  Headers:
    Host: localhost:8000
    Content-Type: application/json
    Content-Length: 18
  Body: {"name": "Alice"}

Implementation Hints:

HTTP requests follow this format:

METHOD PATH HTTP/1.1\r\n
Header1: Value1\r\n
Header2: Value2\r\n
\r\n
[Optional Body]

Questions to guide your implementation:

How do you detect the end of the request line? (CRLF = \r\n)
How do you detect the end of headers? (empty line = \r\n\r\n)
What if there’s no Content-Length header? How do you know where the body ends?
Headers are case-insensitive, but how do you handle that?
What happens if a header has no colon?

Think about building a state machine:

State 1: Reading the request line
State 2: Reading headers
State 3: Reading body (if present)

Learning milestones:

Parse simple GET requests → You understand basic HTTP format
Parse headers correctly → You understand header syntax
Extract and handle request body → You understand stateful parsing
Handle malformed requests gracefully → You understand error recovery

Project 3: Simple Routing Engine (URL to Handler Mapping)

Main Programming Language: Python
Alternative Programming Languages: C, Rust, Go, JavaScript
Coolness Level: Level 3: Genuinely Clever
Business Potential: 1. The “Resume Gold”
Difficulty: Level 2: Intermediate
Knowledge Area: Routing / Pattern Matching / Data Structures
Software or Tool: Flask (for comparison), curl
Main Book: “Algorithms, Fourth Edition” by Sedgewick & Wayne

What you’ll build: A routing system that maps URL patterns to handler functions. Support static routes (/users), parametric routes (/users/:id), and wildcard routes (/files/*). Given a request path, find the matching handler and extract parameters.

Why it teaches web frameworks: Routing is the heart of a web framework. Every framework has a router. This teaches you how to efficiently map URLs to logic and extract dynamic parameters.

Core challenges you’ll face:

Route registration and lookup (maps to data structure design)
Parameter extraction from paths (maps to parsing and regex)
Route precedence and conflict resolution (maps to algorithm design)
Handling wildcards and regex patterns (maps to pattern matching)
Performance at scale (maps to optimization)

Key Concepts:

Trie data structures: “Algorithms, Fourth Edition” by Sedgewick & Wayne - Chapter 5
Regular expressions: “Mastering Regular Expressions” by Jeffrey Friedl
URL path components: RFC 3986 - “Uniform Resource Identifier (URI)”
Pattern matching algorithms: Knuth-Morris-Pratt or similar

Difficulty: Intermediate Time estimate: 1-2 weeks Prerequisites: Comfortable with data structures (trees, maps), regex basics, Python

Real world outcome:

router = Router()
router.add("/", home_handler)
router.add("/users", list_users)
router.add("/users/:id", get_user)
router.add("/files/*", serve_file)
router.add("/api/:version/items/:id", api_handler)

# Test the router
match = router.match("/users/42")
print(match["handler"])      # get_user
print(match["params"])       # {"id": "42"}

match = router.match("/files/docs/resume.pdf")
print(match["params"])       # {"*": "docs/resume.pdf"}

Implementation Hints:

Consider building a Trie (prefix tree) for efficient routing:

Each node represents a path segment
Static segments (like “users”) are direct children
Dynamic segments (like “:id”) need a special handler
Wildcard segments (like “*”) need fallback logic

Think about these decisions:

How do you handle route precedence? (static > dynamic > wildcard)
How do you represent :id and * differently in your data structure?
How do you extract the value for :id from the actual path?
What if multiple routes could match? (e.g., /users/:id vs /users/admin)
How do you handle trailing slashes?

Pseudo-approach:

Build a tree where each node is a path segment
For a given path, traverse the tree matching segments
Handle static matches first, then dynamic (:param), then wildcard
Return the handler and collected parameters

Learning milestones:

Static routes work → You understand basic pattern matching
Dynamic routes work → You understand parameter extraction
Wildcard routes work → You understand greedy matching
Route precedence is correct → You understand algorithm design for routing

Project 4: HTTP Response Builder (Creating Valid HTTP)

Main Programming Language: Python
Alternative Programming Languages: C, Rust, Go, JavaScript
Coolness Level: Level 2: Practical but Forgettable
Business Potential: 1. The “Resume Gold”
Difficulty: Level 1: Beginner
Knowledge Area: HTTP Protocol / String Building
Software or Tool: curl, Postman, Wireshark
Main Book: “HTTP: The Definitive Guide” by Gourley & Totty

What you’ll build: A response builder that creates properly formatted HTTP responses. Set status code, headers, body, and cookies. Ensure proper formatting with correct line endings and header syntax.

Why it teaches web frameworks: Requests are only half the story. Frameworks must generate proper HTTP responses. This teaches you response semantics, headers, and edge cases.

Core challenges you’ll face:

Formatting response lines and headers (maps to HTTP specification compliance)
Setting proper Content-Type and Content-Length (maps to metadata handling)
Handling different status codes (maps to semantics)
Cookie and header edge cases (maps to protocol details)
Ensuring CRLF line endings (maps to binary correctness)

Key Concepts:

HTTP Response Format: RFC 7230 Section 3
Status codes and semantics: RFC 7231 Section 6
Headers and metadata: “HTTP: The Definitive Guide” Chapter 5
Cookies: RFC 6265

Difficulty: Beginner Time estimate: Weekend Prerequisites: Understanding of HTTP, Python string handling

Real world outcome:

response = Response(200, "OK")
response.set_header("Content-Type", "application/json")
response.set_body('{"status": "success"}')

print(response.serialize())
# Output:
# HTTP/1.1 200 OK\r\n
# Content-Type: application/json\r\n
# Content-Length: 21\r\n
# \r\n
# {"status": "success"}

Implementation Hints:

An HTTP response needs:

Status line: HTTP/1.1 <CODE> <REASON>
Headers: Key: Value (one per line)
Empty line: \r\n
Body: raw bytes

Think about:

What are common status codes and their meanings?
How do you calculate Content-Length?
What happens if you don’t set a Content-Length?
How do you handle special headers like Set-Cookie (can appear multiple times)?
What line ending is correct? (\r\n, not \n)

Learning milestones:

Basic responses work → You understand response format
Headers are set correctly → You understand metadata
Content-Length is accurate → You understand protocol correctness
Special cases (cookies, redirects) work → You understand edge cases

Project 5: Middleware Pipeline (Request/Response Processing)

Main Programming Language: Python
Alternative Programming Languages: JavaScript/Node, Go, Rust
Coolness Level: Level 3: Genuinely Clever
Business Potential: 1. The “Resume Gold”
Difficulty: Level 2: Intermediate
Knowledge Area: Design Patterns / Pipeline Architecture
Software or Tool: Express.js (for comparison), curl
Main Book: “Design Patterns” by Gamma, Helm, Johnson, Vlissides

What you’ll build: A middleware system where functions run in sequence. Each middleware can read the request, modify it, call the next middleware, and process the response. Implement logging, authentication checking, error handling, and response compression middlewares.

Why it teaches web frameworks: Modern frameworks (Express, Flask, Django) all use middleware pipelines. This teaches you how cross-cutting concerns are implemented cleanly. It’s how frameworks separate concerns.

Core challenges you’ll face:

Function composition and chaining (maps to understanding closures and function passing)
Error propagation through the chain (maps to exception handling strategy)
Request/response modification (maps to mutable state handling)
Order matters (maps to understanding precedence)
Implementing skip/abort (maps to control flow)

Key Concepts:

Middleware patterns: “Release It!” by Nygard - Chapter on patterns
Decorator pattern: “Design Patterns” by Gang of Four - Chapter 4
Chain of Responsibility: “Design Patterns” - Chapter 5
Function composition: “Functional Programming in Python” concepts

Difficulty: Intermediate Time estimate: 1-2 weeks Prerequisites: Understanding of callbacks/closures, basic design patterns, Python

Real world outcome:

app = Application()

@app.use
def logging_middleware(request, next):
    print(f"Incoming: {request.method} {request.path}")
    response = next()
    print(f"Response: {response.status}")
    return response

@app.use
def auth_middleware(request, next):
    if not request.headers.get("Authorization"):
        return Response(401, "Unauthorized")
    return next()

@app.route("/protected", "GET")
def protected_route(request):
    return Response(200, "You are authenticated!")

# Test it
client.get("/protected")
# Output:
# Incoming: GET /protected
# Response: 200

Implementation Hints:

A middleware pipeline is a chain of function calls. Each middleware:

Receives the request (read-only or can modify)
Calls next() to pass to the next middleware
Receives the response
Can modify the response
Returns the response

Think about this pattern:

middleware1(request, next):
  # Before logic
  response = next()
  # After logic
  return response

How do you:

Store the middleware functions?
Execute them in order?
Handle errors in one middleware affecting others?
Allow a middleware to short-circuit (not call next)?
Handle async middleware?

Consider using a decorator-like approach where each middleware wraps the previous one.

Learning milestones:

Middleware executes in order → You understand function composition
Middleware can modify requests → You understand state mutation
Middleware can modify responses → You understand response processing
Error handling works → You understand exception propagation

Project 6: Template Engine (Dynamic HTML Generation)

Main Programming Language: Python
Alternative Programming Languages: JavaScript, Go, Java
Coolness Level: Level 4: Hardcore Tech Flex
Business Potential: 1. The “Resume Gold”
Difficulty: Level 3: Advanced
Knowledge Area: Parsing / Compilers / Template Languages
Software or Tool: Jinja2, Handlebars
Main Book: “Language Implementation Patterns” by Terence Parr

What you’ll build: A template engine that parses HTML with embedded variables and logic. Support variable substitution ({{ name }}), loops ({% for item in items %}), conditionals ({% if user %}), and filters ({{ email | lowercase }}).

Why it teaches web frameworks: Templates are how frameworks generate dynamic HTML. Building a template engine teaches you parsing, compilation, and execution. It’s a mini-language compiler.

Core challenges you’ll face:

Lexical analysis (tokenization) (maps to breaking template into tokens)
Parsing template syntax (maps to understanding grammar)
Rendering context and variable lookup (maps to scoping)
Handling control flow (maps to loops and conditionals)
Escaping and safety (maps to security)
Caching compiled templates (maps to performance)

Key Concepts:

Parsing and tokenization: “Language Implementation Patterns” by Terence Parr - Chapter 3-4
Template engines: “Jinja2 Documentation” - Template Designer Documentation
Abstract Syntax Trees: “Compilers: Principles and Practice” by Dave & Dave
Scoping and contexts: “Programming Language Implementation” concepts

Difficulty: Advanced Time estimate: 1 month+ Prerequisites: Comfortable with parsing, understanding of recursive descent or similar parsing techniques, Python

Real world outcome:

<!-- users.html template -->
<h1>Users</h1>
<ul>
  {% for user in users %}
    <li>{{ user.name }} ({{ user.email | lowercase }})</li>
  {% endfor %}
</ul>
{% if admin %}
  <p>Admin panel: <a href="/admin">Click here</a></p>
{% endif %}

template = Template(open("users.html").read())
html = template.render({
    "users": [
        {"name": "Alice", "email": "ALICE@EXAMPLE.COM"},
        {"name": "Bob", "email": "BOB@EXAMPLE.COM"},
    ],
    "admin": True
})
print(html)
# Output:
# <h1>Users</h1>
# <ul>
#   <li>Alice (alice@example.com)</li>
#   <li>Bob (bob@example.com)</li>
# </ul>
# <p>Admin panel: <a href="/admin">Click here</a></p>

Implementation Hints:

A template engine typically works in these phases:

Tokenization: Break the template into tokens
- Text tokens: raw HTML
- Variable tokens: {{ ... }}
- Block tokens: {% ... %}
Parsing: Create an AST (Abstract Syntax Tree)
- Parse variable expressions
- Parse for/if blocks recursively
- Handle nesting
Compilation: Convert AST to executable code
- Generate Python functions or bytecode
- Cache the compiled form
Rendering: Execute with a context
- Variable lookup in context
- Filter application
- Output generation

Think about:

How do you tokenize {{ user.name | uppercase }}?
How do you handle {% for item in items %} ... {% endfor %}?
How do you handle nested {% if %} inside {% for %}?
How do you escape {{ html }} to prevent XSS attacks?
How do you handle missing variables?

Learning milestones:

Simple variables render → You understand tokenization
Loops work → You understand AST and control flow
Nested structures work → You understand recursive parsing
Filters and escaping work → You understand processing pipelines

Project 7: Request/Response Objects (Abstractions)

Main Programming Language: Python
Alternative Programming Languages: JavaScript, Go, Java
Coolness Level: Level 2: Practical but Forgettable
Business Potential: 1. The “Resume Gold”
Difficulty: Level 2: Intermediate
Knowledge Area: Object-Oriented Design / API Design
Software or Tool: Flask, Django, Express
Main Book: “Clean Code” by Robert C. Martin

What you’ll build: Request and Response classes that abstract HTTP. Request should parse headers, cookies, query strings, and body. Response should handle status, headers, cookies, and serialization.

Why it teaches web frameworks: Web frameworks abstract HTTP into Request/Response objects. Building these teaches you clean API design and how to handle HTTP complexity behind a simple interface.

Core challenges you’ll face:

Lazy parsing (parse on demand) (maps to performance optimization)
Headers case-insensitivity (maps to HTTP subtleties)
Cookie and query string parsing (maps to format handling)
File uploads (multipart) (maps to complex parsing)
Streaming responses (maps to resource efficiency)

Key Concepts:

Request object patterns: Flask/Django source code
API design: “Clean Code” by Martin - Chapter 4
Lazy evaluation: Programming language design concepts
Cookie handling: RFC 6265

Difficulty: Intermediate Time estimate: 1-2 weeks Prerequisites: Object-oriented Python, HTTP understanding

Real world outcome:

# From within a request handler
def handle_request(request: Request) -> Response:
    # Query parameters
    name = request.query.get("name")  # ?name=Alice

    # Headers
    auth = request.headers.get("Authorization")

    # Cookies
    session_id = request.cookies.get("session")

    # Form data
    email = request.form.get("email")

    # File uploads
    avatar = request.files.get("avatar")

    # Create response
    response = Response()
    response.status = 200
    response.headers["Content-Type"] = "application/json"
    response.set_cookie("new_session", "xyz123")
    response.body = '{"status": "ok"}'

    return response

Implementation Hints:

A Request object should:

Parse the HTTP request into attributes
Lazily parse components (only when accessed)
Provide convenient accessors for common operations

Think about:

How do you handle query string parsing? (?name=value&foo=bar)
How do you parse form data? (URL-encoded vs multipart)
How do you handle cookie parsing?
How do you access raw headers vs convenient getters?
How do you handle binary data (file uploads)?

A Response object should:

Track status code and reason phrase
Manage headers
Handle body as string or bytes
Serialize to valid HTTP

Learning milestones:

Basic request parsing works → You understand HTTP abstraction
Query and form data work → You understand format parsing
Response serialization works → You understand response generation
Cookies and files work → You understand edge cases

Project 8: Session Management (Stateful Interactions)

Main Programming Language: Python
Alternative Programming Languages: Go, Node.js, Java
Coolness Level: Level 3: Genuinely Clever
Business Potential: 2. The “Micro-SaaS / Pro Tool”
Difficulty: Level 2: Intermediate
Knowledge Area: State Management / Security / Storage
Software or Tool: Redis, Memcached, SQLite
Main Book: “OWASP Session Management Cheat Sheet”

What you’ll build: A session system where users get session IDs (via cookies), and you store session data server-side. Implement multiple backends: in-memory, file-based, and Redis. Handle expiration, security, and concurrent access.

Why it teaches web frameworks: HTTP is stateless, but web apps need state (user login info). Sessions bridge this gap. Building a session system teaches you security, storage patterns, and the trade-offs between different approaches.

Core challenges you’ll face:

Session ID generation (maps to randomness and security)
Storage backends (maps to abstraction and switching)
Expiration handling (maps to lifecycle management)
Concurrent access (maps to thread safety)
Security (maps to CSRF and session hijacking prevention)

Key Concepts:

Session security: “OWASP Session Management Cheat Sheet”
Storage patterns: “Database Internals” by Alex Petrov
Random number generation: “Serious Cryptography” by Aumasson
Cache patterns: “Designing Data-Intensive Applications” by Kleppmann

Difficulty: Intermediate Time estimate: 1-2 weeks Prerequisites: Understanding of cookies, HTTP, Python, and at least one storage backend

Real world outcome:

session_manager = SessionManager(backend="redis")

# Create session
session_id = session_manager.create()
response.set_cookie("session", session_id)

# Store data
session_manager.set(session_id, "user_id", 42)
session_manager.set(session_id, "username", "alice")

# Retrieve data
user_id = session_manager.get(session_id, "user_id")
print(user_id)  # 42

# Expiration
session_manager.expire(session_id)

# Check if exists
if session_manager.exists(session_id):
    # Still valid

Implementation Hints:

A session system needs:

Session ID generation: Create cryptographically random IDs
Storage backend abstraction: In-memory dict, file, or Redis
Expiration logic: Remove sessions after timeout
Access patterns: Set/get key-value pairs

Think about:

How do you generate secure random session IDs?
How long should sessions last? (30 minutes? 30 days?)
What happens when someone presents an expired session?
How do you handle concurrent requests with the same session?
How do you prevent session fixation attacks?
How do you prevent session hijacking?

Consider implementing multiple backends:

In-memory (good for development)
File-based (simple, works without external services)
Redis (fast, distributed, production-ready)

Learning milestones:

Basic session creation and retrieval → You understand session lifecycle
Expiration works → You understand timeout handling
Multiple backends work → You understand abstraction
Security best practices implemented → You understand web security

Project 9: Form Data Parsing (Request Bodies)

Main Programming Language: Python
Alternative Programming Languages: Go, Java, C
Coolness Level: Level 2: Practical but Forgettable
Business Potential: 1. The “Resume Gold”
Difficulty: Level 2: Intermediate
Knowledge Area: Parsing / Encoding / Binary Formats
Software or Tool: curl, Postman
Main Book: “Computer Networks” by Tanenbaum & Wetherall

What you’ll build: A parser for form data in multiple formats: URL-encoded (application/x-www-form-urlencoded) and multipart (multipart/form-data). Handle file uploads and parse JSON request bodies.

Why it teaches web frameworks: Most web requests have bodies. Frameworks must parse these correctly. Building this teaches you encoding standards, boundary detection, and error handling.

Core challenges you’ll face:

URL decoding (maps to % encoding understanding)
Multipart boundary parsing (maps to stateful parsing)
File upload handling (maps to binary data)
Large body handling (maps to streaming and memory efficiency)
Encoding issues (maps to character sets)

Key Concepts:

URL encoding: RFC 3986
Multipart form data: RFC 2388
MIME types: RFC 2045-2049
Character encodings: “Understanding Character Encodings” by Tom Scott

Difficulty: Intermediate Time estimate: 1-2 weeks Prerequisites: Understanding of HTTP headers, encodings, and string processing in Python

Real world outcome:

# URL-encoded form
$ curl -X POST http://localhost:8000/login \
  -d "username=alice&password=secret123"

# Multipart form with file
$ curl -X POST http://localhost:8000/upload \
  -F "name=Alice" \
  -F "avatar=@/path/to/image.png"

# JSON body
$ curl -X POST http://localhost:8000/api/users \
  -H "Content-Type: application/json" \
  -d '{"name": "Alice", "email": "alice@example.com"}'

# In your handler
def handle_form(request):
    username = request.form.get("username")  # alice
    password = request.form.get("password")  # secret123

    return Response(200, "Login successful")

Implementation Hints:

For URL-encoded forms:

username=alice&password=secret123&tags=python&tags=web

Split by &
Split each by =
URL-decode the values
Handle multiple values for same key

For multipart forms:

Content-Type: multipart/form-data; boundary=----WebKitFormBoundary

------WebKitFormBoundary
Content-Disposition: form-data; name="username"

alice
------WebKitFormBoundary
Content-Disposition: form-data; name="avatar"; filename="pic.png"
Content-Type: image/png

[binary data]
------WebKitFormBoundary--

Think about:

How do you find the boundary in multipart data?
What if the boundary appears in the file data?
How do you handle large files without loading into memory?
How do you detect the encoding of text fields?
What if the Content-Type header specifies a charset?

Learning milestones:

URL-encoded parsing works → You understand simple encoding
Multipart parsing works → You understand complex formats
File uploads work → You understand binary handling
Large files work → You understand streaming

Project 10: Authentication System (User Verification)

Main Programming Language: Python
Alternative Programming Languages: Go, Node.js, Rust
Coolness Level: Level 3: Genuinely Clever
Business Potential: 2. The “Micro-SaaS / Pro Tool”
Difficulty: Level 3: Advanced
Knowledge Area: Security / Cryptography / Authentication
Software or Tool: bcrypt, JWT, SQLite
Main Book: “Serious Cryptography” by Jean-Philippe Aumasson

What you’ll build: A complete authentication system: user registration, password hashing, login, JWT tokens, and protected routes. Implement password strength validation, account lockout after failed attempts, and password reset flows.

Why it teaches web frameworks: Authentication is critical and complex. Building it teaches you security, cryptography, and how frameworks handle sensitive operations. Most apps need this.

Core challenges you’ll face:

Secure password hashing (maps to bcrypt and salt understanding)
Token generation and validation (maps to JWT or session tokens)
Rate limiting (maps to brute force prevention)
Password reset (maps to secure token flows)
Two-factor authentication (maps to multi-step verification)

Key Concepts:

Password hashing: “Serious Cryptography” by Aumasson - Chapter 12
JWT tokens: RFC 7519
OWASP authentication: “OWASP Authentication Cheat Sheet”
Cryptographic randomness: “Serious Cryptography” - Chapter 2

Difficulty: Advanced Time estimate: 1 month+ Prerequisites: Understanding of hashing, cryptography basics, database basics, security concepts

Real world outcome:

auth = AuthenticationSystem()

# Register user
auth.register("alice@example.com", "SecurePass123!")
# Returns: user_id

# Login
token = auth.login("alice@example.com", "SecurePass123!")
# Returns: JWT token

# Verify token
user_id = auth.verify_token(token)
# Returns: 42

# Protected route
@app.route("/profile", "GET")
def profile(request):
    token = request.headers.get("Authorization").replace("Bearer ", "")
    user_id = auth.verify_token(token)
    if not user_id:
        return Response(401, "Unauthorized")

    return Response(200, f"Profile for user {user_id}")

# Reset password
auth.request_reset("alice@example.com")
# Sends email with reset token
reset_token = "..."  # from email
auth.reset_password(reset_token, "NewPassword123!")

Implementation Hints:

An authentication system needs:

User storage: Database with email, hashed password, other info
Registration: Validate input, hash password, store user
Login: Find user, verify password hash, issue token
Token verification: Validate JWT signature, check expiration
Protected routes: Verify token before allowing access

Think about:

What makes a password strong? (length, complexity, no common patterns)
How do you securely hash passwords? (bcrypt with salt)
How do you prevent brute force? (rate limiting, account lockout)
How do you handle password reset securely? (time-limited tokens)
How do you store tokens? (HTTP-only cookies vs headers)
How do you handle token expiration?

Security considerations:

Never store plaintext passwords
Use slow hashing algorithms (bcrypt, argon2)
Rate limit login attempts
Use HTTPS in production
Validate all input
Use secure random token generation

Learning milestones:

User registration works → You understand user storage
Login and verification work → You understand password hashing
Tokens work → You understand stateless authentication
Protected routes work → You understand authorization

Project 11: Error Handling and Logging (Robustness)

Main Programming Language: Python
Alternative Programming Languages: Go, Java, Rust
Coolness Level: Level 2: Practical but Forgettable
Business Potential: 1. The “Resume Gold”
Difficulty: Level 2: Intermediate
Knowledge Area: Debugging / Logging / Error Management
Software or Tool: logging module, syslog, ELK stack
Main Book: “The Art of Debugging with GDB, DDD, and Eclipse” by Matloff & Salzman

What you’ll build: A comprehensive error handling system: custom exception types, error handlers for different exception types, logging at multiple levels (DEBUG, INFO, WARNING, ERROR), and structured logging.

Why it teaches web frameworks: Production systems fail. Frameworks must handle errors gracefully, log them, and recover. This teaches you robustness and debugging.

Core challenges you’ll face:

Custom exception hierarchy (maps to organizing error types)
Error handler registration (maps to dispatch based on exception type)
Proper HTTP error responses (maps to semantic error codes)
Structured logging (maps to searchable logs)
Log levels and filtering (maps to controlling verbosity)

Key Concepts:

Exception handling: Python documentation on exceptions
Logging: Python logging module documentation
Structured logging: “Designing Data-Intensive Applications” - Chapter 3
Error codes and status codes: RFC 7231 Section 6

Difficulty: Intermediate Time estimate: 1-2 weeks Prerequisites: Understanding of exceptions, logging basics, Python

Real world outcome:

# Define custom exceptions
class ApplicationError(Exception):
    def __init__(self, code, message, status=500):
        self.code = code
        self.message = message
        self.status = status

class ValidationError(ApplicationError):
    def __init__(self, message):
        super().__init__("VALIDATION_ERROR", message, 400)

class NotFoundError(ApplicationError):
    def __init__(self, resource):
        super().__init__("NOT_FOUND", f"{resource} not found", 404)

# Register error handlers
app = Application()

@app.error_handler(ValidationError)
def handle_validation(error, request):
    return Response(error.status, json.dumps({
        "error": error.code,
        "message": error.message
    }))

@app.error_handler(NotFoundError)
def handle_not_found(error, request):
    return Response(404, json.dumps({
        "error": error.code,
        "message": error.message
    }))

# Use in handlers
@app.route("/users/:id", "GET")
def get_user(request):
    user_id = request.params.get("id")
    if not user_id.isdigit():
        raise ValidationError("user_id must be numeric")

    user = db.find_user(int(user_id))
    if not user:
        raise NotFoundError("User")

    return Response(200, json.dumps(user))

# Logging
import logging
logging.basicConfig(level=logging.DEBUG)
logger = logging.getLogger(__name__)

logger.debug(f"Processing request: {request.method} {request.path}")
logger.info(f"User 42 logged in")
logger.warning(f"Rate limit approaching for IP {request.ip}")
logger.error(f"Database connection failed: {error}")

Implementation Hints:

An error handling system needs:

Custom exception classes: Organize errors by type
Error handlers: Map exception types to response handlers
Global try/catch: Catch unhandled exceptions in middleware
HTTP response conversion: Exception → HTTP response
Logging: Record what happened

Think about:

Should all exceptions become 500 errors? (No, some are client errors)
How do you preserve the original exception for logging but show safe message to user?
How do you structure logs for easy searching?
What information should logs contain? (timestamp, level, message, stack trace, context)
How do you avoid logging sensitive information?

Log structure idea:

{
  "timestamp": "2024-01-15T10:30:45.123Z",
  "level": "ERROR",
  "logger": "app.handlers",
  "message": "Failed to process payment",
  "request_id": "req-12345",
  "user_id": 42,
  "exception": "PaymentProcessorError",
  "stack_trace": "..."
}

Learning milestones:

Custom exceptions work → You understand exception design
Error handlers dispatch correctly → You understand dispatch patterns
Logging at multiple levels works → You understand log management
Unhandled exceptions don’t crash the server → You understand robustness

Project 12: Database Layer Abstraction (Persistence)

Main Programming Language: Python
Alternative Programming Languages: Go, Java, Rust
Coolness Level: Level 3: Genuinely Clever
Business Potential: 3. The “Service & Support” Model
Difficulty: Level 3: Advanced
Knowledge Area: Database / ORM / Query Building
Software or Tool: SQLite, PostgreSQL, SQLAlchemy
Main Book: “Designing Data-Intensive Applications” by Martin Kleppmann

What you’ll build: A database abstraction layer (mini-ORM) that: builds SQL queries programmatically, maps database rows to Python objects, handles relationships, and provides a clean API. Support multiple backends (SQLite, PostgreSQL).

Why it teaches web frameworks: Most web apps use databases. Building a database layer teaches you SQL, query optimization, and abstraction. Major frameworks (Django, Rails) all have ORMs.

Core challenges you’ll face:

SQL query building (maps to programmatic query construction)
Result mapping to objects (maps to data binding)
Relationships (one-to-many, many-to-many) (maps to graph navigation)
Lazy loading vs eager loading (maps to performance optimization)
Transaction handling (maps to ACID properties)
Connection pooling (maps to resource management)

Key Concepts:

SQL and relational databases: “Designing Data-Intensive Applications” - Chapter 2
ORM patterns: “Patterns of Enterprise Application Architecture” by Fowler
Query building: SQLAlchemy documentation
Database transactions: “Designing Data-Intensive Applications” - Chapter 7

Difficulty: Advanced Time estimate: 1 month+ Prerequisites: SQL knowledge, understanding of databases, Python OOP, design patterns

Real world outcome:

# Define a model
class User(Model):
    __tablename__ = "users"

    id = Column(Integer, primary_key=True)
    username = Column(String(50), unique=True)
    email = Column(String(100))
    posts = Relationship("Post", back_populates="author")

class Post(Model):
    __tablename__ = "posts"

    id = Column(Integer, primary_key=True)
    title = Column(String(200))
    content = Column(Text)
    author_id = Column(Integer, ForeignKey("users.id"))
    author = Relationship("User", back_populates="posts")

# Use the ORM
session = Session()

# Create
user = User(username="alice", email="alice@example.com")
session.add(user)
session.commit()

# Query
user = session.query(User).filter(User.username == "alice").first()
print(user.email)

# Relationships
for post in user.posts:
    print(post.title)

# Update
user.email = "alice.new@example.com"
session.commit()

# Delete
session.delete(user)
session.commit()

Implementation Hints:

A basic ORM needs:

Model definition: Classes that map to tables
Column definitions: Types, constraints, relationships
Query builder: Fluent API for building WHERE clauses
Result mapping: Rows → objects
Relationships: Navigation between models
Session/transaction: Track changes and commit

Think about:

How do you map Python types to SQL types? (int → INTEGER, str → VARCHAR)
How do you handle relationships? (ForeignKey columns)
How do you build WHERE clauses from Python expressions?
How do you track which objects have been modified?
How do you handle lazy loading? (Load related objects on demand)
How do you handle transactions? (Commit/rollback)

Start simple:

Basic CRUD (Create, Read, Update, Delete)
Simple WHERE filters
Single relationships
Expand from there

Learning milestones:

Basic CRUD works → You understand ORM fundamentals
Filtering works → You understand query building
Relationships work → You understand graph navigation
Transactions work → You understand ACID properties

Project 13: Static File Serving (Asset Handling)

Main Programming Language: Python
Alternative Programming Languages: Go, C, Rust
Coolness Level: Level 2: Practical but Forgettable
Business Potential: 1. The “Resume Gold”
Difficulty: Level 1: Beginner
Knowledge Area: File I/O / HTTP Optimization
Software or Tool: Nginx, Apache
Main Book: “HTTP: The Definitive Guide” by Gourley & Totty

What you’ll build: A module that serves static files (CSS, JavaScript, images) efficiently. Support caching headers (ETag, Last-Modified, Cache-Control), range requests for large files, and MIME type detection.

Why it teaches web frameworks: Most web apps serve static assets. Doing this efficiently requires HTTP knowledge. This teaches you performance optimization and HTTP semantics.

Core challenges you’ll face:

File detection and MIME types (maps to file type identification)
Caching headers (maps to HTTP cache control)
Range requests (maps to HTTP 206 Partial Content)
Directory traversal prevention (maps to security)
Efficient file serving (maps to performance)
Gzip compression (maps to bandwidth optimization)

Key Concepts:

MIME types: IANA Registry of MIME types
HTTP caching: RFC 7234
ETag and Last-Modified: “HTTP: The Definitive Guide” - Chapter 17
Range requests: RFC 7233

Difficulty: Beginner Time estimate: Weekend Prerequisites: File I/O in Python, HTTP header understanding

Real world outcome:

# First request - file not cached
$ curl -i http://localhost:8000/style.css
HTTP/1.1 200 OK
Content-Type: text/css
ETag: "abc123def456"
Last-Modified: Mon, 15 Jan 2024 10:00:00 GMT
Cache-Control: public, max-age=3600
Content-Length: 5432

# Second request - browser has it cached
$ curl -i http://localhost:8000/style.css
HTTP/1.1 304 Not Modified
ETag: "abc123def456"

# Range request for large video
$ curl -i --range 0-999 http://localhost:8000/video.mp4
HTTP/1.1 206 Partial Content
Content-Range: bytes 0-999/1000000
Content-Length: 1000

# Large file with compression
$ curl -i -H "Accept-Encoding: gzip" http://localhost:8000/data.json
HTTP/1.1 200 OK
Content-Encoding: gzip
Content-Length: 345  (smaller than original)

Implementation Hints:

Static file serving involves:

Path mapping: URL → file system path
Security check: Prevent path traversal (/../../etc/passwd)
MIME type detection: Based on file extension
File reading: Read the file and send bytes
Cache headers: Calculate ETag, Last-Modified, send cache headers
Conditional requests: Handle If-None-Match, If-Modified-Since
Range requests: Handle Range header, return 206 Partial Content
Compression: Compress if client supports it

MIME type map:

.html -> text/html
.css -> text/css
.js -> text/javascript
.png -> image/png
.jpg -> image/jpeg
.json -> application/json

Caching strategy:

Calculate ETag (hash of file content or modification time)
Set Last-Modified header
Send Cache-Control header
If client sends If-None-Match with matching ETag, return 304
If client sends If-Modified-Since with newer time, return 304

Learning milestones:

Files serve correctly → You understand file reading
MIME types are correct → You understand content negotiation
Caching headers work → You understand HTTP optimization
Range requests work → You understand advanced HTTP

Project 14: Asynchronous Request Handling (Concurrency)

Main Programming Language: Python (async/await)
Alternative Programming Languages: JavaScript (Node.js), Go, Rust
Coolness Level: Level 3: Genuinely Clever
Business Potential: 3. The “Service & Support” Model
Difficulty: Level 3: Advanced
Knowledge Area: Async Programming / Event Loops / Concurrency
Software or Tool: asyncio, uvloop
Main Book: “Fluent Python” by Luciano Ramalho

What you’ll build: An async-capable web framework using Python’s asyncio. Support async route handlers, async middleware, database queries that don’t block, and proper error handling. Compare throughput: blocking vs async.

Why it teaches web frameworks: Modern frameworks (FastAPI, Starlette) are async-first. Building async handling teaches you event loops, coroutines, and how to handle thousands of concurrent connections efficiently.

Core challenges you’ll face:

Async/await syntax and semantics (maps to understanding coroutines)
Event loop management (maps to understanding the reactor pattern)
Async context (request context in async) (maps to threading-like state)
Mixing sync and async code (maps to integration challenges)
Error handling in async (maps to exception propagation)

Key Concepts:

Asyncio documentation: Python asyncio module
Event loops: “Fluent Python” - Chapter 17-18 by Ramalho
Coroutines and async/await: PEP 492
Concurrency patterns: “Designing Data-Intensive Applications” - Chapter 12

Difficulty: Advanced Time estimate: 1 month+ Prerequisites: Understanding of async/await in Python, event loop concepts, Python 3.7+

Real world outcome:

# Async route handler
@app.route("/api/users/:id", "GET")
async def get_user(request):
    user_id = request.params["id"]

    # Async database query (non-blocking)
    user = await db.query("SELECT * FROM users WHERE id = ?", user_id)
    if not user:
        raise NotFoundError("User")

    # Async external API call
    github = await http.get(f"https://api.github.com/users/{user['github']}")

    return Response(200, json.dumps({
        "user": user,
        "github": github
    }))

# Async middleware
@app.use
async def logging_middleware(request, next):
    start = time.time()

    response = await next()

    duration = time.time() - start
    logger.info(f"{request.method} {request.path} took {duration:.2f}s")

    return response

# Run thousands of requests concurrently
# With blocking: one request at a time
# With async: handle thousands simultaneously

Implementation Hints:

An async framework needs:

Async route handlers: Routes can be async functions
Event loop: Run async code
Async middleware: Middleware can await next()
Async database: Database queries are non-blocking
Proper context: Request context per coroutine

Think about:

What’s the difference between async def and regular functions?
What does await do?
Why can async handle more concurrent requests?
How do you prevent blocking operations in async context?
How do you handle errors in async code?

Key insight: With async, one thread can handle thousands of connections:

Request A waits on database → yield to event loop
Request B waits on external API → yield to event loop
Request A’s database responds → resume Request A
etc.

With blocking, one thread = one request.

Learning milestones:

Async route handlers work → You understand async/await
Event loop properly handles multiple requests → You understand concurrency
Async middleware works → You understand composition
Error handling in async works → You understand exception propagation

Project 15: Testing Framework (Quality Assurance)

Main Programming Language: Python
Alternative Programming Languages: Go, Java, JavaScript
Coolness Level: Level 3: Genuinely Clever
Business Potential: 1. The “Resume Gold”
Difficulty: Level 2: Intermediate
Knowledge Area: Testing / Mocking / Test Automation
Software or Tool: pytest, mock, unittest
Main Book: “Test Driven Development: By Example” by Kent Beck

What you’ll build: A testing framework for your web framework itself. Support unit tests, integration tests, fixtures, mocking, and test discovery. Test your router, middleware, handlers, database layer.

Why it teaches web frameworks: Testing is critical for reliability. Building a test framework teaches you how to structure testable code, mock external dependencies, and verify behavior. Major frameworks include testing tools.

Core challenges you’ll face:

Test discovery and execution (maps to test runners)
Fixtures and setup/teardown (maps to test isolation)
Mocking and patching (maps to dependency injection)
Assertion helpers (maps to readable test failures)
Test organization (maps to test structure)

Key Concepts:

Testing strategies: “Test Driven Development: By Example” by Beck
Mocking: “unittest.mock documentation” Python
Fixtures: pytest documentation
Testing best practices: “Growing Object-Oriented Software, Guided by Tests” by Freeman & Pryce

Difficulty: Intermediate Time estimate: 1-2 weeks Prerequisites: Understanding of pytest or unittest, Python testing concepts

Real world outcome:

# Unit test for router
def test_router_static_match():
    router = Router()
    router.add("/users", list_users)

    match = router.match("/users")
    assert match["handler"] == list_users
    assert match["params"] == {}

def test_router_dynamic_match():
    router = Router()
    router.add("/users/:id", get_user)

    match = router.match("/users/42")
    assert match["params"]["id"] == "42"

# Integration test
@pytest.fixture
def client():
    app = create_app()
    return TestClient(app)

def test_login_flow(client):
    # Register
    response = client.post("/register", data={
        "username": "alice",
        "email": "alice@example.com",
        "password": "SecurePass123!"
    })
    assert response.status == 200

    # Login
    response = client.post("/login", data={
        "username": "alice",
        "password": "SecurePass123!"
    })
    assert response.status == 200
    token = response.json["token"]

    # Access protected route
    response = client.get("/profile", headers={"Authorization": f"Bearer {token}"})
    assert response.status == 200
    assert response.json["username"] == "alice"

# Test with mocking
@patch("app.database.query")
def test_get_user_not_found(mock_query):
    mock_query.return_value = None

    client = TestClient(app)
    response = client.get("/users/999")

    assert response.status == 404
    mock_query.assert_called_once_with("SELECT * FROM users WHERE id = ?", 999)

# Run tests
# pytest test_router.py -v
# pytest test_integration.py --cov=app

Implementation Hints:

A testing framework needs:

Test discovery: Find all test files and functions
Test execution: Run each test, collect results
Fixtures: Setup and teardown for each test
Assertions: Readable test failure messages
Mocking: Replace functions/objects for isolation
Coverage: Track which code is tested
Reporting: Show results in readable format

Think about:

How do you discover test files? (naming convention, directory structure)
How do you isolate tests from each other? (fixtures, database cleanup)
How do you mock external dependencies? (patch, Mock objects)
How do you know what to test? (logic, edge cases, error paths)
How do you structure tests for maintainability?

Learning milestones:

Test discovery and execution works → You understand test infrastructure
Fixtures work → You understand test isolation
Mocking works → You understand dependency injection
Tests are readable and maintainable → You understand good test design

Project 16: WebSocket Support (Real-Time Communication)

Main Programming Language: Python (aiohttp, Starlette)
Alternative Programming Languages: JavaScript (Node.js), Go, Rust
Coolness Level: Level 4: Hardcore Tech Flex
Business Potential: 2. The “Micro-SaaS / Pro Tool”
Difficulty: Level 3: Advanced
Knowledge Area: Real-Time Communication / Protocol Handling / Stateful Connections
Software or Tool: WebSocket libraries, browser WebSocket API
Main Book: “Web Development with Node.js” by Ethan Brown

What you’ll build: WebSocket support in your framework. Enable real-time bidirectional communication. Build a simple chat app: clients connect, send messages, see others’ messages in real-time.

Why it teaches web frameworks: Modern apps need real-time updates (chats, notifications, live feeds). WebSockets are the technology for this. Building this teaches you stateful connections, message framing, and keeping connections alive.

Core challenges you’ll face:

WebSocket handshake and upgrade (maps to HTTP upgrade mechanism)
Frame parsing and framing (maps to protocol handling)
Connection management (maps to tracking connected clients)
Broadcasting to multiple clients (maps to pub/sub patterns)
Handling disconnections (maps to connection lifecycle)
Backpressure and flow control (maps to message queuing)

Key Concepts:

WebSocket protocol: RFC 6455
Frame format and masking: RFC 6455 Section 5
Pub/Sub patterns: “Designing Data-Intensive Applications” - Chapter 4
Real-time communication: “Building Real-Time Web Applications” concepts

Difficulty: Advanced Time estimate: 1 month+ Prerequisites: Understanding of HTTP upgrade, event-driven programming, async/await

Real world outcome:

# WebSocket route
@app.websocket("/chat/:room_id")
async def chat_handler(ws, request):
    room_id = request.params["room_id"]

    # Join room
    broadcast = app.rooms.get_or_create(room_id)
    broadcast.subscribe(ws)

    try:
        # Listen for messages from this client
        async for message in ws:
            # Broadcast to all clients in the room
            broadcast.publish({
                "type": "message",
                "user": request.user.id,
                "text": message,
                "timestamp": datetime.now().isoformat()
            })
    finally:
        # Client disconnected
        broadcast.unsubscribe(ws)

# JavaScript client
<script>
const ws = new WebSocket('ws://localhost:8000/chat/room-42');

ws.onmessage = (event) => {
    const message = JSON.parse(event.data);
    console.log(`${message.user}: ${message.text}`);
};

// Send a message
const sendMessage = (text) => {
    ws.send(JSON.stringify({
        type: "message",
        text: text
    }));
};
</script>

Implementation Hints:

WebSocket support requires:

HTTP Upgrade: Detect and handle the WebSocket handshake
Frame parsing: Understand the WebSocket frame format
Connection management: Track active WebSocket connections
Message routing: Route messages to appropriate handlers
Pub/Sub: Broadcast messages to multiple clients
Heartbeat: Keep connections alive

WebSocket handshake (HTTP upgrade):

GET /chat HTTP/1.1
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Key: x3JJHMbDL1EzLkh9GBhXDw==
Sec-WebSocket-Version: 13

HTTP/1.1 101 Switching Protocols
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Accept: HSmrc0sMlYUkAGmm5OPpG2HaGWk=

WebSocket frame format (simplified):

FIN bit (is this the last fragment?)
Opcode (0=continuation, 1=text, 2=binary, 8=close)
Mask bit (client frames are masked)
Payload length
Mask key (if masked)
Payload data

Think about:

How do you upgrade an HTTP connection to WebSocket?
How do you parse WebSocket frames?
How do you handle fragmented messages?
How do you detect when a client disconnects?
How do you broadcast to multiple clients efficiently?
How do you handle backpressure (slow clients)?

Learning milestones:

WebSocket handshake works → You understand HTTP upgrade
Messages send and receive → You understand frame format
Multiple clients connect → You understand connection management
Broadcasting works → You understand pub/sub patterns

Project 17: Middleware Composition (Advanced Patterns)

Main Programming Language: Python
Alternative Programming Languages: JavaScript, Go, Rust
Coolness Level: Level 4: Hardcore Tech Flex
Business Potential: 1. The “Resume Gold”
Difficulty: Level 3: Advanced
Knowledge Area: Functional Programming / Composition / Design Patterns
Software or Tool: Flask, Express, Koa
Main Book: “Domain-Driven Design” by Eric Evans

What you’ll build: Advanced middleware patterns: middleware factories, conditional middleware, parallel middleware execution, and middleware stacks. Implement common middleware (CORS, compression, authentication) using your patterns.

Why it teaches web frameworks: Understanding middleware deeply teaches you functional composition, design patterns, and how to build extensible systems. This is where framework architecture shines.

Core challenges you’ll face:

Middleware factories (maps to higher-order functions)
Conditional execution (maps to decision logic in pipelines)
Parallel execution (maps to concurrent middleware)
Dependency injection (maps to passing context)
Composable middleware (maps to functional composition)

Key Concepts:

Higher-order functions: “Functional Programming in Python”
Design patterns: “Design Patterns” by Gang of Four
Middleware architectures: “Release It!” by Michael Nygard
Composition patterns: “Composing Software” by Eric Elliott

Difficulty: Advanced Time estimate: 1 month+ Prerequisites: Understanding of functional programming, decorators, design patterns

Real world outcome:

# Middleware factory
def cors_middleware(allowed_origins):
    def middleware(request, next):
        if request.headers.get("Origin") in allowed_origins:
            response = next()
            response.headers["Access-Control-Allow-Origin"] = request.headers["Origin"]
            return response
        return Response(403, "CORS not allowed")
    return middleware

# Conditional middleware
def if_method(methods, middleware):
    def conditional(request, next):
        if request.method in methods:
            return middleware(request, next)
        return next()
    return conditional

# Parallel middleware
def parallel_middleware(*middlewares):
    async def parallel(request, next):
        # Run middlewares in parallel before handler
        before_results = await asyncio.gather(*[
            m(request, lambda: None) for m in middlewares
        ])
        response = await next()
        return response
    return parallel

# Usage
app = Application()
app.use(cors_middleware(["https://example.com", "https://app.example.com"]))
app.use(if_method(["POST", "PUT", "DELETE"], auth_middleware))
app.use(parallel_middleware(logging, metrics, tracing))

Implementation Hints:

Advanced middleware patterns:

Middleware factories: Functions that return middleware
Conditional wrapping: Middleware that optionally wraps the handler
Parallel execution: Multiple middleware running concurrently
Dependency injection: Passing configuration to middleware
Middleware composition: Combining middleware into stacks

Think about:

How do you make middleware reusable with configuration?
How do you conditionally apply middleware?
How do you run middleware in parallel?
How do you handle errors in complex stacks?
How do you debug middleware chains?

Learning milestones:

Middleware factories work → You understand higher-order functions
Conditional middleware works → You understand conditional execution
Complex compositions work → You understand functional composition
Error handling in complex stacks works → You understand error propagation

Project 18: Full Framework Integration (Capstone)

Main Programming Language: Python
Alternative Programming Languages: Go, Rust, JavaScript
Coolness Level: Level 5: Pure Magic
Business Potential: 4. The “Open Core” Infrastructure
Difficulty: Level 4: Expert
Knowledge Area: Systems Architecture / Integration / Performance
Software or Tool: All previous projects combined
Main Book: “Building Microservices” by Sam Newman

What you’ll build: Integrate all previous projects into a cohesive, production-ready web framework. Handle everything: routing, middleware, templating, authentication, databases, testing, async, WebSockets. Build a real application (e.g., multi-user blog, project management tool).

Why it teaches web frameworks: This is where it all comes together. You’ll understand how all the pieces interact, optimize for real use cases, and see the design trade-offs.

Core challenges you’ll face:

Integration complexity (maps to systems thinking)
Performance optimization (maps to profiling and tuning)
Extensibility (maps to plugin systems)
Documentation (maps to API clarity)
Real-world constraints (maps to practical considerations)

Key Concepts:

Systems architecture: “Building Microservices” by Newman
Performance optimization: “Systems Performance” by Gregg
Extensibility: “The Architecture of Open Source Applications”
Real-world considerations: Lessons from Django, Flask, FastAPI

Difficulty: Expert Time estimate: 1-2 months Prerequisites: All previous projects completed, systems thinking, full Python expertise

Real world outcome:

A complete, working web framework that you can use to build real applications. Example application running on your framework:

$ my-framework start --port 8000
Application running on http://localhost:8000

# User registers
$ curl -X POST http://localhost:8000/register \
  -d "username=alice&email=alice@example.com&password=Pass123!"

# User creates blog post
$ curl -X POST http://localhost:8000/posts \
  -H "Authorization: Bearer <token>" \
  -H "Content-Type: multipart/form-data" \
  -F "title=My First Post" \
  -F "content=<content.html" \
  -F "featured_image=@/path/to/image.png"

# Real-time notifications via WebSocket
# Browser connects to ws://localhost:8000/notifications
# New comments appear instantly

Implementation Hints:

This is the ultimate integration project. You’re not learning single concepts anymore; you’re learning how they work together.

Consider:

Performance bottlenecks: Which parts are slow? (DB queries? Rendering?)
Extensibility: How can users add custom middleware or handlers?
Debugging: How do developers debug applications built with your framework?
Documentation: How do you teach others to use your framework?
Testing: How do you make it easy to test applications?

Real application example - Blog/CMS:

GET  /                          -> List posts
GET  /posts/:slug               -> View post
POST /posts                      -> Create post (auth required)
PUT  /posts/:id                 -> Edit post (auth required)
DELETE /posts/:id               -> Delete post (auth required)
POST /comments                  -> Add comment
WS   /notifications/:user_id    -> Real-time updates
GET  /static/*                  -> Serve CSS, images, JS

Learning milestones:

All components work together → You understand framework architecture
Application is performant → You understand optimization
Framework is extensible → You understand API design
Others can use your framework → You understand documentation and usability

Project Comparison Table

Project	Difficulty	Time	Depth of Understanding	Cool Factor
1. TCP Server	Beginner	Weekend	Foundation	⭐⭐
2. HTTP Parser	Beginner	Weekend	HTTP Protocol	⭐⭐⭐
3. Router	Intermediate	1-2 weeks	URL Matching	⭐⭐⭐
4. Response Builder	Beginner	Weekend	HTTP Responses	⭐⭐
5. Middleware	Intermediate	1-2 weeks	Design Patterns	⭐⭐⭐
6. Templates	Advanced	1 month+	Parsing/Compiling	⭐⭐⭐⭐
7. Request/Response	Intermediate	1-2 weeks	Abstractions	⭐⭐
8. Sessions	Intermediate	1-2 weeks	State Management	⭐⭐⭐
9. Form Parsing	Intermediate	1-2 weeks	Format Handling	⭐⭐
10. Authentication	Advanced	1 month+	Security	⭐⭐⭐
11. Error Handling	Intermediate	1-2 weeks	Robustness	⭐⭐
12. Database Layer	Advanced	1 month+	ORM Concepts	⭐⭐⭐
13. Static Files	Beginner	Weekend	Caching	⭐⭐
14. Async	Advanced	1 month+	Concurrency	⭐⭐⭐⭐
15. Testing	Intermediate	1-2 weeks	Quality	⭐⭐⭐
16. WebSockets	Advanced	1 month+	Real-Time	⭐⭐⭐⭐⭐
17. Middleware Composition	Advanced	1 month+	Architecture	⭐⭐⭐⭐
18. Full Integration	Expert	1-2 months	Everything	⭐⭐⭐⭐⭐

Recommended Learning Path

Phase 1: Foundation (Weeks 1-3)

Start with the fundamentals. These teach you the basics without being overwhelming.

Do these in order:

TCP Socket Server (understand the foundation)
HTTP Request Parser (understand the protocol)
Response Builder (understand responses)
Static File Serving (understand file serving)

By the end: You’ll have a working HTTP server that can serve files and understand the full request/response cycle.

Phase 2: Framework Basics (Weeks 4-8)

Now build the core framework features.

Do these in order:

Simple Router (map URLs to handlers)
Middleware Pipeline (cross-cutting concerns)
Request/Response Objects (abstractions)
Error Handling & Logging (robustness)

By the end: You’ll have a basic but functional web framework.

Phase 3: Features (Weeks 9-14)

Add important features that real apps need.

Pick and do (in any order):

Form Data Parsing (handle user input)
Session Management (stateful interactions)
Static Files (more advanced serving)
Authentication (user verification)

By the end: Your framework can handle real user interactions.

Phase 4: Advanced (Weeks 15-20)

Tackle the complex stuff.

Pick one or two:

Template Engine (dynamic HTML - most impactful)
Database Layer (data persistence)
Asynchronous Handling (high concurrency)
Testing Framework (code quality)

By the end: Professional-grade features.

Phase 5: Specialization (Weeks 21+)

Go deep on specific areas you’re interested in.

Optional advanced topics:

WebSocket Support (real-time)
Middleware Composition (advanced patterns)
Integration & Capstone (everything together)

Recommendation

Based on the typical developer learning journey:

Start here: Project 1 (TCP Server)

It’s the foundation everything else builds on
You need to understand raw sockets before HTTP abstractions
It’s achievable in a weekend
It builds confidence

Then: Project 2 (HTTP Parser) → Project 3 (Router) → Project 5 (Middleware)

These three are the core of any web framework
They’re doable in sequence
By project 5, you have something that works
Takes about 6-8 weeks

Then: Pick 2-3 from Phase 3 based on interest

Want to build real features? Do authentication + sessions
Want to understand template engines? Do that
Want databases? Do the database layer

Finally: Either template engine or async handling

Templates teach you language implementation
Async teaches you advanced concurrency
Both are career-changing knowledge

Capstone project last: Full integration

Only do this after you’ve learned the pieces
This is where you synthesize everything
The most rewarding and challenging part

Resources & Tools

Essential Tools

HTTP tester: curl (command line), Postman (GUI)
Network debugging: Wireshark, tcpdump
Database: SQLite (lightweight), PostgreSQL (production)
Testing: pytest (Python), unittest (built-in)

Books Referenced

“The Linux Programming Interface” by Michael Kerrisk
“HTTP: The Definitive Guide” by Gourley & Totty
“Fluent Python” by Luciano Ramalho
“Designing Data-Intensive Applications” by Martin Kleppmann
“Serious Cryptography” by Jean-Philippe Aumasson

Online Resources

Python asyncio: https://docs.python.org/3/library/asyncio.html
HTTP RFCs: https://tools.ietf.org/html/rfc7230-7239
WebSocket RFC: https://tools.ietf.org/html/rfc6455
OWASP: https://owasp.org/

Key Insights You’ll Gain

HTTP is simpler than you think - Just text with conventions
Routing is elegant pattern matching - Not magic, just algorithms
Middleware is function composition - Apply the same concept everywhere
Templates are mini-compilers - Parsing + compilation + execution
Async is about efficiency, not raw speed - One thread, many concurrent operations
Databases are complex - ORMs hide a lot of subtlety
Error handling is tedious but critical - Most code is error cases
Testing is design - Testable code is well-designed code
Performance matters - Profile, don’t guess
Everything is tradeoffs - Simplicity vs features, flexibility vs performance

What Makes This Different From Learning Flask/Django/FastAPI

Those frameworks are great, but they hide the implementation details. By building your own:

You understand why features exist
You understand how they work under the hood
You can debug issues because you understand the stack
You appreciate the design decisions frameworks make
You can build custom solutions for unique problems
You develop systems thinking

It’s like the difference between using a car vs understanding how engines work.

Final Thoughts

Building a web framework from scratch is one of the most educational programming projects you can undertake. Each piece teaches you fundamental concepts that apply far beyond web development:

Networking concepts apply to distributed systems
Parsing applies to interpreters, compilers, DSLs
Async/concurrency applies to systems programming
Databases apply to data engineering
Security applies everywhere

The projects get progressively harder, but each builds on the previous. Start with the TCP server, and by the time you reach the capstone, you’ll have a deep understanding of how modern web systems work.

The journey is the destination. Enjoy!

Summary of All Projects

#	Project	Language	Difficulty	Time	Domain
1	TCP Socket Server	Python	Beginner	Weekend	Networking
2	HTTP Request Parser	Python	Beginner	Weekend	Protocol
3	Simple Router	Python	Intermediate	1-2 weeks	Routing
4	Response Builder	Python	Beginner	Weekend	HTTP
5	Middleware Pipeline	Python	Intermediate	1-2 weeks	Architecture
6	Template Engine	Python	Advanced	1 month+	Parsing
7	Request/Response Objects	Python	Intermediate	1-2 weeks	Abstraction
8	Session Management	Python	Intermediate	1-2 weeks	State
9	Form Data Parsing	Python	Intermediate	1-2 weeks	Parsing
10	Authentication System	Python	Advanced	1 month+	Security
11	Error Handling & Logging	Python	Intermediate	1-2 weeks	Robustness
12	Database Layer	Python	Advanced	1 month+	Persistence
13	Static File Serving	Python	Beginner	Weekend	Optimization
14	Async Request Handling	Python	Advanced	1 month+	Concurrency
15	Testing Framework	Python	Intermediate	1-2 weeks	Quality
16	WebSocket Support	Python	Advanced	1 month+	Real-Time
17	Middleware Composition	Python	Advanced	1 month+	Architecture
18	Full Framework Integration	Python	Expert	1-2 months	Systems