Project 7: Mini SMTP Server (Receive and Store)

Build an SMTP server that accepts inbound mail and stores it in Maildir format.

Quick Reference

Attribute Value
Difficulty Advanced
Time Estimate 3-4 weeks
Language C (Alternatives: Go, Rust)
Prerequisites TCP servers, concurrency, file I/O
Key Topics SMTP server state machine, Maildir, queueing

1. Learning Objectives

  1. Implement a robust SMTP server state machine.
  2. Safely accept and store messages in Maildir format.
  3. Handle concurrent connections with proper isolation.
  4. Return correct SMTP reply codes for errors and edge cases.

2. Theoretical Foundation

2.1 Core Concepts

  • SMTP server states: Connection, HELO/EHLO, MAIL FROM, RCPT TO, DATA.
  • Maildir storage: Messages stored in new/cur/tmp directories for atomic delivery.
  • Queueing and durability: Ensure messages are saved before acknowledging.
  • Backpressure: Use 4xx for temporary errors when overloaded.

2.2 Why This Matters

A server needs to be correct under concurrent load, handle partial input, and remain secure against malformed clients. This is the core of real mail infrastructure.

2.3 Historical Context / Background

SMTP servers predate modern security layers, so strict protocol adherence and careful parsing are essential to avoid abuse.

2.4 Common Misconceptions

  • Misconception: Accepting DATA means the message is stored. Reality: You must confirm durable storage before 250.
  • Misconception: You can ignore 4xx. Reality: Proper 4xx responses allow senders to retry.

3. Project Specification

3.1 What You Will Build

A TCP server that implements SMTP commands, receives messages, and stores them in Maildir. It should log sessions and protect against malformed input.

3.2 Functional Requirements

  1. Support HELO/EHLO, MAIL FROM, RCPT TO, DATA, RSET, QUIT.
  2. Enforce proper state transitions.
  3. Store messages in Maildir format with unique filenames.
  4. Limit message size and reject oversized input.
  5. Return proper 4xx/5xx codes for errors.

3.3 Non-Functional Requirements

  • Performance: Handle 50 concurrent connections locally.
  • Reliability: Never ack a message before it is stored.
  • Security: Reject suspicious inputs and line length abuse.

3.4 Example Usage / Output

$ ./mini-smtp --listen 2525 --maildir /tmp/maildir
[READY] Listening on :2525
[CONN] 127.0.0.1
[SMTP] EHLO client
[SMTP] MAIL FROM:<a@example.com>
[SMTP] RCPT TO:<b@example.net>
[SMTP] DATA
[STORE] /tmp/maildir/new/1700000000.12345.local
[SMTP] 250 Message accepted

3.5 Real World Outcome

You can connect with any SMTP client and store inbound mail locally, observing the same protocol behavior used by production MTAs.

4. Solution Architecture

4.1 High-Level Design

TCP Listener
  -> Session Handler
    -> SMTP State Machine
    -> Message Receiver
    -> Maildir Writer

4.2 Key Components

Component Responsibility Key Decisions
Listener Accept connections Thread-per-conn or event loop
Session Track SMTP state Explicit state enum
Receiver Read DATA with dot termination Enforce size limit
Storage Write to Maildir Write to tmp then rename

4.3 Data Structures

typedef enum {
    STATE_NEW,
    STATE_HELO,
    STATE_MAIL,
    STATE_RCPT,
    STATE_DATA
} smtp_state_t;

4.4 Algorithm Overview

Key Algorithm: DATA Collection

  1. After 354, read lines until a lone dot.
  2. Un-dot-stuff any line starting with ‘..’.
  3. Write to temp file, then atomically move to new/.

Complexity Analysis:

  • Time: O(n) for message length
  • Space: O(n) streamed to disk

5. Implementation Guide

5.1 Development Environment Setup

cc -Wall -Wextra -o mini-smtp mini_smtp.c

5.2 Project Structure

mini-smtp/
├── server.c
├── session.c
├── maildir.c
└── README.md

5.3 The Core Question You’re Answering

“How does a mail server safely accept and store inbound messages?”

5.4 Concepts You Must Understand First

Stop and research these before coding:

  1. SMTP state transitions
  2. Maildir format
  3. CRLF handling and line limits
  4. Concurrency model

5.5 Questions to Guide Your Design

  1. How will you handle pipelined commands?
  2. What is the maximum message size you will accept?
  3. How will you enforce per-session state?

5.6 Thinking Exercise

If the client sends MAIL FROM twice without RSET, how should the server respond? Which RFC section clarifies this?

5.7 The Interview Questions They’ll Ask

  1. “Why does SMTP use dot-termination for DATA?”
  2. “How does Maildir ensure atomic delivery?”
  3. “What is the difference between 4xx and 5xx errors?”

5.8 Hints in Layers

Hint 1: Implement line reader with limits

  • Protect against very long lines.

Hint 2: Track state carefully

  • Reject invalid command sequences.

Hint 3: Use tmp then rename

  • For Maildir atomicity.

5.9 Books That Will Help

Topic Book Chapter
SMTP RFC 5321 Section 4
File I/O TLPI Ch. 4-5
Concurrency CS:APP Ch. 12

5.10 Implementation Phases

Phase 1: Foundation (1 week)

Goals:

  • Listener and basic command parsing

Tasks:

  1. Accept TCP connections.
  2. Read lines and respond to HELO.

Checkpoint: Handles HELO and QUIT.

Phase 2: Core Functionality (1-2 weeks)

Goals:

  • Full SMTP state machine

Tasks:

  1. Implement MAIL, RCPT, DATA.
  2. Store messages in Maildir.

Checkpoint: Receives and stores test message.

Phase 3: Polish and Edge Cases (1 week)

Goals:

  • Limits, errors, concurrency

Tasks:

  1. Enforce message size and line length.
  2. Add concurrency model.

Checkpoint: Handles multiple simultaneous clients.

5.11 Key Implementation Decisions

Decision Options Recommendation Rationale
Concurrency threads vs event loop threads Simpler for learning
Storage mbox vs Maildir Maildir Safer and atomic
Limits no limits vs strict strict Prevent abuse

6. Testing Strategy

6.1 Test Categories

Category Purpose Examples
Unit Tests State transitions invalid command order
Integration Tests SMTP send use swaks or smtp-client
Edge Case Tests Large messages size limit enforcement

6.2 Critical Test Cases

  1. DATA without RCPT returns 503.
  2. Oversized message returns 552.
  3. Dot-stuffing is handled correctly.

6.3 Test Data

From: a@example.com
To: b@example.net
Subject: Test

Body line
.

7. Common Pitfalls and Debugging

7.1 Frequent Mistakes

Pitfall Symptom Solution
Ack before writing Message loss Write then 250
Broken dot-stuffing Truncated message Unescape leading dots
Weak parsing crashes Enforce line length and timeouts

7.2 Debugging Strategies

  • Log every state transition and command.
  • Use swaks to simulate edge cases.

7.3 Performance Traps

  • Per-line disk flush in DATA. Buffer writes.

8. Extensions and Challenges

8.1 Beginner Extensions

  • Add basic SMTP AUTH (PLAIN over TLS).
  • Support multiple recipients.

8.2 Intermediate Extensions

  • Add a queue for deferred delivery.
  • Implement STARTTLS with local cert.

8.3 Advanced Extensions

  • Add SPF checks during SMTP.
  • Build a policy engine for rejections.

9. Real-World Connections

9.1 Industry Applications

  • MTAs use the same state machine and storage patterns.
  • Security gateways inspect and filter before acceptance.
  • Postfix: https://www.postfix.org/
  • Exim: https://www.exim.org/

9.3 Interview Relevance

  • SMTP server design and safe storage are common systems interview topics.

10. Resources

10.1 Essential Reading

  • RFC 5321 - SMTP server behavior
  • Maildir documentation - delivery format

10.2 Video Resources

  • SMTP server implementation walkthroughs

10.3 Tools and Documentation

  • swaks for SMTP testing
  • openssl s_client for TLS testing

11. Self-Assessment Checklist

11.1 Understanding

  • I can explain SMTP server states
  • I understand Maildir atomic writes
  • I can interpret SMTP reply codes

11.2 Implementation

  • Stores messages safely before ack
  • Handles malformed input
  • Enforces size limits

11.3 Growth

  • I can reason about SMTP backpressure
  • I can extend to add auth and filtering

12. Submission / Completion Criteria

Minimum Viable Completion:

  • Accepts SMTP and stores messages in Maildir

Full Completion:

  • Handles concurrency and errors correctly

Excellence (Going Above and Beyond):

  • Adds STARTTLS and SPF checks
  • Implements basic queue management

This guide was generated from EMAIL_SYSTEMS_DEEP_DIVE_PROJECTS.md. For the complete learning path, see the parent directory.