Project 11: POP3 and IMAP Client

Build a client that can list and fetch email via POP3 and IMAP, comparing their state models.

Quick Reference

Attribute Value
Difficulty Intermediate
Time Estimate 2-3 weeks
Language Python (Alternatives: Go, Rust)
Prerequisites TCP clients, TLS basics
Key Topics POP3 states, IMAP commands, MIME parsing

1. Learning Objectives

  1. Implement POP3 and IMAP connections with TLS.
  2. List and fetch messages with both protocols.
  3. Handle IMAP mailboxes, flags, and search.
  4. Compare stateless POP3 to stateful IMAP behavior.

2. Theoretical Foundation

2.1 Core Concepts

  • POP3: Simple, stateless, download-and-delete model.
  • IMAP: Stateful, server-side folders, flags, and search.
  • TLS: STARTTLS or implicit TLS on 995/993.
  • MIME parsing: Email messages often contain multipart content.

2.2 Why This Matters

Understanding retrieval protocols reveals how email clients sync, how flags work, and why IMAP is dominant for modern clients.

2.3 Historical Context / Background

POP3 was designed for intermittent connectivity. IMAP evolved to support multiple clients and persistent server state.

2.4 Common Misconceptions

  • Misconception: POP3 and IMAP are interchangeable. Reality: IMAP supports complex server-side state.
  • Misconception: Message IDs are always stable. Reality: IMAP UID and sequence numbers differ.

3. Project Specification

3.1 What You Will Build

A CLI tool that connects to a mailbox, lists messages, and fetches message headers and bodies via POP3 and IMAP.

3.2 Functional Requirements

  1. Support POP3 USER/PASS and STAT/LIST/RETR/QUIT.
  2. Support IMAP LOGIN/SELECT/SEARCH/FETCH/LOGOUT.
  3. Handle STARTTLS or implicit TLS.
  4. Parse basic MIME structure for multipart messages.

3.3 Non-Functional Requirements

  • Performance: Fetch headers for 100 messages quickly.
  • Reliability: Handle server errors gracefully.
  • Usability: Provide clear output with message counts.

3.4 Example Usage / Output

$ ./mail-client --imap imap.example.com --user a --pass b --list
Mailbox INBOX: 32 messages
UID 1001  \Seen  Subject: Hello
UID 1002  \Recent Subject: Invoice

3.5 Real World Outcome

You can retrieve and inspect messages using both protocols and explain the tradeoffs between POP3 and IMAP.

4. Solution Architecture

4.1 High-Level Design

Client
  -> Connection (TLS)
  -> POP3 Handler
  -> IMAP Handler
  -> MIME Parser

4.2 Key Components

Component Responsibility Key Decisions
Connection TCP/TLS setup STARTTLS vs implicit
POP3 Implement POP3 states transaction/update states
IMAP Tag-based commands tag generation strategy
MIME Parse multipart basic boundary parsing

4.3 Data Structures

class ImapResponse:
    def __init__(self, tag, lines):
        self.tag = tag
        self.lines = lines

4.4 Algorithm Overview

Key Algorithm: IMAP Command Parsing

  1. Send tagged command (A001 LOGIN …).
  2. Read until tagged response is received.
  3. Parse untagged responses for data.

Complexity Analysis:

  • Time: O(n) for response size
  • Space: O(n)

5. Implementation Guide

5.1 Development Environment Setup

python -m venv .venv
source .venv/bin/activate

5.2 Project Structure

mail-client/
├── pop3.py
├── imap.py
├── mime.py
└── README.md

5.3 The Core Question You’re Answering

“How do email clients retrieve messages, and why do POP3 and IMAP behave so differently?”

5.4 Concepts You Must Understand First

Stop and research these before coding:

  1. POP3 states: authorization, transaction, update
  2. IMAP tags and untagged responses
  3. TLS for mail retrieval
  4. MIME boundaries and parts

5.5 Questions to Guide Your Design

  1. How will you generate and track IMAP tags?
  2. How do you map POP3 message numbers to IMAP UIDs?
  3. How will you parse multipart bodies?

5.6 Thinking Exercise

If a new message arrives while an IMAP client is connected, how does IMAP notify the client? How does POP3 handle this?

5.7 The Interview Questions They’ll Ask

  1. “What is the difference between IMAP UID and sequence number?”
  2. “Why is IMAP considered stateful?”
  3. “How does POP3 delete messages?”

5.8 Hints in Layers

Hint 1: Start with POP3

  • It is simpler and good for validating socket code.

Hint 2: Implement IMAP tags carefully

  • Each command must be tagged and matched.

Hint 3: Use a basic MIME parser

  • Do not try to implement full RFC 2045 immediately.

5.9 Books That Will Help

Topic Book Chapter
POP3 RFC 1939 All
IMAP RFC 3501 Sections 3-6
MIME RFC 2045 Sections 5-6

5.10 Implementation Phases

Phase 1: Foundation (4-5 days)

Goals:

  • Connect with TLS and implement POP3

Tasks:

  1. POP3 USER/PASS and STAT.
  2. LIST and RETR.

Checkpoint: Download a message via POP3.

Phase 2: Core Functionality (1 week)

Goals:

  • Implement IMAP list and fetch

Tasks:

  1. LOGIN and SELECT.
  2. SEARCH and FETCH headers.

Checkpoint: List messages with UIDs.

Phase 3: Polish and Edge Cases (4-5 days)

Goals:

  • MIME parsing and robust errors

Tasks:

  1. Parse multipart bodies.
  2. Handle server errors and timeouts.

Checkpoint: Fetch and display multipart content.

5.11 Key Implementation Decisions

Decision Options Recommendation Rationale
TLS implicit vs STARTTLS both common in real servers
IMAP parsing line-based vs full parser line-based simpler for learning
MIME handling simple vs full simple scope control

6. Testing Strategy

6.1 Test Categories

Category Purpose Examples
Unit Tests Tag parsing correct tag matching
Integration Tests Real server local IMAP/POP3
Edge Case Tests Large mailbox handle paging

6.2 Critical Test Cases

  1. IMAP tagged response properly matched.
  2. POP3 retrieval gets full message.
  3. MIME multipart parsed into parts.

6.3 Test Data

Content-Type: multipart/alternative; boundary=abc

7. Common Pitfalls and Debugging

7.1 Frequent Mistakes

Pitfall Symptom Solution
Wrong IMAP tag handling hangs read until tag line
Misreading POP3 dot termination truncated message handle dot-stuffing
MIME parsing errors missing parts parse boundaries carefully

7.2 Debugging Strategies

  • Use server logs or test servers.
  • Log raw responses for inspection.

7.3 Performance Traps

  • Fetching full messages when headers only are needed. Use IMAP FETCH BODY.PEEK[HEADER].

8. Extensions and Challenges

8.1 Beginner Extensions

  • Add support for TOP in POP3.
  • Add IMAP IDLE for notifications.

8.2 Intermediate Extensions

  • Implement IMAP flags updates.
  • Support folder listing and selection.

8.3 Advanced Extensions

  • Build a sync cache for offline access.
  • Implement OAuth2 auth for IMAP.

9. Real-World Connections

9.1 Industry Applications

  • Mail clients use IMAP for sync.
  • Legacy systems still use POP3 for batch retrieval.
  • OfflineIMAP: https://www.offlineimap.org/
  • mu: https://github.com/djcb/mu

9.3 Interview Relevance

  • IMAP state management and protocol parsing are common in networking interviews.

10. Resources

10.1 Essential Reading

  • RFC 1939 - POP3
  • RFC 3501 - IMAP
  • RFC 2045 - MIME

10.2 Video Resources

  • IMAP and POP3 protocol walkthroughs

10.3 Tools and Documentation

  • openssl s_client for testing TLS
  • mailutils for reference behavior

11. Self-Assessment Checklist

11.1 Understanding

  • I can explain POP3 vs IMAP
  • I understand IMAP tags
  • I can parse MIME boundaries

11.2 Implementation

  • Lists and fetches via POP3
  • Lists and fetches via IMAP
  • Handles TLS correctly

11.3 Growth

  • I can explain why IMAP is stateful
  • I can design a sync strategy

12. Submission / Completion Criteria

Minimum Viable Completion:

  • Connect to POP3 and retrieve a message

Full Completion:

  • Implement IMAP listing and fetching

Excellence (Going Above and Beyond):

  • Add IMAP IDLE and offline sync

This guide was generated from EMAIL_SYSTEMS_DEEP_DIVE_PROJECTS.md. For the complete learning path, see the parent directory.