Project 15: Terminal Emulator with PTY

Build a fully-functional terminal emulator that creates PTY pairs, forks shells, parses ANSI escape sequences, and renders output using ncurses. Understand exactly how xterm, iTerm, and gnome-terminal work internally.


Quick Reference

Attribute Value
Difficulty Level 5 - Master
Time 3-4 Weeks
Language C (alt: Rust)
Prerequisites Projects 4, 6, 9; Understanding of fork/exec, signals, sessions
Key Topics PTY master/slave, posix_openpt, termios, ANSI escape codes, raw mode, job control
APUE Chapters 18, 19
Coolness Level 5 - Pure Magic
Portfolio Value Hardcore Tech Flex

Learning Objectives

By completing this project, you will:

  1. Master pseudo-terminal architecture: Understand the PTY master/slave relationship and how terminals connect to processes
  2. Control terminal modes: Switch between raw and canonical modes, control echo, signals, and line discipline
  3. Parse ANSI escape sequences: Implement a state machine to decode cursor movement, colors, and screen control
  4. Implement job control: Understand how the shell manages foreground/background processes through the controlling terminal
  5. Handle window resizing: Propagate SIGWINCH and update terminal dimensions with ioctl
  6. Build a complete terminal stack: Connect keyboard input to shell input, shell output to screen rendering
  7. Debug complex I/O flows: Trace data through multiple layers (user input -> PTY master -> line discipline -> PTY slave -> shell -> PTY slave -> PTY master -> your renderer)

Theoretical Foundation

What Is a Pseudo-Terminal?

A pseudo-terminal (PTY) is a pair of virtual devices that provide a bidirectional communication channel that looks like a terminal to programs. It consists of:

                    The PTY Architecture

    Your Terminal         Kernel           Shell/Program
    Emulator                             (e.g., bash, vim)
    ┌─────────────┐    ┌─────────────┐    ┌─────────────┐
    │             │    │   Line      │    │             │
    │  PTY Master │◄──►│ Discipline  │◄──►│  PTY Slave  │
    │   (fd)      │    │  (in kernel)│    │  (/dev/pts/N)│
    │             │    │             │    │             │
    └─────────────┘    └─────────────┘    └─────────────┘
          │                  │                   │
          │                  │                   │
    ┌─────▼─────┐       ┌────▼────┐       ┌──────▼──────┐
    │ You read/ │       │ Echo,   │       │ Shell thinks│
    │ write here│       │ signals,│       │ it's a real │
    │           │       │ editing │       │ terminal    │
    └───────────┘       └─────────┘       └─────────────┘

Master side (your terminal emulator):

  • You open with posix_openpt() or /dev/ptmx
  • Receives output from programs (what to display)
  • Sends input to programs (keystrokes)

Slave side (the shell and its children):

  • Opened as /dev/pts/N
  • Becomes stdin/stdout/stderr for the shell
  • The shell thinks it’s talking to a real terminal

Line discipline:

  • Kernel layer between master and slave
  • Handles echo (typing shows characters)
  • Handles line editing (backspace works)
  • Generates signals (Ctrl+C -> SIGINT)
  • Converts CR to NL, etc.

Why This Architecture?

Historical context: In the 1960s-70s, terminals were physical devices connected via serial lines. Programs expected to talk to these devices. When we moved to software terminals (xterm, gnome-terminal), we needed a way to:

  1. Make the shell think it’s talking to a real terminal
  2. Give the terminal emulator control over the “terminal”
  3. Keep the kernel’s terminal processing (line discipline) working

The PTY achieves all three. It’s a “pseudo” terminal because it’s not a real hardware device, but it behaves identically from the program’s perspective.

Terminal Modes: Raw vs Canonical

The terminal can operate in different modes:

Canonical Mode (Cooked)              Raw Mode
┌─────────────────────────┐         ┌─────────────────────────┐
│ • Input buffered by line│         │ • Each char available   │
│ • Backspace works       │         │   immediately           │
│ • Echo is on            │         │ • No line editing       │
│ • Special chars work    │         │ • No echo               │
│   (Ctrl+C, Ctrl+D)      │         │ • No signal generation  │
│                         │         │                         │
│ Good for: shells, most  │         │ Good for: vim, less,    │
│ command-line tools      │         │ games, your emulator    │
└─────────────────────────┘         └─────────────────────────┘

Your terminal emulator needs to:

  1. Put YOUR stdin (from the window system) in raw mode
  2. Let the PTY’s line discipline handle the slave side normally

ANSI Escape Sequences

Programs control the terminal by sending special byte sequences. The most common format is CSI (Control Sequence Introducer):

ESC [ <params> <final_byte>
 │   │    │         │
 │   │    │         └── Single char: command type (A, B, H, m, K, J, etc.)
 │   │    │
 │   │    └── Semicolon-separated numbers: parameters
 │   │
 │   └── Left bracket: CSI introducer
 │
 └── Escape character (0x1B)

Examples:
  ESC[2J     → Clear entire screen
  ESC[H      → Move cursor to home (0,0)
  ESC[5;10H  → Move cursor to row 5, column 10
  ESC[31m    → Set text color to red
  ESC[0m     → Reset attributes
  ESC[?25h   → Show cursor
  ESC[?25l   → Hide cursor
  ESC[K      → Clear from cursor to end of line

The Escape Sequence State Machine

                    ANSI Escape Sequence Parser

     ┌─────────────────────────────────────────────────────────┐
     │                                                         │
     │   ┌──────────┐    ESC     ┌──────────┐                 │
     │   │  NORMAL  │───────────►│   ESC    │                 │
     │   │  STATE   │            │  STATE   │                 │
     │   └────┬─────┘            └────┬─────┘                 │
     │        │                       │                       │
     │   printable              '[' (0x5B)                    │
     │   char -> emit                 │                       │
     │        │                       ▼                       │
     │        │               ┌──────────┐                    │
     │        │               │   CSI    │                    │
     │        │               │  STATE   │                    │
     │        │               └────┬─────┘                    │
     │        │                    │                          │
     │        │         ┌──────────┼──────────┐               │
     │        │         │          │          │               │
     │        │     '0'-'9'      ';'      '@'-'~'             │
     │        │     digit      separator   final              │
     │        │         │          │       (command)          │
     │        │         ▼          ▼          │               │
     │        │    ┌────────┐ ┌────────┐      │               │
     │        │    │ Build  │ │ Push   │      │               │
     │        │    │ number │ │ param  │      │               │
     │        │    └────────┘ └────────┘      │               │
     │        │         │          │          │               │
     │        │         └────┬─────┘          │               │
     │        │              ▼                ▼               │
     │        │         ┌─────────────────────────┐           │
     │        │         │   Execute command with   │           │
     │        │         │   collected parameters   │           │
     │        │         └─────────────────────────┘           │
     │        │                    │                          │
     │        └────────────────────┴────────► back to NORMAL  │
     │                                                         │
     └─────────────────────────────────────────────────────────┘

The Complete Data Flow

When user presses a key:

1. User presses 'A'
       │
       ▼
2. Window system delivers keypress to your process
       │
       ▼
3. Your emulator: read(STDIN_FILENO) returns 'A'
       │
       ▼
4. Your emulator: write(master_fd, "A", 1)
       │
       ▼
5. Kernel line discipline (on PTY):
   - If ECHO set: copy 'A' back to master (for display)
   - Buffer in line buffer (if canonical mode)
   - Or immediately pass to slave (if raw mode)
       │
       ▼
6. Shell: read(STDIN_FILENO) returns 'A'
   (Shell's stdin is the PTY slave)
       │
       ▼
7. Shell processes input, eventually runs command
       │
       ▼
8. Command writes output to stdout (PTY slave)
       │
       ▼
9. Your emulator: read(master_fd) returns command output
       │
       ▼
10. Your emulator: parse for escape sequences, render to screen

Project Specification

What You Will Build

A terminal emulator named myterm that:

  1. Creates a PTY pair (master/slave)
  2. Forks a child process running a shell
  3. Displays a curses-based terminal window
  4. Handles keyboard input and sends to PTY master
  5. Reads PTY master output and renders it
  6. Parses ANSI escape sequences for proper rendering
  7. Supports window resize (SIGWINCH)
  8. Runs full-screen programs (vim, htop, less)

Requirements

Core Requirements:

  • Create PTY with posix_openpt(), grantpt(), unlockpt(), ptsname()
  • Fork child, establish session with setsid()
  • Connect PTY slave to child’s stdin/stdout/stderr
  • Put host stdin in raw mode
  • Handle SIGWINCH and propagate size with TIOCSWINSZ
  • Parse basic ANSI escape sequences (cursor movement, clear, colors)
  • Render output using ncurses or direct terminal control

Advanced Requirements:

  • Scrollback buffer (configurable size)
  • Support for 256 colors and true color
  • Mouse input passthrough
  • Copy/paste integration
  • Unicode support (UTF-8)

Example Output

# 1. Start your terminal emulator
$ ./myterm
┌────────────────────────────────────────────────────────┐
│ myterm v1.0                                             │
│                                                         │
│ user@host:~$                                            │
│                                                         │
└────────────────────────────────────────────────────────┘

# 2. Run commands with color
user@host:~$ ls -la --color
drwxr-xr-x  5 user user 4096 Mar 15 10:00 .
-rw-r--r--  1 user user  123 Mar 15 09:00 file.txt  # (colors shown!)

# 3. Run vim - full screen app works!
user@host:~$ vim test.txt
┌────────────────────────────────────────────────────────┐
│ ~                                                       │
│ ~                                                       │
│ ~                                                       │
│ ~                                                       │
│ "test.txt" [New File]                  1,1          All│
└────────────────────────────────────────────────────────┘

# 4. Run htop - complex curses app
user@host:~$ htop
┌────────────────────────────────────────────────────────┐
│  CPU[|||||||||                          20.0%]          │
│  Mem[||||||||||||||||||||               2.1G/8.0G]      │
│                                                         │
│  PID USER      PRI  NI  VIRT   RES   SHR S CPU% MEM%    │
│  123 user       20   0  450M   94M   12M S  5.0  2.3    │
└────────────────────────────────────────────────────────┘

# 5. Window resize - SIGWINCH propagated
(Window resized)
[Shell and programs receive SIGWINCH, redraw correctly]

Solution Architecture

High-Level Design

                        Terminal Emulator Architecture

┌────────────────────────────────────────────────────────────────────┐
│                        YOUR TERMINAL EMULATOR                       │
│                                                                     │
│  ┌─────────────┐    ┌─────────────┐    ┌─────────────────────────┐ │
│  │   Input     │    │   Main      │    │      Renderer           │ │
│  │   Handler   │───►│   Loop      │◄───│   (ncurses or raw)      │ │
│  │             │    │             │    │                         │ │
│  │ - Raw mode  │    │ - select()  │    │ - Screen buffer         │ │
│  │ - Keyboard  │    │ - Dispatch  │    │ - Escape parser         │ │
│  │ - Signals   │    │   events    │    │ - Color support         │ │
│  └─────────────┘    └──────┬──────┘    └─────────────────────────┘ │
│                            │                                        │
│                     ┌──────▼──────┐                                 │
│                     │  PTY Master │                                 │
│                     │     FD      │                                 │
│                     └──────┬──────┘                                 │
└─────────────────────────────┼──────────────────────────────────────┘
                              │
                     ┌────────▼────────┐
                     │  Kernel (Line   │
                     │  Discipline)    │
                     └────────┬────────┘
                              │
┌─────────────────────────────┼──────────────────────────────────────┐
│                     ┌───────▼───────┐                               │
│                     │   PTY Slave   │                               │
│                     │  (/dev/pts/N) │                               │
│                     └───────┬───────┘                               │
│                             │                                       │
│               ┌─────────────┼─────────────┐                         │
│               │             │             │                         │
│               ▼             ▼             ▼                         │
│            stdin         stdout        stderr                       │
│               │             │             │                         │
│               └─────────────┼─────────────┘                         │
│                             │                                       │
│                      ┌──────▼──────┐                                │
│                      │    SHELL    │                                │
│                      │   (bash)    │                                │
│                      └──────┬──────┘                                │
│                             │ fork/exec                             │
│                      ┌──────▼──────┐                                │
│                      │  COMMANDS   │                                │
│                      │ (ls, vim)   │                                │
│                      └─────────────┘                                │
│                                                                     │
│                          CHILD PROCESS                              │
└─────────────────────────────────────────────────────────────────────┘

Key Data Structures

/* Screen cell - what to display at each position */
typedef struct {
    uint32_t codepoint;       /* Unicode character */
    uint8_t fg_color;         /* Foreground color (0-255) */
    uint8_t bg_color;         /* Background color (0-255) */
    uint8_t attrs;            /* Bold, italic, underline, etc. */
} cell_t;

/* Terminal screen state */
typedef struct {
    int rows;                 /* Terminal height */
    int cols;                 /* Terminal width */
    int cursor_row;           /* Current cursor row */
    int cursor_col;           /* Current cursor column */
    cell_t **cells;           /* 2D array of cells [row][col] */
    cell_t **scrollback;      /* Scrollback buffer */
    int scrollback_lines;     /* Lines in scrollback */
    int scrollback_size;      /* Max scrollback lines */

    /* Current attributes for new characters */
    uint8_t current_fg;
    uint8_t current_bg;
    uint8_t current_attrs;

    /* Saved cursor state (for ESC 7 / ESC 8) */
    int saved_row;
    int saved_col;
} screen_t;

/* Escape sequence parser state */
typedef enum {
    STATE_NORMAL,             /* Normal character processing */
    STATE_ESC,                /* Just saw ESC */
    STATE_CSI,                /* In CSI sequence (ESC [) */
    STATE_OSC,                /* Operating System Command (ESC ]) */
    STATE_DCS,                /* Device Control String */
    STATE_CHARSET             /* Charset selection */
} parser_state_t;

typedef struct {
    parser_state_t state;
    int params[16];           /* Numeric parameters */
    int param_count;          /* Number of parameters */
    char intermediate;        /* Intermediate character */
    int private_mode;         /* ? prefix for private modes */
} parser_t;

/* Main terminal context */
typedef struct {
    int master_fd;            /* PTY master file descriptor */
    pid_t child_pid;          /* Shell process PID */
    screen_t screen;          /* Display buffer */
    parser_t parser;          /* Escape sequence parser */
    struct termios saved_tios; /* Original terminal settings */
    int running;              /* Main loop flag */
} terminal_t;

Module Structure

myterm/
├── src/
│   ├── main.c              # Entry point, main loop
│   ├── terminal.c          # Terminal context management
│   ├── terminal.h
│   ├── pty.c               # PTY creation and management
│   ├── pty.h
│   ├── screen.c            # Screen buffer management
│   ├── screen.h
│   ├── parser.c            # ANSI escape sequence parser
│   ├── parser.h
│   ├── render.c            # ncurses or raw rendering
│   ├── render.h
│   ├── input.c             # Keyboard input handling
│   ├── input.h
│   └── signals.c           # Signal handlers (SIGWINCH, SIGCHLD)
├── tests/
│   ├── test_parser.c       # Unit tests for escape parser
│   ├── test_screen.c       # Unit tests for screen operations
│   └── escape_sequences.txt # Test vectors
├── Makefile
└── README.md

Implementation Guide

Setup

# Required packages (Ubuntu/Debian)
sudo apt-get install libncurses-dev

# macOS
brew install ncurses

# Create project
mkdir -p myterm/src myterm/tests
cd myterm

The Core Question You’re Answering

“How does a terminal emulator translate between a shell process and what you see on screen?”

This is the deepest dive into UNIX TTY handling. You’ll understand PTY pairs, line discipline, terminal attributes, and the entire escape code protocol that powers every terminal.

Concepts You Must Understand First

Stop and research these before coding:

  1. Pseudo Terminals (APUE Ch. 19)
    • What is the difference between PTY master and slave?
    • Why does the shell need a controlling terminal?
    • What happens when you open the PTY slave?
  2. Terminal Attributes - termios (APUE Ch. 18)
    • What are c_iflag, c_oflag, c_cflag, c_lflag?
    • How do you set raw mode?
    • What is the difference between ICANON and ~ICANON?
  3. Line Discipline
    • What processing happens between master and slave?
    • When is ECHO applied?
    • How are signals generated from Ctrl+C?
  4. ANSI Escape Codes
    • What is a CSI sequence?
    • How do cursor movement commands work?
    • How are colors specified (SGR - Select Graphic Rendition)?

Questions to Guide Your Design

Before implementing, think through:

  1. PTY Creation Order
    • Why must you call grantpt() and unlockpt() before fork()?
    • Why must the child call setsid() before opening the slave?
  2. Input Handling
    • How do you detect special keys (arrows, function keys)?
    • What byte sequences do arrow keys generate?
    • How do you handle Ctrl+C without terminating your emulator?
  3. Output Parsing
    • How do you handle incomplete escape sequences at buffer boundaries?
    • What do you do with unknown/unsupported sequences?
    • How do you handle malformed sequences gracefully?
  4. Screen Buffer
    • How do you represent the terminal screen in memory?
    • How do you handle scrolling and scrollback?
    • When do you need to redraw vs. incremental update?

Thinking Exercise

Trace a Keystroke Through the System:

User presses 'A'
     │
     ▼
┌────────────────┐
│ OS/Window Sys  │ → Hardware interrupt, delivered to your process
└───────┬────────┘
        │ (via X11/Wayland/direct terminal)
        ▼
┌────────────────┐
│ Your Terminal  │ → read(STDIN_FILENO) returns 'A' (0x41)
│ Emulator       │
└───────┬────────┘
        │ write(master_fd, "A", 1)
        ▼
┌────────────────┐
│ PTY Master     │ → Kernel receives byte
└───────┬────────┘
        │ (through line discipline)
        ▼
┌────────────────┐
│ Line Discipline│ → If ECHO set: copies 'A' back to master
│                │ → If ICANON set: buffers until newline
│                │ → If not: passes through immediately
└───────┬────────┘
        ▼
┌────────────────┐
│ PTY Slave      │ → Available for read()
└───────┬────────┘
        ▼
┌────────────────┐
│ Shell (bash)   │ → read() returns 'A'
└───────┬────────┘
        │ (Shell waits for Enter, then executes)
        │
        │ ... user presses Enter ...
        │
        │ Shell runs command, writes output to stdout
        ▼
┌────────────────┐
│ PTY Slave      │ → write(stdout, output)
└───────┬────────┘
        ▼
┌────────────────┐
│ Line Discipline│ → Output processing (if any)
└───────┬────────┘
        ▼
┌────────────────┐
│ PTY Master     │ → Becomes readable
└───────┬────────┘
        │ Your emulator: select() returns, read(master_fd)
        ▼
┌────────────────┐
│ Your Terminal  │ → Parse output for escape sequences
│ Emulator       │ → Update screen buffer
│                │ → Render to display
└────────────────┘

Questions to consider:
- What if ECHO is disabled? (e.g., password prompt)
- What happens when user presses Ctrl+C?
- Where do escape sequences like ESC[H come from?
- What if the shell is in raw mode (like vim)?

Hints in Layers

Hint 1: Creating a PTY

#include <stdlib.h>
#include <fcntl.h>
#include <unistd.h>

int create_pty(char *slave_name, size_t name_len) {
    int master = posix_openpt(O_RDWR | O_NOCTTY);
    if (master < 0) return -1;

    if (grantpt(master) < 0) {
        close(master);
        return -1;
    }

    if (unlockpt(master) < 0) {
        close(master);
        return -1;
    }

    char *name = ptsname(master);
    if (!name) {
        close(master);
        return -1;
    }

    strncpy(slave_name, name, name_len);
    return master;
}

Hint 2: Setting Raw Mode on Your Stdin

#include <termios.h>

struct termios saved_termios;

void set_raw_mode(void) {
    struct termios raw;

    tcgetattr(STDIN_FILENO, &saved_termios);
    raw = saved_termios;

    /* Input flags: no break, no CR to NL, no parity check,
       no strip char, no start/stop output control */
    raw.c_iflag &= ~(BRKINT | ICRNL | INPCK | ISTRIP | IXON);

    /* Output flags: disable post processing */
    raw.c_oflag &= ~(OPOST);

    /* Control flags: set 8 bit chars */
    raw.c_cflag |= (CS8);

    /* Local flags: no echo, no canonical, no extended,
       no signal chars (^C, ^Z, etc.) */
    raw.c_lflag &= ~(ECHO | ICANON | IEXTEN | ISIG);

    /* Control chars: return immediately with any amount of data */
    raw.c_cc[VMIN] = 1;
    raw.c_cc[VTIME] = 0;

    tcsetattr(STDIN_FILENO, TCSAFLUSH, &raw);
}

void restore_terminal(void) {
    tcsetattr(STDIN_FILENO, TCSAFLUSH, &saved_termios);
}

Hint 3: Fork and Connect Shell to PTY

pid_t spawn_shell(int master_fd, const char *slave_name) {
    pid_t pid = fork();

    if (pid < 0) {
        return -1;
    }

    if (pid == 0) {
        /* Child process */

        /* Create new session - become session leader */
        if (setsid() < 0) {
            _exit(1);
        }

        /* Open slave - this becomes controlling terminal */
        int slave = open(slave_name, O_RDWR);
        if (slave < 0) {
            _exit(1);
        }

        /* Set window size */
        struct winsize ws;
        ioctl(STDOUT_FILENO, TIOCGWINSZ, &ws);  /* Get parent's size */
        ioctl(slave, TIOCSWINSZ, &ws);          /* Set on slave */

        /* Duplicate slave to stdin/stdout/stderr */
        dup2(slave, STDIN_FILENO);
        dup2(slave, STDOUT_FILENO);
        dup2(slave, STDERR_FILENO);

        if (slave > STDERR_FILENO) {
            close(slave);
        }

        /* Close master in child */
        close(master_fd);

        /* Execute shell */
        char *shell = getenv("SHELL");
        if (!shell) shell = "/bin/sh";

        execlp(shell, shell, NULL);
        _exit(1);
    }

    return pid;
}

Hint 4: Escape Sequence Parser Structure

typedef enum {
    STATE_NORMAL,
    STATE_ESC,
    STATE_CSI,
} parser_state_t;

void process_byte(terminal_t *term, unsigned char c) {
    parser_t *p = &term->parser;

    switch (p->state) {
    case STATE_NORMAL:
        if (c == 0x1B) {  /* ESC */
            p->state = STATE_ESC;
        } else if (c >= 0x20 && c < 0x7F) {
            /* Printable character */
            screen_put_char(&term->screen, c);
        } else if (c == '\n') {
            screen_newline(&term->screen);
        } else if (c == '\r') {
            screen_carriage_return(&term->screen);
        } else if (c == '\b') {
            screen_backspace(&term->screen);
        }
        break;

    case STATE_ESC:
        if (c == '[') {
            p->state = STATE_CSI;
            p->param_count = 0;
            p->params[0] = 0;
            p->private_mode = 0;
        } else {
            /* Unknown sequence, return to normal */
            p->state = STATE_NORMAL;
        }
        break;

    case STATE_CSI:
        if (c == '?') {
            p->private_mode = 1;
        } else if (c >= '0' && c <= '9') {
            p->params[p->param_count] *= 10;
            p->params[p->param_count] += (c - '0');
        } else if (c == ';') {
            if (p->param_count < 15) {
                p->param_count++;
                p->params[p->param_count] = 0;
            }
        } else if (c >= 0x40 && c <= 0x7E) {
            /* Final byte - execute command */
            p->param_count++;  /* Count includes last param */
            execute_csi(term, c);
            p->state = STATE_NORMAL;
        }
        break;
    }
}

void execute_csi(terminal_t *term, char cmd) {
    parser_t *p = &term->parser;
    int n = p->params[0] ? p->params[0] : 1;  /* Default is usually 1 */

    switch (cmd) {
    case 'A':  /* Cursor Up */
        screen_move_cursor(&term->screen, -n, 0);
        break;
    case 'B':  /* Cursor Down */
        screen_move_cursor(&term->screen, n, 0);
        break;
    case 'C':  /* Cursor Forward */
        screen_move_cursor(&term->screen, 0, n);
        break;
    case 'D':  /* Cursor Back */
        screen_move_cursor(&term->screen, 0, -n);
        break;
    case 'H':  /* Cursor Position */
    case 'f': {
        int row = p->params[0] ? p->params[0] : 1;
        int col = (p->param_count > 1 && p->params[1]) ? p->params[1] : 1;
        screen_set_cursor(&term->screen, row - 1, col - 1);
        break;
    }
    case 'J':  /* Erase in Display */
        screen_erase_display(&term->screen, p->params[0]);
        break;
    case 'K':  /* Erase in Line */
        screen_erase_line(&term->screen, p->params[0]);
        break;
    case 'm':  /* SGR - Select Graphic Rendition (colors) */
        for (int i = 0; i < p->param_count; i++) {
            screen_set_attr(&term->screen, p->params[i]);
        }
        break;
    }
}

The Interview Questions They’ll Ask

  1. “Explain the difference between a PTY master and slave.”
    • Master: the controlling side (your emulator), receives output, sends input
    • Slave: looks like a terminal to programs, provides stdin/stdout/stderr
  2. “What is the line discipline in UNIX terminals?”
    • Kernel layer that processes data between master and slave
    • Handles echo, line editing, special characters, signal generation
  3. “How do terminal programs like vim know the window size?”
    • TIOCGWINSZ/TIOCSWINSZ ioctls on the terminal fd
    • SIGWINCH signal when size changes
    • Programs call ioctl(0, TIOCGWINSZ, &ws) to query size
  4. “What happens when you press Ctrl+C in a terminal?”
    • Line discipline sees ETX (0x03)
    • Generates SIGINT to foreground process group
    • Unless ISIG is cleared in termios (raw mode)
  5. “How would you implement a terminal multiplexer like tmux?”
    • Multiple PTY pairs (one per window/pane)
    • Single controlling terminal for user input
    • Route input to “focused” pane’s master
    • Combine output from all masters for rendering

Books That Will Help

Topic Book Chapter
Terminal I/O “APUE” by Stevens Ch. 18
Pseudo terminals “APUE” by Stevens Ch. 19
PTY programming “The Linux Programming Interface” by Kerrisk Ch. 64
VT100/ANSI codes “Video Terminal Information” (vttest) Reference

Testing Strategy

Unit Tests

/* test_parser.c - Test escape sequence parsing */

void test_cursor_movement() {
    terminal_t term;
    init_terminal(&term, 24, 80);

    /* Test CSI A - Cursor Up */
    process_string(&term, "\x1b[5A");
    assert(term.screen.cursor_row == 24 - 5);

    /* Test CSI H - Cursor Position */
    process_string(&term, "\x1b[10;20H");
    assert(term.screen.cursor_row == 9);   /* 0-indexed */
    assert(term.screen.cursor_col == 19);

    /* Test default parameters */
    process_string(&term, "\x1b[H");  /* No params = 1,1 */
    assert(term.screen.cursor_row == 0);
    assert(term.screen.cursor_col == 0);

    cleanup_terminal(&term);
    printf("PASS: cursor movement\n");
}

void test_colors() {
    terminal_t term;
    init_terminal(&term, 24, 80);

    /* Test SGR 31 - Red foreground */
    process_string(&term, "\x1b[31m");
    assert(term.screen.current_fg == 1);  /* Red */

    /* Test SGR 0 - Reset */
    process_string(&term, "\x1b[0m");
    assert(term.screen.current_fg == 7);  /* Default */

    cleanup_terminal(&term);
    printf("PASS: color handling\n");
}

Integration Tests

#!/bin/bash
# test_integration.sh

# Test 1: Basic echo
echo "Testing basic echo..."
./myterm -e "echo hello" 2>/dev/null | grep -q "hello" || { echo "FAIL: echo"; exit 1; }
echo "PASS: echo"

# Test 2: Color output
echo "Testing color..."
./myterm -e "ls --color /tmp" 2>/dev/null | grep -q $'\x1b\[' || echo "WARN: no colors detected"

# Test 3: Window size
echo "Testing window size..."
./myterm -e "stty size" 2>/dev/null | grep -qE '^[0-9]+ [0-9]+$' || { echo "FAIL: stty size"; exit 1; }
echo "PASS: window size"

# Test 4: Interactive mode (manual)
echo "Manual test: run './myterm' and try vim, htop, less"

Escape Sequence Test Vectors

# escape_sequences.txt - Common sequences to test

# Cursor movement
ESC[A        # Up 1
ESC[5A       # Up 5
ESC[B        # Down 1
ESC[C        # Right 1
ESC[D        # Left 1
ESC[10;20H   # Position row 10, col 20
ESC[H        # Home (1,1)

# Erase
ESC[2J       # Clear screen
ESC[J        # Clear from cursor to end
ESC[1J       # Clear from start to cursor
ESC[K        # Clear to end of line
ESC[2K       # Clear entire line

# Colors (SGR)
ESC[0m       # Reset
ESC[1m       # Bold
ESC[31m      # Red foreground
ESC[44m      # Blue background
ESC[31;44m   # Red on blue

# Private modes
ESC[?25h     # Show cursor
ESC[?25l     # Hide cursor
ESC[?1049h   # Alternate screen buffer
ESC[?1049l   # Normal screen buffer

Common Pitfalls & Debugging

Problem 1: “Shell doesn’t get signals (Ctrl+C doesn’t work)”

Symptom: Pressing Ctrl+C does nothing to the running program.

Why: The shell isn’t the session leader, or doesn’t have a controlling terminal.

Fix: Ensure child process:

  1. Calls setsid() to become session leader
  2. Opens the PTY slave (this becomes the controlling terminal)
  3. The open must happen AFTER setsid
/* Correct order in child: */
setsid();                              /* 1. New session */
int slave = open(slave_name, O_RDWR);  /* 2. Becomes controlling tty */

Problem 2: “Programs output garbage”

Symptom: Running vim or htop shows garbage characters.

Why: Not parsing escape sequences, or parsing incorrectly.

Fix:

  1. Implement CSI parser state machine
  2. Handle all cursor movement commands
  3. Handle clear screen/line commands
  4. Print unparsed sequences to debug log
/* Debug: log unknown sequences */
case STATE_CSI:
    if (/* final byte */) {
        if (/* unknown command */) {
            fprintf(stderr, "Unknown CSI: ESC[");
            for (int i = 0; i < p->param_count; i++) {
                fprintf(stderr, "%d;", p->params[i]);
            }
            fprintf(stderr, "%c\n", cmd);
        }
    }

Problem 3: “Window size wrong - programs display incorrectly”

Symptom: vim thinks terminal is 24x80 when it’s not, or programs don’t resize.

Why: Didn’t set TIOCSWINSZ on slave, or not handling SIGWINCH.

Fix:

/* Set size on slave after fork */
struct winsize ws;
ws.ws_row = term_height;
ws.ws_col = term_width;
ioctl(slave_fd, TIOCSWINSZ, &ws);

/* Handle SIGWINCH in parent */
void sigwinch_handler(int sig) {
    winch_received = 1;
}

/* In main loop */
if (winch_received) {
    winch_received = 0;
    struct winsize ws;
    ioctl(STDOUT_FILENO, TIOCGWINSZ, &ws);  /* Get new size */
    ioctl(master_fd, TIOCSWINSZ, &ws);       /* Set on PTY */
    screen_resize(&term->screen, ws.ws_row, ws.ws_col);
}

Problem 4: “Output is buffered - appears in chunks”

Symptom: Output appears delayed or all at once.

Why: stdio buffering on PTY, or not reading all available data.

Fix:

/* Read all available data from master */
while (1) {
    char buf[4096];
    ssize_t n = read(master_fd, buf, sizeof(buf));
    if (n <= 0) break;
    process_output(term, buf, n);
}

/* Use non-blocking reads or select() with timeout */

Problem 5: “Can’t type special characters or arrow keys”

Symptom: Pressing Up arrow shows ^[[A instead of moving cursor.

Why: Not in raw mode, or not forwarding escape sequences correctly.

Fix:

/* Ensure raw mode is set */
tcgetattr(STDIN_FILENO, &raw);
raw.c_lflag &= ~(ECHO | ICANON);
tcsetattr(STDIN_FILENO, TCSAFLUSH, &raw);

/* Forward all bytes to master, including escape sequences */
while (1) {
    char buf[64];
    ssize_t n = read(STDIN_FILENO, buf, sizeof(buf));
    if (n > 0) {
        write(master_fd, buf, n);  /* Forward everything */
    }
}

Extensions & Challenges

Beginner Extensions

  • Scrollback buffer: Store N lines of history, scroll with Shift+PgUp/PgDn
  • Session logging: Write all output to a file
  • Title bar: Parse OSC sequences to update window title

Intermediate Extensions

  • 256-color support: Parse ESC[38;5;Nm and ESC[48;5;Nm
  • True color support: Parse ESC[38;2;R;G;Bm
  • Unicode/UTF-8: Handle multi-byte characters and wide characters
  • Mouse support: Parse mouse tracking escape sequences

Advanced Extensions

  • Terminal multiplexer: Multiple panes/tabs like tmux
  • Sixel graphics: Display inline images
  • Hyperlinks: OSC 8 hyperlink support
  • GPU rendering: OpenGL/Vulkan accelerated rendering

Resources

Essential Reading

  • APUE Chapter 18: Terminal I/O
  • APUE Chapter 19: Pseudo Terminals
  • The Linux Programming Interface, Chapter 64: Pseudoterminals
  • ECMA-48 - Control functions for coded character sets

Online References


Self-Assessment Checklist

Understanding

  • I can explain the difference between PTY master and slave
  • I understand what the line discipline does
  • I can describe how raw mode differs from canonical mode
  • I know why setsid() must be called before opening the slave
  • I can trace a keystroke through the entire PTY stack
  • I understand how SIGWINCH propagates window size changes

Implementation

  • PTY creation works correctly (posix_openpt, grantpt, unlockpt)
  • Shell spawns and runs with correct terminal
  • Basic escape sequences are parsed (cursor, clear, colors)
  • Window resize is handled correctly
  • Full-screen programs (vim, htop) work correctly
  • My terminal is responsive and doesn’t hang

Testing

  • All escape sequence test vectors pass
  • vim can be used for basic editing
  • htop displays correctly
  • less scrolls correctly
  • man pages render correctly
  • Color output (ls –color) works

This project guide is part of the Advanced UNIX Programming Deep Dive. Complete Projects 1-14 before attempting this project.