Project 2: The "Pipe" Text Processor

Project 2: The “Pipe” Text Processor

Build a text processing pipeline where small functions compose into powerful transformations.

Quick Reference

Attribute	Value
Difficulty	Beginner
Time Estimate	Weekend
Language	TypeScript
Prerequisites	Basic TypeScript, Project 1 (Immutability)
Key Topics	Function Composition, Pipelines, Currying, Point-Free Style

---

1. Learning Objectives

By completing this project, you will:

1. Understand function composition as the core of functional programming
2. Build pipe and compose utilities from scratch
3. Learn currying and why it enables composition
4. Master point-free style (functions without explicit arguments)
5. Transform text through composable, reusable functions
6. See how Unix pipes and functional composition share the same idea

---

2. Theoretical Foundation

2.1 Core Concepts

What is Function Composition?

Function composition is combining two or more functions to produce a new function:

// Two simple functions
const double = (x: number) => x * 2;
const increment = (x: number) => x + 1;

// Composition: apply increment, then double
const incrementThenDouble = (x: number) => double(increment(x));
incrementThenDouble(5);  // double(6) = 12

// Mathematical notation: (double ∘ increment)(x) = double(increment(x))

The key insight: You build complex operations from simple ones.

Simple Functions           Composed Function
    ┌─────────┐
    │ double  │
    └────┬────┘
         │
         ├──────────▶  incrementThenDouble = double ∘ increment
         │
    ┌────┴────┐
    │increment│
    └─────────┘

Pipe vs Compose

Both combine functions, but in opposite directions:

// compose: right-to-left (mathematical order)
// compose(f, g, h)(x) = f(g(h(x)))
const compose = (...fns) => (x) =>
  fns.reduceRight((acc, fn) => fn(acc), x);

// pipe: left-to-right (reading order)
// pipe(f, g, h)(x) = h(g(f(x)))
const pipe = (...fns) => (x) =>
  fns.reduce((acc, fn) => fn(acc), x);

Visual comparison:

compose(f, g, h)(x):
  x ──▶ h ──▶ g ──▶ f ──▶ result
        └──────────────────────┘
        Right-to-left execution

pipe(f, g, h)(x):
  x ──▶ f ──▶ g ──▶ h ──▶ result
        └──────────────────────┘
        Left-to-right execution

Most developers find pipe more intuitive because it matches reading order.

Why Composition Matters

Compare imperative vs composed approaches:

// IMPERATIVE: Step-by-step mutation
function processText(input: string): string {
  let result = input;
  result = result.toLowerCase();
  result = result.trim();
  result = result.replace(/\s+/g, '-');
  result = result.replace(/[^a-z0-9-]/g, '');
  return result;
}

// COMPOSED: Pipeline of transformations
const processText = pipe(
  toLowerCase,
  trim,
  replaceSpaces('-'),
  removeNonAlnum
);

Benefits of composition:

1. Testable: Each function is independently testable
2. Reusable: Functions can be combined in different pipelines
3. Readable: Pipeline reads like a description of the process
4. Flexible: Easy to add, remove, or reorder steps

Currying: Enabling Composition

Currying transforms a function that takes multiple arguments into a sequence of functions that each take one argument:

// Uncurried: Takes all arguments at once
const add = (a: number, b: number): number => a + b;
add(2, 3); // 5

// Curried: Returns a function for each argument
const curriedAdd = (a: number) => (b: number): number => a + b;
curriedAdd(2)(3); // 5

// Partial application: Fix some arguments
const add10 = curriedAdd(10);
add10(5); // 15

Why currying enables composition:

// Without currying - can't compose
const replace = (search: string, replacement: string, str: string) =>
  str.replace(search, replacement);

// This won't work in a pipe:
pipe(
  toLowerCase,
  replace(' ', '-', ???)  // Where does str come from?
);

// With currying - composable!
const replace = (search: string) => (replacement: string) => (str: string) =>
  str.replace(search, replacement);

// Now it works:
pipe(
  toLowerCase,
  replace(' ')('-')  // Returns (str) => str.replace(' ', '-')
);

Point-Free Style

Point-free means defining functions without explicitly mentioning their arguments:

// Pointed: Explicit argument 'x'
const double = (x: number) => x * 2;

// Point-free with composition
const processNumbers = (numbers: number[]) =>
  numbers.map(x => x * 2).filter(x => x > 10);

// Point-free version
const processNumbers = pipe(
  map(double),        // No explicit mention of 'numbers'
  filter(greaterThan(10))
);

Point-free style emphasizes what the function does, not how it receives data.

2.2 Why This Matters

In Real Code Bases:

• RxJS: Built entirely on composing operators
• Redux: Middleware is composed functions
• React Hooks: Combine simple hooks into complex behavior
• Express.js: Middleware is a composition pipeline

// Express middleware is function composition!
app.use(
  logger,          // Log request
  authenticate,    // Check auth
  parseBody,       // Parse JSON
  validate,        // Validate data
  handleRequest    // Process request
);

In Data Processing:

• ETL pipelines compose transformations
• Stream processing (Kafka, Flink) uses composed operators
• CLI tools use Unix pipes for the same reason

2.3 Historical Context

Function composition comes from mathematics:

• 19th Century: Mathematical function composition (g ∘ f)
• 1936: Lambda Calculus (Alonzo Church) - composition as fundamental operation
• 1958: LISP introduced compose-like operations
• 1977: Unix pipes (|) brought composition to shell scripting
• 2015+: JavaScript libraries (Ramda, Lodash/fp) popularized FP-style composition

The Unix philosophy: “Do one thing well, combine tools with pipes.”

# Unix pipe - same concept!
cat file.txt | grep "error" | sort | uniq -c | head -10

2.4 Common Misconceptions

“Composition is just calling functions in sequence”

• Reality: Composition creates a NEW function from existing ones
• The composed function doesn’t exist until you call it with input

“You need a library for composition”

• Reality: pipe and compose are ~3 lines of code each
• But libraries like Ramda provide type-safe, optimized versions

“Point-free is always better”

• Reality: Point-free can become unreadable when overused
• Use it when it clarifies intent, not for cleverness

---

3. Project Specification

3.1 What You Will Build

A text processing toolkit with:

• pipe and compose utilities
• A library of composable text transformation functions
• Curried utility functions for partial application
• A CLI tool that accepts piped input
• Preset pipelines for common transformations (slug, sanitize, format)

3.2 Functional Requirements

1. Core Utilities:
   • pipe(...fns): Compose functions left-to-right
   • compose(...fns): Compose functions right-to-left
   • curry(fn): Convert multi-argument function to curried form
2. String Transformers (all composable):
   • toLowerCase: Convert to lowercase
   • toUpperCase: Convert to uppercase
   • trim: Remove leading/trailing whitespace
   • replace(search)(replacement): Replace occurrences (curried)
   • split(separator): Split into array (curried)
   • join(separator): Join array to string (curried)
   • take(n): Take first n characters (curried)
   • drop(n): Drop first n characters (curried)
   • reverse: Reverse string
   • words: Split into words
   • unwords: Join words with spaces
3. Preset Pipelines:
   • slugify: “Hello World!” → “hello-world”
   • titleCase: “hello world” → “Hello World”
   • snakeCase: “Hello World” → “hello_world”
   • camelCase: “hello world” → “helloWorld”
4. CLI Tool:
   • Accept input via stdin or argument
   • Apply named pipeline to input
   • Output result to stdout

3.3 Non-Functional Requirements

• Type Safety: Full generic types for pipe and compose
• Immutability: No string mutation (strings are immutable anyway)
• Composability: Every transformer works in any pipeline
• Testability: Each function testable in isolation

3.4 Example Usage / Output

// Using pipe to create transformations
const slugify = pipe(
  toLowerCase,
  trim,
  replace(/\s+/g)('-'),
  replace(/[^a-z0-9-]/g)('')
);

slugify("  Hello, World!  ");  // "hello-world"

// Compose multiple transformers
const processEmail = pipe(
  toLowerCase,
  trim,
  replace(/\s/g)('')
);

processEmail("  USER@Example.COM  ");  // "user@example.com"

// Using compose (right-to-left)
const emphasize = compose(
  (s: string) => s + "!",
  toUpperCase,
  trim
);

emphasize("  hello  ");  // "HELLO!"

// Curried functions enable partial application
const addPrefix = replace(/^/)('>>> ');
const addSuffix = replace(/$/)(' <<<');

const frame = pipe(addPrefix, addSuffix);
frame("message");  // ">>> message <<<"

// CLI usage
// $ echo "Hello World" | npx ts-node cli.ts slugify
// hello-world

---

4. Solution Architecture

4.1 High-Level Design

┌───────────────────────────────────────────────────────────┐
│                    Text Processing Library                 │
├───────────────────────────────────────────────────────────┤
│                                                           │
│  ┌─────────────┐    ┌─────────────┐    ┌─────────────┐   │
│  │   Input     │───▶│   Pipeline  │───▶│   Output    │   │
│  │  (string)   │    │   (pipe)    │    │  (string)   │   │
│  └─────────────┘    └──────┬──────┘    └─────────────┘   │
│                            │                              │
│         ┌──────────────────┼──────────────────┐          │
│         ▼                  ▼                  ▼          │
│    ┌─────────┐       ┌─────────┐       ┌─────────┐       │
│    │Transform│  ───▶ │Transform│  ───▶ │Transform│       │
│    │   fn1   │       │   fn2   │       │   fn3   │       │
│    └─────────┘       └─────────┘       └─────────┘       │
│                                                           │
│  Each transform: (input: string) => string                │
│                                                           │
└───────────────────────────────────────────────────────────┘

4.2 Key Components

Component	Responsibility	Key Decisions
pipe	Compose functions left-to-right	Generic types preserve type flow
compose	Compose functions right-to-left	Uses reduceRight
curry	Convert multi-arg to single-arg chain	Recursively wrap functions
Transformers	Individual string operations	All curried for composability
Presets	Common transformation pipelines	Built from transformers
CLI	Command-line interface	Read stdin, apply preset, write stdout

4.3 Data Structures

// Type for a single-argument function
type Unary<A, B> = (a: A) => B;

// Pipe type: chains functions
type Pipe = {
  <A>(a: A): A;
  <A, B>(a: A, fn1: Unary<A, B>): B;
  <A, B, C>(a: A, fn1: Unary<A, B>, fn2: Unary<B, C>): C;
  // ... more overloads for type safety
};

// Or simplified:
type PipeFunction = <T>(...fns: Array<(x: any) => any>) => (x: T) => any;

// Transformer: string → string
type StringTransformer = (input: string) => string;

// Curried transformer factory
type TransformerFactory<A> = (config: A) => StringTransformer;

4.4 Algorithm Overview

Pipe Implementation:

const pipe = <T>(...fns: Array<(x: any) => any>) => (x: T) =>
  fns.reduce((acc, fn) => fn(acc), x);

How it works:

1. Accept an array of functions
2. Return a new function that takes initial value
3. Reduce through functions, passing each result to next

pipe(f, g, h)(x)

Step 1: acc = x,        fn = f  →  f(x)
Step 2: acc = f(x),     fn = g  →  g(f(x))
Step 3: acc = g(f(x)),  fn = h  →  h(g(f(x)))

Result: h(g(f(x)))

Curry Implementation:

const curry = <T extends (...args: any[]) => any>(fn: T) => {
  const arity = fn.length;  // Number of expected arguments

  return function curried(...args: any[]): any {
    if (args.length >= arity) {
      return fn(...args);  // All arguments received, call original
    }
    return (...moreArgs: any[]) =>
      curried(...args, ...moreArgs);  // Partial application
  };
};

Complexity Analysis:

• Pipe/Compose: O(n) where n = number of functions
• Individual transformers: O(m) where m = string length
• Overall pipeline: O(n × m)

---

5. Implementation Guide

5.1 Development Environment Setup

mkdir pipe-processor && cd pipe-processor
npm init -y
npm install --save-dev typescript ts-node @types/node
npx tsc --init
mkdir src

5.2 Project Structure

pipe-processor/
├── src/
│   ├── core/
│   │   ├── pipe.ts        # pipe and compose
│   │   └── curry.ts       # curry utility
│   ├── transformers/
│   │   ├── basic.ts       # toLowerCase, trim, etc.
│   │   ├── advanced.ts    # replace, split, join
│   │   └── index.ts       # Export all transformers
│   ├── presets/
│   │   ├── slugify.ts
│   │   ├── cases.ts       # camelCase, snakeCase, etc.
│   │   └── index.ts
│   ├── cli.ts             # Command-line interface
│   └── index.ts           # Main exports
├── tests/
│   ├── pipe.test.ts
│   ├── transformers.test.ts
│   └── presets.test.ts
├── package.json
└── tsconfig.json

5.3 Implementation Phases

Phase 1: Core Utilities (2-3 hours)

Goals:

• Implement pipe and compose
• Implement basic curry
• Test with simple functions

Tasks:

1. Create pipe.ts with basic pipe implementation
2. Create compose.ts using reduceRight
3. Create curry.ts with recursive curry
4. Write tests for all three utilities

Checkpoint: These tests pass:

const add = (a: number, b: number) => a + b;
const multiply = (a: number, b: number) => a * b;

const curriedAdd = curry(add);
expect(curriedAdd(2)(3)).toBe(5);

const pipeline = pipe(
  (x: number) => x + 1,
  (x: number) => x * 2
);
expect(pipeline(5)).toBe(12);

Phase 2: String Transformers (3-4 hours)

Goals:

• Build library of composable string functions
• All functions must be curried where appropriate
• Functions must be pure (no side effects)

Tasks:

1. Implement basic transformers (toLowerCase, toUpperCase, trim, reverse)
2. Implement curried transformers (replace, split, join, take, drop)
3. Implement word-based transformers (words, unwords)
4. Test each transformer individually

Checkpoint: Transformers compose correctly:

const transform = pipe(
  toLowerCase,
  trim,
  replace(' ')('-')
);
expect(transform("  Hello World  ")).toBe("hello-world");

Phase 3: Presets and CLI (2-3 hours)

Goals:

• Create preset pipelines for common use cases
• Build CLI tool for pipeline processing

Tasks:

1. Build slugify preset
2. Build case conversion presets (title, snake, camel)
3. Create CLI that reads stdin or argument
4. Support selecting preset by name

Checkpoint: CLI works:

echo "Hello World" | npx ts-node src/cli.ts slugify
# Output: hello-world

5.4 Key Implementation Decisions

Decision	Options	Recommendation	Rationale
Pipe types	Overloads vs Generic	Overloads	Better type inference
Curry implementation	Manual vs Library	Manual	Learning exercise
Regex in replace	String vs RegExp	Both	RegExp for flexibility
CLI parsing	Manual vs Library	Manual	Keep dependencies minimal

---

6. Testing Strategy

6.1 Test Categories

Category	Purpose	Examples
Unit Tests	Test individual functions	toLowerCase("ABC") === "abc"
Composition Tests	Test function combinations	pipe(f, g)(x) === g(f(x))
Property Tests	Verify mathematical laws	Identity, associativity

6.2 Critical Test Cases

1. Pipe Identity Law: ```typescript test(‘pipe with identity returns original’, () => { const identity = (x: T) => x; const f = (x: number) => x * 2;

// pipe(identity, f) should equal f expect(pipe(identity, f)(5)).toBe(f(5)); // pipe(f, identity) should equal f expect(pipe(f, identity)(5)).toBe(f(5)); });

2. **Pipe Associativity**:
```typescript
test('pipe is associative', () => {
  const f = (x: number) => x + 1;
  const g = (x: number) => x * 2;
  const h = (x: number) => x - 3;

  // pipe(pipe(f, g), h) === pipe(f, pipe(g, h))
  expect(pipe(pipe(f, g), h)(5)).toBe(pipe(f, pipe(g, h))(5));
});

1. Curried Function Equivalence: ```typescript test(‘curried function equals original’, () => { const add = (a: number, b: number, c: number) => a + b + c; const curriedAdd = curry(add);

expect(curriedAdd(1)(2)(3)).toBe(add(1, 2, 3)); expect(curriedAdd(1, 2)(3)).toBe(add(1, 2, 3)); expect(curriedAdd(1)(2, 3)).toBe(add(1, 2, 3)); });

### 6.3 Test Data

```typescript
const testStrings = [
  "hello world",
  "  PADDED STRING  ",
  "MixedCase",
  "with-dashes",
  "with_underscores",
  "special!@#$%chars",
  "",                  // Edge case: empty
  "   ",               // Edge case: only whitespace
];

---

7. Common Pitfalls & Debugging

7.1 Frequent Mistakes

Pitfall	Symptom	Solution
Non-composable functions	TypeScript errors about types	Ensure (string) => string signature
Missing curry on multi-arg functions	Pipeline fails at runtime	Curry all multi-argument transformers
Wrong argument order in curried functions	Logic errors	Data-last convention: config first, data last
Forgetting pipe returns a function	undefined results	Call the returned function: pipe(f, g)(x) not pipe(f, g, x)

7.2 Debugging Strategies

• Trace Each Step: Insert tap function to log intermediate values

const tap = <T>(label: string) => (x: T): T => {
  console.log(`${label}:`, x);
  return x;
};

const debugPipeline = pipe(
  tap('input'),
  toLowerCase,
  tap('after lowercase'),
  trim,
  tap('after trim')
);

• Type Annotations: Add explicit types to find mismatches
• Simplify Pipeline: Remove functions until it works, add back one at a time

7.3 Performance Traps

• Creating functions in loops: Pre-compose pipelines, don’t rebuild them
• Large string operations: For huge texts, consider streaming
• Deep curry nesting: Limit curry depth, or use a library

---

8. Extensions & Challenges

8.1 Beginner Extensions

• More Presets: kebab-case, CONSTANT_CASE, sentence case
• Truncate: truncate(maxLength)(ellipsis) - truncate with “…”
• Pad: padStart(length)(char), padEnd(length)(char)

8.2 Intermediate Extensions

• Async Pipe: pipeAsync that handles Promise-returning functions
• Error Handling: Wrap transformers to catch and report errors
• Memoization: Add memoization to frequently-used pipelines

8.3 Advanced Extensions

• Type-Safe Pipe: Full TypeScript inference through any number of functions
• Transducers: Efficient composition that fuses multiple passes
• Parser Combinators: Use composition to build a simple parser (leads to Project 5)

---

9. Real-World Connections

9.1 Industry Applications

• RxJS Operators: pipe(map, filter, reduce) - same pattern!
• Redux Middleware: applyMiddleware(logger, thunk, router)
• Build Tools: Gulp tasks compose transformations
• Data Pipelines: ETL tools use composed transformations

9.2 Related Open Source Projects

• Ramda: https://ramdajs.com - FP library with auto-curried functions
• Lodash/fp: https://github.com/lodash/lodash/wiki/FP-Guide - FP variant of Lodash
• RxJS: https://rxjs.dev - Reactive Extensions with pipe operator

9.3 Interview Relevance

• “Implement pipe”: Common interview question - you’ll have it ready
• “Explain function composition”: Core FP concept
• “What is currying?”: Shows FP knowledge

---

10. Resources

10.1 Essential Reading

• “Professor Frisby’s Mostly Adequate Guide to FP” - Chapter 5: Coding by Composing
• “Functional-Light JavaScript” by Kyle Simpson - Chapter 4: Composing Functions
• “Hands-On Functional Programming with TypeScript” - Chapter 3: Function Composition

10.2 Video Resources

• “Composition over Inheritance” - Fun Fun Function (YouTube)
• “Transducers” - Rich Hickey (YouTube) - Advanced composition

10.3 Tools & Documentation

• TypeScript Utility Types: https://www.typescriptlang.org/docs/handbook/utility-types.html
• Ramda Documentation: https://ramdajs.com/docs/ - See how composition is used

10.4 Related Projects in This Series

• Previous Project: P01 - Immutable Todo List - Immutability foundation
• Next Project: P03 - Maybe Monad - Composition meets containers

---

11. Self-Assessment Checklist

Before considering this project complete, verify:

Understanding

• I can implement pipe from memory
• I can explain why currying enables composition
• I understand the difference between pipe and compose
• I can recognize point-free style and know when to use it

Implementation

• pipe and compose work correctly
• All transformers are composable
• Curried functions work with partial application
• CLI processes input correctly
• All tests pass

Growth

• I’ve identified RxJS patterns that use the same concepts
• I can spot opportunities for composition in existing code
• I understand when NOT to use point-free style

---

12. Submission / Completion Criteria

Minimum Viable Completion:

• pipe and compose working
• At least 5 string transformers
• One preset pipeline (slugify)
• Basic tests passing

Full Completion:

• All transformers implemented
• All presets (slugify, titleCase, snakeCase, camelCase)
• CLI tool working
• Comprehensive tests with edge cases

Excellence (Going Above & Beyond):

• Type-safe pipe with full inference
• Async pipe variant
• Performance benchmarks
• Published to npm

---

This guide was generated from FUNCTIONAL_PROGRAMMING_TYPESCRIPT_LEARNING_PROJECTS.md. For the complete learning path, see the parent directory.