Project 10: The Search-Replace Master

Apply safe, confirmed substitutions and global commands across a file.

Quick Reference

Attribute Value
Difficulty Intermediate
Time Estimate 10-20 hours
Main Programming Language Plain text / code
Alternative Programming Languages N/A
Coolness Level Level 3: Batch Editor
Business Potential Level 4: Cleanup Accelerator
Prerequisites Search basics, Regex fundamentals, Command-line mode comfort
Key Topics Ex Commands, Ranges, and Substitution, Search and Find Motions, Repeatability and the Dot Command

1. Learning Objectives

By completing this project, you will:

  1. Execute the core workflow for The Search-Replace Master without Visual mode.
  2. Apply motion-based edits to achieve a measurable output.
  3. Explain the reasoning behind each key command you used.
  4. Validate results against a deterministic outcome.

2. All Theory Needed (Per-Concept Breakdown)

Ex Commands, Ranges, and Substitution

Fundamentals Ex commands are Vim’s command-line language. They operate on ranges of lines and use patterns, which makes them ideal for batch edits. The most common Ex command is substitution (:s), but there are also global commands (:g), move (:m), copy (:t), and delete (:d). Ranges allow you to target specific lines, the whole file (:%), or a visual selection (:'<,'>). Mastering ranges and substitutions gives you the power to transform large files with a single command.

Ex commands are line-oriented by default, which makes them ideal for batch operations. They are composable: you can run a global command that executes another command, effectively creating a pipeline. This is why Ex is often the fastest path for large transformations.

Deep Dive The command-line mode in Vim is a separate language with its own grammar. You enter it with : for Ex commands, / for forward search, or ? for backward search. Ex commands apply to a range of lines. If no range is specified, the command applies to the current line. A range can be explicit (:1,10), symbolic (:% for whole file), or mark-based (:'<,'> for the last visual selection). Understanding ranges is essential, because it determines the scope of your command. A substitution on the wrong range can cause large unintended changes, so ranges are a safety feature as much as a power feature.

Substitution is the most commonly used Ex command. Its basic form is :s/old/new/, which replaces the first match on the line. Flags like g (global) and c (confirm) change behavior. The confirm flag is particularly important for safety; it allows you to review each match before replacing. Vim’s regex flavor includes special atoms and modes like \v (very magic) to reduce escaping. This allows complex transformations like swapping fields or normalizing log entries. However, complex regex can be brittle, so you should start with the simplest possible pattern and test on a small range before scaling.

Global commands (:g) are another powerful tool. :g/pattern/command runs a command on all lines matching the pattern. This is a second-order operation: you are not just moving to matches, you are performing an action on them. For example, :g/ERROR/d deletes all lines containing ERROR. Combined with :v (inverse global), you can filter files by inclusion or exclusion. These commands are fast and precise but can be dangerous; always test with :g/pattern/p to print the lines first.

Ex commands also interact with Normal mode. You can run Normal mode commands over a range with :normal, which allows you to apply motions and operators to multiple lines. This is useful when a single substitution is not enough. For example, you can use :g/pattern/normal to apply a Normal mode edit to all matching lines. This bridges the gap between linewise commands and structural edits. It also means your Normal mode grammar can scale to batch operations.

The command-line language is best understood as a batch editor. It is not just a fallback; it is a complementary interface. Visual mode can define a range, Ex commands operate on that range, and the results can be repeated with @: or &. This makes command-line operations repeatable in the same way as Normal mode edits. In practice, you should use Ex commands for large, line-oriented transformations, and Normal mode for structural edits within lines. Learning when to switch between them is the difference between a novice and an expert.

A safe workflow is to preview target lines with :g/pattern/p before running a destructive command. The inverse global :v is useful for deleting everything except matches, which is a common cleanup task. Ex commands can also be combined with marks and line numbers to build precise ranges. When you are unsure, use the confirm flag or narrow the range to a few lines, then expand once the pattern is proven.

How this fit on projects Projects 4, 5, and 10 use Ex commands for line operations and substitutions.

Definitions & key terms

  • Range: a set of lines (:1,10, :%, :'<,'>)
  • Substitute: :s/old/new/
  • Global: :g/pattern/command
  • Confirm flag: c prompts before each replacement

Mental model diagram

Range + Command + Pattern = batch editing

How it works

  1. Define a range.
  2. Choose a command (:s, :g, :m, :t).
  3. Provide a pattern or target.
  4. Execute and verify.

Minimal concrete example

:%s/foo/bar/gc
:1,10s/\v(\w+), (\w+)/\2 \1/
:g/ERROR/d

Common misconceptions

  • ”:s only works on one line.” (Use ranges.)
  • “Regex must be escaped heavily.” (Use \v very magic.)
  • ”:g is a search.” (:g runs a command on matches.)

Check-your-understanding questions

  1. What does % mean in a range?
  2. How do you confirm each substitution?
  3. What does :g/pat/normal dd do?

Check-your-understanding answers

  1. The whole file.
  2. Add the c flag.
  3. Deletes lines matching pat.

Real-world applications

  • Transforming CSV, logs, and data dumps
  • Cleaning trailing whitespace
  • Bulk renaming identifiers

Where you’ll apply it Projects 4, 5, 10.

References

  • Vim :help cmdline.txt
  • Vim :help :substitute
  • Vim :help :global

Key insights Ex commands are Vim’s batch processor.

Summary Ranges and substitutions unlock fast, safe bulk editing.

Homework/Exercises to practice the concept

  1. Replace all TODO with DONE in a file.
  2. Swap “Last, First” to “First Last” with one command.

Solutions to the homework/exercises

  1. :%s/TODO/DONE/g.
  2. :%s/\v([^,]+), (.+)/\2 \1/.

Search and Find Motions

Fundamentals Search is a motion. Find (f, t) is a precision tool on the current line. Together they allow you to jump to targets by pattern rather than by counting. Search motions (/, ?) move the cursor to the next match and can be combined with operators to delete, change, or yank through a match. Find motions (f, t, F, T) move to or just before a character on the current line and can be repeated with ; or ,. Mastering these motions removes the need for repeated j/k and makes edits scalable across large files.

Search is more efficient when you can see the matches. Options like hlsearch and incsearch provide immediate feedback, which reduces mistakes. Find motions are best for visible, single-line targets.

Deep Dive Search is the most flexible motion because the target can be defined by a pattern rather than a fixed distance. When you type /pattern, Vim searches forward; ?pattern searches backward. This motion becomes a selection when combined with an operator: d/pattern deletes up to the match, c?pattern changes text backward to the match. Unlike raw motions, search adapts to different file structures because the pattern is content-based. The n and N keys repeat the search in the same or opposite direction, creating a fast loop: search, change, repeat. This is essential for repeated edits across a file.

Find motions are line-scoped and quick. f jumps to a character; t jumps before it. This is perfect for editing delimited fields like CSV, function arguments, or log lines. The repeat keys ; and , allow you to move to the next or previous occurrence of the same character without retyping. These motions are also operator-friendly: dt, deletes up to the next comma, leaving the comma intact; df) deletes through the next closing parenthesis. The distinction between f and t matters because inclusive versus exclusive deletion determines whether delimiters remain.

Search patterns are regular expressions in Vim’s flavor. This means you can target complex structures, but it also introduces complexity. For example, you can use \v (very magic) to reduce escaping, or \c for case-insensitive matches. The key is to choose the simplest pattern that uniquely identifies the target. Overly complex patterns are slow and hard to debug. The hlsearch and incsearch options make search more visible and interactive, which improves accuracy and confidence. In projects, you should use these options to see your matches before you act on them.

Search is also a navigation tool for larger structures. You can search for function names, error codes, or log prefixes. Combined with the jump list, search becomes a navigation memory: you can use Ctrl-O to return to previous positions. This means search is not just about reaching the next target, it is also about creating return points. In a workflow, you can search to jump, perform an edit, then jump back. This reduces the cognitive load of “where was I?”

Find motions are best when the target is visible and the line is short. Search is best when the target is repeated or located beyond the current line. Knowing when to use each is part of Vim mastery. A good rule: if the target is on the current line and visible, use f or t. If the target is elsewhere, use / or ?. Both motions are more powerful when combined with operators and the dot command. If your edit is repeatable, the combination of search + change + n + . becomes a fast pipeline for repetitive tasks.

The * and # commands search for the word under the cursor forward and backward, which is faster than typing the pattern. g* and g# do the same but allow partial matches. Search behavior is also controlled by ignorecase and smartcase, which let lowercase patterns be case-insensitive while uppercase patterns stay case-sensitive. These settings make search predictable across code and prose and are worth enabling in a motion-focused setup.

How this fit on projects Projects 1, 3, 5, and 10 rely on search and find to target edits precisely.

Definitions & key terms

  • Search (/, ?): forward or backward pattern search
  • Find (f, t, F, T): find a character on the current line
  • Repeat search (n, N): move to next/previous match
  • Repeat find (;, ,): move to next/previous find target

Mental model diagram

Search = anywhere in file
Find   = on this line

How it works

  1. Choose search or find based on scope.
  2. Execute the motion (/pattern, f,).
  3. Repeat with n or ;.
  4. Combine with operators for changes.

Minimal concrete example

/Timeout<Enter>
ndt,
f)

Common misconceptions

  • “Search is only for navigation.” (It is also a motion for edits.)
  • “t equals f.” (t stops before the character.)
  • “You must retype the search.” (n repeats it.)

Check-your-understanding questions

  1. What does dt) do?
  2. What does ; repeat?
  3. How do you search backward for a word?

Check-your-understanding answers

  1. Deletes until ) but not including it.
  2. The last f/t search.
  3. ?word.

Real-world applications

  • Editing CSV fields or log entries
  • Jumping between error lines
  • Repeating edits across matching patterns

Where you’ll apply it Projects 1, 3, 5, 10.

References

  • Vim :help search
  • Vim :help f
  • “Learning the vi and Vim Editors” - Ch. 3

Key insights Search is a motion, not just a navigation tool.

Summary Search and find are the fastest ways to target text without counting.

Homework/Exercises to practice the concept

  1. Use dt, to delete until commas in a CSV line.
  2. Use /TODO then n + . to fix multiple TODOs.

Solutions to the homework/exercises

  1. dt, deletes up to the comma.
  2. /TODO, edit once, then n. repeatedly.

Repeatability and the Dot Command

Fundamentals Vim rewards repeatable edits. The dot command (.) repeats the last change, which can include insertions, deletions, or operator-based edits. If you design a clean, single change, . becomes a multiplier. This means the fastest way to edit is often to design the change once and then repeat it across targets. Repeatability also applies to command-line operations (@: for last Ex command) and macros (@@ for last macro). Understanding repeatability turns Vim from a text editor into a small automation engine.

Counts can multiply repeatability: 5. repeats the last change five times, which is useful in aligned lists. Repeatability is a design choice, not an accident.

Deep Dive Repeatability is the core of Vim efficiency. The dot command replays the last change, which is more powerful than it appears. A “change” in Vim includes an operator-based edit, an insert sequence, or a combination of both, as long as it starts and ends in Normal mode. This means a clean edit must be designed with dot in mind. For example, if you want to change a word and then repeat that change elsewhere, use cw and type the new word, then return to Normal mode. Now . will replay the change on the next word. If you use multiple disparate commands, the dot command might only repeat part of the change or might not behave as expected. Designing edits with repeatability in mind is a skill.

Repeatability also interacts with search and motions. A common workflow is: search for a pattern (/foo), make a change (cw bar), then use n to find the next match and . to repeat. This creates a tight loop: search, change, repeat. The same principle applies to f and t motions. If your change is associated with a find motion, you can often use ; to move to the next occurrence and . to repeat. This is a powerful pattern for editing repeated fields on a line or across lines.

The dot command only repeats the last change, not the last navigation. This is a feature, not a bug. It means you can control where the change applies by moving to the next target yourself. This division of labor makes repeated edits predictable. It also means you should avoid mixing multiple changes into one sequence if you want to repeat them. If you need a multi-step change, you should consider a macro instead. Macros can capture more complex sequences, but they are heavier than dot. Dot should be your first tool; macros are for when dot is insufficient.

Repeatability is also about data integrity. When you can repeat an edit reliably, you reduce the chance of inconsistent changes. This matters in code refactors, configuration edits, and log transformations. It also supports verification: you can quickly apply changes, then review, then undo or adjust if needed. Dot uses the same undo history as normal changes, so you can undo repeated edits with u and redo with Ctrl-r. This makes repeated changes safe to experiment with.

Finally, repeatability extends beyond the dot command. The @: command repeats the last Ex command, which is useful for substitutions. The & command repeats the last substitution with the same flags. Macros can be repeated with @@, and counts can be used to repeat dot or macros multiple times. Understanding this ecosystem gives you a ladder of automation: dot for simple repetition, @: for command-line repetition, macros for multi-step repetition. The key is to choose the lightest tool that solves the problem.

Another repeatable tool is the substitute repeat &, which replays the last substitution with the same flags. Combined with @: (repeat last Ex command), you get a command-line analog to the dot command. This is helpful when the last change was a command-line operation rather than a Normal mode edit. Understanding these parallel repeat mechanisms helps you choose the fastest repetition path.

How this fit on projects Projects 1-3 and 7 depend on repeatable edits and dot-driven workflows.

Definitions & key terms

  • Dot (.): repeat last change
  • @:: repeat last command-line command
  • @@: repeat last executed macro

Mental model diagram

One good change -> infinite repeats

vim_repeat_principle_flow

Repeat principle

How it works

  1. Perform a clean, single change.
  2. Move to the next target.
  3. Press . to repeat.
  4. Use counts to repeat multiple times if appropriate.

Minimal concrete example

cw bar<Esc>
n .

Common misconceptions

  • “Dot repeats navigation too.” (It repeats the change only.)
  • “You can dot-repeat command-line.” (Use @: or &.)
  • “Any edit works well with ..” (Design edits to be repeatable.)

Check-your-understanding questions

  1. What does . repeat?
  2. How do you repeat the last :s command?
  3. Why might x be a bad change for repetition?

Check-your-understanding answers

  1. The last change (insert, delete, change, operator).
  2. @: (or & for last substitute).
  3. x deletes one character; a motion-based change is more repeatable.

Real-world applications

  • Replacing identifiers across a file
  • Reformatting repeated lines
  • Applying a change across a search result set

Where you’ll apply it Projects 1-3, 7.

References

  • Vim :help repeat.txt
  • “Practical Vim” - Ch. 1

Key insights Design edits to be repeatable; dot is free power.

Summary The dot command is the simplest automation tool in Vim.

Homework/Exercises to practice the concept

  1. Replace “foo” with “bar” in 10 places using / and ..
  2. Change a list of words to uppercase using one change and dot.

Solutions to the homework/exercises

  1. /foo, cw bar<Esc>, then n . repeatedly.
  2. gUw then n ..

3. Project Specification

3.1 What You Will Build

A batch-edit workflow that normalizes data with substitutions and global commands.

Included:

  • Ranges
  • Confirm flag
  • Previewing matches

Excluded:

  • External scripting tools

3.2 Functional Requirements

  1. Core workflow: Run a safe substitution with confirmation
  2. Repeatability: Use a range to limit scope
  3. Validation: Use a global command to operate on matches

3.3 Non-Functional Requirements

  • Performance: Batch edits complete in seconds on sample files.
  • Reliability: No unintended lines are changed.
  • Usability: Commands are documented with before/after snapshots.

3.4 Example Usage / Output

Transform "User: 42, Name: Ada" into "Ada (id=42)" with a single command.

3.5 Data Formats / Schemas / Protocols

  • Lines with labeled fields: “User: id, Name: value”

3.6 Edge Cases

  • Missing fields
  • Extra spacing
  • Non-matching lines

3.7 Real World Outcome

This is the deterministic output you can compare against directly.

3.7.1 How to Run (Copy/Paste)

  • vim messy.txt

3.7.2 Golden Path Demo (Deterministic)

Normalize three sample lines with a confirmable substitution.

3.7.3 If CLI: provide an exact terminal transcript

$ vim messy.txt
# :%s/.../.../gc then verify output

4. Solution Architecture

4.1 High-Level Design

Input file -> Vim workflow plan -> Verification checklist

4.2 Key Components

Component Responsibility Key Decisions
Input File Provides deterministic data Keep it stable and versioned
Vim Workflow Plan Documents motions and operators Prefer repeatable sequences
Verification Checklist Confirms correctness Use before/after snapshots

4.4 Data Structures (No Full Code)

  • Entry: a structured line or block with fields relevant to the task
  • Target: the specific token or structure you will move to or edit
  • Checklist: steps to verify the output

4.4 Algorithm Overview

Key Algorithm: Motion-First Editing Loop

  1. Identify the target structure (word, field, block, or line).
  2. Choose a motion or text object that selects it safely.
  3. Apply the operator or edit and verify the result.

Complexity Analysis:

  • Time: O(n) over lines edited
  • Space: O(1) additional space

5. Implementation Guide

5.1 Development Environment Setup

vim --version
vimtutor

5.2 Project Structure

project-root/
|-- input/
|   `-- sample.txt
|-- notes/
|   `-- keystrokes.md
`-- outputs/
    `-- expected.txt

5.3 The Core Question You’re Answering

“Apply safe, confirmed substitutions and global commands across a file.”

5.4 Concepts You Must Understand First

Stop and research these before editing:

  • Search basics
  • Regex fundamentals
  • Command-line mode comfort

5.5 Questions to Guide Your Design

  1. Which motion or text object targets the structure directly?
  2. Can the change be repeated with dot or a macro?
  3. How will you verify correctness after the change?

5.6 Thinking Exercise

Before editing, sketch the steps needed to complete the task on paper.

5.7 The Interview Questions They’ll Ask

  1. “Which motion did you choose and why?”
  2. “How did you ensure the edit was repeatable?”
  3. “What is the risk of using Visual mode here?”
  4. “How did you validate the output?”

5.8 Hints in Layers

Hint 1: Start with a stable cursor position Use 0, ^, or a search to align before editing.

Hint 2: Choose the smallest safe unit If a word is enough, use a word object; if not, use a larger object.

Hint 3: Make it repeatable Design the first change so . works on the next target.

Hint 4: Validate Check before/after snapshots after each batch of edits.

5.9 Books That Will Help

Topic Book Chapter
Core workflow “Practical Vim” Ch. 1-4
Motions “Practical Vim” Ch. 8
Editing language “Learning the vi and Vim Editors” Ch. 3

5.10 Implementation Phases

Phase 1: Foundation (10-20 hours)

Goals:

  • Load the input file and identify targets
  • Verify core motions and search behavior

Tasks:

  1. Create a short checklist of target patterns
  2. Practice on 3-5 lines

Checkpoint: You can complete the smallest edit without mistakes.

Phase 2: Core Functionality (10-20 hours)

Goals:

  • Execute the main workflow end-to-end
  • Keep edits repeatable

Tasks:

  1. Apply the main edit sequence to the full file
  2. Record keystrokes or a macro if needed

Checkpoint: Output matches the golden path demo.

Phase 3: Polish & Edge Cases (10-20 hours)

Goals:

  • Handle edge cases
  • Document the workflow

Tasks:

  1. Test edge cases from section 3.6
  2. Write a short summary of decisions

Checkpoint: Edge cases are handled or documented.

5.11 Key Implementation Decisions

Decision Options Recommendation Rationale
Targeting strategy Search vs find vs motion Choose the most stable Stability beats speed early on
Repeatability Dot vs macro Use dot first Lower complexity
Verification Visual check vs checklist Use a checklist Prevents silent errors

6. Testing Strategy

6.1 Test Categories

Category Purpose Examples
Manual Checks Validate edits Before/after snapshots
Repeatability Tests Ensure dot/macro works Run on 3+ targets
Edge Case Tests Handle boundary conditions Missing fields or empty lines

6.2 Critical Test Cases

  1. Nominal case: Apply the workflow to a standard line.
  2. Duplicate target: Handle two targets on the same line.
  3. Irregular line: Verify behavior when a field is missing.

6.3 Test Data

Use the provided sample input file and create 3 additional lines with variations.

7. Common Pitfalls & Debugging

7.1 Frequent Mistakes

Pitfall Symptom Solution
Over-using Visual mode Changes are hard to repeat Use operator + motion
Wrong motion choice Target missed Use a larger text object
No validation step Silent errors Use a checklist

7.2 Debugging Strategies

  • Replay slowly: Step through the workflow one command at a time.
  • Use undo: Roll back and re-apply with a clearer motion.

7.3 Performance Traps

Overusing j/k on large files instead of search can make the workflow slow.

8. Extensions & Challenges

8.1 Beginner Extensions

  • Repeat the workflow on a smaller file
  • Document the exact keystroke sequence

8.2 Intermediate Extensions

  • Apply the workflow to a real project file
  • Reduce keystroke count by 20%

8.3 Advanced Extensions

  • Build a macro to automate the workflow
  • Create a custom mapping to speed up a frequent step

9. Real-World Connections

9.1 Industry Applications

  • Remote server editing: Vim is common on production systems
  • Incident response: quick log edits without GUI tools
  • Vim: core editor reference
  • Neovim: modernized modal editor
  • Universal Ctags: tag generation tool

9.3 Interview Relevance

  • Motion grammar questions
  • Repeatability and macro questions
  • Command-line editing scenarios

10. Resources

10.1 Essential Reading

  • “Practical Vim” by Drew Neil - focus on motion and change workflows
  • “Learning the vi and Vim Editors” - foundational navigation

10.2 Video Resources

  • “Vim as a Language” talk (searchable title)
  • “Practical Vim” author talks (searchable title)