← Back to all projects

LEARN WINDOWS ARCHITECTURE

Learn Windows Architecture: From API to HAL

Goal: To deeply understand the core architecture of the Windows NT operating system, from the separation of User and Kernel mode to the system call mechanism and the role of the Hardware Abstraction Layer (HAL).

Why Learn Windows Architecture?

Most developers interact with Windows through its high-level APIs. They call CreateFile and a file handle appears. But what happens inside the OS during that call? Understanding this process is the key to mastering systems programming. It allows you to:

  • Debug Intelligently: When an API call fails, you’ll understand the potential reasons, from user-mode permissions to kernel-mode driver conflicts.
  • Write High-Performance Code: You’ll know the “cost” of crossing the user/kernel boundary and design your applications to do it efficiently.
  • Enhance Security: You’ll understand why security boundaries exist and how they are enforced by the hardware and the kernel.
  • Build Powerful Tools: It’s the foundation for writing debuggers, security software, and custom drivers.

After completing these projects, you will be able to trace a single API call from your application, through the system libraries, across the user/kernel boundary, into the heart of the Windows kernel, and back.

Core Concept Analysis: The Windows NT Design

The Windows NT architecture (the foundation for everything from Windows XP to Windows 11) is built on a highly-structured, layered design that enforces security and stability.

1. The Great Divide: User Mode vs. Kernel Mode

The most important concept is the separation between the unprivileged User Mode and the all-powerful Kernel Mode. This separation is enforced by the CPU itself (using protection rings).

  • User Mode (Ring 3): This is where all your applications run (explorer.exe, chrome.exe, your game).
    • Protected: Each process gets its own private virtual address space. One application cannot see or modify the memory of another.
    • Restricted: User-mode code cannot directly access hardware (like the disk or network card) or manipulate critical OS data structures.
    • Resilient: If a user-mode application crashes, the operating system is unaffected.
  • Kernel Mode (Ring 0): This is the heart of the OS.
    • Privileged: Code running in kernel mode has unrestricted access to all system memory and all hardware.
    • Shared: The kernel’s code and data live in a single, shared address space used by all processes.
    • Fragile: A bug or crash in kernel-mode code is catastrophic, leading to a system-wide halt known as the “Blue Screen of Death” (BSOD).

2. The System Call: Crossing the Boundary

If user mode can’t access hardware, how does it save a file? It must ask the kernel to do it. This formal, highly-controlled request is a system call.

The Path of CreateFile:

      USER MODE
+--------------------+   1. Calls documented Win32 API
|     Your App       |──────►+-----------------+
|   (MyApp.exe)      |       |  KERNEL32.DLL   |
+--------------------+       | `CreateFileW`   |
                             +-------+---------+
                                     │ 2. This is just a wrapper...
                                     ▼
                             +-------+---------+
                             |   NTDLL.DLL     |
                             | `NtCreateFile`  |
                             +-------+---------+
                                     │ 3. The "gateway" to the kernel.
                                     │    It prepares for the system call...
                                     ▼
  ================= CPU instruction `syscall` or `int 2E` =================
                                     ▲
                             +-------+---------+   6. The kernel returns
      KERNEL MODE            |  NTOSKRNL.EXE   |      control here.
                             | System Call     |
                             | Dispatcher      |
                             +-------+---------+
                                     │ 4. Kernel receives control, looks up
                                     │    the real function, and executes it.
                                     ▼
                             +-------+---------+
                             |  NTOSKRNL.EXE   |
                             | `NtCreateFile`  |
                             | (Internal Impl) |
                             +-----------------+
                                     │ 5. Interacts with other managers
                                     │    (I/O, Object, Security...)
                                     ▼
                             +-----------------+
                             | DRIVERS / HAL   |
                             +-----------------+
  1. Your application calls a friendly function in a subsystem DLL like kernel32.dll.
  2. kernel32.dll calls a lower-level, mostly undocumented function in ntdll.dll.
  3. ntdll.dll places the system call’s unique number in a CPU register (EAX) and triggers a software interrupt.
  4. The CPU sees the interrupt, switches into kernel mode, and passes control to the kernel’s central dispatcher (KiSystemCall).
  5. The dispatcher uses the number from EAX to find and execute the real kernel function inside ntoskrnl.exe.
  6. The kernel function performs the work and returns a status code. The CPU then switches back to user mode and returns control to ntdll.dll, which propagates the result back to your application.

3. HAL: The Hardware Abstraction Layer

Why can the same version of Windows run on thousands of different motherboards from different manufacturers? The answer is the Hardware Abstraction Layer (HAL).

  • The Goal: To provide a consistent interface for the kernel to interact with hardware-specific components, like interrupt controllers, timers, and DMA channels.
  • How it Works: The kernel doesn’t know the specifics of your motherboard’s chipset. When it needs to perform a hardware-specific action, it calls a generic HAL function (e.g., HalGetInterruptVector). The HAL is a specific DLL (hal.dll) that is loaded at boot time and contains the actual code to talk to your specific hardware. This isolates the core kernel from hardware variations.

Project List

These projects are designed to let you observe and interact with these architectural concepts directly. The primary tools will be Visual Studio, the Windows Driver Kit (WDK), and the WinDbg debugger.

Project 1: Tracing a System Call with WinDbg

  • File: LEARN_WINDOWS_ARCHITECTURE.md
  • Main Programming Language: N/A (Debugger commands)
  • Alternative Programming Languages: C (for the target app)
  • Coolness Level: Level 4: Hardcore Tech Flex
  • Business Potential: 1. The “Resume Gold”
  • Difficulty: Level 2: Intermediate
  • Knowledge Area: Debugging / OS Internals
  • Software or Tool: WinDbg, Visual Studio
  • Main Book: “Windows Internals, Part 1 & 2” by Russinovich, Solomon, and Ionescu

What you’ll build: This is not a coding project, but an investigation. You will write a tiny C program that does nothing but call CreateFile. You will then use the WinDbg debugger to step through the execution path, from your C code all the way to the edge of the kernel.

Why it teaches Windows architecture: This makes the system call diagram real. You will personally witness the call travel from your code to kernel32.dll and then to ntdll.dll, proving the layered design and seeing the setup for the final syscall instruction.

Core challenges you’ll face:

  • Setting up WinDbg and symbols → maps to configuring the debugger to download the necessary debugging information for the Windows system DLLs
  • Setting and using breakpoints → maps to using the bp command to stop execution at specific functions
  • Stepping through code → maps to using p (step over), t (step into), and viewing the call stack with k
  • Navigating modules → maps to understanding the difference between your code, kernel32, and ntdll

Difficulty: Intermediate (The concepts are advanced, but the execution is following a recipe) Time estimate: Weekend Prerequisites: Basic C knowledge.

Real world outcome: A profound “aha!” moment. You will see the call stack in WinDbg showing the exact path from your application to the kernel’s gatekeeper, ntdll.

WinDbg Output (simplified):

0:000> k
 # Child-SP          RetAddr           Call Site
00 0000002ab4afe838  00007ffc64b612c4  ntdll!NtCreateFile+0x14  <-- The syscall instruction
01 0000002ab4afe840  00007ff7559e102d  KERNELBASE!CreateFileW+0x84
02 0000002ab4afeaf0  00007ff7559e13bd  MyApp!main+0x2d          <-- Your code
...

Implementation Hints:

  1. Install WinDbg Preview from the Microsoft Store.
  2. In Visual Studio, create a simple C++ console app. In main, call CreateFileW with some dummy parameters. Compile it in Debug mode.
  3. Launch your compiled .exe under WinDbg.
  4. Set a breakpoint on your main function: bp MyApp!main. Run with g.
  5. When it hits, set a breakpoint on CreateFileW: bp kernel32!CreateFileW. Run again.
  6. When it hits, look at the call stack: k. You are now inside kernel32.dll.
  7. Set one last breakpoint: bp ntdll!NtCreateFile. Run again.
  8. You are now in ntdll.dll. Step through the instructions (t) and you will eventually see the syscall instruction that jumps to the kernel.

Learning milestones:

  1. You can launch an app and break on a function inside it → You understand basic WinDbg usage.
  2. You can view the call stack and see the layers: MyApp -> kernelbase -> ntdll → You have visualized the user-mode part of a system call.
  3. You can locate the syscall instruction in the disassembly → You have found the exact user/kernel transition point.

Project 2: Your First Kernel Driver

  • File: LEARN_WINDOWS_ARCHITECTURE.md
  • Main Programming Language: C
  • Coolness Level: Level 5: Pure Magic (Super Cool)
  • Business Potential: 1. The “Resume Gold”
  • Difficulty: Level 3: Advanced
  • Knowledge Area: Kernel Development / Drivers
  • Software or Tool: Visual Studio, Windows Driver Kit (WDK), VMWare/VirtualBox
  • Main Book: “Windows Kernel Programming” by Pavel Yosifovich

What you’ll build: A minimal “Hello, World” kernel driver. This driver does nothing but print a message to the kernel’s debug output when it’s loaded and another when it’s unloaded.

Why it teaches Windows architecture: This project demystifies kernel mode. You will leave the safety of user-mode and write code that runs in Ring 0. This forces you to set up a proper kernel development and debugging environment, the barrier to entry for all driver development.

Core challenges you’ll face:

  • Installing the WDK and integrating with Visual Studio → maps to setting up the professional environment for driver development
  • Setting up a test virtual machine → maps to understanding that you NEVER test drivers on your main machine
  • Configuring kernel debugging → maps to connecting WinDbg on your host machine to the kernel of your guest VM
  • Disabling Test Signing → maps to bypassing Windows’ strict driver signing requirements for development
  • Writing DriverEntry and DriverUnload routines → maps to the main() and cleanup functions of a driver

Difficulty: Advanced (The setup is 90% of the challenge) Time estimate: 1-2 weeks Prerequisites: Project 1, patience, and careful attention to detail.

Real world outcome: You will load your compiled .sys file into your test VM, and a “Hello from the kernel!” message will appear in your WinDbg window on your host machine.

Debug Output in WinDbg:

... (MyDriver) Hello from the kernel! DriverEntry is running.
...

(When you unload the driver)

... (MyDriver) Goodbye from the kernel! Driver is unloading.
...

Implementation Hints:

  1. You must use a virtual machine (VMWare Player/Workstation or VirtualBox are fine).
  2. Install Visual Studio and the matching WDK on your host machine.
  3. In the VM, you need to enable test signing mode by running bcdedit /set testsigning on as an administrator.
  4. Set up a virtual serial port in the VM settings. Configure kernel debugging in the VM to use this serial port (bcdedit /debug on, bcdedit /dbgsettings serial ...).
  5. In WinDbg on your host machine, connect to the VM’s kernel over the named pipe corresponding to the virtual serial port.
  6. In Visual Studio, use the “Kernel Mode Driver (Empty)” template.
  7. Your main C file will have a DriverEntry function. This is your entry point. Use the DbgPrintEx function to print your hello message.
  8. Assign a function to the DriverUnload member of the driver object to handle unloading.
  9. Compile the driver and copy the .sys file to your VM. Use a tool like OSR’s Driver Loader to load and unload it.

Learning milestones:

  1. Your host WinDbg successfully connects to your VM’s kernel → Kernel debugging is set up correctly.
  2. Your driver project compiles without errors → The WDK is integrated.
  3. Your driver loads into the VM kernel without a BSOD → You have a stable, minimal driver.
  4. You see your “Hello” message in the debugger → You have successfully executed code in kernel mode and produced output.

Project 3: User-Kernel Communication with IOCTLs

  • File: LEARN_WINDOWS_ARCHITECTURE.md
  • Main Programming Language: C/C++
  • Coolness Level: Level 5: Pure Magic (Super Cool)
  • Business Potential: 3. The “Service & Support” Model
  • Difficulty: Level 4: Expert
  • Knowledge Area: Kernel Development / I/O System
  • Software or Tool: WDK, WinDbg
  • Main Book: “Windows Kernel Programming” by Pavel Yosifovich

What you’ll build: A matched pair: a kernel driver that creates a “device” and a user-mode console application that talks to it. The console app will send a message (e.g., a number), and the driver will receive it, print it to the debug output, and send a response back to the app.

Why it teaches Windows architecture: This project teaches the primary, sanctioned method of communication across the user/kernel boundary: I/O Request Packets (IRPs) sent via DeviceIoControl (IOCTLs). You’ll understand how applications talk to drivers for everything from graphics cards to USB drives.

Core challenges you’ll face:

  • Creating a device object in the driver → maps to using IoCreateDevice to make your driver visible to the I/O Manager
  • Creating a symbolic link → maps to using IoCreateSymbolicLink so user-mode code can find and open your device
  • Handling IRPs → maps to implementing a dispatch routine for IRP_MJ_DEVICE_CONTROL
  • Defining a custom IOCTL code → maps to using the CTL_CODE macro to create a unique identifier for your request
  • Opening the device from user mode → maps to calling CreateFile on the symbolic link you created
  • Sending the IOCTL → maps to calling DeviceIoControl from your console app with your custom code

Difficulty: Expert Time estimate: 2-3 weeks Prerequisites: Project 2.

Real world outcome: You will run a console app. It will print “Sending ‘42’ to driver…”. Seconds later, in your kernel debugger, the message “[MyDriver] Received data from user mode: 42” will appear. The console app will then print “Driver responded with success!”

Implementation Hints:

  1. In the driver (DriverEntry):
    • Call IoCreateDevice to create a DEVICE_OBJECT.
    • Call IoCreateSymbolicLink to create a user-friendly name (e.g., \??\MyDriver) that points to the device object.
    • Set up dispatch routines, especially for IRP_MJ_CREATE, IRP_MJ_CLOSE, and IRP_MJ_DEVICE_CONTROL. For the IOCTL handler, you will parse the IRP, get the IOCTL code, access the input buffer, and complete the request.
  2. In a shared header file: Define your custom IOCTL code using the CTL_CODE macro. This ensures both user-mode and kernel-mode code use the exact same value.
  3. In the user-mode app:
    • Call CreateFile with the path \\.\MyDriver to get a handle to your device.
    • Allocate buffers for your input and output data.
    • Call DeviceIoControl, passing it the device handle, your custom IOCTL code, and pointers to your buffers.
    • Check the return value to see if it succeeded.

Learning milestones:

  1. Your user-mode app successfully gets a handle to your device → Your driver’s device creation and symbolic link are correct.
  2. The driver’s IOCTL dispatch routine is triggered when the app calls DeviceIoControl → The IRP is being routed correctly by the I/O Manager.
  3. The driver can read the data sent from the user-mode app → You are correctly accessing the IRP buffers.
  4. The user-mode app receives a success/failure code back from the driver → You have completed a full, end-to-end communication cycle across the user/kernel boundary.

Project Comparison Table

Project Difficulty Time Core Concept Fun Factor
1. Tracing with WinDbg Intermediate Weekend System Call Path ★★★★☆
2. First Kernel Driver Advanced 1-2 weeks Kernel Environment ★★★★★
3. User-Kernel IOCTL Expert 2-3 weeks I/O Subsystem ★★★★★

Recommendation

For this topic, the path is clear and sequential.

  1. Start with Project 1: Tracing a System Call with WinDbg. This non-coding project is the most crucial first step. It will build a strong mental model of the user-to-kernel transition that is essential for understanding why the other projects are necessary. It provides the “map” for your journey.

  2. Next, tackle Project 2: Your First Kernel Driver. The goal here is not the driver itself, but mastering the complex development and debugging environment. Getting “Hello, World” to print from the kernel is a major achievement that proves your toolchain is working.

  3. Finally, with the foundation from the first two projects, build Project 3: User-Kernel Communication with IOCTLs. This is the capstone project that ties everything together, forcing you to write both user-mode and kernel-mode code that interact in a standard, structured way. Completing this project demonstrates a true, practical understanding of Windows’ core architecture.

Summary

  • Project 1: Tracing a System Call with WinDbg: Debugger Commands
  • Project 2: Your First Kernel Driver: C
  • Project 3: User-Kernel Communication with IOCTLs: C/C++

```