Project 1: The Manual Pin Projector

Project 1: The Manual Pin Projector

Understanding the Pin Contract at the Deepest Level

• Main Programming Language: Rust
• Coolness Level: Level 4: Hardcore Tech Flex
• Difficulty: Level 3: Advanced
• Knowledge Area: Memory Management / Safety Contracts
• Estimated Time: 3-5 days
• Prerequisites: Ownership, Borrowing, Basic Async/Await

---

Learning Objectives

After completing this project, you will be able to:

1. Explain Pin at the memory level - Draw diagrams showing how self-referential structs break when moved and how Pin prevents this
2. Distinguish Unpin from !Unpin - Understand why most types are Unpin and when you need PhantomPinned
3. Implement structural pinning manually - Create safe projection methods without using pin-project crate
4. Build self-referential structs safely - Use unsafe code correctly to maintain the Pin contract
5. Implement Future for pinned types - Write the poll method with proper Pin<&mut Self> handling
6. Verify address stability - Use pointer comparisons to prove your implementation upholds guarantees
7. Understand the relationship between Pin and unsafe - Know when unsafe is required and how to audit it
8. Apply Pin knowledge to real async code - Debug pinning errors in production async applications
9. Distinguish structural from non-structural pinning - Know when projections preserve Pin guarantees
10. Connect Pin theory to Tokio/async-std internals - Understand how production runtimes use these concepts

---

Deep Theoretical Foundation

What Problem Does Pin Solve?

Before understanding Pin, you must understand the fundamental problem it solves: self-referential structs are broken by Rust’s move semantics.

The Self-Referential Problem Visualized

Consider a struct that contains a pointer to its own field:

struct SelfRef {
    value: String,
    ptr_to_value: *const String,  // Points to `value` above
}

When this struct is created and initialized correctly:

Stack Memory (Address 0x1000)
+---------------------------+
| SelfRef                   |
+---------------------------+
| value: String             | <-- 0x1000
|   ptr: 0x5000 (heap)      |
|   len: 5                  |
|   cap: 8                  |
+---------------------------+
| ptr_to_value: 0x1000  ----+---> Points to value field above
+---------------------------+
                            |
                            v
Heap Memory (Address 0x5000)
+---------------------------+
| "Hello"                   |
+---------------------------+

Everything is valid. But now, what happens when Rust MOVES this struct?

let a = create_self_ref();  // Created at 0x1000
let b = a;                  // MOVED to 0x2000!

After the move:

OLD Location (0x1000) - DEALLOCATED
+---------------------------+
| (garbage/reused memory)   |
+---------------------------+

NEW Location (0x2000)
+---------------------------+
| SelfRef                   |
+---------------------------+
| value: String             | <-- 0x2000 (NEW address!)
|   ptr: 0x5000 (heap)      |
|   len: 5                  |
|   cap: 8                  |
+---------------------------+
| ptr_to_value: 0x1000  ----+---> DANGLING! Points to old location!
+---------------------------+

                            |
                            v
                   0x1000 is now garbage!
                   UNDEFINED BEHAVIOR!

The pointer wasn’t updated because Rust moves are bitwise copies. The pointer still holds the old address 0x1000, but value now lives at 0x2000.

The History: Why Pin Was Invented

Pin was introduced in RFC 2349 (merged December 2018) to solve a specific problem: async/await needed self-referential types to work safely.

When you write:

async fn example() {
    let data = vec![1, 2, 3];
    some_io().await;  // <-- Suspension point
    println!("{:?}", data);  // data must still be valid!
}

The compiler transforms this into a state machine struct:

enum ExampleFuture {
    State0,
    State1 { data: Vec<i32> },  // data stored across await
    Done,
}

The problem: If this state machine contains references to its own fields (common in complex async code), moving the future breaks those references.

Before Pin, async runtimes had to Box every future (heap allocation). Pin enables stack-allocated futures with self-references.

The Pin Type: A Contract, Not a Lock

Pin<P> where P is a pointer type (like &mut T, Box<T>, etc.) represents a contractual guarantee:

“The value pointed to by P will never be moved until it is dropped.”

Important: Pin doesn’t physically prevent movement at the CPU level. It’s a type-system contract enforced by:

1. Not providing safe access to &mut T for !Unpin types
2. Requiring unsafe code to get &mut T from Pin<&mut T>

Understanding Unpin: The Opt-Out Trait

Unpin is an auto trait. Almost every type in Rust implements it automatically:

// These all implement Unpin automatically:
struct Normal { x: i32, y: i32 }
struct WithBox { data: Box<String> }
struct WithVec { items: Vec<u8> }

Why? Because moving these types doesn’t cause undefined behavior. Their internal pointers (if any) point to heap data, not to themselves.

Normal struct (Unpin):
Stack                    Heap
+-------+
|  x: 5 |                (no heap data)
|  y: 10|
+-------+

After move - still valid:
+-------+
|  x: 5 |
|  y: 10|
+-------+

When is a type !Unpin?

A type is !Unpin when:

1. It contains PhantomPinned (explicit opt-out)
2. It contains any !Unpin field (negative trait propagation)
3. Compiler-generated futures often contain self-references

use std::marker::PhantomPinned;

struct MustNotMove {
    value: String,
    ptr: *const String,
    _pin: PhantomPinned,  // Makes this type !Unpin
}

Memory Layout: Why Self-References Break

Let’s trace through memory step-by-step:

STEP 1: Struct created on stack frame A (function foo)
========================================================

Stack Frame A (foo) at 0x7FFF_0100
+----------------------------------+
| MustNotMove                      |
+----------------------------------+
| value: String                    | Address: 0x7FFF_0100
|   - ptr:    0x6000_0000 (heap)   |
|   - len:    11                   |
|   - cap:    16                   |
+----------------------------------+
| ptr: *const String               | Address: 0x7FFF_0118
|   - value:  0x7FFF_0100 ----+    |
+----------------------------------+  |
| _pin: PhantomPinned (0 bytes)    |  |
+----------------------------------+  |
                                      |
                   +------------------+
                   |
                   v
         Points to 0x7FFF_0100 (the value field)


STEP 2: Function foo returns, struct MOVED to caller's frame
=============================================================

Stack Frame B (caller) at 0x7FFF_0200
+----------------------------------+
| MustNotMove                      |
+----------------------------------+
| value: String                    | NEW Address: 0x7FFF_0200
|   - ptr:    0x6000_0000 (heap)   |
|   - len:    11                   |
|   - cap:    16                   |
+----------------------------------+
| ptr: *const String               |
|   - value:  0x7FFF_0100 ----+    | STALE! Still points to old address!
+----------------------------------+  |
                                      |
                   +------------------+
                   |
                   v
        Points to 0x7FFF_0100 (DEALLOCATED!)


Stack Frame A (DEALLOCATED)
+----------------------------------+
| ?????  GARBAGE / REUSED  ?????   |
+----------------------------------+


STEP 3: Dereferencing ptr causes UNDEFINED BEHAVIOR
====================================================

let s: &String = unsafe { &*self.ptr };
//                           ^
//                           |
//                     Reads from 0x7FFF_0100
//                     which is garbage now!

Stack Pinning vs Heap Pinning

There are two ways to pin data:

Heap Pinning with Box::pin

let pinned: Pin<Box<MyType>> = Box::pin(my_value);

Stack                          Heap (stable address!)
+------------+                 +------------------+
| Pin<Box<T>>|                 | MyType           |
| ptr: 0x5000+---------------->| value: ...       |
+------------+                 | ptr: 0x5000 -----+--> Points to itself
                               +------------------+

The Box can be moved on the stack, but the data
on the heap stays at 0x5000 forever!

Stack Pinning with pin! macro (or unsafe)

use std::pin::pin;

let pinned: Pin<&mut MyType> = pin!(my_value);
// my_value is now shadowed and pinned to current stack frame

Stack (current frame - cannot leave!)
+------------------+
| MyType           | <-- Pinned here until scope ends
| value: ...       |
| ptr: 0x7FFF_0100 +--> Points to self (valid!)
+------------------+

WARNING: This Pin<&mut T> cannot outlive the current stack frame!

Key difference:

• Box::pin - Data lives on heap, can be returned from functions
• pin!() - Data lives on stack, cannot outlive current function

Structural vs Non-Structural Pinning

When you have Pin<&mut MyStruct>, can you access the fields? It depends on structural pinning.

Structural Pinning (fields inherit Pin)

A field is structurally pinned if:

1. The field is itself !Unpin
2. You want to maintain Pin guarantees through projections

struct MyFuture {
    // Structurally pinned - accessing this should give Pin<&mut InnerFuture>
    inner: InnerFuture,

    // NOT structurally pinned - this is just data
    ready: bool,
}

Projection for structurally pinned field:

impl MyFuture {
    fn project(self: Pin<&mut Self>) -> Pin<&mut InnerFuture> {
        // UNSAFE: We promise inner won't be moved out
        unsafe { self.map_unchecked_mut(|s| &mut s.inner) }
    }
}

Non-Structural Pinning (fields don’t inherit Pin)

A field is not structurally pinned if:

1. The field is Unpin
2. You want to allow moving/replacing the field

impl MyFuture {
    fn get_ready_mut(self: Pin<&mut Self>) -> &mut bool {
        // SAFE: bool is Unpin, no pin projection needed
        unsafe { &mut self.get_unchecked_mut().ready }
    }
}

The Projection Struct Pattern

The pin-project crate automates this, but let’s understand it manually:

struct SelfRefFuture {
    data: String,
    data_ptr: *const String,
    state: FutureState,
    _pin: PhantomPinned,
}

// Projection struct - what we return from project()
struct SelfRefFutureProjection<'a> {
    data: Pin<&'a mut String>,      // Structurally pinned
    data_ptr: &'a mut *const String, // Not structurally pinned (raw ptr)
    state: &'a mut FutureState,      // Not structurally pinned (Unpin type)
    // Note: _pin is not exposed (it's a ZST anyway)
}

Pin and Drop: The Subtle Danger

If your type implements Drop, there’s a subtle issue:

impl Drop for SelfRefFuture {
    fn drop(&mut self) {
        //     ^^^^^^^^^
        // You get &mut self, not Pin<&mut Self>!
        // This means you could move fields in drop!
    }
}

The Drop guarantee: If your type is !Unpin, you must NOT move fields in Drop. The compiler cannot enforce this - it’s a safety invariant you must uphold.

// WRONG - breaks Pin contract!
impl Drop for SelfRefFuture {
    fn drop(&mut self) {
        let moved_out = std::mem::take(&mut self.data); // BAD!
    }
}

// CORRECT - don't move anything
impl Drop for SelfRefFuture {
    fn drop(&mut self) {
        // Just let fields drop in place, don't move them
        // Or do cleanup that doesn't involve moving
    }
}

---

What You’ll Build

A self-referential struct without using the pin-project crate. You will manually implement structural pinning and projection, handling the safety invariants required to prevent data movement. You will create a custom Future that stores a pointer to its own fields.

Why This Project Teaches Pinning

Most developers use the pin-project macro without understanding why it exists. By implementing the projection manually, you’ll see how Unpin bounds are checked, why PhantomPinned is necessary, and how to safely access fields of a pinned struct.

---

Solution Architecture

Your implementation will consist of these components:

1. The Self-Referential Struct

struct SelfRefFuture {
    // The data we're processing
    data: String,

    // Raw pointer to data - this is the self-reference!
    data_ptr: *const String,

    // State machine for the Future
    state: State,

    // Opt-out of Unpin
    _pin: PhantomPinned,
}

enum State {
    Initial,
    Processing,
    Complete(String),
}

Why this design?

• data_ptr creates the self-reference that would break on move
• PhantomPinned ensures the type is !Unpin
• State allows the Future to progress through stages

2. The Projection Method

impl SelfRefFuture {
    fn project(self: Pin<&mut Self>) -> SelfRefFutureProjection<'_> {
        unsafe {
            let this = self.get_unchecked_mut();
            SelfRefFutureProjection {
                data: Pin::new_unchecked(&mut this.data),
                data_ptr: &mut this.data_ptr,
                state: &mut this.state,
            }
        }
    }
}

Safety invariants you must maintain:

1. Never move data out of the struct
2. Never expose &mut String for data (only Pin<&mut String>)
3. Never implement Unpin for SelfRefFuture

3. The Future Implementation

impl Future for SelfRefFuture {
    type Output = String;

    fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output> {
        let mut this = self.project();

        match this.state {
            State::Initial => {
                // Initialize self-reference
                *this.data_ptr = this.data.as_ref().get_ref() as *const String;
                *this.state = State::Processing;
                cx.waker().wake_by_ref();
                Poll::Pending
            }
            State::Processing => {
                // Verify pointer validity (demonstrates safety)
                let data_ref = unsafe { &**this.data_ptr };
                let result = format!("Processed: {}", data_ref);
                *this.state = State::Complete(result.clone());
                Poll::Ready(result)
            }
            State::Complete(result) => Poll::Ready(result.clone()),
        }
    }
}

4. Safe Construction

impl SelfRefFuture {
    pub fn new(data: String) -> Pin<Box<Self>> {
        let future = SelfRefFuture {
            data,
            data_ptr: std::ptr::null(),  // Initialize as null
            state: State::Initial,
            _pin: PhantomPinned,
        };

        let mut boxed = Box::pin(future);

        // Initialize self-reference while pinned
        unsafe {
            let mut_ref = Pin::as_mut(&mut boxed);
            let this = mut_ref.get_unchecked_mut();
            this.data_ptr = &this.data as *const String;
        }

        boxed
    }
}

---

Phased Implementation Guide

Phase 1: Create Basic Self-Referential Struct (Unsafe)

Goal: Understand why self-references are dangerous by creating one without Pin.

1. Create a struct with a String field and a raw pointer to it
2. Initialize the pointer to point to the String
3. Move the struct and observe the dangling pointer
4. Use addr_of! to safely check addresses

Key code to write:

use std::ptr::addr_of;

struct UnsafeSelfRef {
    value: String,
    ptr: *const String,
}

impl UnsafeSelfRef {
    fn new(s: String) -> Self {
        let mut this = UnsafeSelfRef {
            value: s,
            ptr: std::ptr::null(),
        };
        this.ptr = addr_of!(this.value);
        this
    }

    fn verify(&self) -> bool {
        self.ptr == addr_of!(self.value)
    }
}

What to observe:

let a = UnsafeSelfRef::new("hello".into());
println!("Before move: valid = {}", a.verify()); // true

let b = a; // MOVE!
println!("After move: valid = {}", b.verify()); // FALSE!

Phase 2: Add PhantomPinned Marker

Goal: Make the type !Unpin so Pin can enforce its guarantees.

1. Add PhantomPinned field
2. Observe that Pin::new() no longer works (requires Unpin)
3. Use Box::pin() for heap pinning

Key changes:

use std::marker::PhantomPinned;

struct PinnedSelfRef {
    value: String,
    ptr: *const String,
    _pin: PhantomPinned,
}

// This won't compile anymore:
// let pinned = Pin::new(&mut my_struct); // Error: !Unpin

// Must use Box::pin or unsafe:
let pinned = Box::pin(my_struct);

Phase 3: Implement Safe Projection Methods

Goal: Create methods that maintain Pin invariants while allowing field access.

1. Write a project() method
2. Return different types for pinned vs unpinned fields
3. Document safety requirements

The Projection struct:

struct PinnedSelfRefProjection<'a> {
    // Pinned field - returns Pin<&mut T>
    value: Pin<&'a mut String>,
    // Non-pinned field - returns &mut T
    ptr: &'a mut *const String,
}

impl PinnedSelfRef {
    fn project(self: Pin<&mut Self>) -> PinnedSelfRefProjection<'_> {
        // SAFETY:
        // - We never move `value` out of the struct
        // - We never expose &mut String for `value`
        // - The struct is !Unpin due to PhantomPinned
        unsafe {
            let this = self.get_unchecked_mut();
            PinnedSelfRefProjection {
                value: Pin::new_unchecked(&mut this.value),
                ptr: &mut this.ptr,
            }
        }
    }
}

Phase 4: Implement the Future Trait

Goal: Make your type usable with async/await.

1. Implement Future with poll(self: Pin<&mut Self>, ...)
2. Use projection to access fields
3. Manage state transitions

State machine pattern:

enum FutureState {
    NotStarted,
    WaitingForData,
    Complete,
}

impl Future for PinnedSelfRef {
    type Output = String;

    fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output> {
        let proj = self.project();

        match *proj.state {
            FutureState::NotStarted => {
                // Initialize pointer now that we're pinned
                *proj.ptr = proj.value.get_ref() as *const String;
                *proj.state = FutureState::WaitingForData;
                cx.waker().wake_by_ref();
                Poll::Pending
            }
            FutureState::WaitingForData => {
                // Demonstrate self-reference works
                let data = unsafe { &**proj.ptr };
                *proj.state = FutureState::Complete;
                Poll::Ready(format!("Got: {}", data))
            }
            FutureState::Complete => {
                panic!("Polled after completion")
            }
        }
    }
}

Phase 5: Test Address Stability

Goal: Prove your implementation maintains the Pin contract.

1. Print addresses before and after operations
2. Poll multiple times and verify stability
3. Compare with non-pinned version

Address verification pattern:

#[test]
fn test_address_stability() {
    let mut future = PinnedSelfRef::new("test".into());

    let addr_before = {
        let proj = future.as_mut().project();
        proj.value.get_ref() as *const String
    };

    // Poll the future
    let waker = futures::task::noop_waker();
    let mut cx = Context::from_waker(&waker);
    let _ = future.as_mut().poll(&mut cx);
    let _ = future.as_mut().poll(&mut cx);

    let addr_after = {
        let proj = future.as_mut().project();
        proj.value.get_ref() as *const String
    };

    assert_eq!(addr_before, addr_after, "Address must not change!");
}

---

Testing Strategy

1. Address Stability Tests

#[test]
fn test_pin_prevents_movement() {
    use std::ptr::addr_of;

    let mut future = SelfRefFuture::new("hello".into());

    // Get address of data field
    let data_addr = unsafe {
        addr_of!(Pin::as_mut(&mut future).get_unchecked_mut().data)
    };

    // Poll multiple times
    block_on(async {
        let _ = (&mut future).await;
    });

    // Address should be unchanged
    let data_addr_after = unsafe {
        addr_of!(Pin::as_mut(&mut future).get_unchecked_mut().data)
    };

    assert_eq!(data_addr, data_addr_after);
}

2. Self-Reference Validity Tests

#[test]
fn test_self_reference_remains_valid() {
    let mut future = SelfRefFuture::new("test data".into());

    // After construction, pointer should point to data
    unsafe {
        let this = future.as_mut().get_unchecked_mut();
        assert_eq!(this.data_ptr, addr_of!(this.data));
    }

    // After polling, pointer should still be valid
    let waker = noop_waker();
    let mut cx = Context::from_waker(&waker);
    let _ = future.as_mut().poll(&mut cx);

    unsafe {
        let this = future.as_mut().get_unchecked_mut();
        assert_eq!(&*this.data_ptr, &this.data);
    }
}

3. Miri Tests for Undefined Behavior

Run with Miri to detect UB:

cargo +nightly miri test

Miri will catch:

• Dangling pointer dereferences
• Invalid memory access
• Violations of aliasing rules

4. Compile-Time Guarantee Tests

// This should NOT compile - proves our type is !Unpin
// Uncomment to verify the error
/*
#[test]
fn should_not_compile() {
    let mut future = SelfRefFuture::new("test".into());
    // Error: the trait `Unpin` is not implemented for `SelfRefFuture`
    let pinned = Pin::new(&mut *future);
}
*/

5. Future Completion Tests

#[test]
fn test_future_completes_correctly() {
    let future = SelfRefFuture::new("important data".into());
    let result = block_on(future);
    assert!(result.contains("important data"));
}

---

Common Pitfalls and Debugging

Pitfall 1: Forgetting PhantomPinned

Problem: Your type is Unpin, Pin provides no safety!

// WRONG
struct MySelfRef {
    data: String,
    ptr: *const String,
    // No PhantomPinned!
}

// This compiles but is DANGEROUS:
let mut s = MySelfRef::new();
let pinned = Pin::new(&mut s);  // Works because Unpin!
let unpinned = Pin::into_inner(pinned);  // Can unpin!
let moved = unpinned;  // Move breaks the struct!

Solution: Always include PhantomPinned:

_pin: PhantomPinned,

Pitfall 2: Moving Fields in Drop

Problem: Drop gives &mut self, not Pin<&mut Self>.

// WRONG
impl Drop for MySelfRef {
    fn drop(&mut self) {
        let stolen = std::mem::take(&mut self.data);  // MOVED!
        // Now ptr is dangling!
    }
}

Solution: Never move fields in Drop:

impl Drop for MySelfRef {
    fn drop(&mut self) {
        // Only cleanup, no moving!
        // Fields drop automatically in place
    }
}

Pitfall 3: Exposing &mut to Pinned Fields

Problem: Returning &mut T allows callers to std::mem::swap.

// WRONG
fn get_data_mut(self: Pin<&mut Self>) -> &mut String {
    unsafe { &mut self.get_unchecked_mut().data }  // Caller can swap!
}

// Caller does:
std::mem::swap(future.get_data_mut(), &mut other_string);  // Moved!

Solution: Return Pin<&mut T> for pinned fields:

fn get_data(self: Pin<&mut Self>) -> Pin<&mut String> {
    unsafe { self.map_unchecked_mut(|s| &mut s.data) }
}

Pitfall 4: Initializing Pointer Before Pinning

Problem: If you set the pointer before the struct is pinned, a move can happen.

// WRONG
fn new(data: String) -> Self {
    let mut s = MySelfRef {
        data,
        ptr: std::ptr::null(),
        _pin: PhantomPinned,
    };
    s.ptr = &s.data;  // Set pointer
    s  // RETURNED! Caller can move before pinning!
}

Solution: Only set pointer after pinning:

fn new(data: String) -> Pin<Box<Self>> {
    let s = MySelfRef {
        data,
        ptr: std::ptr::null(),  // Null initially
        _pin: PhantomPinned,
    };
    let mut pinned = Box::pin(s);

    // NOW set the pointer, we're pinned
    unsafe {
        let this = pinned.as_mut().get_unchecked_mut();
        this.ptr = &this.data;
    }

    pinned
}

Pitfall 5: Using get_unchecked_mut Incorrectly

Problem: get_unchecked_mut returns &mut Self - you can move things!

// WRONG
fn do_something(self: Pin<&mut Self>) {
    let this = unsafe { self.get_unchecked_mut() };
    let moved = std::mem::replace(&mut this.data, String::new());  // BAD!
}

Solution: Only use get_unchecked_mut for projection, never to move:

fn project(self: Pin<&mut Self>) -> Projection<'_> {
    unsafe {
        let this = self.get_unchecked_mut();
        Projection {
            // Only create references, never move!
            data: Pin::new_unchecked(&mut this.data),
            state: &mut this.state,
        }
    }
}

Pitfall 6: Struct Destructuring

Problem: Pattern matching can implicitly move fields.

// WRONG
fn consume(self: Pin<&mut Self>) {
    let MySelfRef { data, ptr, _pin } = unsafe { self.get_unchecked_mut() };
    //              ^^^^ MOVED!
}

Solution: Use ref patterns or field access:

fn access(self: Pin<&mut Self>) {
    let this = unsafe { self.get_unchecked_mut() };
    let data_ref = &mut this.data;  // Borrow, don't move
}

---

Extensions and Challenges

Extension 1: Implement a Self-Referential Linked List

Create a linked list where each node contains a pointer to its own data:

struct Node {
    data: String,
    self_ptr: *const String,
    next: Option<Pin<Box<Node>>>,
    _pin: PhantomPinned,
}

Challenges:

• How do you insert at the end?
• How do you traverse without moving nodes?
• Can you implement an iterator?

Extension 2: Build a Pin-Compatible Intrusive List

Implement an intrusive linked list where nodes store the links:

struct IntrusiveNode {
    links: Links,
    data: MyData,
    _pin: PhantomPinned,
}

struct Links {
    next: *mut IntrusiveNode,
    prev: *mut IntrusiveNode,
}

This is how Tokio’s internal linked lists work!

Extension 3: Create Your Own pin_project! Macro

Write a procedural macro that generates projection methods automatically:

#[my_pin_project]
struct MyFuture {
    #[pin]  // Generate Pin<&mut T> projection
    inner: InnerFuture,

    // Generate &mut T projection
    state: State,
}

This teaches:

• Proc macro development
• AST analysis
• Code generation

Extension 4: Implement Generator/Coroutine

Create a generator that yields values across suspensions:

struct Generator<Y, R> {
    state: State,
    yielded: Option<Y>,
    resume_arg: Option<R>,
    _pin: PhantomPinned,
}

impl Generator<i32, ()> {
    fn resume(self: Pin<&mut Self>, arg: ()) -> GeneratorState<i32, ()> {
        // Implement state machine
    }
}

---

Real-World Connections

How Tokio Uses Pin

Tokio’s task system is built on these exact concepts:

1. Tasks are pinned futures: JoinHandle wraps a Pin<Box<dyn Future>>
2. Wakers contain self-references: The waker points back to the task
3. The scheduler never moves tasks: Once spawned, a task stays at its address

// Simplified Tokio task structure
struct Task {
    future: Pin<Box<dyn Future<Output = ()>>>,
    waker: RawWaker,  // Contains pointer to self!
    next: *mut Task,   // Intrusive linked list
}

How async-std Uses Pin

async-std uses similar patterns:

1. LocalSet pins tasks to the current thread
2. TimerWheel uses intrusive lists of pinned timers
3. Channel implementations use pinned waiter lists

How the Compiler Uses Pin

When you write async fn, the compiler generates:

async fn example(data: String) -> usize {
    some_io(&data).await;
    data.len()
}

// Becomes something like:
enum ExampleFuture {
    State0 { data: String },
    State1 { data: String, io_future: SomeIoFuture },
    // Note: State1 might contain &data if io_future borrows it!
}

The generated future is !Unpin because it may contain self-references.

The pin-project Crate

Your project recreates what pin-project does:

use pin_project::pin_project;

#[pin_project]
struct MyFuture {
    #[pin]
    inner: InnerFuture,
    state: State,
}

// Generates projection methods automatically!

After this project, you’ll understand exactly what pin-project generates.

---

The Core Question You’re Answering

“Why can’t I just use a regular reference inside my own struct?”

Before you write any code, sit with this question. In Rust, structs are moveable by default. If a struct contains a reference to its own field, and that struct is moved (e.g., returned from a function), the pointer inside it now points to the old memory location, creating a dangling pointer. Pin is the mechanism that forbids this move.

---

Concepts You Must Understand First

Stop and research these before coding:

1. Unpin vs. !Unpin
   • What makes a type “move-safe” (Unpin)?
   • Why do most types implement Unpin automatically?
   • Book Reference: “Rust for Rustaceans” Ch. 8 - Jon Gjengset
2. Pointer Aliasing & Dereferencing
   • Why does Rust forbid multiple mutable references to the same location?
   • How does Pin interact with &mut access?
   • Book Reference: “The Rust Programming Language” Ch. 19
3. Self-Referential Structs
   • Why are they inherently dangerous in a language with move semantics?
   • Book Reference: “Programming Rust” Ch. 21 (Context of FFI/Pinning)

---

Questions to Guide Your Design

1. Safety Invariants
   • Why is get_unchecked_mut marked as unsafe?
   • What happens if you implement Drop for a pinned type and move a field?
2. Structural Projection
   • How do you convert Pin<&mut MyStruct> to Pin<&mut PinnedField>?
   • When is it safe to allow a &mut UnpinnedField (non-pinned) access?

---

Thinking Exercise

The Moving Target

Consider this snippet:

struct SelfRef {
    value: String,
    ptr_to_value: *const String,
}

Questions:

• If I put SelfRef in a Vec and the Vec reallocates, what happens to ptr_to_value?
• How does Pin prevent Vec from moving it? (Hint: It doesn’t, it prevents you from putting it in a Vec in a way that allows movement).

---

The Interview Questions They’ll Ask

1. “What is the difference between Pin<Box<T>> and Pin<&mut T>?”
2. “Why does a Future need to be pinned before it can be polled?”
3. “Can you explain ‘structural pinning’ vs ‘non-structural pinning’?”
4. “Why is PhantomPinned a zero-sized type?”
5. “What are the safety requirements for implementing Drop on a !Unpin type?”

---

Hints in Layers

Hint 1: The Marker Start by adding std::marker::PhantomPinned to your struct. This tells the compiler your type is !Unpin.

Hint 2: Safe vs Unsafe Realize that to get a reference to the fields of a pinned struct, you must use unsafe code or a crate like pin-project. Try writing a method fn project(self: Pin<&mut Self>) -> Projection.

Hint 3: The Projection Struct The Projection struct should hold Pin<&mut Field> for pinned fields and &mut Field for unpinned fields.

Hint 4: Verification Use std::ptr::addr_of! to verify addresses without triggering moves or creating invalid references.

---

Books That Will Help

Topic	Book	Chapter
Pin Internals	“Rust for Rustaceans” by Jon Gjengset	Ch. 8: Asynchronous Programming
Unsafe Safety	“The Rust Programming Language”	Ch. 19: Advanced Features
Memory Layout	“Programming Rust” by Blandy & Orendorff	Ch. 21: Unsafe Code
Computer Memory	“Computer Systems: A Programmer’s Perspective”	Ch. 3.9: Heterogeneous Data Structures
Async Deep Dive	“Asynchronous Programming in Rust”	Full Book

---

Real World Outcome

You will have a working self-referential struct that can be safely polled as a Future. You will verify its address stability by printing the memory address of the struct before and after an operation that would normally trigger a move. This project demonstrates complete mastery of Rust’s pinning guarantees.

Example Build & Run:

$ cargo new --lib manual-pin-projector
     Created library `manual-pin-projector` package

$ cd manual-pin-projector

$ cargo add futures
    Updating crates.io index
      Adding futures v0.3.30 to dependencies.features
             futures
             -- features: async-await, std

$ cargo build
   Compiling proc-macro2 v1.0.78
   Compiling unicode-ident v1.0.12
   Compiling syn v2.0.48
   Compiling futures-core v0.3.30
   Compiling futures-task v0.3.30
   Compiling futures-util v0.3.30
   Compiling futures v0.3.30
   Compiling manual-pin-projector v0.1.0 (/Users/you/manual-pin-projector)
    Finished dev [unoptimized + debuginfo] target(s) in 8.23s

$ cargo run --example self_ref_future
   Compiling manual-pin-projector v0.1.0 (/Users/you/manual-pin-projector)
    Finished dev [unoptimized + debuginfo] target(s) in 1.05s
     Running `target/debug/examples/self_ref_future`

=== Manual Pin Projector Demo ===

[Step 1] Creating self-referential struct on stack...
  struct SelfRefFuture {
    data: String = "Hello, Pinning!"
    ptr_to_data: *const String = 0x7ffee8b3c5a0 (points to own 'data' field)
    _pin: PhantomPinned
  }

[Step 2] Verifying self-reference integrity...
  Address of 'data' field:     0x7ffee8b3c5a0
  Pointer field points to:     0x7ffee8b3c5a0
  [OK] Self-reference is VALID (pointer matches actual address)

[Step 3] Moving struct to heap with Box::pin...
  Before pin: Stack address = 0x7ffee8b3c5a0
  After pin:  Heap address  = 0x600001f04020
  Pinned pointer field now:  0x600001f04020
  [OK] Pointer updated correctly during heap move

[Step 4] Attempting unsafe move (this should fail in safe code)...
  // In safe Rust, this line would not compile:
  // let moved = pinned_future;
  // ERROR: cannot move out of `pinned_future` because it is behind a Pin

[Step 5] Polling the pinned future...
  Poll attempt #1: Poll::Pending
    Waker registered at: 0x600001f04088
    Future state: Waiting

  Poll attempt #2: Poll::Pending
    Waker address stable: 0x600001f04088 (unchanged)
    Future state: Waiting

  Poll attempt #3: Poll::Ready("Data processed successfully!")
    [OK] Future completed without memory corruption

[Step 6] Address stability verification...
  Initial heap address:    0x600001f04020
  Address after polling:   0x600001f04020
  [OK] NO MOVEMENT OCCURRED (Pin guarantee upheld)

[Step 7] Manual projection demonstration...
  Using unsafe projection to access fields:
    Projecting to 'data' field: Pin<&mut String>
    Projecting to 'ptr_to_data': *const String (raw pointer, non-structural)

  Modifying 'data' through projection...
    Old value: "Hello, Pinning!"
    New value: "Modified through Pin projection!"
    Pointer still valid: 0x600001f04020
    [OK] Structural pinning preserved invariants

[Step 8] Comparison with Unpin types...
  Creating normal (Unpin) struct...
    Address before move: 0x7ffee8b3c7d0
    Address after move:  0x7ffee8b3c8a0
    [OK] Unpin types can move freely (80 bytes moved)

[Summary]
[OK] Self-referential struct created successfully
[OK] Pin<Box<T>> prevented unsafe movement
[OK] Manual projection worked without UB
[OK] Future polled to completion with stable addresses
[OK] Demonstrated difference between Pin and Unpin

Memory layout visualization:
+-------------------------------------+
|  Heap Allocation (0x600001f04020)   |
+-------------------------------------+
|  +0x00: data (String)               | <--+
|         - ptr: 0x600001e08000       |    |
|         - len: 16                   |    |
|         - cap: 16                   |    |
|  +0x18: ptr_to_data                 | ---+ (self-reference)
|         - *const String: 0x600001f04020
|  +0x20: _pin (PhantomPinned)        |
|         - zero-sized marker         |
+-------------------------------------+

$ cargo test
   Compiling manual-pin-projector v0.1.0 (/Users/you/manual-pin-projector)
    Finished test [unoptimized + debuginfo] target(s) in 0.89s
     Running unittests src/lib.rs (target/debug/deps/manual_pin_projector-a1b2c3d4e5f6)

running 5 tests
test tests::test_pin_guarantees ... ok
test tests::test_self_reference_validity ... ok
test tests::test_projection_safety ... ok
test tests::test_address_stability ... ok
test tests::test_future_completion ... ok

test result: ok. 5 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.02s

$ cargo doc --open
 Documenting manual-pin-projector v0.1.0 (/Users/you/manual-pin-projector)
    Finished dev [unoptimized + debuginfo] target(s) in 2.15s
     Opening /Users/you/manual-pin-projector/target/doc/manual_pin_projector/index.html

---

Quick Reference: Pin API Cheat Sheet

+------------------------------------------------------------------+
|                     PIN API QUICK REFERENCE                       |
+------------------------------------------------------------------+

CREATING PINS:
  Pin::new(&mut t)         -- Only works if T: Unpin
  Box::pin(t)              -- Always works, heap allocates
  pin!(t)                  -- Stack pins (macro from std)
  Pin::new_unchecked(p)    -- Unsafe, for !Unpin types

ACCESSING PINNED DATA:
  pin.as_ref()             -- Pin<&T>
  pin.as_mut()             -- Pin<&mut T>
  pin.get_ref()            -- &T (safe, gives shared ref)
  pin.get_mut()            -- &mut T (only if T: Unpin!)
  pin.get_unchecked_mut()  -- &mut T (unsafe, any T)
  pin.into_inner()         -- T (only if T: Unpin!)
  pin.into_inner_unchecked() -- T (unsafe, consumes)

PROJECTING TO FIELDS:
  // For Unpin fields:
  pin.get_unchecked_mut().field  -- Get &mut Field

  // For !Unpin fields:
  pin.map_unchecked_mut(|x| &mut x.field)  -- Get Pin<&mut Field>

SAFETY REQUIREMENTS FOR !UNPIN TYPES:
  1. Never move the value after pinning
  2. Never expose &mut T for pinned fields
  3. Never move fields in Drop
  4. Initialize self-references AFTER pinning

COMMON PATTERNS:
  // Constructor for self-referential types
  fn new() -> Pin<Box<Self>> {
      let mut boxed = Box::pin(Self { ptr: null(), ... });
      unsafe {
          let this = boxed.as_mut().get_unchecked_mut();
          this.ptr = &this.data;
      }
      boxed
  }

  // Projection method
  fn project(self: Pin<&mut Self>) -> Projection<'_> {
      unsafe {
          let this = self.get_unchecked_mut();
          Projection {
              pinned_field: Pin::new_unchecked(&mut this.pinned),
              normal_field: &mut this.normal,
          }
      }
  }
+------------------------------------------------------------------+

---

Conclusion

Pin is one of Rust’s most misunderstood features, but it’s essential for safe async programming. By building a manual pin projector, you’ve learned:

1. Why self-referential structs are dangerous - Moving breaks internal pointers
2. How Pin provides safety - Type-system contract preventing moves
3. The role of Unpin - Auto trait for “moveable” types
4. Structural vs non-structural pinning - When projections preserve guarantees
5. The unsafe requirements - What you must uphold manually

This knowledge will serve you every time you debug a confusing async error, implement a custom Future, or need to understand how production runtimes like Tokio work internally.

Next Steps: Try Project 2 (Box-less Async Trait) to see how Pin interacts with GATs for zero-allocation async.