← Back to all projects

INTERNAL DEVELOPER PLATFORM ENGINEERING DEEP DIVE

In the early days of DevOps, the mantra was you build it, you run it. But as cloud-native ecosystems exploded (Kubernetes, microservices, serverless, multi-cloud), the cognitive load on individual developers became unbearable. A single dev now needs to understand Docker, K8s, Terraform, IAM, VPCs, CI/CD, and monitoring just to deploy a Hello World.

Learn Platform Engineering: From Zero to IDP Master

Goal: Deeply understand the shift from “DevOps as a job” to “Platform Engineering as a Product.” You will learn how to build an Internal Developer Platform (IDP) that reduces cognitive load, creates “Golden Paths,” and enables true developer self-service through infrastructure abstraction and automation.

Why Platform Engineering Matters

In the early days of DevOps, the mantra was “you build it, you run it.” But as cloud-native ecosystems exploded (Kubernetes, microservices, serverless, multi-cloud), the “cognitive load” on individual developers became unbearable. A single dev now needs to understand Docker, K8s, Terraform, IAM, VPCs, CI/CD, and monitoring just to deploy a “Hello World.”

Platform Engineering is the industrial revolution of software delivery. It’s about building a Product for an internal audience (the developers).

  • Historical Context: Evolved from SRE (Google) and “Product-as-a-Platform” (Netflix) to solve the “DevOps Silo” problem.
  • Real-World Impact: Companies with high platform maturity see 20x higher deployment frequency and 50% lower change failure rates.
  • Why it remains relevant: Infrastructure is getting more complex, not simpler. Abstraction is the only way to scale.
  • What understanding this unlocks: The ability to design systems that allow 1,000 developers to work as efficiently as 10.

Core Concept Analysis

The Developer’s Dilemma vs. The Platform’s Promise

Before Platform Engineering (High Cognitive Load):

Developer -> [Docker] -> [K8s] -> [Terraform] -> [IAM] -> [VPC] -> [DB] -> [Monitoring] -> PROD
(Developer must master 10+ tools just to ship code)

After Platform Engineering (The Golden Path):

Developer -> [ IDP (Self-Service) ] -> [ Magic Happens (Platform) ] -> PROD
                 ^
                 |
          "I need a Node.js app 
           with a Postgres DB"

The IDP Architecture

An Internal Developer Platform isn’t just one tool; it’s a layer that sits between the Developer and the Infrastructure.

+-------------------------------------------------------+
|                DEVELOPER INTERFACE                    |
|  (CLI / Portal / API / GitOps / Backstage)            |
+--------------------------+----------------------------+
                           |
+--------------------------v----------------------------+
|                PLATFORM ORCHESTRATOR                  |
|  (The "Brain": Processes requests, applies policy)     |
+--------------------------+----------------------------+
                           |
+--------------------------v----------------------------+
|                INFRASTRUCTURE ADAPTERS                |
|  (Terraform / Crossplane / Pulumi / Cloud SDKs)       |
+--------------------------+----------------------------+
                           |
+--------------------------v----------------------------+
|             THE INFRASTRUCTURE (AWS/GCP/K8s)          |
+-------------------------------------------------------+

Fundamental Building Blocks

  1. Self-Service: Developers shouldn’t wait for a ticket. They should be able to provision resources via API/CLI/UI instantly.
  2. Infrastructure as Code (IaC): Everything the platform does must be versioned, reproducible, and automated.
  3. Golden Paths (not Golden Cages): Opinionated, supported ways to do things (e.g., “The Standard Java App Template”) while still allowing experts to “eject” if needed.
  4. Environment Management: Handling the lifecycle of Dev, Stage, and Prod (and ephemeral “Preview” environments).
  5. Role-Based Access Control (RBAC): Ensuring the right people have the right permissions without manual intervention.

Concept Summary Table

Concept Cluster What You Need to Internalize
Cognitive Load The primary enemy. Every field a dev doesn’t need to fill out is a win.
Self-Service Success is when the platform team is NOT a bottleneck in the daily flow.
Abstraction Hide the “how” (YAML, CIDRs, IAM JSON) and expose the “what” (App Name, Size, DB Type).
Golden Paths Standardized templates that include security, logging, and monitoring by default.
State Management How the platform tracks what exists vs. what was requested (The “Truth”).

Deep Dive Reading by Concept

Platform Strategy & Organization

Concept Book & Chapter
Team Topologies Team Topologies by Matthew Skelton — Ch. 5: “Evolution of Team Patterns”
Developer Experience The Phoenix Project by Gene Kim — The concept of “The Four Types of Work”

Infrastructure & Automation

Concept Book & Chapter
IaC Principles Infrastructure as Code by Kief Morris — Ch. 2: “Principles of Cloud Age Infrastructure”
Self-Service Infra Infrastructure as Code by Kief Morris — Ch. 12: “Governance and Policy”
Site Reliability Site Reliability Engineering (Google) — Ch. 1: “Introduction” & Ch. 18: “Software Engineering in SRE”

Essential Reading Order

  1. Foundation (Week 1):
    • Team Topologies (Understanding why we need a platform team)
    • Infrastructure as Code Ch. 1-4 (The mechanics of automation)
  2. Advanced (Week 2):
    • Accelerate (The metrics that prove platform engineering works)
    • Designing Data-Intensive Applications Ch. 1 (Thinking about system reliability)

Project 1: The “plat” CLI (The First Abstraction)

  • File: INTERNAL_DEVELOPER_PLATFORM_ENGINEERING_DEEP_DIVE.md
  • Main Programming Language: Go
  • Alternative Programming Languages: Python (Typer), Rust (Clap)
  • Coolness Level: Level 2: Practical but Forgettable
  • Business Potential: 1. The “Resume Gold”
  • Difficulty: Level 1: Beginner
  • Knowledge Area: CLI Tools / Abstraction
  • Software or Tool: Cobra (Go), AWS SDK or GCP SDK
  • Main Book: “The Pragmatic Programmer” by Hunt & Thomas

What you’ll build: A CLI tool that abstracts cloud commands into simple developer intents. plat create bucket --name my-app-assets.

Why it teaches Platform Engineering: This project introduces the concept of abstraction. You aren’t just running scripts; you are defining a higher-level language for your developers.

Real World Outcome: A binary that devs can install to provision valid resources without needing to know the cloud provider’s syntax.

Project 2: The Infrastructure State Machine (IaC Wrapper)

  • File: INTERNAL_DEVELOPER_PLATFORM_ENGINEERING_DEEP_DIVE.md
  • Main Programming Language: Python
  • Alternative Programming Languages: Go, TypeScript
  • Coolness Level: Level 3: Genuinely Clever
  • Business Potential: 3. The “Service & Support” Model
  • Difficulty: Level 2: Intermediate
  • Knowledge Area: IaC / Orchestration
  • Software or Tool: Terraform (CLI), Python Subprocess
  • Main Book: “Infrastructure as Code” by Kief Morris

What you’ll build: A “wrapper” service that takes a JSON request and executes the corresponding Terraform modules to build it. It must handle the “Plan” and “Apply” lifecycle and store state centrally.

Project 3: DBaaS (Database as a Service) Automator

  • File: INTERNAL_DEVELOPER_PLATFORM_ENGINEERING_DEEP_DIVE.md
  • Main Programming Language: Go
  • Alternative Programming Languages: Python, Rust
  • Coolness Level: Level 4: Hardcore Tech Flex
  • Business Potential: 4. The “Open Core” Infrastructure
  • Difficulty: Level 3: Advanced
  • Knowledge Area: Databases / API Design
  • Software or Tool: Docker, Postgres, SQL
  • Main Book: “Designing Data-Intensive Applications” by Martin Kleppmann

What you’ll build: A tool that manages database instances. A dev says “I need a Postgres DB named ‘orders’”, and your tool: 1. Provisions the instance, 2. Creates users, 3. Configures schema, 4. Returns connection string.

Project 4: Ephemeral Environments (Namespace as a Service)

  • File: INTERNAL_DEVELOPER_PLATFORM_ENGINEERING_DEEP_DIVE.md
  • Main Programming Language: Go
  • Alternative Programming Languages: Python
  • Coolness Level: Level 4: Hardcore Tech Flex
  • Difficulty: Level 3: Advanced
  • Knowledge Area: Kubernetes / Orchestration
  • Software or Tool: Kubernetes, Helm, client-go
  • Main Book: “Kubernetes: Up and Running” by Burns, Beda, and Hightower

What you’ll build: A system that creates “Preview” environments. When a developer creates a Pull Request, your system creates a temporary Kubernetes Namespace, deploys their code via Helm, and provides a unique URL.

Project 5: The Golden Path Template Engine

  • File: INTERNAL_DEVELOPER_PLATFORM_ENGINEERING_DEEP_DIVE.md
  • Main Programming Language: TypeScript
  • Alternative Programming Languages: Go (text/template), Python (Jinja2)
  • Coolness Level: Level 3: Genuinely Clever
  • Difficulty: Level 2: Intermediate
  • Knowledge Area: Templating / Scaffolding
  • Software or Tool: Cookiecutter, Yeoman
  • Main Book: “Team Topologies” by Skelton & Pais

What you’ll build: A generator that creates a “Ready-to-Deploy” service. It creates the code, Dockerfile, Helm Chart, and CI/CD Pipeline definition.

Project 6: Governance Guardrails (Policy as Code)

  • File: INTERNAL_DEVELOPER_PLATFORM_ENGINEERING_DEEP_DIVE.md
  • Main Programming Language: Rego (Open Policy Agent)
  • Alternative Programming Languages: Go
  • Coolness Level: Level 4: Hardcore Tech Flex
  • Difficulty: Level 4: Expert
  • Knowledge Area: Compliance / Security
  • Software or Tool: Open Policy Agent (OPA)
  • Main Book: “Infrastructure as Code” by Kief Morris

What you’ll build: A validation engine that inspects incoming Kubernetes manifests or Terraform plans and rejects them if they violate “Guardrails” (e.g., “No LoadBalancer should be public”).

Project 7: The Developer Portal (UI/API Layer)

  • File: INTERNAL_DEVELOPER_PLATFORM_ENGINEERING_DEEP_DIVE.md
  • Main Programming Language: TypeScript (React/Next.js)
  • Alternative Programming Languages: Go (Backend) + Vue.js
  • Coolness Level: Level 3: Genuinely Clever
  • Business Potential: 5. The “Industry Disruptor”
  • Difficulty: Level 3: Advanced
  • Knowledge Area: Web Development / API Integration
  • Software or Tool: Backstage (Spotify)
  • Main Book: “Software Architecture in Practice” by Bass et al.

What you’ll build: A centralized dashboard where developers can see all their services, health status, logs, and a “Create New Service” button.

Project 8: Secret Management as a Service

  • File: INTERNAL_DEVELOPER_PLATFORM_ENGINEERING_DEEP_DIVE.md
  • Main Programming Language: Go
  • Alternative Programming Languages: Python
  • Coolness Level: Level 4: Hardcore Tech Flex
  • Difficulty: Level 4: Expert
  • Knowledge Area: Security / Cryptography
  • Software or Tool: HashiCorp Vault, AWS Secrets Manager
  • Main Book: “Serious Cryptography” by Jean-Philippe Aumasson

What you’ll build: A system that automatically injects secrets (DB passwords) into a running application via sidecars or volume mounts.

Project 10: The Kubernetes Operator (Custom Resources)

  • File: INTERNAL_DEVELOPER_PLATFORM_ENGINEERING_DEEP_DIVE.md
  • Main Programming Language: Go
  • Alternative Programming Languages: Rust (Kube-rs)
  • Coolness Level: Level 5: Pure Magic (Super Cool)
  • Difficulty: Level 5: Master
  • Knowledge Area: Kubernetes Internals
  • Software or Tool: Kubebuilder, Operator SDK
  • Main Book: “Programming Kubernetes” by Hausenblas & Schimanski

What you’ll build: A custom Kubernetes resource called InternalApp. When a dev creates this resource, your Operator automatically creates the Deployment, Service, HPA, Ingress, and Database.

Project 11: The Cloud Janitor (Automated Garbage Collection)

  • File: INTERNAL_DEVELOPER_PLATFORM_ENGINEERING_DEEP_DIVE.md
  • Main Programming Language: Python
  • Alternative Programming Languages: Go
  • Coolness Level: Level 2: Practical but Forgettable
  • Difficulty: Level 2: Intermediate
  • Knowledge Area: Automation / Cost Control
  • Software or Tool: AWS SDK (Boto3)

What you’ll build: A background worker that scans your cloud account for unattached resources (EBS, EIPs) and old Dev instances, notifying the owner before deleting them.

Final Overall Project: The “Self-Service Metropolis”

What you’ll build: A complete, end-to-end Internal Developer Platform that integrates everything you’ve learned.

  1. Frontend: A custom React portal (Project 7) where a developer can request a new “Service.”
  2. Backend: A Go-based orchestrator (Project 2/3) that receives the request.
  3. Infrastructure: It creates a GitHub Repo (via API), scaffolds the code (Project 5), and provisions a Cloud Database (Project 3).
  4. Environment: It creates a dedicated K8s Namespace and Ingress (Project 4).
  5. Security: It injects a unique Vault token (Project 8) and applies OPA policies (Project 6).
  6. Delivery: It sets up a CI/CD pipeline that builds and deploys the app on every push.

Why it teaches Platform Engineering: This is the culmination of all concepts. You aren’t just building tools; you are building a Workflow. You’ll understand how all these pieces fit together to create a seamless experience for a developer.

Summary

This learning path covers Platform Engineering through 11 hands-on projects. Here’s the complete list:

# Project Name Main Language Difficulty Time Estimate
1 The “plat” CLI Go Level 1 Weekend
2 IaC State Machine Python Level 2 1 week
3 DBaaS Automator Go Level 3 2 weeks
4 Ephemeral Environments Go Level 3 2 weeks
5 Golden Path Templates TypeScript Level 2 1 week
6 Governance Guardrails Rego/Go Level 4 2 weeks
7 The Developer Portal TypeScript Level 3 2 weeks
8 Secrets-as-a-Service Go Level 4 2 weeks
10 K8s Custom Operator Go Level 5 1 month
11 The Cloud Janitor Python Level 2 1 week

For beginners: Start with projects #1, #5, and #11. For intermediate: Focus on #2, #3, and #7. For advanced: Master #4, #6, #8, and #10.

Expected Outcomes

After completing these projects, you will:

  • Understand how to treat infrastructure as a product.
  • Be able to build complex abstractions that reduce developer cognitive load.
  • Master Kubernetes at a level where you can extend its core functionality.
  • Know how to integrate security and cost control into the developer workflow.

You’ll have built 11 working projects that demonstrate deep understanding of Internal Developer Platforms from first principles.