# Ship Fast, Hold No Keys: A Builder’s Guide to Least Privilege, JIT, and ZSP **Who this is for:** security engineers, engineering managers, and developers **Promise:** By the end, you’ll know exactly what each term means, how they differ, and how to apply them—in the right order—without slowing the team. ## TL;DR (read this, then build) * **PoLP** = give every identity only the **minimum** permissions needed to do its job. * **JIT** = grant **temporary** elevated access **only when needed**, with approvals and auto-revoke. * **ZSP** = run with **no always-on admin roles**; everything privileged happens **on demand** via JIT. > Think of a seatbelt (PoLP), an airbag that deploys only in a crash (JIT), and a car designed so no one can start it without a key that expires (ZSP). You don’t choose one; they **layer**. ## Why you should care (even if you’re busy) * Most real incidents get worse because someone had **more power than they needed for longer than they needed**. * Attackers love **standing** (always-on) admin paths, stale tokens, and forgotten “temporary” roles. * Fixing this **doesn’t have to slow you down**. If JIT is faster than asking a teammate for help, delivery **speeds up** and risk drops. ## Why These Three Concepts Matter (Business Lens) * Most breaches aren’t “movie hacks”; they’re **credential reuse**, **phishing**, **tokens leaked in CI**, or **excessive permissions** misused. * **Standing admin** turns any compromise into a **catastrophe**. * You need a model that reduces blast radius **every day** (PoLP), still lets people **do the rare powerful thing** (JIT), and ensures **permanent admin simply doesn’t exist** (ZSP). * The end result: **less risk**, **clean audit trails**, and **no slowdown** because JIT is built to be fast. ## Clean Definitions * **PoLP (Principle of Least Privilege)** Give every human, service account, and workload identity only the **minimum** permissions needed for **routine** tasks. Think **scope** and **granularity**: read vs write vs admin; per environment; per resource. * **JIT (Just-in-Time Access)** Provide **temporary**, **approved**, **time-boxed** elevation when the baseline isn’t enough. Always enforce **MFA**, **expiry**, **auto-revoke**, and **logging**. * **ZSP (Zero Standing Privileges)** Keep **no one** in privileged roles by default. Admin is not a permanent state; it’s a **short-lived event** via JIT. Maintain a **sealed break-glass** for emergencies with the shortest possible TTL and mandatory review. > **Not the same as Zero Trust:** Zero Trust is an architectural philosophy (never trust, always verify). **ZSP is a privileged access outcome** you can objectively measure (“Who has standing admin right now?” → “No one.”). ## The Mental Model That Never Fails **Access = Who × What × When × Why** * **Who**: human user, service account, workload identity, GitHub App, CI runner * **What**: the **exact** permissions on a **specific** resource (not “admin”), ideally split into read / change / admin * **When**: baseline (PoLP, no expiry) vs elevated (JIT, **expires**) * **Why**: ticket/incident/PR/CRQ that explains **business need** If any dimension is fuzzy, your control will be fuzzy. ## The ideas in plain language ### 1) Least Privilege (PoLP) — your safe default * **What it means:** Give each person and each service the smallest set of permissions they need to do their **normal** job. * **Why it helps:** If an account gets phished or a token leaks, the damage is limited. * **Example:** Developers can read production logs, not change production settings. A microservice can read a single S3 bucket, not all buckets. ### 2) Just-in-Time (JIT) Access — safe exceptions * **What it means:** When someone needs extra power (e.g., to migrate a database), they **ask for it**, get it for **15–60 minutes**, then it **auto-revokes**. * **Why it helps:** It collapses risk from “always” to **minutes**. * **Example:** An engineer gets temporary “DB migrator” for 30 minutes tied to a ticket. When time’s up, the extra permission disappears automatically. ### 3) Zero Standing Privileges (ZSP) — the goal state * **What it means:** There are **no** permanent admins. Admin power appears only through JIT or a break-glass account kept in a vault. * **Why it helps:** Attackers can’t find “always-on” superusers or forever tokens. * **Example:** No one is a permanent GitHub org owner. If someone needs owner power, they elevate via JIT. One emergency owner is kept in a vault; using it pages the on-call and gets reviewed. **Memory model:** * **PoLP** = small everyday keys. * **JIT** = borrow the big key briefly. * **ZSP** = no big keys left lying around. ## Same Task, Three Ways (So You See the Delta) Below, the **identical** tasks under (1) PoLP only, (2) PoLP+JIT, and (3) ZSP. This is the cleanest way to internalize the differences. ### A) Change branch protections on a sensitive repo (GitHub) * **PoLP only:** Maintainers have `maintain`, not `admin`. They’re blocked until an Owner intervenes. If an Owner exists permanently, that’s a risk. * **PoLP + JIT:** Maintainer requests **45m** `repo:admin` with reason **RELEASE-124**; approver OK; **auto-revoke** at 45m. * **ZSP:** No standing Owners. Org changes require **JIT Owner** ≤30m with two approvals. Attackers cannot find a “sleeping Owner.” ### B) Restore a production S3 bucket (Cloud IAM) * **PoLP only:** Devs have `ReadOnly`; someone with standing `Administrator` must act. * **PoLP + JIT:** On-call assumes `Administrator` **for 60m** via SSO; session recorded; auto-revoke. * **ZSP:** `Administrator` group kept **empty**; even platform leads elevate via JIT. ### C) Hotfix a schema in prod (Postgres) * **PoLP only:** Engineers have `SELECT`. You page a standing `SUPERUSER`. * **PoLP + JIT:** DBA gets **30m** `ALTER` on **one schema**; keystrokes recorded; `DROP` blocked without emergency flag + second approver. * **ZSP:** No standing `SUPERUSER`. Break-glass **15–30m** max, mandatory post-use review. ### D) Cordon/Drain nodes (Kubernetes) * **PoLP only:** Engineers `view`; you hunt for someone with standing `cluster-admin`. * **PoLP + JIT:** On-call elevates to `cluster-admin` for **20m** with incident link; logs kept. * **ZSP:** `cluster-admin` group is empty 24×7 unless JIT. ### E) Modify a sensitive GPO (Active Directory) * **PoLP only:** Helpdesk has OU-scoped rights; a standing `Domain Admin` flips the switch. * **PoLP + JIT:** Senior admin elevates to `Domain Admins` **for 30m**; two approvals. * **ZSP:** `Domain Admins` has no humans day-to-day; elevation only. **Key takeaway:** PoLP reduces constant risk; JIT allows necessary exceptions **with control**; ZSP ensures exceptions are **the only** way to be powerful. ## How to Implement—In the Right Order (Without Slowing the Team) ### Phase 1 — PoLP (Days 0–10): Make the baseline safe 1. **Inventory**: Cloud roles, GitHub roles, DB roles, K8s RBAC, AD groups, SaaS admin roles. 2. **De-admin**: Remove dormant users; shrink “God groups.” 3. **Split roles**: For each system define **Read / Change / Admin**; grant the **lowest** necessary to keep work moving. 4. **Protect CI**: Default to `permissions: read-all` in GitHub Actions; request granular writes per job. 5. **Segment environments**: Separate prod vs non-prod roles; never share credentials. **Guardrails:** MFA everywhere, device posture checks for risky apps, disallow local admin on day-to-day laptops where feasible. ### Phase 2 — JIT (Days 11–30): Add a fast, audited exception path 1. **One door only**: a chat command (e.g., Slack) or a web form that captures **who/what/when/why**. 2. **Policy**: * Low-risk roles (read/write in non-prod) can be **auto-approved**. * High-risk roles (prod write/admin) require **one approver** (on-call or EM). * Tier-0 (org/cloud root) requires **two approvers** (Security + EM). 3. **Controls**: enforce **MFA at request time**, **expiry** (15–120 min), **auto-revoke**, and session **recording** for shells, DBs, and K8s. 4. **Telemetry**: push events to SIEM with correlation IDs linking request → approval → elevation → actions. **UX principle:** JIT must be **faster than filing a ticket** or people will route around it. Aim for minutes, not hours. ### Phase 3 — ZSP (Days 31–60): Remove permanent admin 1. **Empty** standing admin groups (`Owners`, `Administrators`, `Domain Admins`, `cluster-admin`). 2. **Break-glass**: maintain one sealed, hardware-key-protected path with **10–30 min TTL** and mandatory post-use review. 3. **Enforce**: require JIT for all admin changes; build alerts for any elevation outside the broker. 4. **Publish KPIs**: * **Standing admins** → trending to **0** * **Median JIT wait** < **10 minutes** * **% privileged sessions recorded** → **100%** **Result:** Admin is **not a property** someone has; it’s a **short event** with receipts. ## “How I Implemented All Three for GitHub” (End-to-End, Reproducible) This section shows the complete arc—PoLP baseline, JIT exceptions, and ZSP target—specifically for **GitHub**. Adapt the patterns, not the brand names. ### Objectives * Eliminate org-wide blast radius. * Keep maintainers **fast** for day-to-day tasks. * Pass audits with **clear**, **correlated** logs. ### Step 1 — PoLP on GitHub (baseline that doesn’t slow teams) * **Org roles** * Almost everyone is **Member**; no one sits as **Owner** for daily work. * **Team-to-repo mapping** * Teams map to products/components. Permissions are **repo-scoped**: `triage` or `write` for most repos, `maintain` for a small set; **no blanket** `admin`. * Sensitive repos (infra, SSO, org policy) live in dedicated teams. Maintainers have `maintain`, not `admin`. * **Branch & rulesets** * Protected branches: required reviews, status checks, code owners for risky paths, signed commits in critical repos. * Rulesets block force-push on default branches and optionally block secrets/large binaries. * **Actions (CI) hardening** * Org default: `permissions: read-all`. * Jobs request the minimum (e.g., `contents: write` **only** in publish jobs; `packages: write` only when needed). * Prefer **GitHub OIDC** to cloud roles for deploy—**no long-lived cloud keys** in secrets. * Allowlist critical third-party actions and pin by commit SHA for sensitive workflows. * **Apps & tokens** * Prefer **GitHub Apps** (repo-scoped) over fine-grained PATs. * If PATs exist, enforce **short TTL** and minimal scopes; alert on PAT creation. **Outcome:** day-to-day is safe and fast; nothing is blocked. Admin is still needed occasionally—enter JIT. ### Step 2 — JIT for GitHub (fast, auditable, time-boxed) * **ChatOps flow** * Engineer runs: ```plaintext /gh-elevate role:repo-admin repo: duration:45m reason:"release policy change" ``` * Bot posts a card in `#priv-access` with **who/what/when/why** and “Approve/Reject.” * On approval, the bot uses the GitHub API to add the user to a **temporary grant** (e.g., a `JIT-Admins-` team) with an **expiry** timestamp. * A scheduled job (or webhook) **auto-revokes** at TTL and posts a summary. * **Org-level changes (high risk)** * Use **JIT Owner** for ≤30 min with **two approvals** (Security + EM). * Enforce **MFA at request time** and device posture if available. * **Audit & telemetry** * Every request → approval → elevation → GitHub event → revocation gets a **correlation ID**. * Logs stream to SIEM; weekly report lists the top elevated actions and their reasons. **Outcome:** maintainers change protections, rotate secrets, or adjust settings **without** permanent Owners. Admin windows are short and provably justified. ### Step 3 — ZSP on GitHub (the steady state) * **Standing Owners: Zero** * GitHub requires at least one Owner; we satisfy this with a **sealed break-glass Owner** account: hardware key, strong passphrase, distinct hardware, not used for daily work. * Rotation requires **two people** (dual control) and is logged. * **Everything privileged goes through JIT** * Repo admin? JIT. * Org policy changes? JIT Owner with 2 approvals. * Secret rotation? JIT. * Emergency? Break-glass for **≤30 min**, loud banner, mandatory post-use review within 24 hours. * **Monitoring & drift control** * Alert if any user is added to `Owners` or to a repo with `admin` outside the JIT bot. * Weekly **drift report**: * Who has standing admin? (should be **none**) * JIT latency (P50, P95) * % of elevated sessions correlated to a ticket/PR (should be **100%**) **Result:** on a random Tuesday, **no human is an Owner**. The only path to power is **JIT**—fast, logged, and temporary. ## Practical Guardrails You Should Not Skip * **MFA + device posture** checks for any elevation (don’t approve from an untrusted laptop). * **Short TTLs** (15–120 min); incident windows can be a bit longer but still expire automatically. * **Two-person approvals** for Tier-0 (org/cloud root, SSO). * **Session recording** for shells, DBs, and K8s where legally permissible; immutable storage. * **Auto-revoke + watchdogs**; assume revocation can fail and verify it did not. * **No long-lived secrets** for humans **or** CI; prefer OIDC/STS short-lived credentials. * **Speed matters**: if JIT takes 30 minutes, engineers will bypass it. Optimize the workflow, not your principles. ## Quick-Start Checklists ### PoLP Checklist (baseline) * Inventory admin roles/groups: Cloud, GitHub, DB, K8s, AD, SaaS * Remove dormant users; enforce MFA org-wide * Split **Read / Change / Admin** for each system; **grant minimum** * Lock CI to `permissions: read-all`; request writes per job * Separate prod vs non-prod identities and roles * Implement repo/environment scoping for robots; no shared credentials ### JIT Checklist (controlled exceptions) * Single elevation path (chat or web) captures **who/what/when/why** * Approval policy by risk tier; on-call approver for speed * Enforce **MFA**, **expiry**, **auto-revoke**, and **session recording** * Stream request/approval/elevation/action/revocation to SIEM with correlation IDs * SLO: P50 JIT < **10m**, P95 < **30m**; report weekly ### ZSP Checklist (target posture) * Empty standing admin groups (Owners, Administrators, Domain Admins, cluster-admin) * Maintain sealed **break-glass**: hardware keys, dual control, **≤30m** TTL * Alerts for any admin assignment outside JIT * Weekly drift report: standing admins, JIT latency, coverage of recorded sessions * Quarterly test of break-glass + revocation watchdogs ## Ideal State (What “Done” Looks Like) * **Humans and robots** operate with **least privilege** by default. * **All** privileged actions are **requested**, **approved** (policy-based where possible), **time-boxed**, and **auto-revoked**. * On any given day, **no one** holds org-level destructive power; **break-glass** is sealed and rarely touched. * **CI/CD** uses **ephemeral**, **scoped** credentials (OIDC/STS), not static keys. * **Metrics** show: standing admins ≈ **0**, JIT is **fast**, and audit coverage is **complete**. * Engineers consider access **boring**—because it “just works,” quickly and safely. ## Borrow, Fix, Expire: The Break-Glass Model > JIT isn’t a ticket queue; it’s the engineered emergency lane. In SRE literature this is called “break-glass.” > > “**A breakglass mechanism provides access to your system in an emergency situation.**” — *Building Secure and Reliable Systems* (Google SRE), Chapter 5 > > In practice, that means: > > * **PoLP:** Engineers work with minimal, routine access. No one keeps standing admin by default. > > * **JIT:** When normal paths fail, the team invokes the break-glass path—a temporary, approved, time-boxed elevation that logs every action and then automatically revokes access. > > * **ZSP:** Because there are no permanent admin accounts, the only way to gain high privilege is through this designed, auditable, short-lived path. > > > With this model, ZSP becomes practical: admin is an event, not a state, and every privileged action leaves a clear, reviewable trail. > > *Source: Building Secure and Reliable Systems — Chapter 5.* ## Frequently Asked (and Frank) Answers **Is PoLP enough?** No. PoLP blocks routine excess but fails the moment someone **needs** extra power. You need JIT for those moments. **Is JIT just ticket hell?** Bad JIT is. Good JIT is **two clicks** in chat, under 10 minutes, with auto-revoke and logs. **Will ZSP slow incidents?** Not if implemented properly. ZSP **speeds** incidents by giving a **standard**, **audited**, **fast** path to power—no hunting for a random admin. **Is ZSP the same as Zero Trust?** No. Zero Trust is a **philosophy**. ZSP is a **measurable outcome**: “no standing admin” as an invariant. ## Closing: The Only Order That Works * **PoLP** makes your **everyday** safe. * **JIT** makes your **exceptions** safe **and fast**. * **ZSP** ensures **exceptions are the only way** to be powerful. Apply them in that order, keep JIT under 10 minutes, and you’ll lower risk **and** keep velocity. If you want, I can turn your GitHub or cloud setup into a one-page PoLP/JIT/ZSP access map and a 30-day rollout—so your team can execute without debate. ### **Let’s Talk: Practical Security for Shipping Teams** If you’ve got a challenge, a question, or just want a different perspective, let’s connect. 📅 **Book a 30-min slot:** [**cal.com/rahulj/30min**](https://cal.com/rahulj/30min) Happy to share what I know, swap ideas, or just listen.