Vol. I · № 1
Study
Back to the index

Inside Git's Credential Protocol

What a `ghs_` token actually is, how the helper protocol works, and the small places both of them leak.

GitHub’s installation token is alphanumeric, prefixed with ghs_, and lives for exactly one hour. It is minted by a small signed token whose issuer field must equal a specific numeric integer. The protocol that gets it onto the wire is short, well-documented, and quietly surprising in a few places once you start building on top of it.

When you run git push, the part you watch — the progress bar, the SHA being sent up — is the easy half. The interesting half happens just before it, in a quiet conversation between git, a small helper program on your machine, and the GitHub server. That conversation has rules. The rules are short. The places those short rules combine into something surprising are not.

This is a walk through that conversation.

What an installation token actually is

The token most people meet first on GitHub is the PAT — personal access token, the long string you create in your settings and paste into a CI variable. PATs work and are obvious, but they belong to you, not to the work. A token meant to belong to the work — a CI pipeline, a deploy bot, an automation — is a GitHub App installation token, prefixed with ghs_.

There are four prefixes in the wild, and they’re worth knowing on sight:

PrefixWhat it is
ghp_Personal access token. Belongs to a user.
gho_OAuth user-to-server token. Issued when a user authorizes an app.
ghu_User-to-server token, similar register.
ghs_Installation token. Belongs to a GitHub App on a specific install.

You don’t create a ghs_ token in a UI. You mint one, on demand, from your code. The mint sequence has three moving parts: a small signed token you carry as proof of who you are, a POST to a specific endpoint, and a short-lived response token that’s the one you actually use.

The proof-of-identity token is a JWT — a JSON Web Token, the kind that has three dot-separated parts where the third part is a signature over the first two. GitHub requires it signed with RS256 (RSA + SHA-256, using the private key you generated when you created the App), with three field values it checks precisely:

  • iss (issuer): the numeric App ID. Not the App slug. Not the slug cast to a string. The integer.
  • iat (issued-at): a recent timestamp. Roughly “now”.
  • exp (expiry): no more than iat + 600s. The server caps the JWT lifetime at ten minutes — your code can ask for less, never more.
build RS256 JWT(iss = app_id, exp ≤ iat + 600s) POST /app/installations/{id}/access_tokens verify signature, iss, exp { token: "ghs_…", expires_at: "+1h" } git fetch / push using ghs_ token Authorization: Bearer <JWT> your code github API
Minting an installation token: a JWT goes in, a ghs_ token comes out.

You POST that JWT to /app/installations/{installation_id}/access_tokens as a Bearer header. GitHub responds with a ghs_… token good for one hour. The one-hour lifetime is hard-coded — there is no parameter to make it shorter or longer.

There is one small piece of cleverness available at mint time. The body of the POST optionally accepts repositories and permissions. If you pass them, the token GitHub mints is weaker than the installation’s full grant — it can see only those repos, with only those permissions. This is the right move in any automation that doesn’t need full access. It’s also one of those features nobody uses on the first build of their tooling, then everyone uses on the rewrite a year later when the postmortem demands it.

The credential helper protocol

Embedding tokens in URLs works for one-off clones, but anything long-lived shouldn’t put secrets on disk in .git/config. The supported path is a credential helper — a small program git runs on demand to fetch (or store, or erase) secrets.

The protocol is unusually small. Git invokes the helper with exactly one argument — get, store, or erase — and writes the input data to the helper’s standard input as a sequence of key=value lines. The helper responds by writing key-value lines back to stdout. Git reads them, uses the password, moves on.

HTTPS request, no auth 401 Unauthorized spawn helper, argv: "get" stdout: username + password lines HTTPS retry, Basic auth 200, the data you asked for stdin: protocol/host/path lines git remote credential helper
Git tries the request first. The helper is consulted only after a 401.

Concretely, a helper invocation looks like this:

bashwhat git writes to the helper's stdin, and what the helper writes back
$ ./git-credential-mything get
protocol=https
host=github.com
path=org/repo
<EOF>

# the helper writes to stdout:
username=x-access-token
password=ghs_AAAA…BBBB

A few things in the protocol are worth knowing well enough to stop them surprising you:

  • The helper runs after a 401, not before. Git tries the request anonymous first. Only when the server says “no” does git look for a helper. (Git 2.46 added http.proactiveAuth for the rare sites where you’d rather pre-attach the credential — but the default is reactive, and most code paths still are.)
  • Exit codes are ignored. Git doesn’t care if your helper exits 0 or 17. The only signal git takes from a helper is what it wrote to stdout. A helper that writes nothing is a helper that failed, regardless of how proudly it exited.
  • Multi-line values are rejected. A password line with an embedded newline gets dropped. This sounds pedantic until you realize it’s the protection that stops a crafted repo URL from injecting an HTTP header into your auth. It is a security feature pretending to be a parser quirk.
  • fill in the docs and get on the CLI mean the same thing. Both names appear in the wild; they’re aliases.
  • A leading ! in credential.helper is shell-evaluated. If your config value starts with !, git hands the rest to your shell, so quoting matters. Without the !, git looks for a binary named git-credential-<name> on PATH — the same convention as git subcommands.

Where it leaks: the precedence chain

Multiple helpers can be configured at once — system, global, repo, even multiple within one scope. They’re consulted in declaration order; the first one to return a password wins.

There are two useful overrides and one dangerous one.

The useful pair is the empty-helper reset:

ini~/.gitconfig — drop the system-wide chain, then name our own
[credential]
    helper =
    helper = manager

The bare helper = clears every helper inherited from system or global config above this scope. Useful when something baked into the OS gitconfig (corporate laptop, container base image) is in your way and you can’t or shouldn’t edit it.

The dangerous one is the GITHUB_TOKEN environment variable. It does not come from your config. It comes from your shell. And it beats every configured helper, silently, with no warning printed by git or by the gh CLI. You can have ten helpers set up perfectly and one stale GITHUB_TOKEN lingering in your environment, and git will use the stale one. There is no command that shows you it’s winning. The only way to find out is to print the environment.

I have lost more time to this than any other credential bug I’ve hit. The fix is one line — unset GITHUB_TOKEN — once you know to look.

Where it leaks: token expiry, mid-stream

The one-hour token lifetime is enforced server-side, per request. It does not become friendlier when there’s a long-lived connection. A clone of a large monorepo that takes seventy minutes can — and does — die mid-stream when the token crosses its expiry. The connection is open. The bytes are flowing. The server enforces the lifetime anyway.

Three mitigations matter, in order of importance:

  1. Pre-mint before the boundary. Helpers can emit a password_expiry_utc field alongside the username and password. Git reads it, and if it would expire mid-operation, asks the helper for a fresh credential before making the request. This costs you nothing if your helper supports it, and recovers a whole class of mid-stream failure.
  2. Mint behind a mutex for parallel ops. Submodule clones, git fetch --all, parallel fetches in CI — they invoke the credential helper concurrently. If your helper mints a new token per call, you create N tokens in parallel, each consuming a slice of GitHub’s mint-rate budget. Cache and lock around it. One token at a time.
  3. Don’t reuse one token across long jobs. The token is meant to be cheap. Mint frequently, throw away aggressively.

Where it leaks: gh in containers

The gh CLI ships with a keyring abstraction that does the right thing on a developer’s laptop — Keychain on macOS, libsecret on Linux, wincred on Windows. In a container, none of those exist. gh falls back to a file at ~/.config/gh/hosts.yml, encrypted with a key derived from the machine ID.

This works, and it also surprises people, because the machine ID is host-stable but container-stable in unintuitive ways. A token written in one container layer might be unreadable in another container started from the same image, depending on whether the runtime forwards /etc/machine-id or generates a new one. gh auth git-credential re-reads the keyring (or file) on every invocation — there is no in-process cache — so the moment you find out it’s broken is the next moment git asks for a credential, which is also the moment your pipeline is doing useful work.

Two practical paths:

  • Mount /etc/machine-id from the host into the container, so the derived key is stable.
  • Skip gh’s keyring entirely in CI, and write a tiny helper that mints installation tokens directly from a private key kept in a secret. More work the first time, less work every time after.

What to carry forward

The credential machinery is a small protocol stacked on a smaller protocol stacked on a documented endpoint. Each layer is honest. The surprises live at the boundaries: an environment variable that silently wins, a username that has to be the literal string x-access-token, a token clock that doesn’t care about your in-flight clone.

The lesson — to the extent there’s one — is to spend an hour reading the actual protocol before building anything serious on top of it. The protocol is short. An hour is enough.

The interesting part of git push is the half-second before anything is pushed.

Previous move
17.
The Heap Was Fine. The Container Wasn't.
Next move
18.
What DELETE Doesn't Delete