How to Commit and Push via GitHub REST API When git CLI Fails in VM Environments

If you've ever run Claude Code in a scheduled task or sandbox environment, you've probably seen something like this:

fatal: detected dubious ownership in repository at '/tmp/repos/mysite'
error: could not lock config file .git/config: Permission denied
Another git process seems to be running in this repository

A stale index.lock, a repository owned by nobody, no write access to /tmp — these aren't bugs, they're the reality of VM and container environments. As an indie developer who runs several sites on autopilot, I ran into every one of them while building automated article publishing pipelines for my Claude Code agents.

The solution I landed on: skip git CLI entirely and push directly through the GitHub REST API.

Why GitHub REST API?

The git CLI assumes it can freely access a local repository: read the object store, acquire locks, verify ownership. When any of those assumptions break, you get cryptic errors.

The GitHub REST API works differently. It's just HTTP. The state of your local .git directory is irrelevant — you're talking directly to GitHub's servers.

git CLI → local .git → GitHub  ← breaks when local state is bad
REST API          → GitHub  ← always works, regardless of local state

The Four-Step Commit Flow

Creating a commit via the API requires four sequential calls:

1. Create blobs   → upload file contents to GitHub
2. Create a tree  → arrange blobs into a directory structure
3. Create a commit → attach a message and parent to the tree
4. Update the ref  → advance the branch pointer to the new commit

Each step returns a SHA you pass into the next step. It sounds verbose, but each call is idempotent and easy to retry on failure.

Shell Script Implementation

Here's a working bash + curl example, suitable for use in Claude Code scheduled tasks or any CI environment:

#!/bin/bash
set -e
 
GITHUB_TOKEN="YOUR_GITHUB_TOKEN"
OWNER="your-username"
REPO="your-repo"
BRANCH="main"
COMMIT_MSG="Add: new article (JA+EN)"
 
# ── Step 1: Get current HEAD SHA ──────────────────
HEAD_SHA=$(curl -sf \
  -H "Authorization: token ${GITHUB_TOKEN}" \
  "https://api.github.com/repos/${OWNER}/${REPO}/git/refs/heads/${BRANCH}" \
  | python3 -c "import sys,json; print(json.load(sys.stdin)['object']['sha'])")
 
BASE_TREE=$(curl -sf \
  -H "Authorization: token ${GITHUB_TOKEN}" \
  "https://api.github.com/repos/${OWNER}/${REPO}/git/commits/${HEAD_SHA}" \
  | python3 -c "import sys,json; print(json.load(sys.stdin)['tree']['sha'])")
 
echo "HEAD: ${HEAD_SHA}"
echo "BASE_TREE: ${BASE_TREE}"
 
# ── Step 2: Create a blob for each file ───────────
FILE_PATH="content/articles/en/claude-code/my-article.mdx"
 
BLOB_SHA=$(curl -sf \
  -X POST \
  -H "Authorization: token ${GITHUB_TOKEN}" \
  -H "Content-Type: application/json" \
  "https://api.github.com/repos/${OWNER}/${REPO}/git/blobs" \
  -d "$(python3 -c "
import sys, json
content = open('${FILE_PATH}', 'r').read()
print(json.dumps({'content': content, 'encoding': 'utf-8'}))
")" | python3 -c "import sys,json; print(json.load(sys.stdin)['sha'])")
 
echo "BLOB: ${BLOB_SHA}"
 
# ── Step 3: Create a tree ─────────────────────────
TREE_SHA=$(curl -sf \
  -X POST \
  -H "Authorization: token ${GITHUB_TOKEN}" \
  -H "Content-Type: application/json" \
  "https://api.github.com/repos/${OWNER}/${REPO}/git/trees" \
  -d "$(python3 -c "
import json
tree = {
    'base_tree': '${BASE_TREE}',
    'tree': [{
        'path': '${FILE_PATH}',
        'mode': '100644',
        'type': 'blob',
        'sha': '${BLOB_SHA}'
    }]
}
print(json.dumps(tree))
")" | python3 -c "import sys,json; print(json.load(sys.stdin)['sha'])")
 
echo "TREE: ${TREE_SHA}"
 
# ── Step 4: Create the commit ─────────────────────
COMMIT_SHA=$(curl -sf \
  -X POST \
  -H "Authorization: token ${GITHUB_TOKEN}" \
  -H "Content-Type: application/json" \
  "https://api.github.com/repos/${OWNER}/${REPO}/git/commits" \
  -d "$(python3 -c "
import json
commit = {
    'message': '${COMMIT_MSG}',
    'tree': '${TREE_SHA}',
    'parents': ['${HEAD_SHA}']
}
print(json.dumps(commit))
")" | python3 -c "import sys,json; print(json.load(sys.stdin)['sha'])")
 
echo "COMMIT: ${COMMIT_SHA}"
 
# ── Step 5: Update the branch ref ─────────────────
curl -sf \
  -X PATCH \
  -H "Authorization: token ${GITHUB_TOKEN}" \
  -H "Content-Type: application/json" \
  "https://api.github.com/repos/${OWNER}/${REPO}/git/refs/heads/${BRANCH}" \
  -d "{\"sha\": \"${COMMIT_SHA}\"}" > /dev/null
 
echo "✅ Push complete: ${COMMIT_SHA}"

Bundling Multiple Files in One Commit

When you want to commit a JA/EN article pair in a single commit — the right way to do it — add both files to the tree array in Step 3:

python3 -c "
import json
 
tree_items = [
    {
        'path': 'content/articles/ja/claude-code/my-article.mdx',
        'mode': '100644',
        'type': 'blob',
        'sha': '${BLOB_JA_SHA}'
    },
    {
        'path': 'content/articles/en/claude-code/my-article.mdx',
        'mode': '100644',
        'type': 'blob',
        'sha': '${BLOB_EN_SHA}'
    }
]
 
payload = {
    'base_tree': '${BASE_TREE}',
    'tree': tree_items
}
print(json.dumps(payload))
"

You can create blobs in parallel since they're independent. The tree and commit steps must be sequential, but that's only two more requests.

Forgetting base_tree Wipes the Repository

The scariest pitfall in the whole flow is omitting base_tree when you create the tree in Step 3.

base_tree tells GitHub to build your new tree on top of the existing directory structure. Leave it out, and GitHub creates a tree containing only the files you listed — so a commit meant to add one article silently deletes everything else in the repo.

I reproduced this once in a throwaway repository, and it genuinely rattled me. It has the same destructive power as force-pushing over your entire directory.

# ❌ no base_tree → every other file disappears
tree = {'tree': [{'path': FILE, 'mode': '100644', 'type': 'blob', 'sha': BLOB}]}
 
# ✅ with base_tree → layer your change onto the existing tree
tree = {'base_tree': BASE_TREE, 'tree': [{'path': FILE, ...}]}

Always pass the tree SHA from the latest commit you fetched in Step 1. Think of base_tree as the safety catch for incremental commits.

Deleting and Renaming Files

Set a tree entry's sha to null and that path gets deleted. Removing an article — say, returning a 410 — uses the exact same flow as adding one.

tree_items = [
    # delete the old file
    {'path': 'content/articles/en/claude-code/old.mdx',
     'mode': '100644', 'type': 'blob', 'sha': None},
    # add the same content at a new path (= rename)
    {'path': 'content/articles/en/claude-code/new.mdx',
     'mode': '100644', 'type': 'blob', 'sha': BLOB_SHA},
]

A rename is just "delete the old path + add the new path" in one tree. There's no dedicated git mv endpoint, but this combination covers it completely.

Common Errors and Fixes

422 Unprocessable Entity (blob creation): Usually a binary file or encoding mismatch. For plain text files, "encoding": "utf-8" is always safe.

409 Conflict (ref update): Someone else pushed since you fetched HEAD. Re-fetch the latest HEAD SHA and try again — this is the equivalent of git pull --rebase.

401 Unauthorized: Your Personal Access Token is expired or missing the repo scope. GitHub fine-grained tokens need "Contents: read and write" permission.

A Hybrid Strategy

If your VM environment allows cloning to $HOME/repos/ (outside of the problematic /tmp area), a reasonable approach is to try git CLI first and fall back to the REST API only on failure:

push_article() {
  local repo_path="$1"
  
  if cd "$repo_path" && git push origin main 2>/dev/null; then
    echo "✅ git push succeeded"
    return 0
  fi
  
  echo "⚠️ git push failed → falling back to REST API"
  # call your REST API push function here
}

Since I added this fallback to my Claude Code pipelines, environment-related push failures have effectively disappeared.

Always Verify the Push Landed

One more thing the REST API won't tell you: every step can return 200 and the branch still might not have moved where you expect. Mixing up a SHA variable or malforming the ref payload fails silently, without raising an error.

After the PATCH, I re-fetch HEAD and confirm it matches the commit SHA I just created:

ACTUAL=$(curl -sf -H "Authorization: token ${GITHUB_TOKEN}" \
  "https://api.github.com/repos/${OWNER}/${REPO}/git/refs/heads/${BRANCH}" \
  | python3 -c "import sys,json; print(json.load(sys.stdin)['object']['sha'])")
 
if [ "${ACTUAL}" = "${COMMIT_SHA}" ]; then
  echo "✅ Verified: branch advanced to ${COMMIT_SHA}"
else
  echo "❌ Mismatch: expected ${COMMIT_SHA}, got ${ACTUAL}"
fi

That one extra request is what turns "I think it pushed" into "I know it pushed" — exactly the confidence you want when a scheduled task runs unattended.

Next Steps

If you're running automated publishing workflows with Claude Code, consider wrapping the REST API push into a reusable shell function and sourcing it across your site scripts. The GitHub API rate limit for authenticated requests (5,000 per hour) is more than enough for typical article publishing pipelines.

The four-step flow becomes second nature quickly, and the reliability gain — no more wondering whether a push failed because of a stale lock file — is worth the verbosity.