⬡ API & SDK/2026-06-23Advanced

An Unattended Agent That Wakes Up to a Blank Machine Every Time

A scheduled, unattended agent wakes up on a fresh disposable machine every run: paths it cannot write to, a filesystem that has not finished booting, a working directory that has vanished. Here is how to design state recovery into the first thirty seconds, drawn from real operational logs.

Claude Agent SDK¹³unattended agentsstate recoveryidempotency⁶ operations³⁰

✦ Premium Article

The morning the scheduled agent woke up, the first thing it returned was neither success nor failure. It was a single line: could not create work tree dir: Permission denied.

The /tmp/repos path it had written to yesterday was not writable today.

The cause was quick to find. The machine the agent runs on is rebuilt from scratch every time — a disposable sandbox — and the directory's owner happened to be assigned to a different user that run. As an indie developer, I run unattended article generation across four sites at Dolice Labs, and whenever I take the "the environment is different every time" assumption lightly, the nightly task ends quietly with zero output.

The reliability of an unattended agent is decided not by clever core logic, but by the first thirty seconds. This piece is about those thirty seconds: how to design startup and state recovery right after a cold boot.

Compute is disposable; state lives somewhere else

The first mental shift is to abandon the naive expectation that "the same machine remembers where it left off."

Most scheduled runs spin up a new sandbox each time. Files you wrote last run are gone, packages you installed do not persist, and cwd and environment variables do not carry over. The only state you can rely on lives outside the compute.

In my own operation, I split durable state into three layers.

Layer	Where it lives	Role
Code / artifacts	Git remote	The work itself. It only counts once pushed
Reference data / config	Cloud-synced workspace	Policy, keywords, tokens. Treated as near read-only
Run history	Per-date log files	The only way to tell your future self what today already did

With these three layers in place, the sandbox can be burned down as often as it likes. The agent rebuilds its state from external truth on every run. Conversely, the moment any of these three relies on a local temp directory, that information is lost on the next cold boot.

Write the startup sequence defensively

Before any real work, I always run a short sequence to prepare the environment. Cut corners here and the later stages fail silently.

The order matters: (1) wait for the filesystem to come up, (2) discover the working directory dynamically, (3) secure a writable work area, (4) retrieve credentials.

#!/usr/bin/env bash
set -euo pipefail
 
# (1) Wait for the mount — right after a cold boot, files are invisible for a few seconds
WS=""
for i in $(seq 1 10); do
  WS="$(ls -d /sessions/*/mnt/Workspace 2>/dev/null | head -1)"
  [ -n "$WS" ] && [ -d "$WS" ] && break
  sleep 3
done
[ -n "$WS" ] || { echo "FATAL: workspace not mounted after 30s"; exit 1; }
 
# (2) Check the first-choice work dir is writable; fall back to $HOME if not
WORK_BASE="/tmp/repos"
if ! mkdir -p "$WORK_BASE/.probe" 2>/dev/null; then
  echo "WARN: $WORK_BASE not writable, falling back to \$HOME/repos"
  WORK_BASE="$HOME/repos"
  mkdir -p "$WORK_BASE"
fi
rmdir "$WORK_BASE/.probe" 2>/dev/null || true
 
# (3) Read credentials from synced storage, not env vars (survives reboots)
TOKEN="$(grep -A1 '^MySite' "$WS/_tokens/tokens.txt" | tail -1 | tr -d '[:space:]')"
[ -n "$TOKEN" ] || { echo "FATAL: token missing"; exit 1; }
 
echo "ready: WS=$WS WORK_BASE=$WORK_BASE"

The key is to never assume /tmp is writable. Actually create a small directory; if it fails, quietly escape to $HOME. The Permission denied I hit on this run was exactly the kind of failure that one .probe test absorbs.

Searching with a wildcard like ls -d /sessions/*/mnt/... is also deliberate. The session name changes every run, so a hardcoded path points at a directory that will not exist next time. Dynamic discovery is the cheapest insurance against an environment that keeps changing.

✦

Thank you for reading this far.

Continue Reading

What follows includes implementation code, benchmarks, and practical content we hope you'll find useful. This site runs without ads — server and development costs are supported entirely by members like you. If it's been helpful, we'd be truly grateful for your support.

WHAT YOU'LL LEARN

✦A copy-pasteable bash startup sequence built for disposable environments: mount detection, boot waiting, dynamic working-directory discovery, and fallback paths

✦An idempotency guard that asks durable storage whether today's work is already done, plus the two-signal check that stops duplicate output

✦The three traps I hit running Dolice Labs unattended — write permissions, cloud-synced files, and boot latency — written up as field-tested operational guidance

Secure payment via Stripe · Cancel anytime

✦

Unlock This Article

Get full access to the rest of this article. Buy once, read anytime. This site is ad-free — your support goes directly toward keeping it running.

Unlock all articles with Membership →

Decide idempotently whether today's work is already done

The other danger of disposable environments is retries. If the scheduler misreads a timeout and launches the same task twice, or you re-run manually after a partial failure, the agent repeats the same work "as if for the first time." In unattended operation, that surfaces as duplicate publishing.

The defense is simple: place an idempotency guard that queries durable logs before doing real work. In my case, whatever I wrote that day is always recorded in a per-date log, so the log can serve as the source of truth.

import os, sys, datetime, glob
 
def already_done(log_dir: str, marker: str) -> bool:
    """True if today's log contains marker (= already done).
    Determine "today" in JST. A bare date runs as UTC and can mislabel the day."""
    jst = datetime.timezone(datetime.timedelta(hours=9))
    today = datetime.datetime.now(jst).strftime("%Y-%m-%d")
    for path in glob.glob(os.path.join(log_dir, f"{today}*.txt")):
        with open(path, encoding="utf-8") as f:
            if marker in f.read():
                return True
    return False
 
if already_done(sys.argv[1], sys.argv[2]):
    print("SKIP: already produced today")
    sys.exit(0)

Pinning the timezone is not a stylistic flourish. If you build log filenames with a bare date, the sandbox clock stays on UTC, and a run crossing the date boundary overwrites yesterday's log — making completed work look undone. I once nearly created two logs for the same day because of this mix-up. Always fix the idempotency key to the timezone you actually operate in.

Make it two-signal for extra safety. Beyond the log, confirm by the existence of the artifact itself. For article generation, that means checking the Git log for a near-duplicate slug in today's commits. Agree on both the log and the artifact, and a duplicate stops even if one signal is corrupted.

It only counts once you have pushed

Do not relax just because you committed. In a disposable environment, locally stacked commits evaporate with the sandbox. Only what reached the remote is durable.

So judge a push not by "did the command return zero," but by whether the local and remote commit hashes match.

git push origin main 2>&1 | tail -3
LOCAL="$(git rev-parse HEAD)"
REMOTE="$(git ls-remote origin -h refs/heads/main | cut -f1)"
if [ "$LOCAL" = "$REMOTE" ]; then
  echo "PUSH OK: $LOCAL"
else
  echo "PUSH FAILED: local=$LOCAL remote=$REMOTE"; exit 1
fi

I added this check because I once watched git commit fail silently with no identity set, after which the push appeared to "succeed" with nothing to send. In a freshly cloned disposable environment, both user.email and user.name are empty. Set identity before committing, and trust success only on a hash match. Those two habits almost entirely eliminate fake successes.

What the docs do not tell you

Here are a few instincts that live between the lines, learned from actually running this.

First, do not wait for boot with a fixed delay. A hardcoded sleep 30 wastes time on fast days and falls short on slow ones. Retrying on a "ready" condition (the loop above) shrinks the average wait. In my environment, switching from fixed to conditional waiting alone cut the felt startup overhead by more than half.

Second, a cloud-synced file is not necessarily on disk. Files on synced storage may only materialize on first access. If a cat appears to fail for a moment, it is often a materialization delay rather than a permission issue, and reading through a download-aware path resolves it. Before touching many or large files at once, be deliberate about what you are forcing to materialize, and you avoid wasteful downloads.

Third, do not reuse temp files under fixed names. A fixed name like /tmp/insert.txt silently mixes in last run's leftovers when a write fails. Place uniquely named files (including a slug) under $HOME, and grep-verify the contents right after writing, so contamination cannot sneak in.

Fourth, log failures too. The night that ended with zero output is precisely when "why did it stop" is the only clue you have the next morning. Always write a _FAILED-suffixed log in the same place as success logs, so your future self can reconstruct the situation.

Choosing the right level for the situation

Not every unattended task needs full state recovery. Match the design to the stakes.

Nature of the run	Recommended safeguards
Several times a day, side effect is public	Three-layer split + idempotency guard + hash-match check, all of it
Once a day, read-mostly aggregation	Boot wait and fallback path only. Overwriting the output target is idempotent enough
Manually triggered, interactive	Dynamic discovery only; a human is watching, so simplify recovery

Excessive defense is itself complexity, and complexity is another source of failure. I recommend starting with just two: the idempotency guard and the hash-match check. Those two stop the quietest, hardest-to-notice failures — duplicate publishing, and a push you only thought you made.

A mechanism that runs unattended is invisible to everyone while it works. That is exactly why I take care to write down, for those first thirty seconds after waking, how the agent rebuilds itself from a blank machine. It is a quiet, dependable investment in stacking up calm successes, morning after morning.

I hope it helps with your own implementation. Thank you for reading to the end.

Thank You for Reading

Claude Lab is ad-free, supported entirely by members like you. We publish practical guides daily with implementation code, benchmarks, and production-ready patterns. If you've found it useful, we'd love to have you on board.

✦Copy-paste ready implementation code
✦New advanced guides published daily
✦$5/mo or $10 for lifetime access

View Membership →