'Success' That Counted as Failure: Debugging Three Overlapping Bugs in a Claude Code / Codex Pipeline

I run an in-house system that has AI agents write code. It uses Claude Code and Codex for different roles, and automates everything from planning to implementation starting from a GitHub issue. Related design context is in Sonnet 4.6 Is Out — I Found Out From a Chat, Not a Changelog.

Recently, I hit a bug in that system where three distinct problems happened to stack up at once. Each one had a reasonable root cause on its own, but the way they overlapped made debugging painful. Writing it down while it's fresh.

What was happening

Looking at the execution log, the implementation phase finished in 79 turns over about 16 minutes. TypeScript type-check passed, the build passed, tests passed 3751 / 3788. By every visible signal, it succeeded.

And yet the overall workflow exited as failed.

Staring at it long enough, I realized it wasn't one problem. The eventual PR #862 ended up bundling three fixes together.

Problem 1: Claude Code SDK's tail-end error

Two [AGENT RESULT] lines in a row

Near the end of the log, these lines showed up back-to-back:

08:06:23 [AGENT RESULT] status=success, turns=79, duration_ms=959990
08:06:23 [AGENT RESULT] status=error_during_execution, turns=0, duration_ms=0
08:06:23 [ERROR] Execute step failed: Claude Code process exited with code 1

The first status=success is the real thing — 79 turns of successful implementation. The second status=error_during_execution appears immediately after, with zero turns and zero milliseconds.

"Zero-turn failure" doesn't happen in normal execution. This wasn't a processing failure; it had to be something emitted in teardown.

Following the SDK source

Digging into @anthropic-ai/claude-agent-sdk, the main loop looked roughly like this (simplified):

try {
  for await (let d of agentLoop({ prompt, tools, ... })) {
    // type:"result", subtype:"success" is emitted here too
    stream.write(d);
  }
} catch (g) {
  try {
    stream.write({
      type: "result",
      subtype: "error_during_execution",
      duration_ms: 0,
      num_turns: 0,
      is_error: true,
      ...
    });
  } catch {}
  process.exit(1);
}

After agentLoop yields success, the loop's finally block and other teardown code still run. If something throws there, catch(g) writes a second result and the process exits with code 1.

So turns=0, duration_ms=0 doesn't mean "failed in zero turns" — those are just the default field values in the catch block.

ProcessTransport.readMessages uses waitForExit() to check the child process exit code and throws on non-zero. Even if the artifacts are fully produced, a non-zero exit code flips the whole thing to failure.

Organizing this:

The root trigger might be on my side or environmental, but the fact that it cascades directly into "phase failed" is a design problem on the SDK side.

Fix: downgrade post-success stream errors to warn

Minimal fix: flag sawSuccessResult, and downgrade stream errors after success to warnings.

// claude-agent-client.ts
const messages: string[] = [];
let sawSuccessResult = false;

try {
  for await (const message of stream) {
    const sanitizedMessage = sanitizeObjectStrings(message) as SDKMessage;
    messages.push(JSON.stringify(sanitizedMessage));

    // Only count this as "real success" when type=result, subtype=success, AND is_error=false.
    // The Claude CLI can return subtype=success even on auth errors (401), so is_error
    // is what actually distinguishes real success from fake success.
    const m = message as { type?: string; subtype?: string; is_error?: boolean };
    if (m.type === 'result' && m.subtype === 'success' && m.is_error !== true) {
      sawSuccessResult = true;
    }

    if (verbose) this.logMessage(message);
  }
} catch (err) {
  // If we've already received a successful result, SDK teardown failures
  // ("Claude Code process exited with code 1" etc.) stay as warnings,
  // not phase-level failures.
  if (sawSuccessResult) {
    logger.warn(
      `Claude SDK stream error after successful result (downgraded to warning): ${getErrorMessage(err)}`
    );
  } else {
    throw err;
  }
}

return messages;

The reason for the is_error !== true condition came out later (see the next problem).

I also added stderr capture, to leave a trail for diagnosing the real cause next time:

const stream = query({
  prompt,
  options: {
    // ...
    // Capture the Claude CLI child process stderr so we can see
    // the exceptions the SDK otherwise swallows.
    stderr: (data: string) => {
      const trimmed = data.trim();
      if (trimmed) {
        logger.warn(`[Claude CLI stderr] ${trimmed}`);
      }
    },
  },
});

Problem 2: is_error=true with subtype=success

After shipping the fix above, a different problem surfaced on the next run:

[AGENT THINKING] API Error: 401 ... "Invalid authentication credentials" · Please run /login
[AGENT RESULT] status=success, turns=1, duration_ms=1437
[WARNING] Claude SDK stream error after successful result (downgraded to warning): ...

Claude Code's OAuth token had become invalid (401), but the SDK still returned subtype=success. My new fix was treating this "apparent success" as real success and downgrading the subsequent error to a warning.

Checking the SDK type definitions, SDKResultMessage does have an is_error: boolean field. The Claude CLI returns subtype:'success' + is_error:true on auth errors — a slightly confusing design.

// SDKResultMessage type (excerpt)
type SDKResultMessage = SDKMessageBase & {
  subtype: 'success';
  is_error: boolean;
  // ...
}

So adding the is_error !== true condition makes sawSuccessResult=true only for real success.

I also updated the log formatter to surface is_error=true, so future diagnoses are easier:

// After the fix, log output looks like:
[AGENT RESULT] status=success, turns=1, duration_ms=1437, is_error=true

With that field visible, "fake success caused by a 401" is recognizable at a glance.

Problem 3: Codex failing every phase before falling back

While chasing the Claude-side issue, I noticed something unrelated in the Codex logs:

[INFO ] Using model override for Codex Agent: gpt-5.1-codex-mini
[CODEX ERROR] "The 'gpt-5.1-codex-mini' model is not supported when using Codex with a ChatGPT account."
[WARNING] Falling back to Claude Agent.

gpt-5.1-codex-mini was no longer usable via a ChatGPT account. Every phase was burning 2–3 seconds on a Codex failure before falling back to Claude.

Why it stopped working

This was a spec change. The day before (2026-04-14), OpenAI pulled gpt-5.2-codex and gpt-5.1-codex-mini from ChatGPT accounts — confirmed via community discussion and GitHub Issues. Bad timing: I walked right into a change made the day before.

Via ChatGPT account (codex login), the usable models are now gpt-5.3-codex and the gpt-5.4 family. The old models still work through API keys (CODEX_API_KEY).

Which model to migrate to

I needed to change the alias targets for max (complex tasks) and mini (lightweight / low-cost).

mini was an easy call — gpt-5.4-mini. For max, I spent a bit of time weighing gpt-5.3-codex vs gpt-5.4.

This system runs lots of long, multi-file, multi-turn tasks (today's implementation task was 79 turns / 16 min), so total tokens consumed matters more than per-input price. Token-efficient gpt-5.4 is likely to come out cheaper overall. And since mini is going to gpt-5.4-mini, keeping both on the 5.4 family felt cleaner.

// codex-agent-client.ts
export const DEFAULT_CODEX_MODEL = 'gpt-5.4';

export const CODEX_MODEL_ALIASES: Record<string, string> = {
  max: 'gpt-5.4',         // Default, flagship for complex multi-step projects
  mini: 'gpt-5.4-mini',   // Lightweight, cost-effective
  '5.1': 'gpt-5.1',       // General-purpose (legacy)
  legacy: 'gpt-5-codex',  // Legacy (backward compatibility)
};

Implementing session-wide disable

Alongside the model migration, I added logic so that once we know "the model isn't available on this ChatGPT account," later phases don't even try Codex.

The catch: each phase creates its own Codex client instance. Disabling Codex in phase 1 doesn't carry over to phase 2, because phase 2 gets a fresh instance and hits the same failure.

So I used a static flag to share the disabled state across the whole process.

// codex-agent-client.ts
class CodexAgentClient {
  // Session-wide (process-wide) disable flag.
  // Once we detect a permanent account-level failure, other instances
  // should not retry the same Codex CLI binary.
  private static sessionDisabled = false;
  private static sessionDisabledReason = '';

  public static markSessionDisabled(reason: string): void {
    if (!CodexAgentClient.sessionDisabled) {
      CodexAgentClient.sessionDisabled = true;
      CodexAgentClient.sessionDisabledReason = reason;
      logger.warn(`Codex agent disabled for this session (all instances): ${reason}`);
    }
  }

  public markDisabled(reason: string): void {
    if (!this.disabled) {
      this.disabled = true;
      this.disabledReason = reason;
      logger.warn(`Codex agent disabled for this session: ${reason}`);
    }
    // Instance disable also propagates to the session-wide flag.
    CodexAgentClient.markSessionDisabled(reason);
  }

  public isDisabled(): boolean {
    return this.disabled || CodexAgentClient.sessionDisabled;
  }
}

The caller calls markDisabled whenever it detects the "not supported on ChatGPT account" error message:

// agent-executor.ts
if (
  this.codex?.markDisabled &&
  /not supported when using Codex with a ChatGPT account/i.test(message)
) {
  this.codex.markDisabled(
    'ChatGPT account: configured Codex model is not supported',
  );
}

So the static flag is set the moment phase 1 fails, and from phase 2 onwards isDisabled() returns true immediately — we don't even spawn the binary. That 2–3 seconds of waste per phase becomes a one-time cost.

A fourth problem: metadata state contamination

Separate from those three fixes, another nasty aspect of this incident was metadata state contamination.

After several failed runs stack up, you end up in this state:

12:10:45 [INFO] Phase implementation: skipping 'execute' step (already completed)
                ↑ empty implementation.md, recognized as "done"

12:12:12 [WARNING] Review failed: Phase 4 not implemented (still skeleton)
12:12:12 [ERROR] Retry count already at maximum (3/3)
                 ↑ past failures already consumed the revise retry budget

"Execute is done," "implementation file is empty," "revise is maxed out" — a three-way stuck state.

The solution is to roll back the phase to reset the state.

node dist/index.js rollback --issue 854 --phase implementation --auto

This moves the implementation phase back to pending and also resets revise_count.

Final check

After the rollback and a rerun, I could see all three fixes working:

✅ Codex model migration (gpt-5.4 / gpt-5.4-mini): review completed
✅ Codex session-disabled: didn't fire (today succeeded), but will skip later phases on failure
✅ Claude SDK stream-error downgrade + is_error check: didn't fire today, but is armed

What I took away

"The process returned success" and "exit code was zero" are two different things. That's true in normal software, but with AI agents there's an extra layer: the artifact was produced, and the internal teardown finished — two completion states layered on top of each other — which makes it worse.

If you design the phase-completion check to verify the actual artifact (in this case, the implementation files and build output) rather than relying on exit code alone, this class of problem goes away.

Equally memorable was how state accumulation and failure chains compound. The first failure left things "half-done but marked complete," runs 2–3 burned through the retry budget, and by the time run 4 got to a clean diagnosis the system was already locked up.

In recovery design, having a reliable way to "reset the state and start over" matters more than I had appreciated. I'd already implemented the rollback command some time back, but I didn't expect it to have such a clear, immediate use case.

The CLAUDE_CODE_OAUTH_TOKEN 401 issue (Claude's auth) is still unresolved. Codex is carrying things for now so I'm not hitting the Claude fallback path, but the underlying problem is still waiting for a real fix.

Further reading

For readers who want to build a systematic understanding of how to use Claude Code in real workflows, the following books are useful references (Japanese editions).

The first focuses on hands-on practices and working patterns; the second is a broader introduction to AI-driven development. When you're tracing agent behavior like in this post, having a solid mental model of how the tool is meant to work helps the diagnosis go faster.