Why I Built This

I'm building DevLoop Runner, an AI agent automation service. It takes GitHub Issues and calls Claude Code or Codex to automatically create pull requests.

This tool uses credentials for external services like GitHub, Codex, and Claude Code.

The problem was: you wouldn't know credentials had expired until you ran a job.

Run the job, it fails, check the logs, wonder "why did it fail?", and 30 minutes later realize "oh, the credentials expired." I wanted to eliminate this wasted time.

So I added a validate-credentials command that pre-validates credentials.

🔗 https://devloop-runner.app/

The Guiding Principle: "Don't Confuse the User"

While implementing, I kept thinking about UI consistency.

Running a check and seeing results that leave users confused is not good. I wanted users to see results and immediately know what action to take.

Here's what I focused on:

1. Don't Expose Security Information

In the initial implementation, JWT tokens and email addresses were being output in plain text in the validation results.

const isJsonString = authJsonPath.startsWith('{') || authJsonPath.startsWith('[');
const displayValue = isJsonString ? '[JSON content - masked for security]' : authJsonPath;

Validation results just need to show "is it configured?" and "can it connect?". No need to show the actual values.

2. Match Execution and Validation Environments

The validation command was checking the CODEX_AUTH_JSON environment variable, but the actual execution command used the ~/.codex/auth.json file.

This meant validation could pass but execution could still fail.

I changed it to check under the same conditions as the execution command.

3. Distinguish Required from Optional

OpenAI and Anthropic API keys are optional features for follow-up issue generation. The main functionality works without them.

But the initial implementation showed status: failed.

checks.push({
  name: 'OPENAI_API_KEY',
  status: 'skipped',  // Changed from 'failed'
  message: 'Not configured (optional for follow-up issue generation)',
});

I changed it to skipped so users wouldn't think something had failed.

4. Request Only the Minimum Required Scopes

For GitHub token validation, I initially checked for three scopes: repo, workflow, and read:org.

But when I checked the code:

I ended up reducing it to just the repo scope.

If validation is too strict, you get "authentication error" even when things would actually work.

5. Make It Debuggable

The Codex connectivity check was timing out in the Jenkins environment. But no logs were being output, so I couldn't figure out why.

await codexClient.executeTask({
  prompt: 'ping',
  maxTurns: 1,
  verbose: true,  // Enable logging
});
setTimeout(() => reject(new Error('Timeout after 30 seconds')), 30000)

I enabled verbose logging so the cause can be identified on the next run.

If the validation tool itself is hard to debug, that defeats the purpose.

Where Things Stand Now

I just merged this, so we'll see how useful it actually is.

From my testing, it seems to work as expected.

But whether it actually helps "catch auth errors early" is something I won't know until I use it in production.

When credentials expire, will I notice before running a job?

Will users see results and immediately know what to do?

That's what I'll find out going forward.

[📩 敆擁ăƒȘンク: moshimo-backlog-banner-728x90]