The "Usability Issues" of AI Agents

I'm developing a web app called DevLoopRunner. It's a tool where AI agents automatically handle everything from requirements definition to implementation, testing, and documentation creation, starting from a GitHub Issue.

The backend is implemented as a Node.js CLI, executed by Claude (AI). When you open a repository with the Claude Code tool, the AI automatically executes commands.

This tool is useful, but I felt stressed every time I used it.

For example, when starting a workflow with a new Issue, I need to execute a command like this:

node dist/index.js init \
  --issue-url https://github.com/tielec/ai-workflow-agent/issues/123 \
  --base-branch main \
  --language ja

It looks simple at first glance, but in reality:

I often thought "Wait, what was this option again?" and checked CLAUDE.md (instruction file for AI) every time.

The phase execution command is even more complex.

node dist/index.js execute \
  --issue 123 \
  --preset review-design \
  --agent auto \
  --codex-model max \
  --claude-model opus \
  --auto-model-selection

What presets were available for --preset? Is --agent supposed to be auto? What's the difference between --codex-model and --claude-model?

I had to check and wonder every time.

Honestly, it was tedious.

Discovering "Skills"

While thinking "there must be a better way," I looked into Claude Code's latest features.

That's when I encountered the Skills mechanism.

Skills is, roughly speaking, "specialized commands you teach the AI." You write command descriptions in a folder called ~/.claude/skills, and they become callable in the form /command-name.

For example, like this:

~/.claude/skills/init-workflow/
└── SKILL.md  # Write command description

Once created, just typing /init-workflow makes the AI:

  1. Ask for the Issue number
  2. Suggest default values
  3. Allow interactive selection
  4. Confirm before execution

I thought "This could be really good."

Before/After: How Would It Change?

Let me compare actual usage examples to see how it would change concretely.

Current Usage

User: "I want to start workflow for Issue #123"

Claude: "Let me check CLAUDE.md...
        I'll use the init command.
        The options are... --issue-url is needed...
        --base-branch too... and --language..."

        node dist/index.js init \
          --issue-url https://github.com/tielec/ai-workflow-agent/issues/123 \
          --base-branch main \
          --language ja

        Should I execute this?

User: "Yes"

(Executes)

I need to confirm and approve this long command every time.

After Skills Implementation (Image)

User: "Start workflow for Issue #123"

Claude: "I'll use /init-workflow"

        Please provide Issue number: 123

        Confirming settings:
        ✓ Repository: tielec/ai-workflow-agent (auto-detected)
        ✓ Branch name: ai-workflow/issue-123 (auto-generated)
        ✓ Base branch: main (default)
        ✓ Language: ja (default)

        Execute with these settings? [Y/n]

User: "y"

(Executes)

Changed points:

  1. Shorter command: Just /init-workflow
  2. Interactive: Asks for Issue number
  3. Default values: Automatically fills commonly used settings
  4. Easy confirmation: Can review settings in a list

Let me also look at a phase execution example.

Current: Option Hell

User: "Execute from Planning to Design"

Claude: "Let me check CLAUDE.md...
        There's a review-design preset.
        Should I enable auto-model-selection?
        Is agent supposed to be auto?
        For codex-model..."

        node dist/index.js execute \
          --issue 123 \
          --preset review-design \
          --agent auto \
          --codex-model max \
          --claude-model opus \
          --auto-model-selection

        Should I execute this?

User: (Too many options to confirm...)

After Skills Implementation (Image)

User: "Execute from Planning to Design"

Claude: "I'll use /execute-phase"

        Detected Issue #123

        Preset: review-design
        Execution phases: Planning → Requirements → Design

        Recommended settings:
        ✓ auto-model-selection: ON (auto-select by difficulty)
        ✓ agent: auto (Codex priority, auto-switch to Claude on failure)
        ✓ codex-model: max (highest performance model)
        ✓ claude-model: opus (highest performance model)

        Change settings? [y/N]

User: "n"

(Executes)

It suggests recommended settings, so you can proceed as-is unless you have specific preferences. Customize only when you want to change something.

This reduces the effort of confirming options every time.

Also Discovered Subagent Feature

While researching Skills, I found another interesting feature: Subagent.

Subagent is a mechanism for parallel task execution. Apparently, it can run up to 10 simultaneously.

"Maybe this could..." I thought.

In DevLoopRunner, we execute sequentially from Phase 0 (Planning) to Phase 7 (Documentation). If parallel execution were possible, couldn't we drastically reduce execution time?

I estimated the expected benefits.

Item Current After Subagent Use Improvement
Phase 0-2 execution time 45 min 15 min 67% reduction
auto-issue execution time 30 min 10 min 66% reduction

67% reduction! That's amazing.

However, thinking calmly, Phase 0-2 have dependencies.

Phase 0 (Planning)
  ↓ References output
Phase 1 (Requirements)
  ↓ References output
Phase 2 (Design)

Requirements can't be written without Planning's output. Design can't be written without Requirements' output.

Can't run in parallel after all.

Furthermore, reviewing the existing architecture, each phase was already functioning as a "specialized agent." They had independent prompts, received only necessary information, and generated independent outputs.

The main functionality of Subagent was already realized.

What can actually be parallelized:

However, the processing time for these is already relatively short, and it's not a situation where "it's slow and problematic."

Final Decision: Decided Against Everything

After all this consideration, I ultimately decided to not adopt Skills or Subagent at all.

The reason is simple: I judged I'm not ready to use them yet.

Why I Decided Against Skills

Skills certainly seems convenient. Commands become shorter, default values are filled in, and interactive selection is possible.

However, when I actually tried to write Skills, several questions emerged.

Even after reading the official documentation, I only understood "sort of," and couldn't gain confidence that "I've got this perfectly."

Honestly, I need to study a bit more.

Why I Decided Against Subagent

Same with Subagent. While I understood the parallel execution mechanism, I couldn't imagine how to actually implement it.

Weighing implementation risks against benefits, I felt it was too early to jump in at this stage.

Importance of Careful Judgment

The desire to "use new technology" was strong. However, introducing it without being able to use it properly only increases complexity.

The current CLAUDE.md is admittedly lengthy, but at least it works. It's stable.

Do I need to migrate to a new mechanism even if it breaks this? Thinking calmly, I judged now is not the time.

Lessons: The Courage Not to Jump

I learned three things from this evaluation process.

1. "Looks Convenient" and "Can Actually Use It" Are Different

When seeing new technology, we think "this looks convenient." However, whether you can actually use it is a different story.

If you can't say "Yes" to these, it's still too early.

2. Don't Underestimate Current Designs

We tend to think "new technology solves everything," but existing designs have value too.

DevLoopRunner's each phase was already functioning essentially as subagents. Even without Skills, it works sufficiently with CLAUDE.md.

It's important not to underestimate the current state and ask "do we really need to change?"

3. The Option to "Study First"

There's no need to rush thinking "I must introduce this now." There's also the option of "study a bit more first."

Skills and Subagent aren't going anywhere. It's not too late to introduce them after reading documentation, trying samples, and deepening understanding.

Sometimes courage not to jump is necessary.

Conclusion

I researched Claude Code 2.1.0's new features and evaluated Skills and Subagent implementation.

Both were attractive features, but I ultimately decided against everything. The reason is that I judged I'm not ready to use them yet.

The balance between the desire to "use new technology" and the calm judgment of "can I really use it?" ultimately leads to better decision-making, I think.

I'll study a bit more and reconsider later.

Related Books

For those who want to learn more about Claude Code and AI development tools:

[📦 商品リンク: moshimo-book-9CeFY]

[📦 商品リンク: moshimo-book-fINTJ]