From Log Streaming to JSON Output — Adding Structured Results to a CLI Tool

When building CLI commands for automation tools, it's easy to stop at "it runs, logs are flowing, done."
Honestly, that's exactly what I was doing.

Then one day, while reviewing the CI/CD pipeline configuration, I noticed something:

There was a setting to save a results file, but nobody was actually creating the file.

This is the record of adding JSON output functionality to a CLI command to solve the "logs flow but files don't stick" problem. The implementation itself is straightforward, but the "how to build it" decisions turned out to be the interesting part, so that's what I want to focus on here.

What's the Actual Problem?

Let me explain the situation more concretely.

The automation tool I'm developing has a CLI command that inspects GitHub Issues and auto-closes them. An AI agent reads each Issue's content and automatically closes the ones it determines are ready to be closed.

Until now, the results of this command only appeared in logs. They'd flow through the terminal but never get saved as a file anywhere.

Meanwhile, on the CI/CD pipeline (Jenkins) side, this configuration already existed:

// Setting to save execution results as build artifacts
archiveArtifacts artifacts: 'results.json'

This says "if there's a file called results.json, save it as a build artifact." But since the command never generated that file, the setting existed but had nothing to work with.

Functionally, this wasn't a problem. The setting was configured to not error out when the file was missing. But it created pain points:

"How many Issues did we close in last week's run?" → Have to dig through logs
"Why was this Issue closed?" → Logs have scrolled away
Want to keep an audit trail of "when, under what conditions, what was done" → Nothing saved

Logs are for checking the "now." Files are for looking back "later." They serve different purposes.

How to Build It: Two Options

There were two main implementation approaches:

Option A: Stream JSON to stdout
A simple --json flag that outputs results in JSON format to standard output. Easy to implement.

Option B: Write to a file (follow existing pattern)
An --output-file <path> option that saves results as a JSON file. Another command in the same tool (the auto-create-issue command) was already using this approach.

After deliberation, I chose Option B.

graph TD
    A{How to output JSON?}
    A -->|Option A: stdout| B["--json flag<br/>Easy to implement"]
    A -->|Option B: File output| C["--output-file path<br/>Follow existing pattern"]

    B --> D["❌ Risk of mixing with logs<br/>❌ Awkward CI artifact integration<br/>❌ Inconsistent option patterns"]
    C --> E["✅ Directly connects to CI artifacts<br/>✅ Logs and output stay separated<br/>✅ Consistent UX across commands"]

    style C fill:#e8f5e9,stroke:#4caf50
    style E fill:#e8f5e9,stroke:#4caf50

The deciding factor was keeping the CLI option patterns consistent.

If Command A uses --output-file but Command B uses --json, the user (future me and CI/CD) gets confused. "Wait, which one does this command use again?" — that kind of subtle friction adds up.

My SRE days of wrestling with mountains of inconsistent configuration files paid off here. Keeping things aligned makes life easier down the road.

Three Things I Kept in Mind During Implementation

The changes spanned 5 files, but rather than walk through each one, I'll focus on the three design decisions that mattered most.

1. Include "Execution Context" in the JSON Schema

The output JSON structure looks like this:

{
  "execution": {
    "timestamp": "2026-02-16T06:54:00.000Z",
    "repository": "your-org/your-repo",
    "category": "followup",
    "dryRun": false,
    "confidenceThreshold": 0.7
  },
  "summary": {
    "totalInspected": 10,
    "recommendedClose": 3,
    "actualClosed": 3,
    "skipped": 5,
    "errors": 2
  },
  "issues": [
    {
      "issueNumber": 42,
      "title": "Fix login bug",
      "action": "closed",
      "inspection": {
        "recommendation": "close",
        "confidence": 0.95,
        "reasoning": "..."
      }
    }
  ]
}

The key is the execution section. It saves not just the results, but the conditions under which the command ran.

Without this, when you look at the JSON later, you can't tell "Was this a dry run? Or a production run?" You'd have to go back to the logs to check the conditions — defeating the entire purpose. A results file should be self-contained in terms of context. This is a subtle but important design decision.

2. Separate "Building" from "Writing"

The output module was split into two functions:

buildJsonPayload() — Constructs the JSON structure from result data (pure function)
writeOutputFile() — Writes JSON to a file (has side effects)

Why separate them? Testability.

buildJsonPayload() doesn't touch the filesystem, so it can be tested without mocks. Tests like "if I pass 10 results, are the summary numbers correct?" become simple. For file writing tests, you only need to mock fs in writeOutputFile().

The existing output module already used this separation, so I followed suit. Looking at it fresh, I appreciated the pattern even more.

3. Don't Break Existing Behavior

Integration into the command body looks like this:

// Regular log output (existing behavior)
reportResults(results, options);

// JSON output (only runs when --output-file is specified)
await exportJsonIfRequested(results, options, repository);

If --output-file isn't specified, nothing happens. Zero impact on existing behavior.

For option parsing, path.resolve() converts the path to absolute. This ensures files are created in the right place regardless of which directory the CLI is called from. Passing relative paths can create files in unexpected locations when the current directory changes — a common gotcha in CI/CD environments.

CI/CD Pipeline Integration

On the Jenkins side, the only change was adding --output-file results.json to the command invocation:

// Before: No file generated
sh 'node dist/index.js auto-close-issue --category followup'

// After: Results saved to JSON file
sh 'node dist/index.js auto-close-issue --category followup --output-file results.json'

Since the archiveArtifacts setting was already in place, the moment the command started generating the file, the pipeline's artifact saving kicked in automatically. "Expectation" and "implementation" finally aligned.

Verification Results

Build, type checking, and CLI --help display all passed without issues. The test suite had a few pre-existing failures, but none were caused by this change. The existing failures stemmed from option changes in a previous commit that tests hadn't caught up with, plus local environment permission issues — to be addressed separately.

Retrospective

What I actually did was simple: "add a JSON output option to a CLI."

But looking back, most of the decisions boiled down to "follow the existing pattern or take the easier route." This time, I chose to follow the pattern. The changes scattered across 5 files might look tedious at first glance, but when another command needs the same functionality later, having an established pattern means no deliberation needed.

I'm genuinely grateful to my past self for properly establishing the output module pattern in the first command.

By the way, this CLI tool runs via Jenkins pipelines on devloop-runner.app. Building out these automation pieces one by one is honestly the most enjoyable part of the work for me.

What's Still Left

Adding unit tests for JSON output
Updating existing tests to catch up with option changes from the previous commit
Output format versioning (not needed yet, but might become necessary as more external tools start consuming the JSON)