Streaming & NDJSON

Two commands — research and automate — return streaming responses. The API uses Server-Sent Events (SSE). The CLI converts them to NDJSON for shell consumption.

Why NDJSON, not SSE?

SSE is a browser protocol. Shell tools speak lines. jq, grep, awk, and every other Unix text tool expects newline-delimited records, not event: prefixed frames with data: fields.

NDJSON (newline-delimited JSON) is the natural translation: one {"event": "...", "data": {...}} per line. Each line is valid JSON. jq handles it with no special flags.

# SSE frame (what the API sends internally):
# event: citation
# data: {"url":"https://...","title":"..."}
#
# NDJSON (what tabstack outputs):
# {"event":"citation","data":{"url":"https://...","title":"..."}}

Filtering events with jq

Every command has a complete event (or task:completed for automate) that carries the final result. Filter to it:

# Research — get the report
tabstack research "approaches to browser automation" 
  | jq -r 'select(.event=="complete") | .data.report'

# Research — get cited pages
tabstack research "EU AI regulation" 
  | jq -r 'select(.event=="complete") | .data.metadata.citedPages[] | .url'

# Automate — get the final answer
tabstack automate "find trending repos" --url https://github.com/trending 
  | jq -r 'select(.event=="task:completed") | .data.answer'

Intermediate events (useful for progress or debugging):

# Research citations as they arrive
tabstack research "quantum computing 2026" 
  | jq 'select(.event | startswith("citation"))'

# Automate iteration progress
tabstack automate "scrape the pricing table" --url https://example.com 
  | jq 'select(.event | startswith("task:iteration"))'

Saving the full stream

Pipe to a file first, then query it multiple times without re-running the API call:

tabstack research "CRDTs vs OT" > /tmp/research.ndjson

# Extract report
jq -r 'select(.event=="complete") | .data.report' /tmp/research.ndjson > adr.md

# Append citations
jq -r 'select(.event=="complete") | .data.metadata.citedPages[] | "- [(.title)]((.url))"' 
  /tmp/research.ndjson >> adr.md

The edge cases that required hardening

The Tabstack API sends SSE over HTTP/1.1. In practice, three things went wrong during development that required explicit fixes:

CRLF line endings

HTTP/1.1 uses \r\n for headers. SSE frames can arrive with \r\n line endings from some servers. The naive buffer.split('\n') approach misses these — \r\n\r\n double-newlines (the SSE frame separator) never match \n\n, so frames never fire.

Fix: normalize \r\n and lone \r to \n as bytes arrive, with heldCR tracking to handle a \r that lands at the end of one chunk and the \n at the start of the next.

Unterminated final frame

The SSE spec requires frames to end with \n\n. Some implementations close the stream without the trailing double-newline. The last event stays in the buffer and never fires, so research would return an empty stream.

Fix: after the stream closes, flush the decoder and parse any remaining buffer content as a final frame even without the trailing \n\n.

Two SSE event shapes

The Tabstack API sends SSE in two formats depending on the endpoint:

# Format A (standard SSE)
event: complete
data: {"report":"..."}

# Format B (event field inside data JSON)
data: {"event":"complete","data":{"report":"..."}}

The parser handles both: first checks for a standard event: field; if absent, tries to unwrap the event key from the data JSON payload.

These fixes are covered by the test suite — the mock server sends CRLF frames, unterminated frames, and both SSE shapes, and the tests verify the correct events arrive.