CVE-2026-39414: Oversized S3 Select Records and a SIMD Bypass

Status: Released; second-round fix completed in June
Initial fix release: RELEASE.2026-04-17T00-00-00Z
Complete fix release: RELEASE.2026-06-18T00-00-00Z
Fix commits: 3252d5b, 5dc705936 (fd69c89)
GitHub issue: pgsty/silo#25

Our initial April patch (3252d5b) reused the existing 1 MiB maxCharsPerRecord limit for both CSV and standard JSON Lines in S3 Select, stopping unbounded buffering while waiting for a record delimiter and returning the S3 OverMaxRecordSize error code to clients.

A follow-up call-chain audit in June uncovered a hardware-dependent gap: on x86_64 CPUs with AVX2/AVX-512 support where simdjson.SupportedCPU() returned true, JSON Lines took a separate simdjson fast path that bypassed the bounded json.PReader completely. The June fix (5dc705936 / fd69c89) routed JSON Lines through the bounded reader on every CPU architecture, while also correcting streaming error codes, parser error wrapping, and pre-error output queue flushing.

Threat model

An authenticated caller with s3:GetObject permission can issue an S3 Select (SelectObjectContent) query against an object containing an arbitrarily long single record without a newline delimiter. If the record reader buffers bytes without an upper bound until it encounters a delimiter, a single query can exhaust server memory and CPU.

Hardware-dependent dispatch makes this class of bug harder to catch: the same S3 Select query can execute different parser implementations depending on the host CPU’s instruction-set flags. Passing a test on a non-SIMD machine does not guarantee that production hosts with AVX2/AVX-512 are protected.

First round: reusing the existing 1 MiB invariant

Rather than introducing a new configuration knob, the April patch (3252d5b) applied the existing maxCharsPerRecord = 1 MiB (1048576 bytes) constant across CSV and standard line-delimited JSON:

• the CSV splitter and line-delimited JSON reader rejected records exceeding 1 MiB before buffering or parsing them further;
• the reader preserved the earliest splitter error instead of letting a subsequent partial decode overwrite it;
• the error propagated to the client as OverMaxRecordSize rather than collapsing into a generic InternalError so clients and monitoring can distinguish an enforced record-size limit from a server fault.

Second round: the hardware-dependent SIMD bypass

Tracing the full S3 Select JSON dispatch chain in June revealed a second path for JSON Lines (DOCUMENT type LINES):

JSON Lines -> simdj.NewReader -> simdjson.ParseNDStream

Whenever simdjson.SupportedCPU() returned true, JSON Lines bypassed the bounded json.PReader and called simdj.NewReader. Inside simdjson.ParseNDStream, the third-party parser continued reading across chunk boundaries until it found a newline. Wrapping the underlying io.Reader in a byte-counting adapter could not solve the problem cleanly: because ParseNDStream reads ahead in chunks, a generic wrapper cannot simultaneously preserve already-completed records in the buffer and enforce an exact 1 MiB cap on the next record.

Rather than layering a brittle byte-counting wrapper around simdjson.ParseNDStream, commits 5dc705936 and fd69c89 disabled the SIMD NDJSON path and routed JSON Lines through the bounded json.PReader on all CPUs. If a SIMD fast path is reintroduced in the future, the SIMD parser itself must enforce the 1 MiB per-record limit and pass the same CPU-independent regression suite.

Stream semantics corrected in the same round

Reviewing the S3 Select streaming pipeline during the June fix also resolved four adjacent protocol issues:

• Interface-based error matching: Used errors.As to pass through any error implementing SelectError, rather than matching only a single concrete struct type.
• Typed JSON parser errors: Wrapped syntax failures in the JSON worker as JSONParsingError instead of returning untyped errors.
• Pre-error batch flushing: Flushed completed records still waiting below the batch threshold in the output queue before emitting a terminal error event, matching S3 Select streaming semantics.
• Input-order error priority: Preserved the first error encountered in input order so a later oversized record cannot overwrite an earlier parse failure on a preceding line.

Deliberately kept out of scope

To keep the security patch focused and avoid speculative behavior changes, three adjacent topics were left out of scope:

• the historical interaction between CSV AllowQuotedRecordDelimiter and the outer physical-newline splitter;
• whether \r in CRLF counts toward the 1 MiB record length limit;
• re-implementing a bounded SIMD NDJSON parser before having a design that enforces record boundaries inside simdjson.

Verification and release

The regression suite in internal/s3select tests oversized CSV and JSON Lines records, OverMaxRecordSize and JSONParsingError propagation, pre-error record flushing, and parser behavior across both SIMD and non-SIMD code paths.

go test ./internal/s3select/... -count=1

The initial fix shipped in RELEASE.2026-04-17T00-00-00Z (3252d5b), and the complete SIMD-independent fix shipped in RELEASE.2026-06-18T00-00-00Z (5dc705936 / fd69c89).

Final trade-offs

• Throughput vs. predictable bounds: Routing JSON Lines through json.PReader gives up SIMD NDJSON acceleration on AVX2/AVX-512 hosts in exchange for a strict 1 MiB per-record memory bound on every CPU.
• 1 MiB record ceiling: S3 Select queries against CSV or JSON Lines objects with individual records larger than 1 MiB are rejected with OverMaxRecordSize; callers must split records exceeding 1 MiB.
• Quoted multiline CSV: Quoted multiline CSV delimiter semantics remain tracked separately from this CVE.