SILO 20261026: Server, mcli and KES

SILO, mcli, and PGSTY KES 20261026 are now released, alongside SILO Console 2.5.1 and silo-pkg 3.15.2. This article consolidates the October 24–25 maintenance records and summarizes all changes since the 20261018 release. All three executables share the same release date and package version; KES remains an optional, independently deployed key encryption service.

Released components

All five projects build with Go 1.27.3. Server, Console, and mcli select silo-pkg v3.15.2 and the pinned upstream commit minio-go v7.4.2-0.20261025165926-320af050c445 (a pseudo-version rather than an upstream v7.4.2 tag). Server explicitly embeds the released Console and mcli sources; upgrading standalone Console does not replace the UI embedded inside an older Server binary. See the component matrix and downloads.

Server changes since 20261018

• LDAP and IAM (#250): Restores group-based login for LDAP users whose normalized DNs exceed 255 bytes in a single ASCII path component by treating missing optional IAM records on those paths as absent, while continuing to fail closed on unreadable revocation records or storage errors (writing overlong path records remains unsupported on the object backend). Policy reloads no longer hold the authentication-cache lock during storage I/O, swap the cache only after a complete load, and preserve IAM write ordering. Request cancellation now propagates to KMS and SSE-S3 decryption paths without shortening the caller’s policy-query deadline.
• Built-in policy Reset and live bucket inventory: Resetting an overridden built-in policy immediately restores the native policy definition on the next read, while delayed peer/etcd notifications reload committed state without clobbering newer edits (in-use checks, delete revisions, and custom-policy deletion remain unchanged). Console’s account-info endpoint now reads the live bucket inventory so newly created or deleted buckets appear immediately on refresh, while object usage totals remain scanner-derived.
• Multi-pool deletion safety (#257, #256): Ordinary and bulk deletes clean non-authoritative duplicate pools before removing the authoritative copy, select an available pool when writing new delete markers, and protect remote-tier payloads still referenced by surviving copies.
• Versioned lifecycle expiration (#253, #252): Retains remote tier payloads whenever lifecycle expiration only inserts a delete marker (including non-null versions in versioning-suspended buckets), reclaiming remote tier objects only when a historical version is permanently purged and no surviving reference needs its payload.
• Multipart replication (#255, #254): Forwards the FULL_OBJECT checksum type when replicating CRC multipart objects so the target cluster does not reject the replica with BadDigest.
• Storage RPC and timeout semantics: Forwards storage RPC deletion options (including the previous data directory needed for overwrite rollback) while preserving existing drive/RPC deadlines, request cancellation, object locking, read quorum, and healing rules. The final release withdrew the 20261025 draft’s admitted-I/O draining waits and per-object mutation guards (including #258) in dcc2f20ba; see the multi-pool object consistency record.
• Dependencies, packaging, and delivery: Updates SFTP, etcd, NATS, cloud SDKs, and other Go modules; preserves Kafka’s default protocol 2.1, CLI formatting, and NetBSD compatibility; and fixes legacy DARE v1 unsealing while keeping v2 writes. Container images, the verified installer, and Helm client jobs bundle mcli 20261026. Windows archives now ship as .zip, static curl is updated to 8.23.0 with the certificate-trust mitigation described below, and RPM signer verification is fixed during the final SBOM/provenance refresh.
• Documentation and migration: Incorporates the single-node MinIO migration guide (#228), reconciles operational defaults, and consolidates upgrade instructions under the migration guide.

This release retains all Object Lock, IAM authorization, durable STS revocation, pool-migration, and LIST recovery repairs shipped in 20261018. The dependency security inventory below separates newly adopted fixes from earlier repairs and scanner findings in unused code paths.

mcli changes since 20261018

• Client identity cache isolation (#50, #55): Isolates cached S3 and admin clients by credentials, TLS trust, proxy, custom resolver, and request headers while preserving credential refresh and token-file rotation.
• Replication resync status and throughput (#48, #49, #52): admin replicate resync status --json drains the status stream through terminal Completed, Failed, or Canceled states so completed jobs emit their final JSON record; explicitly reports when replication is disabled; stops waiting on Failed or Canceled in TTY mode; and fixes integer overflow in throughput calculations past ~8.6 GiB.
• Honest exit codes (#51): resync start and resync cancel retain per-bucket error details and return a nonzero exit code when any bucket fails, while resync status continues to exit 0 when successfully inspecting an already Failed or Canceled job. admin info --json also exits nonzero on request failure while preserving its JSON error output.
• Batch job observation (#53, #54): batch status continues watching when live metrics are not yet available, emits a pending status instead of treating queued or missing samples as complete, falls back to bounded persisted-status queries when the metric stream ends early, and fixes IOPs integer overflow on large object counts.
• Alias regions (#44): Honors persisted alias region settings with precedence MC_REGION → AWS_REGION → alias configuration → bucket-location discovery, and isolates signing regions across cached clients (without adding an alias set --region CLI flag).
• SDK fixes: Rejects truncated sequential or parallel multipart streams; preserves ReadAt EOF/negative-offset semantics and post-Stat read/seek bounds; deduplicates fallback batch-delete errors; and accepts both HTTP 200 and 204 when removing bucket replication configurations (while still reporting 403 and other real errors).
• Packaging and help (#45): Builds with Go 1.27.3 and GoReleaser 2.19.0, switches Windows archives to .zip, refreshes help URLs and disabled server-update/SUBNET messages, and preserves existing CLI configuration and MC_* environment variables.

Automation scripts should handle the pending batch state and nonzero exit codes on partially failed resync start/cancel commands. Upgrading mcli requires no Server protocol change. Mid-stream download interruptions still fail immediately; SIGINT during a multipart upload exits 130, and incomplete upload parts may require explicit cleanup. See the complete client commit history.

KES changes since 20261018

KES 20261026 rolls up the Go 1.27.3 toolchain and cloud-SDK dependency maintenance and aligns the executable, Linux packages, and container images with the 20261026 release batch. Its HTTP API, configuration schema, and stored-key format remain unchanged.

The 25 module updates also cover AWS support packages and Google generated APIs, while keeping Lip Gloss v1 and gopkg.in/yaml.v3. Packaging uses GoReleaser 2.19.0 and nFPM 2.48.0; Linux packages install /usr/local/bin/kes without auto-starting an unconfigured service.

Compatibility checks verified OpenBao KV v2 and local Moto Secrets Manager fixtures across pre-upgrade and post-upgrade encrypted objects, version rollbacks, and cold Server restarts. Preserve persistent keys, certificates, and configuration across upgrades, and verify object decryption after restarting Server. See the complete KES commit history.

Console 2.5 and silo-pkg 3.15

Console 2.5.1 delivers the 2.5 interface refresh, permission-aware navigation, Overview dashboard, bounded object previews, Tasks center, identity management, and high-latency diagnostic stream fixes; the 2.5.1 patch additionally preserves group members during status changes, OIDC string-array group permissions, secure session cookies behind TLS proxies, exact UTF-8 credential exports, streaming ZIP error statuses, and keyboard focus/accessibility labels. Server 20261026 embeds Console 2.5.1 directly.

silo-pkg 3.15.2 combines the 3.15.0 Go 1.27.3 and dependency refresh, 3.15.1 upstream SDK fixes, and 3.15.2 HTTP 204 handling when removing bucket replication, while retaining existing policy semantics and the 3.14.3 admin-action lookup optimization. As a Go source module, silo-pkg is adopted by rebuilding dependent applications.

Upgrade and failure boundaries

1. Coordinated upgrade: Back up IAM state, configuration, and encryption keys, and upgrade all Server nodes and replication peers together following the IAM upgrade guide. Running mixed old/new Server versions against a shared IAM backend or performing rolling downgrades remains unsupported.
2. Post-upgrade checks: Verify LDAP/OIDC login, built-in policy Reset, bucket list refresh, versioned reads, cross-site replication, and encrypted-object decryption. The FULL_OBJECT multipart replication fix requires upgrading the sending Server first and then checking its pending replication queue.
3. Archive formats and scripts: Windows archives now use .zip (Linux and macOS retain .tar.gz); update automation scripts for mcli batch status pending outputs and nonzero exit codes on partially failed resync start/cancel commands. Existing environment variables, configuration filenames, and on-disk formats remain unchanged.
4. Known multipart and issue scope: Public ListMultipartUploads continues to use the node-local in-memory cache and exact-key lookup, ignoring the retired strict switch with a startup warning. Issues #79 and #217 remain open, webhook hot-reload PR #249 is not included, and broader performance symptoms remain tracked in #251.

Storage timeout and consistency boundary: A client-visible request timeout does not guarantee that underlying disk or RPC I/O has stopped or that the write failed to commit. Late commits, delayed rollbacks, and stale values or tags remain possible after partial drive failures, lock lease expiration, or network partitions. This release does not add distributed lock fencing and cannot automatically recover previously lost objects, remote tier payloads, or metadata.

Downloads and verification

See the download matrix for platform archive filenames, SHA-256 checksums, Linux packages, and verification commands. Versioned container images are available on Docker Hub:

docker pull docker.io/pgsty/silo:RELEASE.2026-10-26T00-00-00Z
docker pull docker.io/pgsty/mc:RELEASE.2026-10-26T00-00-00Z
docker pull docker.io/pgsty/kes:RELEASE.2026-10-26T00-00-00Z
docker pull docker.io/pgsty/silo-console:v2.5.1

• Server standard image (Linux amd64/arm64 manifest): sha256:7bb2aff6b89efe1cb1303c10fc21d579447434ed06649ac9ec05a5e2dbf30ad7 (matches latest)
• Server distroless image (Linux amd64/arm64 manifest): sha256:e6544a902f426f2b4e775b89f3c1cd78310dbffc262d1ed82e9800c1cb1b5885 (matches distroless, with native silo healthcheck ready configured on both architectures; see the Server image workflow)

Each component’s artifacts are identified by its release commit, archive checksums, package signatures, and build provenance:

• Server artifact verification: Verified 36 release assets and 34 attested subjects (including six platform binaries and six Linux packages) built from c3ad836aed0d. Both RPM packages are GPG-signed, and the checksum manifests, SBOMs, and provenance attestations match the final packages (see the Go CI, VulnCheck, Release, and Finalize workflows).
• Local multi-node and fault-injection verification: Passed 88 baseline four-node, 88 upgraded four-node, and 136 eight-node multi-pool S3 checks on Linux arm64, preserving historical objects, tags, encrypted objects, and in-flight multipart uploads across the coordinated upgrade. Rolling restarts, SIGKILL, and a 75-second pause/recovery test passed with all 408 acknowledged writes read back with matching hashes and complete listings; the native trusted-TLS mcli suite and 16 persistence checks also passed. Because health endpoints evaluate reported drive quorum rather than per-node installed peer connections, run a data-path check before resuming traffic after a restart (see Startup Readiness Window).
• KES artifact and integration verification: Verified all 19 public assets, 11 attested subjects, and both signed RPMs. OpenBao KV v2 and Moto fixtures exercised the Darwin arm64 KES binary across upgrade, rollback, and cold Server restarts—first against SILO 20261018 and then against the final Server/mcli 20261026 Linux arm64 binaries. Both KES Linux container images match the archive binaries, with multi-architecture manifest digest sha256:ea36fcdd260c606eb89e693b0949aea73b149359605273a98fcf13c8433049cd (Release workflow · Image workflow).

Contributors and change history

Thanks to Denis Babchuk / @botdeniska for lifecycle, replication, and multi-pool deletion repairs; @910patin for long-DN LDAP login; Azat / @azataiot for reproducing high-latency health-report truncation; @MikeVL for reporting resync throughput overflow and JSON status output gaps; @mrjavadseydi for alias-region support and the webhook reload proposal; Gerald Combs / @geraldcombs for Windows archive feedback; @yupoet for the single-node migration guide; and @GhaziTriki, @devTeam-AiReg, and @sargarass for their bug reports and feedback.

The table below summarizes the community Issues and PRs merged, adopted, or tracked across the five component repositories during this release cycle:

The community contributor roll recognizes every human issue and PR author across all repositories. Full commit comparisons: Server, mcli, KES, Console, and pkg.

The sections below summarize the October 22–25 dependency and security audit records (dependencies shared across multiple components are listed once under the corresponding package):

Additional Go dependency updates on October 24

The Go toolchain remains 1.27.3. Compared with the October 22 candidate dependencies, this update includes 26 newly released third-party module versions, plus Arrow and codescan updates (28 modules in total):

Public API behavior and output formats remain unchanged; see the component table above for the final pinned module versions.

Included runtime security fixes

Go and certificate trust

CVE-2026-111943 / GO-2026-6809: Go 1.27.3 rejects known untrustworthy intermediate certificates in crypto/x509, protecting TLS verification even when host trust stores have not yet been updated. All five primary projects and the standalone debugging modules are built with Go 1.27.3.

In addition, the UBI container trust stores for SILO, mcli, and KES apply the root certificate blocklist from Red Hat RHSB-2026-012. Because blocking a root rejects every certificate chain anchored under that root, this container-level mitigation is broader than Go 1.27.3’s intermediate-certificate check; external services still using certificates chained to those roots must rotate their certificate chains. See Locally mitigated items below for the UBI CA package status.

SFTP dependency hardening

pkg/sftp 1.13.12 fixes three security issues disclosed on October 23. At the time of this review, none has a CVE identifier. They are recorded separately by GHSA and are not counted as new CVEs:

Because the latter two upstream advisories have inconsistent affected-range and patched-version metadata, this batch upgrades directly to 1.13.12 containing the code fixes and verifies them with regression tests (the Go vulnerability database snapshot remains at October 21 and has not yet ingested these October 23 advisories).

Static curl and linked libraries

The static curl binary bundled in SILO container images is upgraded from 8.22.0 to 8.23.0 (built with HTTP/2, SSH, and Brotli enabled over the OpenSSL backend). The table below lists all 21 CVEs from the official curl 8.22.0 vulnerability summary alongside their applicability to SILO’s static build:

The statically linked Alpine libraries are also updated to the following patched revisions (-rN denotes the Alpine security backport revision):

The previous SILO arm64 image already linked OpenSSL 3.5.9-r0, nghttp2 1.70.0-r0 (CVE-2026-58055), and zlib 1.3.2-r1 (CVE-2026-85091); this update enforces minimum version floors in the build script and verifies both architectures against the official Alpine security metadata.

Build and development security fixes

This section covers the Node.js, Vite, and code-generation toolchain used during SILO Console frontend compilation and testing (Console runs in production as a compiled Go server binary), as well as the standalone Go debugging modules in the server repository.

Node 24.22.1

The build environment updates Node.js from 24.22.0 to 24.22.1, addressing all eight Node 24.x CVEs in the October 2026 Node.js security release (excluding Node-26-only findings):

Node 24.22.1 updates bundled Undici from 7.29.1 to 7.30.1, resolving eight CVEs documented in the Undici v7.30.1 release notes:

Node 24.22.1 also updates bundled OpenSSL from 3.5.8 to 3.5.9, incorporating 13 fixes across DTLS, QUIC, CRLs, CMP, EC/SM2, and handshake memory safety: CVE-2026-84782, CVE-2026-35189, CVE-2026-35191, CVE-2026-42772, CVE-2026-54872, CVE-2026-54873, CVE-2026-54875, CVE-2026-72897, CVE-2026-75804, CVE-2026-75805, CVE-2026-75806, CVE-2026-77696, CVE-2026-84784.

Node 24.22.1 updates bundled c-ares from 1.34.8 to 1.34.9, resolving 11 upstream GHSAs that do not carry separate CVE IDs (GHSA-wpr3-8q3r-4q8c covers a connection-shutdown path omitted from the earlier CVE-2026-33630 fix and remains listed as No known CVE):

Frontend dependencies and code generation

In addition, go-swagger is updated from 0.33.2 to 0.36.6, resolving GHSA-8gmg-wf5j-hvf7 and GHSA-rg22-m539-jwvf (code injection during code generation when processing untrusted OpenAPI content types or enum values; first fixed upstream in 0.35.3 without a separate CVE ID). Generated files are synchronized with the updated tool. The generation process explicitly preserves the reviewed configuration-import binder and multipart consumer, retaining the existing file-size, filename and form-encoding behavior under regression tests.

Independent Go debugging modules

The standalone inspect, s3-verify, and xattr module graphs in the server repository previously lagged behind the root module. Updating their dependencies clears 37 CVE-numbered dependency findings and one non-CVE S2 advisory:

The table below links each finding to the Go vulnerability database and records the first upstream fixed version (the unfixed OpenPGP notice is listed separately under Build-stage and non-CVE advisories):

Outstanding upstream or distribution issues

As of 2026-10-23, the advisories below do not yet have an upstream or distribution release available. Rather than suppressing scanner rules, we record each finding alongside its local mitigation or runtime reachability.

Locally mitigated, awaiting upstream publication

UBI 9.8 findings without an applicable vendor fix

Scans conducted on 2026-10-22 against the SILO, mcli, and KES system-component acceptance images (containing the updated static curl and CA bundle on amd64 and arm64) each report 24 RPM findings across the same 12 unique CVEs (15 Medium and 9 Low per image, zero High/Critical, and no vendor FixedVersion). Red Hat security queries on 2026-10-23 continue to list the RHEL 9 source packages below as Affected or Fix deferred.

Because RPM scanners map advisories by source package (SRPM), defects in uninstalled CLI utilities or other CPU architectures are reported against shared runtime packages such as libgcc and glibc. The applicability of each entry is detailed below:

Build-stage and non-CVE advisories

• Alpine build-stage curl/libcurl remains 8.22.0-r0: As of 2026-10-23, the Alpine package indexes for amd64 and arm64 have not yet published 8.23.0. This host package is used only inside the temporary build stage to fetch SHA-256-pinned source archives; the runtime image ships the newly compiled static curl 8.23.0.
• GO-2026-5932: golang.org/x/crypto/openpgp is deprecated upstream with no fixed version and no CVE alias. While golang.org/x/crypto remains in the module graph, none of the target builds import the affected openpgp packages.

Historical repairs, not newly fixed vulnerabilities

Module-path binary scanners continue to match SILO against CVE-2025-27414, CVE-2026-33322, CVE-2026-33419, CVE-2026-34204, CVE-2026-39414, CVE-2026-40344, CVE-2026-41145, and CVE-2026-42600 because SILO retains the github.com/minio/minio module path and those upstream advisories specify open-ended version ranges. All eight issues were patched and verified in earlier SILO releases; see SFTP, OIDC, LDAP, replication metadata, CSV, Snowball, signature validation, and ReadMultiple.

Compatibility and dependency regression evidence

To prevent reproduced regressions in CLI table formatting, character-width calculation, and NetBSD cross-compilation, this batch pins tablewriter 0.0.5, go-humanize 1.0.1, go-runewidth 0.0.29 in the root modules, and go-systemd/v22 22.6.0 (while xattr retains its existing width-library combination). In addition:

• Kafka notification targets explicitly initialize the protocol version default to 2.1 before applying user settings;
• DARE encryption retains 32-byte keys and v2 writes while fixing buffer handling when unsealing legacy v1 payloads;
• pgzip retains the integrity check that returns unexpected EOF when a gzip checksum trailer is missing.

The October 22–23 candidate validation (built with Go 1.27.3 on both the rollback baseline and the candidate, with persisted-data acceptance run on a single-node four-drive Linux arm64 setup) passed:

• Full compilation and applicable -race and lint suites;
• Encrypted-object read and rollback verification across the old and new dependency stacks;
• Service restart, missing-shard healing recovery, and CLI output checks;
• End-to-end Console object uploads/downloads, ZIP archive previews, WebSocket sessions, and metrics charts.

The candidate sources with the October 24 dependency updates passed these additional checks:

• Full race suites for silo-pkg, mcli, and KES; API / pkg / cmd race suites for Console; the full SILO test suite and focused SFTP, IAM, and Parquet race tests. Lint, module verification, and dependency tidiness checks passed for all five projects.
• SFTP object reads, writes, listing, deletion, Unicode names, transfer block boundaries, and read-only data protection through the SILO driver against HTTP / HTTPS S3 backends. Upstream read-only server and symlink deletion regression tests passed. Both dependency baselines passed the new object-operation tests.
• JWX signature and time-claim boundary comparisons; 68 CLI output comparisons with no new differences, normalizing only existing metadata map ordering.
• Local etcd mTLS, account authentication, transactions, watches, leases, and old → new → old clients reading across server restarts; NATS / STAN message persistence, ACKs, durable subscription cursors, and reconnection.
• Local Azure / GCS protocol fixtures, KES GCP Secret Manager operations and pagination, the Azure SDK’s Arrow decoder tests, and the full upstream procfs parser fixtures. The procfs update corrects bcache units and NVMe sizes, so the corresponding metric values may change to reflect those corrections.
• Native Darwin arm64 and cross-compiled Linux amd64 builds of mcli, Console, SILO, and KES; NetBSD amd64 cross-builds of mcli and SILO.

Linux amd64 package-level Go vulnerability scans of all five projects found no affected imported packages (only the module-level GO-2026-5932 entry remains, with no imports of the affected OpenPGP subpackages).