Skip to content

This is the multi-page printable view of this section. .

Return to the regular view of this page.

Compatibility

Where Silo matches MinIO and where it deliberately differs — server, client, and console.

Silo is a community fork of MinIO. This section records what Silo keeps from MinIO, where it deliberately differs, and what either means when moving between them.

Contents, in reading order: the migration guide (scope and Docker), native package migration (RPM/DEB), the server compatibility audit, and the mcli client notes. The Feature Notes subsection covers the opposite direction: design records for capabilities Silo adds beyond upstream.

1 - Migrate from MinIO to Silo

What changes, what stays, and how to switch a container deployment. Package installations are covered in Native Package Migration.

Migrating from MinIO to Silo is an in-place binary replacement, not a data migration. Nothing is exported or re-imported. In a container deployment the only required change is the image name. For RPM/DEB installations, see Native Package Migration.

What changes

In order of importance:

  1. Container image: minio/minio, quay.io/minio/minio, and pgsty/minio are all replaced by docker.io/pgsty/silo.
  2. Package, systemd service, and server executable: miniosilo.
  3. Upstream services: the in-place updater and MinIO-operated callhome/SUBNET are disabled; upgrades go through packages, images, or your orchestrator.
  4. Default OS service account: silo — fresh installations only; migrations keep running as the existing data owner.
  5. Branding: banners, Console appearance, log wording, and product links say Silo.

What stays

  • Object data and the .minio.sys metadata directory — the on-disk format is unchanged and remains interoperable with MinIO in both directions.
  • Buckets, versions, users, access keys, policies, lifecycle rules, replication state, encryption metadata.
  • S3 API, SigV4 signing, SDKs, mc/mcli, presigned URL behavior.
  • Endpoint hostname, API port 9000, Console port, volume mounts.
  • MINIO_* environment variables and existing server options.
  • /minio/* routes, x-minio-* headers, minio_* metrics.

There is no data-conversion step. If your MinIO build is years old, validate the version distance itself in staging; it is a large software upgrade, not a format change.

Docker migration

Whichever image you run today, replace it with:

docker.io/pgsty/silo:<RELEASE-tag>

Tags: immutable RELEASE.YYYY-MM-DDTHH-MM-SSZ (pin these), rolling latest, and the -distroless variants below. The old pgsty/minio repository stays published, frozen at its final tag.

In Compose, change only the image line:

services:
  minio:                              # service name may stay "minio"
    image: docker.io/pgsty/silo:<RELEASE-tag>
    command: server /data --console-address ":9001"
    environment:                      # MINIO_* unchanged
      MINIO_ROOT_USER: ${MINIO_ROOT_USER}
      MINIO_ROOT_PASSWORD: ${MINIO_ROOT_PASSWORD}
    ports: ["9000:9000", "9001:9001"]
    volumes:
      - minio-data:/data              # same volume, same data
volumes:
  minio-data:
docker compose pull minio && docker compose up -d minio

The entrypoint translates the legacy first argument, so an inherited command: minio server /data keeps working. A hard-coded entrypoint: /usr/bin/minio must change to /usr/bin/silo. Existing mc ready local healthchecks keep working; the native replacement is test: ["CMD", "silo", "healthcheck", "ready"] (reference). Do not run docker compose down -v-v deletes the data volume.

Distroless variant

pgsty/silo:<RELEASE-tag>-distroless ships the silo binary only: no shell, no mc, no curl. It has a built-in HEALTHCHECK (the native probe) and works under any --user:

docker run -d --name silo \
  -p 9000:9000 -p 9001:9001 \
  -e MINIO_ROOT_USER=admin \
  -e MINIO_ROOT_PASSWORD=change-me-long-password \
  -v silo-data:/data \
  docker.io/pgsty/silo:<RELEASE-tag>-distroless \
  server /data --console-address ":9001"

The same deployment as a Compose file:

services:
  silo:
    image: docker.io/pgsty/silo:<RELEASE-tag>-distroless
    command: server /data --console-address ":9001"
    environment:
      MINIO_ROOT_USER: admin
      MINIO_ROOT_PASSWORD: change-me-long-password
    ports: ["9000:9000", "9001:9001"]
    volumes:
      - silo-data:/data
volumes:
  silo-data:

depends_on: condition: service_healthy works against it with no healthcheck: block. The volume format is the same as the classic image and MinIO — the variants are interchangeable over the same data. TLS certificates mount at /tmp/.silo/certs. If command-line flags move the listen address, point the built-in probe with MINIO_HEALTHCHECK_URL. There is no shell inside; debug with docker debug / kubectl debug.

Kubernetes

Kubelet probes are httpGet requests in the pod spec; Docker HEALTHCHECK is ignored, so both image variants are probed identically and existing probe configs keep working. For Helm releases, keep the release identity with nameOverride/fullnameOverride and compare helm template output before applying (details).

Rollback

The disk format is unchanged and works with both servers: set image: back to the recorded MinIO tag and docker compose up -d. The same volume stays attached, and data written by Silo remains readable by MinIO.

One cluster, one binary

Distributed nodes verify each other’s binary at bootstrap. A node started among peers running a different binary does not fail — it waits indefinitely in activating, logging:

Expected Silo binary checksum: ..., seen: ...
Waiting for at least 1 remote servers with valid configuration to be online

This applies to any pair of different binaries: MinIO next to Silo, and one Silo version next to another. So do not migrate — or later upgrade — a cluster node by node. Switch all nodes in one pass: stop the old binary everywhere, start the new one everywhere (in Compose: change the image for all nodes in one edit, docker compose up -d once). Single-node deployments are unaffected. The same applies to rollback. Rolling restarts of the same binary work normally; gate them with silo healthcheck --maintenance cluster (exit 0 = safe to stop this node).

Verification

silo healthcheck ready                   # this node serves; exit 0/1
silo healthcheck cluster                 # cluster-wide write quorum
mc admin info <existing-alias>           # all nodes online, new version, old alias

Then download a known object and compare its checksum, exercise one application through its existing SDK, restart the service once, and re-check.

2 - Silo vs. MinIO Server Compatibility

A code-verified compatibility audit of the Silo server fork: binaries, configuration, S3 and admin behavior, storage internals, packaging, containers, and Helm.

Silo is a maintained fork of the MinIO server. It preserves MinIO’s S3-facing and on-disk compatibility, but it is not a byte-for-byte, operationally invisible rename. This page is the compatibility contract for moving from the upstream baseline to the Silo source prepared on 2026-08-06.

Warning

Read this before replacing a MinIO deployment. The binary, package, service account, systemd unit, default local configuration directory, container path, Helm resource names, embedded Console, update behavior, several authorization decisions, and some error responses changed. Data disks and the MINIO_* configuration namespace did not receive a matching rename.

Audit scope and method

This is a source audit, not a compilation of release-note claims.

Boundary Audited value
Upstream baseline minio/minio@7aac2a2c5b7c882e68c1ce017d8256be2feea27f, 2026-02-11
Silo snapshot pgsty/silo@219670d3176a5b27ded60914390d5ee7e763cf58, 2026-08-06
Commit set 7aac2a2c..219670d3: 96 reachable commits; 93 were on origin/main and the final three were committed locally at audit time
Net source diff 523 files, 36,715 insertions, 21,450 deletions
Interpretation The behavior of the final snapshot. Intermediate changes later replaced or removed are not presented as current behavior

Every commit in the range was inspected. Release and security posts were used as an index of intended changes, then checked against the final implementation, tests, dependency graph, build recipes, package payloads, container entrypoint, and rendered Helm manifests. The complete commit ledger prevents a documentation-only or CI-only commit from silently falling out of scope.

The range is defined by Git reachability, not author-date sorting. It therefore contains d4cd4b433, authored in December 2025 but joined into the post-baseline graph later; this is not an extra undocumented baseline.

The tagged RELEASE.2026-08-04T00-00-00Z ends at d88f46cce, 18 commits before this audit head. Accordingly, this page records the 2026-08-06 prepared source state; it does not claim that the last 18 changes were already present in a public package, image, tag, or deployed website.

Executive compatibility matrix

Surface Status Practical result
S3 wire API Compatible with documented exceptions Routes, XML/JSON schemas, SigV4, S3 headers, ports, and ordinary error codes retain the MinIO contract. Security and correctness fixes below deliberately reject some requests previously accepted
Data disks Compatible .minio.sys, erasure metadata, bucket/object layout, healing, replication, and encryption formats keep their names and schemas. Poisoned or unusable metadata is now rejected earlier
Configuration Mostly compatible Existing MINIO_* variables, config keys, KMS/KES, IAM, notification, and storage settings remain. The default per-user directory becomes ~/.silo, with a deterministic ~/.minio fallback
Metrics and automation APIs Compatible minio_* Prometheus metrics, /minio/* routes, x-minio-* headers, admin/S3 error identifiers, and release tag syntax stay unchanged
Binary and distribution Renamed minio becomes silo; package, unit, image, chart, archives, checksums, and paths move to the Silo identity. There is no installed server-binary alias
Runtime identity Changed CLI text, banners, HTTP Server, User-Agent application names, FTP banner, log names, support links, and some human-readable errors say Silo
Upstream network services Disabled In-place update, update polling, callhome, SUBNET registration, and diagnostic uploads do not contact MinIO services
Authorization/security Intentionally stricter OIDC HMAC tokens, unsafe LDAP failures, forged replication metadata, object-only grants for protected bucket writes, shadowed policy inputs, ambiguous version IDs, and several malformed internode requests change behavior
Embedded UI and Go dependencies Forked behind compatible import paths Silo Console, MCLI, and Silo Pkg are selected with replace directives while github.com/minio/... module/import paths remain
Mixed-version cluster Not supported for this transition The private ReadMultiple storage-REST operation was removed without bumping storage REST v63. Upgrade all nodes as one build

What deliberately stays compatible

Protocol, storage, and configuration names

The following MinIO identifiers are compatibility identifiers, not unfinished branding work, and must remain visible:

  • the Go module path github.com/minio/minio and the inherited github.com/minio/... imports;
  • the MINIO_* environment namespace, including MINIO_ROOT_USER, MINIO_ROOT_PASSWORD, MINIO_VOLUMES, MINIO_OPTS, and notification variables;
  • S3 and admin routes under /minio/*, x-minio-* headers, MinIO-specific S3 extensions, and established API error codes;
  • Prometheus metric names under minio_*;
  • the .minio.sys internal volume and all existing disk metadata names;
  • the default S3 port 9000, existing --address / --console-address flags, and release tags of the form RELEASE.YYYY-MM-DDTHH-MM-SSZ;
  • configuration KV formats, IAM data, KMS/KES configuration, encryption metadata, bucket metadata, replication state, and healing state.

The automated rebrand baseline records 137 compatible imports, 436 environment names, 19 metric namespaces, 84 headers, 330 routes, one internal root, three Grid namespaces, 15 storage-REST identifiers, 58 policy identifiers, and 9,014 exported symbols. The guard treats an unreviewed change to that manifest as a compatibility failure.

No data copy or metadata rewrite is required when the same disks move from MinIO to Silo. This does not mean every malformed historical object is accepted: the storage hardening described below rejects unsafe paths, invalid erasure geometry, negative part sizes, and poisoned metadata that older code could carry farther into the stack.

Source compatibility

The server module remains github.com/minio/minio. Silo selects maintained forks without forcing callers to rewrite imports:

replace github.com/minio/console => github.com/pgsty/silo-console ...
replace github.com/minio/mc      => github.com/pgsty/mc ...
replace github.com/minio/pkg/v3  => github.com/pgsty/silo-pkg/v3 v3.11.0

This preserves most source compatibility, but it is not an assertion that every private or exported Go symbol is frozen. The internal ReadMultiple storage interface was removed, and the selected silo-pkg release has several developer-visible fixes described in Dependencies.

The maintained source remote is github.com/pgsty/silo on branch main; the former minio branch is archived. go install github.com/minio/minio@... still resolves the upstream project, not Silo, so clone the Silo repository or use an explicit module replace. Contributions no longer require MinIO’s CLA, but commits require DCO sign-off (git commit -s).

Identity, binary, and outbound-service changes

Surface Upstream baseline Silo snapshot Compatibility consequence
Server executable minio / minio.exe silo / silo.exe Scripts and absolute paths must change. Archives, packages, and images do not install a /usr/bin/minio server alias
Version output MinIO identity Silo release/commit/runtime, AGPL, upstream copyright, PGSTY modification copyright, and MinIO technology lineage Parsers should rely on stable fields, not grep for MinIO prose
Local config home ~/.minio ~/.silo for a new home See the fallback rules below; data disks are unrelated
HTTP identity Server: MinIO and MinIO application UAs Server: Silo; internal batch/fan-out/perf UAs use silo-* / Silo names Protocol headers such as x-minio-* remain unchanged; identity-sensitive monitoring may need an update
Human text MinIO banner, help, errors, examples, FTP greeting, support links Silo identity; examples prefer mysilo Exact-string log parsers and snapshots may change, not status/error codes unless listed elsewhere
Integration-visible labels MinIO NATS/Redis connection names and Veeam model NATS name Silo Notification, Redis CLIENT SETNAME Silo, Veeam model "Silo <release>" Broker dashboards, connection-name filters, and Veeam inventory display can change
KMS validation prose MinIO-branded conflict messages Brand-neutral “both KMS/KES/static-key configuration” messages Configuration rules are the same; exact-text automation can change
Updater Release polling and in-place update paths Permanently disabled MINIO_UPDATE is parsed but cannot re-enable it; admin update routes remain and fail stably instead of disappearing
Callhome/SUBNET Registration, callhome, support uploads, embedded MinIO support key Configuration is accepted for migration but forced off; no registration/upload/post; no fallback encryption key Remove automation that expects MinIO-operated services. Requester-key inspect encryption remains
Embedded Console Upstream snapshot had the Console stripped Silo Console v2.1.1, English/Chinese UI, Metrics V3, no SUBNET UI Browser behavior and assets change; the S3/Admin API boundary remains the server contract
Bundled client in OCI No maintained fork contract /usr/bin/mcli plus /usr/bin/mc -> mcli This mc is the client compatibility alias, never a server alias
Warm-tier probe Temporary object contains MinIO Same-length probe contains Silo! Only observable through backend inspection or a failed cleanup; protocol semantics do not change
Log rotation default minio-*.log silo-*.log Log collectors matching filenames must change

The deleted /api/health/upload reference was an outbound SUBNET URL path, not a local Silo HTTP endpoint. The compatibility change is that Silo no longer issues that POST; it is incorrect to describe this as removal of a server route.

Default configuration-directory selection

Unless --config-dir is explicit, Silo makes one decision at startup:

Home-directory state Selected directory Message
Neither directory exists ~/.silo none
Only ~/.silo exists ~/.silo none
Only ~/.minio exists ~/.minio informational legacy notice; no files are moved
Both exist ~/.silo ambiguity warning

--config-dir always wins. Unless --certs-dir is also supplied, the certificate directory follows the selected configuration directory. For deterministic automation, set --config-dir instead of depending on filesystem discovery.

The fork adds only three server configuration controls that materially change compatibility behavior; there is no parallel SILO_* replacement namespace:

Setting Purpose Default
MINIO_API_TRUSTED_PROXIES General source-address trust boundary unset: exact historical trust-any behavior
MINIO_IDENTITY_LDAP_STS_TRUSTED_PROXIES / LDAP key sts_trusted_proxies Source buckets used by LDAP STS failure limiting no trusted proxy; use the socket peer
MINIO_API_LEGACY_BUCKET_RESOURCE_MATCH Temporary rollback for the protected bucket/object IAM boundary off

Notification KV registration fixes do not rename their existing environment variables. MINIO_UPDATE, SUBNET, and callhome inputs are retained only as ignored/migration-compatible inputs as described next. SILO_OPTS appears only inside the generated inspect helper, not as a general replacement for MINIO_OPTS.

Updater, callhome, SUBNET, and inspect

  • Startup never polls dl.min.io; the release URL and upstream minisign root are absent.
  • MINIO_UPDATE values that request updates produce a warning and are ignored. Public and peer admin update handlers remain registered, returning MethodNotAllowed or the stable “in-place updates are disabled” error.
  • Legacy subnet and callhome keys remain parseable so an old config can start. Registration state is always false, Console SUBNET variables are unset, callhome is forced off, and no diagnostic or license payload is posted.
  • Inspect output is encrypted only for a requester-provided public key. It no longer falls back to MinIO’s built-in support key. The helper is start-silo.sh, invokes silo, and includes cluster.info only in the requester-key flow.
  • Upgrade through a package manager, an image rollout, or an orchestrator. Do not call mc admin update / mcli admin update against Silo as an upgrade mechanism.

Installation and deployment compatibility

RPM, DEB, and APK

The package is silo for Linux amd64/arm64. Its relevant payload is:

/usr/bin/silo
/usr/lib/systemd/system/silo.service
/etc/default/silo                 (config, noreplace)
/usr/lib/sysusers.d/silo.conf
/usr/share/doc/silo/LICENSE
/usr/share/doc/silo/NOTICE

The package creates a system silo:silo account without a home. It does not chown existing data, migrate ownership, stop a running MinIO service during installation, or declare package-manager Provides, Obsoletes, Replaces, or Conflicts against the minio package. Both packages can therefore be installed, but their services cannot run together through the shipped units.

silo.service has Conflicts=minio.service, runs as silo:silo, and reads /etc/default/minio first and /etc/default/silo second. The shipped Silo file has no active assignments, so a legacy file continues to work until an administrator overrides it in the later file. It starts:

/usr/bin/silo server $MINIO_OPTS $MINIO_VOLUMES

Before switching the unit, make every data, certificate, KMS credential, and environment-file path readable by silo and every writable path writable by it. An old deployment owned by minio:minio will otherwise fail at startup. Package removal stops/disables silo.service; package upgrades do not deliberately stop the running service in the pre-remove hook.

Container image

The source repository is github.com/pgsty/silo; the intended image name is docker.io/pgsty/silo. There is no registry-level promise that pgsty/minio redirects to it.

  • The server exists only at /usr/bin/silo; an explicit /usr/bin/minio ... command breaks.
  • The entrypoint translates a first argv word of minio to silo, and prepends silo when argv starts with server, fmt-gen, or an option. Consequently the common command: minio server /data form continues to work.
  • An explicitly requested shell or utility is left alone.
  • Every privilege path uses exec, so the server becomes PID 1 and receives SIGTERM for graceful shutdown instead of timing out behind the entrypoint.
  • HOME=/tmp is the image default and is normalized to a writable directory for arbitrary-UID and legacy MINIO_USERNAME drop-user execution.
  • Port 9000, /data, and the MINIO_* interface remain. The amd64/arm64 image manifest also contains checksummed MCLI RELEASE.2026-08-04T00-00-00Z and the client-only mc symlink.
  • OCI license material is under /licenses/{LICENSE,NOTICE,CREDITS}.

Helm chart

The inherited helm/minio chart, helm-releases, root chart index, and reindex helper were removed. The maintained chart is helm/silo, chart version 7.0.0.

Most values deliberately keep their established names, including minioAPIPort, minioConsolePort, and all MINIO_* environment settings. The changes that matter during migration are:

  • image repositories become pgsty/silo;
  • generated resource names and labels follow chart name silo;
  • default service account becomes silo-sa;
  • certificate/client mount paths move from /etc/minio/{certs,mc} to /etc/silo/{certs,mc};
  • no insecure console/console123 user is created by default (users: []);
  • post-job examples prefer alias mysilo, while myminio is also registered so inherited customCommands can still resolve it;
  • the new chart executes silo, so an image-only rollback to an old MinIO image is not safe.

To preserve the old Kubernetes object identities while adopting the new chart, start from the exact old values and set at least:

nameOverride: minio
fullnameOverride: <the-old-full-release-name>   # for example my-release-minio
serviceAccount:
  name: minio-sa
image:
  repository: pgsty/silo
mcImage:
  repository: pgsty/silo

Render both charts and compare Services, selectors, StatefulSets/Deployments, PVC templates, Secrets, service account, storage mounts, environment, and ports before applying. Roll back chart and image together.

Archives, provenance, and legal files

Release archives are named silo_<version>_<os>_<arch> and contain the executable, README, LICENSE, and NOTICE. The checksum manifest is silo_<version>_checksums.txt; each archive receives an SPDX JSON SBOM, and the checksum set is accompanied by a keyless Sigstore bundle. Static, CGO-disabled, kqueue-tagged binaries are published only for Linux, macOS, and Windows on amd64/arm64; formerly compile-checked but unshipped architectures are no longer release gates. RPM/DEB/APK are built for Linux amd64/arm64, with timestamp package versions such as YYYYMMDDHHMMSS.0.0 (RPM release 1) and a separate package checksum manifest. Normal builds no longer stamp the build host’s GOPATH/GOROOT, improving reproducibility and removing path leakage.

Unlike the upstream baseline Docker recipe, which downloaded and verified a prebuilt dl.min.io server, the Silo release image consumes the exact source-built, attested release archive at the selected tag. Image publication is a separate, explicitly dispatched workflow after a GitHub release; build success alone does not publish it.

CREDITS is regenerated from the modules actually linked into the server and guarded in CI. It is included in the OCI image; it is intentionally omitted from packages and archives because of its size. Upstream AGPL and copyright notices remain, alongside PGSTY’s modification notice.

Runtime and security behavior changes

These are user-visible changes even when they close a vulnerability. “Stricter” means a request, policy, token, configuration, or corrupted internal message that formerly succeeded or failed differently can now be rejected.

Authentication, IAM, and request identity

Change Final behavior Who must act
OIDC JWT verification (d24f449e0) The client secret is no longer a verification key. Only asymmetric JWKS algorithms RS256/384/512, ES256/384/512, RS3256/3384/3512, and ES3256/3384/3512 are accepted. HS256/384/512 tokens fail; unknown kid still triggers the established JWKS refresh/retry path An IdP signing Silo tokens with HMAC must migrate to an asymmetric JWKS key
LDAP STS errors (3b950f8fa) Unknown user and bad password share one external InvalidParameterValue authentication failure. LDAP infrastructure failures remain server errors and are logged Clients must not distinguish account existence from response text
LDAP STS rate limiting (18b712d49, 9e10f6d9a, f44110890, 5e40665ac) Per-source, per-node in-memory bucket: burst 10, refill one per 6 seconds, idle TTL 15 minutes. Only authentication failures consume tokens; success and infrastructure failures refund them. Exhaustion returns HTTP 429, ThrottlingException, Retry-After: 6 Proxies should configure MINIO_IDENTITY_LDAP_STS_TRUSTED_PROXIES; it is separate from the general source-address setting
LDAP trusted-proxy source For an allow-listed socket peer, clean X-Real-IP is preferred; otherwise XFF is walked right-to-left past trusted hops. RFC 7239 Forwarded is ignored. A proxy must overwrite X-Real-IP Review ingress header sanitation; the limiter is not a distributed account lockout
LDAP service-account lookup “User DN not found” matching is case-insensitive, preserving the intended Admin no-such-user / login-name error classification across dependency message capitalization Only brittle clients that depended on the accidental misclassification see a difference
Bucket/object IAM boundary (97b7d2804) Twelve protected bucket-write actions no longer inherit an Allow from only arn:aws:s3:::bucket/*; the bare bucket ARN is required. Deny/NotResource and built-in * policies retain their semantics Add arn:aws:s3:::bucket to custom policies that legitimately perform protected bucket writes, or use the temporary global escape hatch MINIO_API_LEGACY_BUCKET_RESOURCE_MATCH=on
Protected actions Delete/ForceDelete bucket; put/delete bucket policy; put replication, lifecycle, object-lock, versioning, or CORS; delete CORS; put bucket QoS or inventory configuration Read/list, create bucket, tags, encryption, and notifications keep the inherited matching behavior
Effective policy inputs (2f55347f7) Server-derived conditions cannot be shadowed by a same-named header/query parameter. Exact keys win in the policy package. Existing/request tags, storage class, content/copy/checksum, object-lock, signature age, and list parameters are sourced from the value actually consumed by the operation Policies that accidentally depended on attacker-controlled shadow values stop matching
Request tags PutObject, CreateMultipartUpload, and PutObjectTagging bind s3:RequestObjectTag/* to the parsed input. ExistingObjectTag comes only from stored metadata. Other operation paths keep the inherited header fallback Re-test tag-conditioned write policies
s3:signatureAge Present only for verified presigned SigV4 requests Raw x-amz-signature-age injection no longer creates the condition
s3:versionid (744a9dcd7) Absent means absent, whitespace is normalized, and DeleteObject/MultiDelete uses the effective per-object version. A URL version cannot decoy a different XML version Re-test version-conditioned delete policies and any policy relying on Null
Replication metadata (56fa63bfd) Ordinary PUT/COPY cannot inject internal replication status/time metadata. An authenticated replica request must have ReplicateObjectAction; multipart and Snowball replication flows retain their legitimate metadata Custom replication callers must use the authorized replication path

The bucket-boundary escape hatch is startup-global and all-or-nothing. It exists for migration, not as a permanent mixed-policy mode. An empty or unparseable LDAP source is deliberately not placed into one shared limiter bucket; it is not throttled until a usable source can be derived.

General source-address trust

fe6dc4780 adds MINIO_API_TRUSTED_PROXIES, because the chosen client address feeds aws:SourceIp, audit remotehost, event Host, admin trace, and node-to-node forwarding.

Value Result
unset Exact inherited behavior: trust source headers from any peer; left-most XFF, then X-Real-IP, then Forwarded
none or off Ignore all three source-address headers and use the TCP peer
IP/CIDR list Trust headers only when the TCP peer is listed; walk XFF right-to-left past trusted hops (maximum 100), then use the last X-Real-IP line, then walk Forwarded right-to-left

Malformed entries, a non-empty list naming no proxy, or a remote env:// read failure fail closed and stop startup rather than reverting to trust-any. Invalid values inside a received chain are skipped; no usable address falls back to the peer. Loopback is implicitly trusted only for the FTP/SFTP peer bridge, not skipped as an arbitrary hop inside a client chain.

The inherited _MINIO_API_XFF_HEADER=off retains its exact old semantics and initialization timing: it disables only XFF parsing, not X-Real-IP or Forwarded, and therefore is not a security boundary. Configure every cluster peer that legitimately forwards requests and ensure edge proxies overwrite or strip client-provided source headers.

S3 request and response behavior

Area Change and compatibility effect
Presigned streaming auth A query/presigned SigV4 request declaring STREAMING-UNSIGNED-PAYLOAD-TRAILER is rejected with SignatureVersionNotSupported; it cannot fall through to anonymous authorization. Header SigV4 is verified before body processing
Snowball Authorization happens before tar extraction; the streaming-trailer bypass cannot write objects before a later failure
S3 Select record limits CSV input, JSON Lines input, and output records over 1 MiB return an OverMaxRecordSize event. JSON Lines always uses the bounded reader (possibly slower on SIMD-capable CPUs), JSON parse errors use JSONParsingError, and already completed records can precede the terminal error
Streaming responses The tracking writer implements Flush; Write/Flush records an implicit HTTP 200. ListenBucketNotification/watch streams and S3 Select keepalives reach clients, while audit/status metrics record the committed status correctly
Multipart full-object checksum FULL_OBJECT CRC32/CRC32C/CRC64NVME completion may omit every per-part checksum. If any are supplied they are still checked; COMPOSITE still requires every part. A zero-byte multipart object’s checksum is retained correctly
Multipart part ordering Duplicate or non-increasing part numbers fail with InvalidPartOrder before assembly. Gaps and a first part other than 1 remain legal; the upload stays available for retry
Erasure read pooling Correct shard-buffer ownership is restored, avoiding loss of pooling and a wrong-buffer association that could cause hangs, corruption, or severe performance loss
Update buffers Returned download buffers are owned correctly; the public updater was subsequently disabled, so no current supported upgrade path exercises this code

Distributed storage and private APIs

These changes are normally invisible to an S3 client but are compatibility changes for mixed clusters, custom internal callers, corrupted disks, and adversarial peers.

  • ReadMultiple and its private storage-REST /rmpl endpoint, client method, exported Go types, and metric were removed. External S3 List/Get operations are unchanged. storageRESTVersion remains v63, so do not infer mixed-node compatibility from the version number.
  • Every remote StorageAPI path field, nested metadata name, and raw-volume sink is validated at the storage boundary, including peer-S3 Grid messages. Lexical traversal, volume-root aliases, and Windows separator/drive forms are rejected.
  • Erasure geometry, non-positive blocks, negative part sizes, and unusable stored erasure metadata are rejected at all decoded sinks and during CheckParts/VerifyFile.
  • Internode allocation declarations are bounded: AppendFile caps preallocation at 1 MiB while still accepting the body; DeleteVersions grows as it decodes and rejects negative declarations; legacy ReadFile is capped at 5 GiB.
  • Deadline-bounded work converts worker panics into errors and logs a bounded stack instead of taking down the process.
  • ReadParts keeps the actual backend error across keepalive frames. An empty part list returns a successful empty result without a trace panic or leaked goroutine.
  • HTTP stream helpers left orphaned by ReadMultiple were deleted later; that cleanup creates no additional public behavior change.

This containment is lexical. It does not resolve filesystem symlinks, and native Windows server CI was not available for the audit. Upgrade every node together and keep untrusted clients away from the internode port even though root credentials and validation protect it.

Notification configuration and audit output

  • NATS now registers parser-consumed user_credentials, nkey_seed, and tls_handshake_first; AMQP registers immediate. Existing environment variable names stay unchanged.
  • The old literal MINIO_NOTIFY_NATS_USER_CREDENTIALS remains accepted for NATS only. Precedence is environment, new key, then old migration key.
  • AMQP legacy migration maps immediate correctly. Invalid notification errors identify names but no longer echo credential values.
  • Generated PostgreSQL notification DSNs now quote/escape every libpq value and use the correct user keyword. An explicit connection_string is passed through unchanged.
  • Dangling-object deletion audit events once again include per-drive errors under merrs.

One inherited migration risk is intentionally documented, not disguised as fixed: PostgreSQL/MySQL legacy notification migration can write host/port/user/password/database keys that are not registered by the current target schema, potentially disabling all targets on the next load; historical passwords may be stored in plaintext. Review legacy notification KV state before restarting into Silo.

Toolchain, dependencies, and embedded components

The build declaration moved from Go 1.24 plus a 1.24.8 toolchain to go 1.26.5. That can change TLS, HTTP, DNS, scheduler, garbage-collector, and standard-library edge behavior even where no Silo source line changed. Security-sensitive dependencies were also advanced, including Go-Jose, OpenTelemetry, Go crypto/network modules, cloud SDKs, etcd, NATS, and compression libraries.

Observable edge corrections inherited through those updates include Go TLS/X.509/URL/archive fixes; MQTT oversized UTF-8 packet encoding; malformed Azure NTLM challenge handling; Thrift framed transport and 32-bit compilation; NATS authentication, authorization, identity, and denial-of-service fixes; and Prometheus remote-read/write and UI hardening. They are dependency behavior changes, not a promise that every advisory path is reachable from Silo. The jsonparser CVE-2026-32285 investigation produced no patch: the resolved v1.1.2 already contained the fix and no vulnerable reachable symbol was found, so it creates no compatibility delta in this range.

Important deliberate dependency decisions are:

Component Final selection Compatibility rationale / effect
Console pgsty/silo-console v2.1.1 behind github.com/minio/console Restores the embedded UI, applies Silo branding and bilingual text, adds Metrics V3, removes SUBNET flows, and fixes untranslated metric legends
Client library pgsty/mc behind github.com/minio/mc Keeps Console’s import path while consuming the maintained MCLI fork
Shared package pgsty/silo-pkg/v3 v3.11.0 behind github.com/minio/pkg/v3 Supplies the IAM exact-match half, LDAP TLS/StartTLS/deadline/close fixes, certificate-watcher cleanup, and RNG fixes
Kafka Sarama 1.45.1 Pinned to avoid a breaking broker-negotiation drift
PostgreSQL lib/pq 1.10.9 Pinned to avoid a nil-[]byte / PostgreSQL-before-14 behavior regression; generated DSN quoting is fixed in server code
Compression klauspost/compress 1.18.7 Explicit security/correctness upgrade
Thrift 0.24.0 Fixes 32-bit builds
systemd library require 22.7, replace with 22.6 Retains NetBSD compilation until the monotonic-clock regression is fixed upstream

The LDAP package now honors TLS fields for ldaps://, keeps StartTLS active even with server_insecure, avoids a nil-TLS panic, applies a StartTLS deadline, and closes a connection after failed StartTLS. Certificate-watcher shutdown no longer leaks; on Windows, polling can delay reload by up to about ten seconds. RNG subkey entropy/reset behavior is corrected, although the server does not exercise the reset path.

For external Go consumers of silo-pkg, two changes are broader than this server’s own call paths: xtime.Duration JSON moves from integer nanoseconds to duration strings, and some AIStor action vocabulary / protected-action helpers differ from upstream. In particular, Policy.IsAllowedActions can disagree on protected actions; the server does not call it. The server stores the relevant state through YAML/msgp and does not call the differing AIStor/action helper paths, so no server data migration or authorization delta was found from those library changes.

Generated String() files were regenerated under the new toolchain. Valid enum output remains the same; the diff is generator provenance and invalid-value formatting machinery, not a separately claimed S3 behavior change.

Known residual risks and non-fixes

This audit does not turn inherited limitations into claims of compatibility:

  1. Source IP is still forgeable by default. Unset MINIO_API_TRUSTED_PROXIES deliberately retains upstream trust-any behavior. Set none for direct deployments or an exact proxy allowlist for proxied deployments.
  2. Some version-condition gaps remain. MultiDelete governance-bypass reauthorization still consults the query/absent version rather than each XML entry, and Snowball reads the PAX minio.versionId after per-file authorization. Empty username, userid, signatureversion, and authType condition keys are also still inserted, so Null on them has present-empty semantics.
  3. Multipart parser defense is not complete. Handler-level ordering is fixed, but the object layer has no independent uniqueness defense; XML-root validation and the inherited nonnumeric-part error mapping were not changed.
  4. Legacy notification migration remains risky. Review it as described above.
  5. Storage path validation is lexical. Symlinks are not resolved; native Windows execution was not independently covered.
  6. Private APIs are not a stable compatibility promise. ReadMultiple proves that a same-numbered storage REST protocol can still lose an operation. Do not run a rolling mixed build across this boundary.
  7. A source result is not a released artifact. This page does not assert that GitHub tags, packages, OCI manifests, signatures, or the public site contain the three audit-head-only commits until each channel is verified separately.
  8. Informational HTTP responses remain imperfectly tracked. The response-tracking layer treats a 1xx response as final. The Flush/implicit-200 change did not introduce this behavior and does not claim to fix it.

Migration checklist

For a MinIO-to-Silo move, use this order:

  1. Record the exact MinIO binary/tag, chart and values, image digest, package payload, service unit, environment files, config directory, data ownership, IAM policies, OIDC/LDAP settings, notification targets, and proxy topology.
  2. Back up configuration and IAM metadata. Silo reads existing disks in place, but a rollback still needs the old executable/config/chart and unchanged data ownership available.
  3. Replace invocations of the server binary with silo; do not assume /usr/bin/minio exists. In containers, argv-level minio server is translated, but the absolute path is not.
  4. Decide the config directory explicitly. Reuse --config-dir ~/.minio or let the legacy-only fallback select it; do not create an empty ~/.silo accidentally and then wonder why the old configuration is ignored.
  5. For packages, grant silo:silo access to data, certificates, secrets, and logs. Move deliberate overrides into /etc/default/silo; understand that it overrides /etc/default/minio.
  6. For Helm, render the old and new charts with the complete old values, preserve names with nameOverride / fullnameOverride / serviceAccount.name where required, and change chart plus image atomically.
  7. Remove updater, callhome, SUBNET-registration, and support-upload automation. Replace it with package/image/orchestrator rollout and your own diagnostic transfer path.
  8. Change HMAC-signed OIDC tokens to asymmetric JWKS. Exercise success, bad-password, unknown-user, backend-failure, and rate-limit LDAP paths.
  9. Add bare bucket ARNs for the twelve protected actions. Test effective tag, signature-age, source-IP, and per-version delete conditions. Use the legacy bucket switch only as a temporary rollback lever.
  10. Set MINIO_API_TRUSTED_PROXIES=none or an exact allowlist, sanitize all three source-address headers, and include cluster peers that forward authenticated requests.
  11. Test oversized S3 Select records, streaming notifications, unsigned-trailer rejection, multipart full-object checksums, duplicate parts, replication, healing, KMS, every notification target, audit ingestion, and graceful container shutdown.
  12. Upgrade all distributed nodes as one build. Keep the old chart and image paired for rollback; never roll back only one of them.

Verification evidence

The audit used the final source, not prose alone. At the recorded snapshot:

Check Result / boundary
Commit enumeration 96/96 commits classified in the ledger below; origin/main accounted for 93 and the local prepared head for three more
Net diff review All 523 changed paths classified across server runtime, internal protocol, dependency, delivery, documentation, tests, or superseded changes
Rebrand compatibility guard Passed; compatibility manifest and delivery/runtime assertions unchanged, including Docker argv tests
Go test suite Full go test ./... passed against 219670d31, including cmd, OIDC, LDAP, notify, event targets, Grid, handlers, hash, and all S3 Select packages
Helm migration buildscripts/verify-helm-migration.sh passed: lint, render, legacy upgrade, archive, and identity comparison across seven rendered resources
Package lifecycle buildscripts/package/lifecycle_test.sh passed; package payload/provenance assertions cover empty DEB conflict metadata, unit/default paths, and legal files
Site Strict make check passed: module verification, warning-fatal Hugo render (617 EN / 615 ZH pages), and 388,962 internal references across 1,084 HTML files; git diff --check, bilingual anchors, and 96/96 commit coverage also passed

Security articles contain deeper threat models and test vectors, but their historical “released/unreleased” labels describe their publication date. Where they conflict with the final snapshot, this page’s audited boundary is authoritative.

Complete commit coverage ledger

The hashes are in graph order. Merge, documentation, test, and CI commits are included because delivery behavior and the strength of a compatibility claim are themselves user-relevant; “no independent runtime delta” means exactly that, not “not reviewed.”

Class Commits Verified net effect
Initial fork, Console, CI, dependency base d4cd4b433, 8630937e7, 68521b37f, 00f3cf74f, 5abd9a80f, 377fc616d, f2f9a40dc, ee55e5391, ce1c537eb, 68e0ba997, 1869bd30b, ff58df949, e4fa06394 Go/SDK evolution; embedded Console restoration/fork; MCLI in OCI; replacement CI; LDAP TLS regression fix; security dependency upgrades. The two merge commits add no delta beyond their parents
April security series d24f449e0, 3b950f8fa, 56fa63bfd, 3252d5b7f, f444b6f37, efb6e5b00, db4c0fd5e, 18b712d49, 9e10f6d9a, f44110890, f48dbe777 OIDC, LDAP STS, replication metadata, S3 Select, unsigned-trailer/Snowball, Go 1.26.2, limiter accounting/source hardening, and security documentation
May–June reliability and private API 65795ee1f, 5e40665ac, fd69c89d0, 73ac52472, df627ff89, 3e61b1d3a, d495d30d5 HTTP Flush, final LDAP bucketing, full S3 Select bound, ReadMultiple removal, Go 1.26.4/dependency update, and documentation-link change
Pre-August component integration ce01ccbdc, 4dfc27ce3, b7f52ca43, 7babc0c39, c1aec0518, 15fcc3c8a, 3f192f3f0 Historical chart image switch, security dependency upgrade, notification-stream merge, portable dependency pins, compression, MCLI replacement, Console v2.0
August runtime correctness/security c8590413f, 3e14733f1, 924717926, 89d346bf5, 8069a32ac, a36fd8fff, ca7baa670, 80e8eaa42, b6f70ab08, 1af351a70, 38366f654, 22c1e41fd, 97b7d2804, 2f55347f7, 744a9dcd7, fe6dc4780, 162ded343, 0c14d8151, 9dd1dc172, 2602177ef Multipart, erasure buffers, response commits, panic containment, path/metadata/allocation/ReadParts containment, orphan cleanup, IAM/effective values/version ID/source trust, notification/libpq, and audit details
Chart hardening and audit documentation dfe669862, 5f4513fd4, b42ee4e8a, 8eae745ab Secure chart user default, portal/doc routing, maintainer ignore rules, and advisories; only the chart default changes runtime delivery behavior
Release engineering through the 20260804 tag 9c799f42d, 10c7670b8, cf7df097b, 32863c852, 632ade111, 1814ae52f, 475236c79, 11d79fddc, 3b8a55dee, ca674a696, 4c185d5a6, 2ca4971d9, e064b5555, aa5139369, 021110b45, d88f46cce RPM/DEB/APK, provenance, OCI publishing gates, pinned lint/generation, S3 Select test-race fix, broad CI, PID-1 signal fix, published-target cross-builds, safe release dispatch, reproducibility, stale-config removal, systemd location, honest gates, and runtime-image shutdown assertion
Silo cutover and 2026-08-06 prepared head 15def34dc, 77bdc4c0c, 15ab10833, 30749911b, e071bb77e, bd8df5166, 6613c2a3c, fd2ca1c6d, c46b16ec6, c47733abc, f1c77d5a2, 62717d7bf, 6740e6978, b57275be3, 05be686b8, a6d6d9b02, 6bd9cf77e, 219670d31 Removes unpublished MinIO delivery residue; Silo runtime identity/offline boundary; renamed packages, OCI and Helm with migration guards; pinned fixtures; docs/repository cutover; Console 2.1.0 then 2.1.1; Node 24 actions; DCO/legal/docs polish; regenerated CREDITS; LICENSE/NOTICE delivery

The ledger totals 96 unique commits. Changes replaced inside the range—such as Console 2.0 → 2.1.0 → 2.1.1, the historical pgsty/minio image/chart state, and updater-buffer code after the updater was disabled—are described only where they leave a final compatibility consequence.

See also

3 - MCLI Client Compatibility Notes

Differences between the pgsty/mc and upstream minio/mc

mcli is Silo’s build of the MinIO Client (mc). This page records where the two are interchangeable and where they differ.

pgsty/mc forked from the upstream minio/mc at its final commit, 77f82e18 (2025-11-06). The upstream repository was archived in July 2026 without ever cutting a release that contains that commit — so every mcli release is strictly newer than any official mc binary ever published. Fork releases to date: 20260313, 20260321, 20260417, 20260804, and 20260806.

Principles

The fork follows one rule: the shipped artifact and its channels are renamed; the tool you use is not.

  • Renamed / replaced — the artifact name on disk (mcli), the product identity in --version and --help, the distribution channels (GitHub pgsty/mc, the Pigsty repository, docker.io/pgsty/mc), and the signing keys. Not the command syntax, and — depending on how you install it — not even the name you type.
  • Unchanged — every command, subcommand, and flag; S3 and admin API behavior, request signing, and protocol headers (x-minio-*); JSON output schemas; exit codes of normal operations; the configuration file format and alias semantics; MC_* environment variables (including MC_HOST_<alias>); the .part.minio resume suffix; and the Go module path github.com/minio/mc.
  • Severed — every connection to MinIO-operated services: the release/update feed, the SUBNET support and licensing portal, telemetry, and the pre-seeded play demo alias. Affected commands remain in the CLI for script compatibility and fail with a stable error rather than disappearing.
  • Preserved — upstream copyright and the AGPL-3.0 license. Runtime output credits both MinIO, Inc. and PGSTY.

A configuration written by upstream mc is readable by mcli unchanged, and vice versa; both clients can talk to MinIO servers, Silo servers, and any other S3-compatible endpoint.

What changed

Ordered by how likely each change is to affect you, most likely first.

1. The name — what you type, and where the config lives

For many users nothing changes here: the container image keeps mc as its entrypoint, and a binary installed under the name mc behaves identically to upstream. What changed is what we ship — archives and Linux packages install the binary as /usr/local/bin/mcli (package name mcli).

Neither name is hardcoded anywhere. Since 2016 the upstream client has derived its runtime identity from the name it is invoked as, and mcli is the exact rename upstream’s own CONFLICT.md recommended (issue #873) for the Midnight Commander clash — this fork merely promoted that suggestion to the official shipping name, with zero code changes. What that mechanism means in practice:

Follows the invoked name Fixed, regardless of the name
Configuration directory: ~/.mc vs ~/.mcli (Windows: %USERPROFILE%\mc\ vs …\mcli\) Environment variables: always MC_* — there is no MCLI_CONFIG_DIR
Program name shown in help and usage text config.json format — identical and interchangeable in both directions
Shell-completion registration All commands, flags, JSON output, exit codes
User-Agent application suffix (mc/… vs mcli/…) --config-dir and MC_CONFIG_DIR overrides

The one real trap: run mcli for the first time and your existing mc aliases are not there — it starts from an empty ~/.mcli. Either keep invoking it as mc (a symlink suffices — argv[0] is what counts), or copy the state once with cp -a ~/.mc ~/.mcli. For automation and configuration templates, set MC_CONFIG_DIR explicitly: the environment prefix does not follow the name, so one template serves both. Details in Migration.

Get it from GitHub Releases (SHA-256 mcli_<version>_checksums.txt), the Pigsty repository (RPMs GPG-signed, key fingerprint 9592A7BC7A682E7333376E09E7935D8DB9BD8B20), or docker.io/pgsty/mc. Upstream’s minisign key does not sign these artifacts, and dl.min.io is never contacted. Release tags (RELEASE.YYYY-MM-DDTHH-MM-SSZ) and package versions (YYYYMMDDHHMMSS.0.0) keep their upstream schemes.

2. mcli update always fails — on purpose

Self-update is removed. mcli update never contacts the network and never replaces its binary; it prints an explicit notice and always exits 1. Upstream mc update exited 0 when already current, so any cron job or script that calls it and treats a non-zero exit as failure will start failing — drop the call and upgrade through your package manager or GitHub Releases instead. The per-invocation version probe against upstream release feeds is also gone, and MC_UPDATE / MINIO_UPDATE are no longer consulted.

(mcli admin update ALIAS — updating the server — still exists, but Silo servers reject in-place updates server-side.)

3. SUBNET, licensing, and telemetry commands

Everything that reached MinIO SUBNET is disabled at build time. Affected commands keep their names and flags, print a stable notice — “MinIO SUBNET services (registration, licensing, uploads) are disabled in this Silo build of mc; diagnostics remain available locally.” — and exit 1:

Command Behavior now Use instead
mcli license register notice, exit 1
mcli license update ALIAS (online renewal) notice, exit 1 mcli license update ALIAS license.key (offline, still works)
mcli support upload notice, exit 1 share files through your own channels
mcli support proxy set notice, exit 1 proxy remove still clears a legacy setting
mcli support callhome enable notice, exit 1 disable / status still work

The diagnostics themselves stay: mcli support diag / perf / profile / inspect always run in local (airgap) mode — results are written to local files, nothing is uploaded, and SUBNET registration is no longer a prerequisite. Two related hardening changes: inspect no longer falls back to encrypting output with an embedded MinIO public key (your archives stay decryptable by you), and since 20260804 --debug output redacts SUBNET credentials — if you ever shared debug logs from older builds, rotate the keys in them. mcli license info and unregister work locally.

4. The play demo alias is no longer pre-seeded

Fresh configurations seed local, s3, and gcs — not play. Tutorials and smoke scripts that assume the demo alias need it added explicitly: mcli alias set play https://play.min.io <access-key> <secret-key> restores the old behavior, since nothing blocks deliberate access to any S3 endpoint. Existing configuration files are never modified.

5. Output text carries the Silo identity

mcli --version keeps its machine-readable first line and adds an identity line plus dual copyright; --help says “Silo client” and examples use mysilo. Command syntax is untouched — only scripts that grep for upstream identity strings (e.g. “MinIO Client”) need adjusting.

6. For developers

The module path stays github.com/minio/mc, so imports compile unchanged — but go install github.com/minio/mc@latest installs the archived upstream, not this fork. Build from source (git clone https://github.com/pgsty/mc && cd mc && make) or consume it via a replace directive. Contributions need no CLA but require a DCO sign-off (git commit -s). Upstream being archived also means inherited defects are only ever fixed here — most notably minio/mc#5139 (mirror --remove --watch on versioned buckets).

Migration

Moving from an official mc binary to mcli:

  1. Install mcli from one of the fork’s channels (see §1 for verification): GitHub Releases archive, yum install mcli / apt install mcli from the Pigsty repository, or docker pull pgsty/mc.
  2. Decide what to call it — this determines which configuration it reads:
    • Keep the mc name (least friction): after confirming no upstream binary remains (command -v mc), install it as mc — e.g. ln -s /usr/local/bin/mcli /usr/local/bin/mc. Invoked as mc, it reads your existing ~/.mc untouched; nothing else to migrate.
    • Adopt the mcli name: carry your state over once with cp -a ~/.mc ~/.mcli, or set MC_CONFIG_DIR=~/.mc. Both clients can also coexist side by side, each with its own directory.
  3. Clean up automation:
    • remove mc update calls — they now always exit 1;
    • remove license register, support upload, support callhome enable, and support proxy set — same stable failure;
    • support diag / perf / profile / inspect keep working and write local files; drop any step that expected a SUBNET upload;
    • review anything that greps --version output beyond the first line.
  4. Re-check play usage in tutorials and smoke scripts (§4).
  5. Verify: mcli --version, mcli alias ls, then mcli ls <alias> and mcli ping <alias> against your servers.
  6. Rollback stays trivial: the configuration format is identical in both directions, so keeping the old mc binary around lets you switch back at any time.

See also

4 - Native Package Migration

How the silo RPM/DEB packages differ from the minio packages: file layout, service account, takeover semantics, and caveats.

Silo publishes silo packages for RPM, DEB, and APK on amd64/arm64 via GitHub Releases, with SHA-256 sums and build-provenance attestations. This page records what changes relative to a minio package installation: the file layout, the service account, and the caveats. General migration scope is in the migration guide.

File layout

MinIO installation Silo package
/usr/bin/minio /usr/bin/silo (also provides silo healthcheck)
minio.service /usr/lib/systemd/system/silo.service
/etc/default/minio Still read, first; /etc/default/silo overrides per variable (noreplace/conffile — upgrades never overwrite edits)
service account minio-user (upstream) / minio (Pigsty) silo:silo, declared in /usr/lib/sysusers.d/silo.conf, created on install
/usr/share/doc/silo/LICENSE, NOTICE (AGPL-3.0-or-later)

Two package properties:

  • Installation never starts or enables the service; postinstall only creates the silo account and reloads systemd.
  • The package installs alongside the minio package — no file conflicts, so the old package stays available for rollback.

Service account

The unit defaults to User=silo, but existing data, TLS keys, and KMS credentials belong to the old MinIO user. Do not chown the data. Run Silo as the current owner via a drop-in:

ls -ld /path/to/your/data              # note the owner, e.g. minio-user
sudo mkdir -p /etc/systemd/system/silo.service.d
sudo tee /etc/systemd/system/silo.service.d/10-legacy-user.conf <<'EOF'
[Service]
User=minio-user
Group=minio-user
EOF
sudo systemctl daemon-reload

This also keeps TLS working: Silo resolves certificates from the runtime user’s home (~/.silo/certs, falling back to the legacy ~/.minio/certs), so the existing public.crt/private.key/CAs/ are found without copying. Without the drop-in, a TLS deployment fails to start:

FATAL Unable to start the server: HTTPS specified in endpoints,
      but no TLS certificate is found on the local machine

Adopting the silo account is an optional later change: move the certificates to a silo-readable path, set --certs-dir in MINIO_OPTS, and transfer data ownership outside the migration window.

Takeover and rollback

The unit is a takeover unit:

[Unit]
After=network-online.target minio.service
Conflicts=minio.service

[Service]
Type=notify
EnvironmentFile=-/etc/default/minio
EnvironmentFile=-/etc/default/silo
ExecStart=/usr/bin/silo server $MINIO_OPTS $MINIO_VOLUMES
Restart=always
  • Conflicts=minio.service: systemd never runs both; starting one stops the other. This implements takeover and rollback in both directions.
  • The EnvironmentFile chain means MINIO_VOLUMES, MINIO_OPTS, credentials, and KMS settings from /etc/default/minio apply unchanged.
  • Type=notify: systemctl start returns success only after the server is actually ready.

Switch over:

sudo systemctl disable --now minio.service
sudo systemctl enable  --now silo.service
silo healthcheck --url https://127.0.0.1:9000 ready    # http:// without TLS
mc admin info <existing-alias>

Roll back (nothing to restore — data ownership, certificates, and the old unit were never touched):

sudo systemctl disable --now silo.service
sudo systemctl enable  --now minio.service

Caveats

  • Clusters switch all nodes together. Two different binaries do not form a cluster — MinIO next to Silo, or one Silo version next to another; a mixed node waits indefinitely in activating (details). Prepare every node first (install package, create drop-in), then flip all nodes in quick succession: systemctl disable --now minio && systemctl enable --now --no-block silo. Rollback and later upgrades likewise: all nodes together.
  • Non-packaged installations work the same way. A /usr/local/bin/minio with a custom unit is taken over identically, as long as its configuration lives in /etc/default/minio.
  • Crash loops rate-limit. A misconfigured start (for example, missing certificates) repeats under Restart=always until systemd’s start limit trips (Start request repeated too quickly). Fix the cause, then systemctl reset-failed silo && systemctl start silo.
  • Keep the rollback window. Leave the minio package, unit, and binary installed until validation completes; a disabled unit costs nothing. Remove the old package afterwards if desired.
  • Rolling restarts after migration: gate each with silo healthcheck --maintenance cluster; exit 0 means stopping this node keeps write quorum, HTTP 412 means it does not.

5 - Feature Notes

Design notes for capabilities Silo adds beyond upstream MinIO — written down before they ship.

The component pages in this section record where Silo matches MinIO. This subsection records the places where Silo deliberately goes beyond it: each page is a design note, written before the implementation lands, and kept afterwards as the authoritative record of what was decided and why.

A note here describes intent, not necessarily shipped behavior — every page carries a status line saying which it is.

5.1 - Native Health Checks and the Distroless Image

Why the silo binary grows a healthcheck subcommand, why mc ready had to be retired as a probe, and how the single-binary distroless image is planned.

Status: P1 (subcommand, 2ff594f4b) and P2 (distroless image + CI gate, 4c34d2309) implemented in pgsty/silo; P3 (Helm probes) and P4 (docs) pending · Decided: 2026-08-06 · Owner: pgsty/silo (command, images, Helm chart), this site (docs)

Silo is getting a native silo healthcheck subcommand and, alongside the existing container image, a new distroless image variant that contains exactly one file that matters: the silo binary. This note records the reasoning and the design decisions before implementation, so the code has a specification to be checked against — and so that “why is it built this way?” has a permanent answer.

Background

Today’s release image (docker.io/pgsty/silo) is built on ubi-micro and ships four moving parts: the silo server, the mcli client (with an mc alias), a statically linked curl, and a POSIX-shell entrypoint script. The compose examples check container health with the bundled client:

healthcheck:
  test: ["CMD", "mc", "ready", "local"]

That arrangement is inherited from upstream MinIO, and its fragility is a matter of record: when mc was briefly missing from the image, users found their health checks failing “with no option but to disable” (#9). Upstream’s own history rhymes — when MinIO moved to ubi-micro in 2023 and lost curl, the maintainers’ answer was to lean harder on mc ready local (minio/minio#18373, #18389), and upstream minio/minio has since been archived with server as the only subcommand its binary ever had. Nobody upstream is going to fix this.

A distroless image forces the question. There is no shell, no curl, no mc — by design. The only program guaranteed to exist inside the container is the server binary itself. If Docker-level health checking is to exist at all in that image, the binary has to provide it.

Why mc ready had to be retired as a probe

Reading the actual mc implementation (cmd/ready-main.go) shows the current health check works by accident, not by design. Four independent defects:

  1. It never fails on its own. mc ready is a wait-until-ready loop: it retries every 5 seconds forever and only ever exits zero, on success. Connection refused does not break the loop. As a Docker healthcheck, the “unhealthy” verdict is produced entirely by Docker’s timeout killing the process — the probe semantics are a side effect of SIGKILL.
  2. It checks the wrong scope. mc ready requests /minio/health/cluster — cluster-wide write quorum. Every container’s “health” therefore reflects the state of the whole cluster, which is precisely the cascading-failure anti-pattern the Kubernetes documentation warns about: lose quorum, and every node is marked unhealthy simultaneously.
  3. It has hidden failure modes. It requires a writable ~/.mc config directory (on a read-only rootfs or an arbitrary OpenShift UID, the probe fails while the server is perfectly healthy), it prints config-creation noise on first run, and its built-in local alias is hardcoded to http://localhost:9000 — wrong the moment TLS is enabled or the port changes, a limitation users complained about upstream.
  4. It is the last functional reason to bundle a second binary. Both mcli and the pinned static curl carry ongoing supply-chain and maintenance cost (the curl pin is stuck on v8.11.0 because a later release dropped the aarch64 build) for what a subcommand of the existing binary can do in ~150 lines.

The decisions

Three tracks, deliberately decoupled:

# Decision
D1 The silo binary gains a healthcheck subcommand — a thin, anonymous HTTP client for the server’s existing /minio/health/* endpoints. It ships in every build, so every image and bare-metal install gains the capability.
D2 The existing image does not change. It keeps mcli, curl, the shell entrypoint, and the mc ready local examples. Users of the current image who want the new probe can opt in by overriding their healthcheck.test — nothing is taken away and no default behavior moves.
D3 A new distroless variant is published alongside it, as a pilot: single binary, no shell, native HEALTHCHECK baked in. If the pilot proves out, it becomes the recommended default later and the switch completes; the classic image remains for compatibility either way.

D2 and D3 answer the obvious “why not just slim the main image?” — because the main image’s contents are a compatibility surface. #9 exists because that surface was changed underneath people once before. The distroless image is a new name with a new contract, so nobody’s existing healthcheck, docker exec mc habit, or entrypoint assumption breaks while it is evaluated.

The silo healthcheck command

silo healthcheck [FLAGS] [CHECK]

CHECK — positional, maps 1:1 onto /minio/health/<path>:
  live          the process is serving (default; touches no external system)
  ready         live + KMS and etcd reachable, when configured
  cluster       cluster-wide write quorum
  cluster-read  cluster-wide read quorum

FLAGS:
  --address value   target host:port  (EnvVar: MINIO_ADDRESS; default ":9000",
                    an empty host is completed to 127.0.0.1)
  --url value       full base URL override (http[s]://host:port); wins over
                    --address and TLS auto-detection (EnvVar: MINIO_HEALTHCHECK_URL)
  --maintenance     cluster only: appends ?maintenance=true — asks "is it safe
                    to take this node down?" (HTTP 412 = no, it would break HA)
  --timeout value   overall deadline; defaults: 5s for live/ready, 15s for cluster*
  plus the inherited global flags: --certs-dir, --config-dir, --json, --quiet

EXIT CODE:  0 = healthy / safe to proceed · 1 = anything else
OUTPUT:     one line, e.g.
  live: ok (200, 2ms)
  cluster: unhealthy (503) server-status=iam-offline write-quorum=3 healing-drives=2

The design principles behind that shape:

  1. Thin client, single source of truth. The command is only ever an HTTP client of the canonical health API. It never re-implements a check in-process, so the semantics of “healthy” live in exactly one place: the server handlers.
  2. CLI vocabulary = API vocabulary. The check names are the endpoint paths. No new concepts to learn, nothing to keep in sync.
  3. Share the server’s own configuration. The port comes from the same --address/MINIO_ADDRESS contract the server uses, and http-vs-https is decided by the same certificate check the server itself performs at startup (public.crt + private.key in the certs directory). This is the Traefik healthcheck pattern — the closest prior art, which resolves its ping endpoint from the same static configuration as its server — with the TLS handling Traefik left as a // TODO actually implemented. It is also the direct fix for mc ready’s port-guessing defect.
  4. The default check is node-local. live answers “is this process serving,” which is the only question a per-container health status should answer. Cluster-scope checks exist, but only behind explicit arguments, mirroring mc ready’s --cluster-read/--maintenance so the operational vocabulary carries over.
  5. Exit codes are 0 and 1, nothing else. The Dockerfile reference explicitly reserves exit code 2 (vault status, which uses 2 for “sealed”, is the cautionary tale). Rich diagnostics belong in the single output line instead — Docker stores the first 4096 bytes of probe output in docker inspect, and the command decodes the server’s diagnostic headers (x-minio-server-status, x-minio-write-quorum, x-minio-healing-drives) into it, which is exactly the detail a bare curl -f throws away.
  6. Skip TLS certificate verification, with no opt-out in v1. This is a loopback self-probe of an anonymous endpoint carrying no data — and the kubelet’s documented behavior for HTTPS httpGet probes is precisely the same. Matching it means one TLS deployment produces one verdict across Docker and Kubernetes; verifying by default would only manufacture false negatives, since self-signed server certs rarely carry a 127.0.0.1 SAN.

Two implementation constraints, discovered in the source, that are load-bearing rather than stylistic:

  • The request must be strictly anonymous. The health routes are exempted from the reserved-path guard only for requests the server classifies as anonymous; attaching an Authorization header reclassifies the request and gets it rejected (ErrAllAccessDisabled) instead of answered.
  • The HTTP transport must set Proxy: nil. Containers routinely inherit HTTP_PROXY without a NO_PROXY entry for 127.0.0.1; a loopback probe must never route through a corporate proxy. (Traefik’s healthcheck does this deliberately, for the same reason.)

And one number that looks arbitrary but is not: the 15-second default timeout for cluster checks exists because the server evaluates cluster health under its own 10-second cluster_deadline — a client that gives up at 5s abandons the request before the server delivers its considered 503, losing every diagnostic header with it. Two details were added after adversarial review: --url is environment-backed (MINIO_HEALTHCHECK_URL) because a probe process cannot see the server’s command line — it is the documented way to point a baked-in HEALTHCHECK at a server whose address or TLS setup comes from CLI arguments; and any outer (Docker) timeout must exceed the probe’s own deadline, or the probe is SIGKILLed before it can print its diagnostic line.

What the endpoints really do

The table below is verified against the handler source, not quoted from documentation — and it corrects a common misreading:

Endpoint Returns 200 when… Fails with… Notes
/minio/health/live almost always — even before the object layer is initialized (that state is only signaled via the x-minio-server-status: offline header) 503 when the request queue is saturated touches no external system; the only endpoint quiet enough for high-frequency probing
/minio/health/ready as live, plus KMS can generate a key and etcd answers a read — each only if configured KMS/etcd failure; queue saturation without KMS or etcd, ready and live are the same code path
/minio/health/cluster object layer, bucket metadata and IAM are initialized, and every erasure set has write quorum 503 with quorum diagnostic headers; with ?maintenance=true, failure is 412 each failed evaluation writes a server-side log line — do not poll it tightly
/minio/health/cluster/read the read-quorum version of the above as above

Consequences worth spelling out: live and ready are liveness-grade signals — they do not tell you the node can serve objects; only the cluster pair does. That is exactly why the cluster pair must stay out of per-container probes (scope, log noise, cascading restarts) and why it is the right tool for operational questions like “may I take this node down?” (--maintenance, where 200 means safe and 412 means you would lose HA).

The distroless variant

Base: gcr.io/distroless/static-debian12 — which is sufficient because silo builds with CGO_ENABLED=0. The base ships the four things the server actually needs from a rootfs: CA certificates (for KMS/webhook/STS egress), tzdata, /tmp, and an /etc/passwd with root/nonroot entries. It ships no shell, no package manager, no libc.

Sketch of the contract:

FROM gcr.io/distroless/static-debian12:latest
COPY silo /usr/bin/silo
COPY LICENSE NOTICE CREDITS /licenses/
ENV HOME=/tmp
# /data is created in the image layer, world-writable — see issue #55:
# there is no entrypoint left to repair ownership at runtime.
VOLUME ["/data"]
EXPOSE 9000
HEALTHCHECK --interval=30s --timeout=10s --start-period=2m --start-interval=2s --retries=3 \
  CMD ["/usr/bin/silo", "healthcheck", "ready"]
ENTRYPOINT ["/usr/bin/silo"]

The decisions folded into that sketch:

  • ENTRYPOINT is the binary itself. docker run pgsty/silo:distroless server /data — no argv-translation script, because there is no shell to run one. The classic image’s MINIO_USERNAME/MINIO_GROUPNAME privilege-drop path (which needs GNU chroot and a writable /etc/passwd) is not supported in this variant; the supported mechanism is --user / Kubernetes runAsUser.
  • /data is created in the layer, mode 0777, and the default user stays root for the pilot. Issue #55 demonstrated that declaring VOLUME ["/data"] without creating it breaks every non-root invocation, and that no entrypoint can repair it after the fact — in distroless there is no entrypoint at all. Creating it world-writable in the layer is the one option that makes all privilege modes work (--user included), keeps drop-in parity with the classic image, and its exposure is bounded by the image running a single process. A nonroot-by-default posture (uid 65532) was considered and deferred: it would break the documented bind-mount workflow on UID mismatch, and the pilot’s job is to measure friction, not maximize it. Revisit at promotion time, possibly as a -nonroot tag.
  • The health check is baked in, exec-form. Shell-form HEALTHCHECK strings need /bin/sh and are impossible here; the JSON-array form is mandatory. Compose inherits an image’s HEALTHCHECK automatically (with disable: true as the escape hatch), so compose users of this variant get working depends_on: condition: service_healthy with zero configuration. ready rather than live because Docker’s health status feeds gating (start ordering), which is readiness semantics — and the two are identical anyway unless KMS/etcd are configured.
  • One release-blocking verification: HEALTHCHECK is a Docker extension, absent from the OCI image spec (opencontainers/image-spec#749 is still open), and OCI-media-type builds drop it silently. The publish pipeline must assert docker inspect shows the Health config on the pushed manifest, or adjust the build’s media types until it does.
  • Naming: docker.io/pgsty/silo:<RELEASE>-distroless, plus a rolling distroless tag. A new Dockerfile.distroless in the server repo — which, having no download stages, is fully offline and can therefore be built and asserted in CI on every release, closing the “the gate tests a synthetic image, not the shipped one” coverage gap that #55 documented for the classic Dockerfile.

What a user gives up in the variant, stated honestly in its docs: no docker exec <container> sh debugging (use docker debug / kubectl debug ephemeral containers), no in-image mc (use the pgsty/mc image or a host-installed mcli), no MINIO_USERNAME path (use --user).

Kubernetes needs no image support at all

Worth stating explicitly, because it bounds the problem: Kubernetes ignores Dockerfile HEALTHCHECK entirely — kubelet probes are configured in the pod spec and executed from outside the container as httpGet requests. Both image variants are therefore probed identically:

startupProbe:            # boot budget: 5s × 60 = 5 minutes for large IAM loads
  httpGet: { path: /minio/health/live, port: 9000 }
  periodSeconds: 5
  failureThreshold: 60
livenessProbe:           # when to restart: process-level signal only
  httpGet: { path: /minio/health/live, port: 9000 }
  periodSeconds: 30
  timeoutSeconds: 5
  failureThreshold: 3
readinessProbe:          # when to unroute: may include hard dependencies (KMS/etcd)
  httpGet: { path: /minio/health/ready, port: 9000 }
  periodSeconds: 15
  timeoutSeconds: 5
  failureThreshold: 3

Three cautions that belong next to any such config: the cluster endpoints must never appear in probes (liveness would restart the whole fleet on quorum loss; readiness would tangle with bootstrap — the chart’s headless service correctly sets publishNotReadyAddresses: true for exactly that reason); live deliberately returns 503 under sustained request-queue saturation, so a saturated node restarts after ~90s by design; and with scheme: HTTPS the kubelet skips certificate verification, so self-signed deployments need nothing extra.

The Silo Helm chart currently ships no probes at all (neither does upstream’s, despite its docs). Adding the three probes above to the chart is planned as an independent follow-up — it depends on neither image track, and it is a differentiator over upstream rather than a compatibility risk.

Rollout

Phase Scope Repo
P1 silo healthcheck subcommand + tests; ships in the next release binary (all images inherit the capability, no image behavior changes) pgsty/silo
P2 Dockerfile.distroless + CI build/health gate + publish -distroless tags as a pilot pgsty/silo
P3 Helm chart: add the three probes; refresh the stale default image tag pgsty/silo
P4 Docs: command reference, probe guide, distroless migration notes; pilot feedback → decide on promoting distroless to the recommended default this site

Compatibility promises across all phases: the classic image’s contents and examples do not change; mc ready local keeps working everywhere it works today; the health HTTP API is untouched (the subcommand is purely additive); and the /minio/health/* paths remain frozen as compatibility surface, same as every other /minio/* route in the fork.

Deferred decisions

Recorded so they are not re-litigated from scratch:

  • --wait mode (block-until-healthy, the one thing mc ready’s loop is genuinely for): deferred — no in-repo consumer needs it yet, and adding it later is backward-compatible. The flag namespace is reserved.
  • Nonroot-by-default for the distroless image: deferred to promotion time, as above.
  • JSON output schema for --json: follows the global-flag convention; the exact schema is fixed at implementation time and documented in the command reference.