Skip to content
Certus

Substrate

Every run boots two sandboxes — one at BASE, one at HEAD. The Substrate view shows how each was provisioned and how it is functioning.

Twin sandboxes

A proof is a comparison, so Certus boots the change twice: base and head, from the same recipe, the same seed data, the same fixtures and the same external-dependency policies. Only the checkout differs.

The two are symmetric by construction. Both are forks of one shared world that contains neither checkout, and both then have exactly one delta applied — BASE's delta being empty. A world where BASE is merely restored and HEAD is restored-then-modified would differ in operation even when identical in content, and every timing- and state-sensitive oracle would see that manufactured difference. Neither side is the primary.

Each sandbox is a hardware-virtualized guest with its own kernel. If a provider cannot give one, the run is BLOCKED; it is never downgraded to a shared-kernel container. The repository under test is treated as hostile input, and the host kernel is not part of its attack surface.

The lifecycle

A sandbox moves through a fixed sequence of states. The Substrate view shows the current state per side and the timeline of transitions with how long each took, so a slow boot is attributable to a step rather than to “startup”.

requested
The launch manifest is verified. No machine is held yet.
snapshot_restored
The cached tier named by the manifest is restored. See snapshots.
forked
The guest is running and its in-sandbox agents have answered.
delta_applied
The workspace is at this side's commit. On BASE the delta is empty, not skipped.
run_disk_attached
A fresh, empty writable disk is mounted.
datastores_ready
Every datastore accepts connections and is seeded from the world snapshot.
config_injected
Every non-secret configuration entry has been materialized.
secrets_leased
Every secret lease is redeemed and bound to the process that needs it.
network_attached
The fixture network is attached and its boot self-test passed.
app_starting
Your processes have been started, in the order their dependencies require.
crash_looping
A process exited and was restarted past the threshold. Its only exit is teardown.
ready
Every declared process satisfies its readiness signal. A partially ready system is not ready.
driving
The harness has begun executing the plan.
collecting
The harness is done and evidence is being flushed.
tearing_down
Leases are revoked, then mutable state is destroyed.
destroyed
Nothing of the run's mutable state remains.

The line at ready

Where a sandbox stopped decides what the run can say. This is the single most useful thing to know when reading a failed boot.

stopped before ready
The verdict is BLOCKED. The experiment never ran, so nothing about your code was established either way.
stopped while driving or collecting
The verdict is INCONCLUSIVE, and only for the obligations whose evidence is missing. Obligations already discharged keep their results.
failed during teardown
The verdict is unchanged. A teardown failure is an operational or security incident for Certus; it does not reflect on your change.

No lifecycle state can produce FAILED. Only an oracle reading evidence can refute a claim. An environment that would not start is a statement about the environment.

Readiness

Certus never infers readiness from quiet. There is no settle heuristic and no sleep. Each service is ready when the strongest signal it declares is satisfied, in this order of preference: a declared health endpoint returning success twice in a row from inside the sandbox; a declared migration process exiting cleanly; a declared queue showing a registered consumer; a declared port accepting a connection; a declared log line matching.

A service that declares nothing falls back to its port if one is named, and failing that the run is BLOCKED naming the service. Declaring a health endpoint for every service in .certus/env.yaml is the single highest-value thing you can do to make boots reliable.

Readiness is re-checked, not latched. A service that becomes unhealthy after the sandbox reached ready sends it back to starting rather than allowing a proof to run against a half-dead system.

Reading the Substrate view

The view is scoped to the run in focus and shows BASE and HEAD side by side: lifecycle state, sandbox identity, the manifest digest, the trust domain, the isolation kind, the snapshot tier and whether it hit, the egress phase and what is allowed, each service and its health, the environment variables required and supplied, and the resources the machine was given.

Compare the two sides. The point of reading substrate is usually to answer “why did these two behave differently”, and the answer is almost always a row that differs where it should not.

Where the engine did not record a fact, the view says so in words. It does not fill the gap with a plausible number.

When it says something unexpected

BLOCKED at app_starting with a readiness timeout
Read the readiness history and the log tails. They name the service, the signal it was waiting for, and the line of .certus/env.yaml that declared it.
crash_looping
Your process is restarting itself. Read the postmortems — exit code, signal, and the tail of each stream. Certus does not retry this: a re-run restarts identically.
A missing environment variable or a refused credential
Fix the environment contract entry the record names. This is not retryable; a re-run without the fix fails the same way.
The isolation floor was unavailable
Nothing on your side. Certus could not get a hardware-virtualized guest and refused to fall back to a container. Re-run later.
The two sides differ where they should not
Compare timelines, snapshot tiers and service lists. A difference in provisioning invalidates the comparison, and it is reportable as a defect in Certus.
A BLOCKED run with no boot failure record
That is a defect in Certus, not an ambiguity. Every terminal path short of ready is required to produce one.