Rollout model
From a build artifact to a confirmed device.
Eight steps, of which only two involve moving bytes. The rest is deciding which devices are allowed to receive them, in what order, and under what conditions the whole thing stops.
The eight steps
-
Upload the build
Push the artifact to https://api.{{DOMAIN}}/v1 from CI. Emberwire computes a SHA-256 digest, chunks the image, and stores only the chunks it has never seen for that product line. A 1.4 GB terminal image that differs from the last build in 96 MB costs 96 MB of new storage, not 1.4 GB.
-
Sign the release
A manifest is produced — image digest, size, version, product line, compatible hardware revisions, minimum agent version — and signed with your release key. The private key can live in your own HSM: Emberwire will hand you the manifest to sign and take the signature back. See Security.
-
Describe the target
A release is not aimed at a device list. It is aimed at a selector over device attributes — hardware revision, current build, region, customer tag, agent version, uptime. Selectors are evaluated continuously, so a device that comes online tomorrow and matches is picked up without anyone editing anything.
-
Plan the wave
Pick a ramp, a soak window per step, maintenance windows, and a per-hour cap. The default ramp is below. Everything is a parameter; the defaults are what we would choose for a fleet we did not know anything about.
-
Devices pull
Nothing is pushed. On its next check-in, a device in the active cohort is told which build it should be running and which nodes to fetch it from. It requests the delta if it can apply one and the full image otherwise, over a resumable ranged transfer.
-
Verify, then write
The agent validates the signature chain against the trust anchor in its ROM, then streams the image into the inactive slot, hashing as it goes. A digest mismatch at the end means the slot is marked invalid and nothing else happens. The device keeps running the build it has.
-
Swap and confirm
The slot pointer flips, the device reboots into the new image, and the application calls ew_confirm() once it is satisfied that it is healthy. No confirm within the watchdog window — default 20 minutes — and the bootloader reverts on the next boot.
-
Watch, and be ready to stop
Confirmations, rollbacks and error codes stream back per device. The wave advances only if the current cohort is healthy. Halting is automatic and does not need an operator to be awake.
The default wave
Fleet exposure over the first 72 hours of a release. The vertical axis is stepped, not linear — each gridline is one cohort.
| Cohort | Share | Devices | Soak | Delta bytes | Advance condition |
|---|---|---|---|---|---|
| 1 — canary | 0.1 % | 480 | 4 h | 84 MB | Zero failed confirms. One rollback stops the release. |
| 2 | 1 % | 4,800 | 6 h | 843 MB | Failed confirms below 0.5 %, no 25-device streak. |
| 3 | 5 % | 24,000 | 8 h | 4.12 GB | Same, plus error-code mix unchanged from cohort 2. |
| 4 | 25 % | 120,000 | 12 h | 20.6 GB | Same. Rate cap becomes the limiting factor here. |
| 5 — remainder | 100 % | 480,000 | — | 82.4 GB | Runs until the selector is empty. Latecomers join for 24 months. |
| Release total | — | 480,000 | 30 h | 82.4 GB | Full images would be 1.46 TB. |
Cohort byte figures are cumulative from the start of the release, at 180 KB per device. At the default cap of 12,000 devices per hour the last cohort takes roughly 30 hours of wall clock; the 72-hour axis above allows for devices that are simply not online when their turn comes.
Deltas, and when they are skipped
A delta is computed between two exact builds, not between two versions. The device tells us what it is running; if we have that pair, it gets a patch.
How the patch is produced
Emberwire keeps the last 12 builds of a product line hot and pre-computes patches from each of them to the new build at publish time. Patch generation for a 1.4 GB image pair takes about four minutes and happens once, not once per device.
The agent applies the patch into the inactive slot. It never patches in place, so a failed patch costs a retry, not a device.
When a device gets the full image anyway
- It is running a build older than the 12 kept hot.
- Its reported slot digest does not match the expected base — flash corruption, a field reflash, or a unit that came back from repair.
- It has no room to stage a patch and a scratch buffer at the same time.
- The product line has delta: false set, which is the right choice for encrypted or compressed images where deltas do not compress well.
- It is enrolling for the first time.
Budget for the full image, not the delta
Across the fleet, 4.1 % of devices receive a full image on a typical release. On a 1.4 GB product line that single-digit percentage is most of your egress bill. The pricing page works this through with real numbers.
Targeting
Selectors are declarative and evaluated on every check-in.
POST https://api.{{DOMAIN}}/v1/releases{
"product_line": "pl_meter_g4",
"image": "img_8f3a21c9",
"version": "4.19.2",
"select": {
"hw_rev": ["G4-B", "G4-C"],
"current_build": { "lt": "4.19.2", "gte": "4.12.0" },
"agent": { "gte": "2.7.0" },
"region": ["eu"],
"tag_not": ["lab", "field-trial", "do-not-update"]
},
"wave": {
"ramp": [0.001, 0.01, 0.05, 0.25, 1.0],
"soak_hours": [4, 6, 8, 12],
"rate_cap_per_hour": 12000,
"halt_on": { "fail_ratio": 0.005, "consecutive_failures": 25 },
"windows": [{ "days": "mon-thu", "from": "22:00", "to": "05:00", "tz": "device" }]
}
}
Devices that match tag_not are never offered the build, including devices tagged after the wave started. Devices that stop matching mid-wave — because an operator retagged them — are dropped from the remaining cohorts, not rolled back.
Failure paths
What the system does, without an operator, in each case.
| What happens | Device behaviour | Platform behaviour | Cost |
|---|---|---|---|
| Transfer interrupted | Resumes with a ranged request from the last complete block on the next check-in. | Any node can serve the remainder; blocks are content-addressed. | 0 bytes re-sent |
| Delivery node unreachable | Falls back to the next node in the candidate list it was given. | Node is removed from candidate lists within 30 s of failing its health check. | one retry |
| Digest mismatch after download | Marks the staged slot invalid, discards it, reports E_DIGEST. | Counts toward the halt threshold. Three from one node quarantines that node's copy. | 1 full re-fetch |
| Signature does not chain | Discards without writing. Reports E_CHAIN and keeps running. | Release is frozen fleet-wide immediately — this is never a transient fault. | wave stops |
| Power lost mid-write | Boots the active slot, which was never touched. Resumes staging afterwards. | Nothing. The device was never counted as updated. | partial re-fetch |
| New image boots but is unhealthy | No ew_confirm() within 20 min → bootloader reverts to the previous slot on the next boot. | Counted as a failed confirm. Halt rule fires at 0.5 % or 25 in a row. | 1 reboot |
| New image does not boot at all | Watchdog reset; bootloader sees an unconfirmed slot and reverts. | Same as above — the platform cannot tell the two apart, and does not need to. | 1 reboot |
| Device clock is wrong | Signature validity is checked against the manifest's monotonic release counter, not wall-clock time, so a dead RTC does not block updates. | Reports skew so you can find fleets with failing batteries. | none |
| Staging area full | Reports E_NOSPACE with free-byte count and does not start. | Device is excluded from the wave and listed for review. | 0 bytes |
| Device offline for a year | Reappears, is placed in a quarantine cohort, walks required waypoint releases in order. | Never handed an untested multi-version jump. | n full images |
Maintenance windows
Device-local time
A window expressed in tz: "device" follows each device's own timezone, which is what you want for a fleet across several countries. A window in a fixed zone is available where the operator, not the device, defines the quiet hours.
Windows do not interrupt
A transfer already in progress when the window closes is allowed to finish. A reboot that has not started yet is deferred to the next window. Nothing is killed halfway.
Emergency override
A release can be marked ignore_windows for a security fix. It is a separate flag from publishing, it is logged with the operator identity, and it appears in the audit export.
What Emberwire will not do
Limits we would rather state than have you discover
- It will not make an unsafe update safe. If your image bricks the device, Emberwire limits how many devices reach it — it cannot repair the ones that already did, unless they can roll back.
- It will not update a device with a single flash slot, no external storage and no room to stage. There is no safe way to do that, and we will not pretend otherwise. See Device agent for the storage layouts that do work.
- It will not push. A device that never connects never updates, and no amount of platform configuration changes that.
- It will not manage your keys for you unless you ask it to. The default is that the release key is yours and the root of trust never touches our infrastructure.
- It is not a device-management platform. No remote shell, no configuration push, no telemetry pipeline. Firmware images and the state machine around them, nothing else.