Skip to content
Emberwire OTA firmware delivery

Rollout model

From a build artifact to a confirmed device.

Eight steps, of which only two involve moving bytes. The rest is deciding which devices are allowed to receive them, in what order, and under what conditions the whole thing stops.

The eight steps

  1. Upload the build

    Push the artifact to https://api.{{DOMAIN}}/v1 from CI. Emberwire computes a SHA-256 digest, chunks the image, and stores only the chunks it has never seen for that product line. A 1.4 GB terminal image that differs from the last build in 96 MB costs 96 MB of new storage, not 1.4 GB.

  2. Sign the release

    A manifest is produced — image digest, size, version, product line, compatible hardware revisions, minimum agent version — and signed with your release key. The private key can live in your own HSM: Emberwire will hand you the manifest to sign and take the signature back. See Security.

  3. Describe the target

    A release is not aimed at a device list. It is aimed at a selector over device attributes — hardware revision, current build, region, customer tag, agent version, uptime. Selectors are evaluated continuously, so a device that comes online tomorrow and matches is picked up without anyone editing anything.

  4. Plan the wave

    Pick a ramp, a soak window per step, maintenance windows, and a per-hour cap. The default ramp is below. Everything is a parameter; the defaults are what we would choose for a fleet we did not know anything about.

  5. Devices pull

    Nothing is pushed. On its next check-in, a device in the active cohort is told which build it should be running and which nodes to fetch it from. It requests the delta if it can apply one and the full image otherwise, over a resumable ranged transfer.

  6. Verify, then write

    The agent validates the signature chain against the trust anchor in its ROM, then streams the image into the inactive slot, hashing as it goes. A digest mismatch at the end means the slot is marked invalid and nothing else happens. The device keeps running the build it has.

  7. Swap and confirm

    The slot pointer flips, the device reboots into the new image, and the application calls ew_confirm() once it is satisfied that it is healthy. No confirm within the watchdog window — default 20 minutes — and the bootloader reverts on the next boot.

  8. Watch, and be ready to stop

    Confirmations, rollbacks and error codes stream back per device. The wave advances only if the current cohort is healthy. Halting is automatic and does not need an operator to be awake.

The default wave

Fleet exposure over the first 72 hours of a release. The vertical axis is stepped, not linear — each gridline is one cohort.

100 % 25 % 5 % 1 % 0.1 % 0 0 h 4 10 18 30 72 h canary · 480 devices cohort 2 · 4,800 cohort 3 · 24,000 cohort 4 · 120,000 full fleet · 480,000 halt example: 0.9 % of cohort 2 failed to confirm — cohort 3 never starts, 4,800 devices exposed, not 480,000 soak windows: 4 h · 6 h · 8 h · 12 h rate cap: 12,000 devices / h example fleet: 480,000 smart meters
Steel: a healthy release. Orange: the same release stopped by the automatic halt rule at the second cohort. The fleet size used here (480,000 meters) matches the worked example on the pricing page.
Default ramp applied to a 480,000-device fleet
Cohort Share Devices Soak Delta bytes Advance condition
1 — canary0.1 %480 4 h84 MB Zero failed confirms. One rollback stops the release.
21 %4,800 6 h843 MB Failed confirms below 0.5 %, no 25-device streak.
35 %24,000 8 h4.12 GB Same, plus error-code mix unchanged from cohort 2.
425 %120,000 12 h20.6 GB Same. Rate cap becomes the limiting factor here.
5 — remainder100 %480,000 —82.4 GB Runs until the selector is empty. Latecomers join for 24 months.
Release total—480,000 30 h82.4 GB Full images would be 1.46 TB.

Cohort byte figures are cumulative from the start of the release, at 180 KB per device. At the default cap of 12,000 devices per hour the last cohort takes roughly 30 hours of wall clock; the 72-hour axis above allows for devices that are simply not online when their turn comes.

Deltas, and when they are skipped

A delta is computed between two exact builds, not between two versions. The device tells us what it is running; if we have that pair, it gets a patch.

How the patch is produced

Emberwire keeps the last 12 builds of a product line hot and pre-computes patches from each of them to the new build at publish time. Patch generation for a 1.4 GB image pair takes about four minutes and happens once, not once per device.

The agent applies the patch into the inactive slot. It never patches in place, so a failed patch costs a retry, not a device.

When a device gets the full image anyway

  • It is running a build older than the 12 kept hot.
  • Its reported slot digest does not match the expected base — flash corruption, a field reflash, or a unit that came back from repair.
  • It has no room to stage a patch and a scratch buffer at the same time.
  • The product line has delta: false set, which is the right choice for encrypted or compressed images where deltas do not compress well.
  • It is enrolling for the first time.

Budget for the full image, not the delta

Across the fleet, 4.1 % of devices receive a full image on a typical release. On a 1.4 GB product line that single-digit percentage is most of your egress bill. The pricing page works this through with real numbers.

Targeting

Selectors are declarative and evaluated on every check-in.

POST https://api.{{DOMAIN}}/v1/releases
{
  "product_line": "pl_meter_g4",
  "image": "img_8f3a21c9",
  "version": "4.19.2",
  "select": {
    "hw_rev":        ["G4-B", "G4-C"],
    "current_build": { "lt": "4.19.2", "gte": "4.12.0" },
    "agent":         { "gte": "2.7.0" },
    "region":        ["eu"],
    "tag_not":       ["lab", "field-trial", "do-not-update"]
  },
  "wave": {
    "ramp":       [0.001, 0.01, 0.05, 0.25, 1.0],
    "soak_hours": [4, 6, 8, 12],
    "rate_cap_per_hour": 12000,
    "halt_on": { "fail_ratio": 0.005, "consecutive_failures": 25 },
    "windows":  [{ "days": "mon-thu", "from": "22:00", "to": "05:00", "tz": "device" }]
  }
}

Devices that match tag_not are never offered the build, including devices tagged after the wave started. Devices that stop matching mid-wave — because an operator retagged them — are dropped from the remaining cohorts, not rolled back.

Failure paths

What the system does, without an operator, in each case.

What happens Device behaviour Platform behaviour Cost
Transfer interrupted Resumes with a ranged request from the last complete block on the next check-in. Any node can serve the remainder; blocks are content-addressed. 0 bytes re-sent
Delivery node unreachable Falls back to the next node in the candidate list it was given. Node is removed from candidate lists within 30 s of failing its health check. one retry
Digest mismatch after download Marks the staged slot invalid, discards it, reports E_DIGEST. Counts toward the halt threshold. Three from one node quarantines that node's copy. 1 full re-fetch
Signature does not chain Discards without writing. Reports E_CHAIN and keeps running. Release is frozen fleet-wide immediately — this is never a transient fault. wave stops
Power lost mid-write Boots the active slot, which was never touched. Resumes staging afterwards. Nothing. The device was never counted as updated. partial re-fetch
New image boots but is unhealthy No ew_confirm() within 20 min → bootloader reverts to the previous slot on the next boot. Counted as a failed confirm. Halt rule fires at 0.5 % or 25 in a row. 1 reboot
New image does not boot at all Watchdog reset; bootloader sees an unconfirmed slot and reverts. Same as above — the platform cannot tell the two apart, and does not need to. 1 reboot
Device clock is wrong Signature validity is checked against the manifest's monotonic release counter, not wall-clock time, so a dead RTC does not block updates. Reports skew so you can find fleets with failing batteries. none
Staging area full Reports E_NOSPACE with free-byte count and does not start. Device is excluded from the wave and listed for review. 0 bytes
Device offline for a year Reappears, is placed in a quarantine cohort, walks required waypoint releases in order. Never handed an untested multi-version jump. n full images

Maintenance windows

Device-local time

A window expressed in tz: "device" follows each device's own timezone, which is what you want for a fleet across several countries. A window in a fixed zone is available where the operator, not the device, defines the quiet hours.

Windows do not interrupt

A transfer already in progress when the window closes is allowed to finish. A reboot that has not started yet is deferred to the next window. Nothing is killed halfway.

Emergency override

A release can be marked ignore_windows for a security fix. It is a separate flag from publishing, it is logged with the operator identity, and it appears in the audit export.

What Emberwire will not do

Limits we would rather state than have you discover

  • It will not make an unsafe update safe. If your image bricks the device, Emberwire limits how many devices reach it — it cannot repair the ones that already did, unless they can roll back.
  • It will not update a device with a single flash slot, no external storage and no room to stage. There is no safe way to do that, and we will not pretend otherwise. See Device agent for the storage layouts that do work.
  • It will not push. A device that never connects never updates, and no amount of platform configuration changes that.
  • It will not manage your keys for you unless you ask it to. The default is that the release key is yours and the root of trust never touches our infrastructure.
  • It is not a device-management platform. No remote shell, no configuration push, no telemetry pipeline. Firmware images and the state machine around them, nothing else.