Webhooks and telemetry

Webhooks

curl -s https://api.zode.dev/v1/webhooks \
  -H "Authorization: Bearer $ZODE_TOKEN" -H "Content-Type: application/json" \
  -d '{"url": "https://hooks.example.com/zode", "event_types": ["sandbox.create", "sandbox.delete", "spend.cap_reached"]}'

Event types

{
  "invoice_id": "in_...",
  "number": "...",
  "status": "open",
  "amount_due_cents": 1250,
  "amount_paid_cents": 0,
  "currency": "usd",
  "hosted_invoice_url": "https://invoice.stripe.com/i/..."
}

What a delivery looks like

• Zode-Event: the event type
• Zode-Delivery: whd_..., unique per delivery
• Zode-Signature: t=<unix seconds>,v1=<hex HMAC-SHA256>

{
  "id": "evt_...",
  "type": "sandbox.create",
  "created_at": "2026-11-11T10:00:00Z",
  "workspace_id": "org_...",
  "actor": "ada@acme.dev",
  "target": { "type": "sandbox", "id": "sbx_..." },
  "data": {}
}

Verify the signature

import { createHmac, timingSafeEqual } from "node:crypto";

export function verifyZodeSignature(secret: string, header: string, rawBody: Buffer, toleranceSec = 300): boolean {
  let ts = "";
  const sigs: string[] = [];
  for (const part of header.split(",")) {
    const [k, v] = part.trim().split("=");
    if (k === "t") ts = v;
    if (k === "v1") sigs.push(v);
  }
  if (!ts || Math.abs(Date.now() / 1000 - Number(ts)) > toleranceSec) return false;
  const want = createHmac("sha256", secret).update(`${ts}.`).update(rawBody).digest("hex");
  return sigs.some((s) => s.length === want.length && timingSafeEqual(Buffer.from(s), Buffer.from(want)));
}

import hashlib, hmac, time

def verify_zode_signature(secret: str, header: str, body: bytes, tolerance: int = 300) -> bool:
    fields = [p.strip().split("=", 1) for p in header.split(",") if "=" in p]
    ts = next((v for k, v in fields if k == "t"), "")
    sigs = [v for k, v in fields if k == "v1"]
    if not ts.isdigit() or abs(time.time() - int(ts)) > tolerance:
        return False
    want = hmac.new(secret.encode(), ts.encode() + b"." + body, hashlib.sha256).hexdigest()
    return any(hmac.compare_digest(s, want) for s in sigs)

Retries, deliveries and tests

• Any 2xx is success. Anything else, a timeout (10 s) or a redirect (not followed) is a failure.
• Failures retry with backoff: 30 s, doubling, capped at 1 hour, with jitter, until 24 hours after the event. Then the delivery is failed.
• An endpoint whose deliveries have all failed for 3 days is turned off (disabled_reason: "failing"). PATCH /v1/webhooks/{id} with {"enabled": true} turns it back on.
• GET /v1/webhooks/{id}/deliveries lists deliveries newest first, with status (pending, delivering, succeeded, failed), attempts, your response status and the first 2,048 bytes of your response body. Page with limit (1 to 100) and before.
• POST /v1/webhooks/{id}/deliveries/{delivery_id}/resend queues the same event and body again with a new delivery id (409 endpoint_disabled on a disabled endpoint).
• POST /v1/webhooks/{id}/test sends one webhook.test event now, without retries, and returns the delivery.

OpenTelemetry export

curl -s -X PATCH https://api.zode.dev/v1/otel \
  -H "Authorization: Bearer $ZODE_TOKEN" -H "Content-Type: application/json" \
  -d '{"endpoint": "https://otlp.example.com", "protocol": "http/protobuf", "headers": {"Authorization": "Bearer ..."}, "enabled": true}'
curl -s -X POST https://api.zode.dev/v1/otel/test -H "Authorization: Bearer $ZODE_TOKEN"

• Data goes to <endpoint>/v1/metrics, /v1/traces and /v1/logs. Protocols: http/protobuf (default) and http/json. gRPC is not offered.
• Up to 20 headers. Their values are stored sealed and never returned; GET /v1/otel shows header_names only.
• Resource attributes: service.name=zode-sandbox, zode.org, zode.sandbox, zode.sandbox.name, zode.region, zode.branch, zode.repo.
• Metrics: zode.sandbox.cpu.utilization, zode.sandbox.memory.usage, zode.sandbox.memory.pressure, zode.sandbox.disk.read.iops, zode.sandbox.disk.write.iops, zode.sandbox.network.receive and .transmit.
• Traces: one span per process and one per AI call (gen_ai.* attributes and cost). Logs: one record per network event, WARN when blocked.
• Delivery is at least once. Span ids derive from recorder event ids, so your backend can deduplicate. Export starts when you enable it; there is no backfill.
• POST /v1/otel/test sends one log record (“Zode connection test”). DELETE /v1/otel removes the configuration.

Flight recorder

curl -s "https://api.zode.dev/v1/sandboxes/$ID/timeline?tracks=process,ai,net&limit=100" \
  -H "Authorization: Bearer $ZODE_TOKEN"
curl -s "https://api.zode.dev/v1/sandboxes/$ID/stats?from=2026-11-11T00:00:00Z" \
  -H "Authorization: Bearer $ZODE_TOKEN"

• from and to take unix milliseconds or RFC 3339. The default window is the last hour; one read covers at most 31 days.
• timeline returns entries in time order and a next value; pass it back as cursor for the next page. limit is 1 to 5,000 (default 500). tracks is any of metrics, process, ai, net.
• Process rows have source: "guest": the VM reports them, so treat them as untrusted. The other tracks come from the host.
• stats summarises the window: samples, average and peak CPU, peak RAM and memory pressure, network bytes, processes, OOM kills, blocked network events, AI calls and AI cost.