SDK reference

The Egg SDK is a TypeScript package, @egg/sdk, for building Egglets. An Egglet is a portable app that talks to the Egg Gateway over loopback HTTP and runs anywhere a modern browser can render it: the Egg desktop app, Chrome, Safari, or any other current browser. Authors install the SDK, export an activate(ctx) function, and receive an EggletContext with every host capability the Egglet’s manifest permits: database, AI (text, image, audio, video), scheduled work, cross-Egglet messaging, refs, credentials, file URLs, the user’s tier, and more.

This page is the field-by-field reference for EggletContext. Sections are grouped into tiers — Primitives (the universal toolkit), Capabilities (model / media / extension access), Runtime services, Egg domains, and the escape hatch. Each method card shows what it does, its parameters, its return shape, the errors it can raise, and the manifest declaration required to call it when one is needed. Calling the Egg Gateway from outside an Egglet (a Chrome extension, an agent skill, the CLI) takes a different path; see the HTTP API reference for that.

import { defineEgglet, type EggletContext } from "@egg/sdk";

export default defineEgglet({
  activate(ctx: EggletContext) {
    // ctx.ai, ctx.database, ctx.log, ...
  },
});

Primitives

The universal toolkit. Every Egglet uses these, and none carry product semantics.

EggletContext.log

Tagged logger scoped to the Egglet’s id. Calls forward to the host console with a [egglet:<id>] prefix. No manifest declaration required. Use this instead of bare console.log so output is attributable.

log.debug(msg, ...args) / log.info(...) / log.warn(...) / log.error(...)

Write a tagged log line to the host console. All four variants share the same shape; they differ only in severity level.

ParameterTypeRequiredDefaultDescription
msgunknownyesPrimary message. Stringified for the console.
...argsunknown[]no[]Additional values appended to the log line.

Returns: void.

Errors: none.

EggletContext.routes

Mount the Egglet’s top-level view at the path declared in manifest.route. The Egglet does not pick the path; the manifest does. The host calls the loader on first activation of the route and keeps the component mounted thereafter (the view participates in the keep-alive panel system; switching tabs does not unmount it). Two Egglets cannot mount the same route.

ctx.routes.mount(() => import("./MyEggletPage"));

routes.mount(loader)

Mount the Egglet's top-level view at its declared route.

ParameterTypeRequiredDefaultDescription
loader() => Promise<{ default?: ComponentType; Component?: ComponentType }>yesDynamic-import callback returning a module that exports default or Component. At least one must be present.

Returns: void.

Errors: throws if another Egglet has already mounted this route, or if the loader resolves to a module with neither a default nor a Component export.

EggletContext.tools

Register named tools the agent and other Egglets can call. Tool names are namespaced by convention: <egglet_id>.<verb>. Every name passed to register must appear in manifest.tools. Cross-Egglet calls require both external: true on the owning manifest and a matching entry in the caller’s permissions array.

ctx.tools.register<{ id: string }, { count: number }>(
  "myEgglet.countItems",
  async ({ id }) => {
    const row = await ctx.database.one<{ n: number }>(
      "SELECT COUNT(*) AS n FROM {{items}} WHERE owner = ?",
      [id],
    );
    return { count: row?.n ?? 0 };
  },
);

tools.register<P, R>(name, handler) manifest.tools

Register a tool the agent and other Egglets can invoke.

ParameterTypeRequiredDefaultDescription
namestringyesFully-qualified tool name (e.g. "feed.summarize"). Must appear in manifest.tools.
handler(params: P) => Promise<R> \| RyesSync or async function that produces the tool result.

Returns: void.

Errors: throws if name is not declared in the manifest, or if another Egglet has already registered the same name.

tools.call<P, R>(name, params?)

Invoke a registered tool by fully-qualified name. Cross-Egglet calls require permission.

ParameterTypeRequiredDefaultDescription
namestringyesFully-qualified tool name to invoke.
paramsPnoundefinedForwarded as the handler’s first argument.

Returns: Promise<R>. whatever the handler returns.

Errors: throws if the tool is unknown. Cross-Egglet calls throw if the target tool is not external: true or the caller’s manifest does not list the tool in permissions.

tools.has(name)

Test whether a tool is registered. Does not apply permission checks.

ParameterTypeRequiredDefaultDescription
namestringyesFully-qualified tool name to test.

Returns: boolean. true if a handler is registered. Permission checks are not applied here.

Errors: none.

EggletContext.database

Read and write the Egglet’s tables. SQL strings reference tables by their logical name wrapped in double curly braces: {{items}}. The host substitutes the placeholder with the physical name from the manifest before sending the query. SQL referencing tables not declared by the manifest is rejected. Reads run against the local connection directly; writes route through the Gateway daemon so its update hooks fire and any sync push runs. From the Egglet’s perspective the calls look identical; the platform handles routing.

const rows = await ctx.database.all<{ id: string; title: string }>(
  "SELECT id, title FROM {{items}} WHERE is_read = 0 ORDER BY ts DESC LIMIT ?",
  [50],
);

await ctx.database.exec(
  "UPDATE {{items}} SET is_read = 1 WHERE id IN (?, ?, ?)",
  ["a", "b", "c"],
);

database.all<T>(sql, params?)

Run a SELECT and return every matching row.

ParameterTypeRequiredDefaultDescription
sqlstringyesSELECT statement. Table names wrapped in {{ }}.
paramsunknown[]no[]Positional bind values for ? placeholders.

Returns: Promise<T[]>. all rows.

Errors: throws if the SQL references a table not in the manifest, or if SQLite reports an error.

database.one<T>(sql, params?)

Run a SELECT and return the first row, or null when empty.

ParameterTypeRequiredDefaultDescription
sqlstringyesSELECT expected to return zero or one row.
paramsunknown[]no[]Positional bind values.

Returns: Promise<T | null>. the first row, or null when the result set is empty. If multiple rows match, only the first is returned.

Errors: same as all.

database.exec(sql, params?) uses.gateway.writes

Run an INSERT, UPDATE, DELETE, or CREATE. Writes route through the daemon so sync push fires.

ParameterTypeRequiredDefaultDescription
sqlstringyesINSERT / UPDATE / DELETE / CREATE statement.
paramsunknown[]no[]Positional bind values.

Returns: Promise<{ changes: number; lastInsertRowid: number }>.

Errors: throws if uses.gateway.writes is not declared in the manifest, if the SQL references undeclared tables, or if SQLite reports an error.

EggletContext.schedule

Recurring work the platform drives. The Egglet supplies the handler; the platform supplies the cadence and survives a closed window when the work is daemon-driven. Requires uses.gateway.schedules: true.

const handle = ctx.schedule.poller({
  name: "refresh",
  interval: 300,
  handler: async () => { await refreshFeeds(); },
});
// later, to stop it manually:
handle.cancel();

schedule.poller(config) uses.gateway.schedules

Register a recurring handler the platform's scheduler drives, surviving window close.

config: PollerConfig

FieldTypeRequiredDefaultDescription
namestringyesStable name scoped to the Egglet. Used as the schedule key for catch-up after a window close.
intervalnumber (seconds)yesTime between handler calls. Minimum 1.
handler() => Promise<void> \| voidyesInvoked every interval. Errors are logged and do not stop the poller.
fireOnActivate"always" \| "if-due" \| "never"no"if-due"Whether the handler fires immediately on register. "if-due" fires only if the recorded last-fire is older than the interval.

Returns: ScheduleHandle. { cancel(): void }.

Errors: throws if uses.gateway.schedules is not declared, or if a poller with the same name is already registered for this Egglet.

EggletContext.fetch

In-process outbound HTTP, distinct from the Gateway-driven headless path. Routes through the platform’s shared HTTP client. Requires uses.network.fetch: true.

const r = await ctx.fetch("https://api.example.com/v1/items", {
  method: "POST",
  headers: { "Content-Type": "application/json" },
  body: JSON.stringify({ q: "today" }),
  timeoutMs: 10_000,
});
if (r.ok) { const items = JSON.parse(r.body); }

fetch(url, init?) uses.network.fetch

Outbound HTTP through the platform's shared, vetted client.

ParameterTypeRequiredDefaultDescription
urlstringyesAbsolute URL. Loopback / file URLs rejected.
init.methodstringno"GET"HTTP method.
init.headersRecord<string, string>no{}Request headers.
init.bodystringnoRequest body. Stringify JSON yourself.
init.timeoutMsnumbernoplatform defaultPer-request timeout.

Returns: Promise<SafeFetchResponse>. { ok: boolean; status: number; statusText: string; headers: Record<string, string>; body: string }. Body is always a string; parse JSON yourself.

Errors: throws if uses.network.fetch is not declared, on network errors, or on timeout. Non-2xx responses are NOT thrown. check ok.

EggletContext.headless

Hidden-browser page fetch. Useful for SPAs, login-walled feeds, anything where ctx.fetch does not produce JS-rendered content. The Gateway spawns a fresh hidden browser per call. Requires uses.gateway.headless: true.

const result = await ctx.headless.fetch("https://example.com/feed", {
  waitMs: 3_000,
  evalJs: "document.title",
  credentialSource: "example",
});
console.log(result.title, result.evalResult, result.html.length);

headless.fetch(url, opts?) uses.gateway.headless

Fetch a fully-rendered page from a hidden browser the Gateway spawns.

ParameterTypeRequiredDefaultDescription
urlstringyesAbsolute URL.
opts.waitMsnumberno0Time to wait after the page’s load event before reading state. Use for SPA hydration.
opts.evalJsstringnoJavaScript expression evaluated in the page context after waitMs elapses. The result lands in evalResult.
opts.credentialSourcestringnoCredential name from manifest.credentials; cookies / auth headers are injected before navigation.

Returns: Promise<HeadlessResult>. { title: string; html: string; evalResult?: unknown; url: string; status: number }.

Errors: throws if uses.gateway.headless is not declared, if credentialSource is unknown, or if the hidden browser cannot reach the URL within the timeout.

EggletContext.web

Client-side web access — a unified surface for fetching, rendering, reading, and searching the web. Every verb runs locally: a plain HTTP request from the user’s machine, or the hidden local browser (with the user’s own session). Nothing is proxied through a cloud service, so requests stay private, cost nothing per call, and see the web as the user’s logged-in browser does. Requires uses.gateway.web: true.

const article = await ctx.web.read("https://example.com/post");
console.log(article.title, article.text?.length);

const hits = await ctx.web.search("mozilla readability library", { limit: 5 });
for (const h of hits.results) console.log(h.title, h.url);

web.fetch(url) uses.gateway.web

Raw HTTP GET from the user’s machine (no JavaScript execution).

ParameterTypeRequiredDescription
urlstringyesAbsolute URL.

Returns: Promise<WebFetchResult>. { url: string; status: number; contentType: string | null; body: string; error: string | null }.

web.render(url, opts?) uses.gateway.web

Load the URL in the hidden local browser (JS-rendered) and return the rendered HTML + visible text. Use for SPAs / JS-heavy pages.

ParameterTypeRequiredDefaultDescription
urlstringyesAbsolute URL.
opts.waitMsnumberno2500Time to wait after load before reading (SPA hydration).

Returns: Promise<WebRenderResult>. { url: string; html: string; text: string; error: string | null }.

web.read(url, opts?) uses.gateway.web

Render locally, then run Mozilla Readability in the page — returns a clean article (title, byline, readable text and HTML). The one-call “give me the content of this page” primitive.

ParameterTypeRequiredDefaultDescription
urlstringyesAbsolute URL.
opts.waitMsnumberno2500Time to wait after load before extracting.

Returns: Promise<WebReadResult>. { url: string; title: string | null; byline: string | null; text: string | null; html: string | null; kind: string | null; error: string | null }. kind is "article" when Readability found an article, else "fallback".

web.search(query, opts?) uses.gateway.web

Search the web by scraping a real results page in the local browser, with a multi-engine fallback (Bing → DuckDuckGo) so one engine’s markup change isn’t an outage.

ParameterTypeRequiredDefaultDescription
querystringyesSearch query.
opts.limitnumberno8Max results (1–25).

Returns: Promise<WebSearchResult>. { query: string; engine: string | null; results: { title: string; url: string; snippet: string }[]; error: string | null }.

Errors: all four throw if uses.gateway.web is not declared. Network/engine failures surface in the error field rather than throwing.

EggletContext.files

List, delete, and resolve URLs for files the daemon has written into its generated-media store (TTS output, generated images, captured video, etc.). The Egglet receives a path back from ctx.ai.*; pass that path to urlFor to get a URL it can embed. No manifest declaration required.

const images = await ctx.files.listGenerated("image");
for (const f of images) {
  console.log(f.name, f.sizeBytes, ctx.files.urlFor(f.path));
}

files.listGenerated(kind)

List files of a given kind in the daemon's generated-media store.

ParameterTypeRequiredDefaultDescription
kind"audio" \| "image" \| "video" \| "document"yesFilter by media kind.

Returns: Promise<GeneratedFile[]>. { id, path, name, fileType, mimeType, sizeBytes, createdAt }, newest first.

Errors: none under normal operation.

files.deleteGenerated(path)

Delete a generated file by its daemon-known path.

ParameterTypeRequiredDefaultDescription
pathstringyesPath returned by an earlier listGenerated or ctx.ai.* call.

Returns: Promise<void>.

Errors: 404 if the file is unknown to the daemon (no error if it’s already gone from disk).

files.urlFor(path)

Resolve a daemon path to a streaming HTTP URL safe for media tags. Synchronous.

ParameterTypeRequiredDefaultDescription
pathstringyesDaemon-known on-disk path.

Returns: string. daemon HTTP URL that streams the file. Safe to drop into <img>, <audio>, <video>, or to pass to a download flow. Synchronous.

Errors: none.

EggletContext.credentials

Read access to user-authorized credentials. Egglets cannot create or revoke credentials; the user manages them in Settings. The source must appear in manifest.credentials.

const entry = await ctx.credentials.get("github");
if (!entry) { ctx.log.warn("no GitHub credential authorized"); return; }
const { token } = JSON.parse(entry.dataJson);

credentials.get(source) manifest.credentials

Fetch a user-authorized credential by source name.

ParameterTypeRequiredDefaultDescription
sourcestringyesCredential source name (e.g. "github"). Must appear in manifest.credentials.

Returns: Promise<CredentialEntry | null>. { credentialType: string; dataJson: string; expiresAt?: string }, or null when no credential is authorized. The Egglet parses dataJson.

Errors: throws if source is not declared in manifest.credentials.

EggletContext.refs

Ref kinds are the cross-Egglet handle system: stable, serializable values like @e:alice-123 that can travel through tables, agent prompts, sync payloads, and other contexts. Each kind has exactly one owning Egglet, declared in manifest.refs. The owning Egglet calls register in activate() with a function that resolves an id to a RefView. Other Egglets call resolve to render or parse to extract refs from text.

ctx.refs.register("e", async (id) => {
  const row = await ctx.database.one<{ id: string; name: string }>(
    "SELECT id, name FROM {{entities}} WHERE id = ?",
    [id],
  );
  return row ? { kind: "e", id: row.id, label: row.name, route: `/people/${row.id}` } : null;
});

refs.register(kind, resolver) manifest.refs

Register the resolver for a ref kind this Egglet owns.

ParameterTypeRequiredDefaultDescription
kindstringyesKind prefix (e.g. "e"). Must appear in manifest.refs.
resolver(id: string) => Promise<RefView | null> \| RefView \| nullyesReturns { kind, id, label, icon?, route?, description? } or null when unresolved.

Returns: void.

Errors: throws if kind is not declared in the manifest or another Egglet owns it.

refs.encode(kind, id)

Build a ref string from a kind + id. Synchronous.

ParameterTypeRequiredDefaultDescription
kindstringyesKind prefix.
idstringyesDomain id.

Returns: string. the serialized ref (e.g. "@e:alice-123"). Synchronous.

Errors: none.

refs.resolve(ref)

Resolve a ref string to its view (label, route, icon), or null.

ParameterTypeRequiredDefaultDescription
refstringyesSerialized ref (e.g. "@e:alice-123").

Returns: Promise<RefView | null>. null when the kind has no registered resolver or the resolver reports unresolved.

Errors: none for malformed input (returns null).

refs.parse(text)

Extract all refs embedded in arbitrary text. Synchronous; does not call resolvers.

ParameterTypeRequiredDefaultDescription
textstringyesArbitrary text to scan for embedded refs.

Returns: ParsedRef[]. each entry is { kind, id, start, end, raw }. Synchronous; does not call resolvers.

Errors: none.

EggletContext.eggletIpc

Fire-and-forget messaging between Egglets. Use this for one-way signals (“new item arrived”, “follow-up needed”); use ctx.tools.call when you need a response. Both methods require uses.eggletIpc: true. Messages are dropped silently if the target has no subscribers; senders cannot tell.

// Sender
ctx.eggletIpc.send("people", { kind: "profileVisited", id: "alice-123" });

// Receiver
const handle = ctx.eggletIpc.subscribe<{ kind: string; id: string }>((msg) => {
  console.log(`from ${msg.from}: ${msg.payload.kind}`);
});

eggletIpc.send<P>(targetEgglet, payload) uses.eggletIpc

Send a fire-and-forget message to another Egglet. Silently dropped if no subscribers.

ParameterTypeRequiredDefaultDescription
targetEggletstringyesRecipient’s Egglet id.
payloadPyesJSON-serializable payload.

Returns: void.

Errors: throws if uses.eggletIpc is not declared. Does NOT throw when the target has no subscribers.

eggletIpc.subscribe<P>(handler) uses.eggletIpc

Register a handler for incoming Egglet messages.

ParameterTypeRequiredDefaultDescription
handler(msg: EggletIpcMessage<P>) => void \| Promise<void>yesCalled for every message addressed to this Egglet. Receives { from, payload }.

Returns: EggletIpcHandle. { cancel(): void } to stop receiving messages.

Errors: throws if uses.eggletIpc is not declared.

EggletContext.tier

The user’s plan tier and per-capability availability/quota. Use this to gate AI calls in the UI and avoid showing buttons the user’s plan can’t honor. Capability strings are the same daemon-side gate names used by ctx.ai.* (text_fast, vision, imagen, veo, tts, stt, etc.).

if (await ctx.tier.isAvailable("imagen")) {
  // safe to call ctx.ai.image.generate(...)
}

tier.getInfo()

Fetch the user's current tier, BYOK status, and per-capability used / limit counters.

No parameters.

Returns: Promise<TierInfo>. { tier: "free" | "plus" | "pro"; byokAllowed: boolean; capabilities: { capability: string; available: boolean; limit: number; used: number }[] }. limit: -1 means unlimited.

Errors: none under normal operation. Network failure surfaces as a rejected promise.

tier.isAvailable(capability)

Check whether a capability is exposed on the user's plan. Fail-open on transport error.

ParameterTypeRequiredDefaultDescription
capabilitystringyesCapability slug.

Returns: Promise<boolean>. true if the user’s tier exposes this capability. Falls back to true on transport failure (fail-open).

Errors: none.

tier.isLimitReached(capability)

Check whether today's quota has been hit. Fail-open on transport error.

ParameterTypeRequiredDefaultDescription
capabilitystringyesCapability slug.

Returns: Promise<boolean>. true when the capability is unavailable OR the daily limit has been hit. Falls back to false on transport failure (fail-open).

Errors: none.

EggletContext.setup

A dependency preflight. The Egglet declares what it needs in manifest.setup; ctx.setup.check() evaluates those declarations against current Gateway state and returns a report the Egglet uses to gate its core UI and degrade optional features. Call it early — typically on activate(), or let the host’s Egglet chrome render the setup banner for you.

Each requirement is satisfied by a single token (need) or any of several (anyOf), and either gates the core app (omit enables) or unlocks one named feature (enables: "feature-id"). Tokens name a Gateway dependency:

TokenSatisfied when
capability:text|vision|tts|stt|embed|imagea model/engine is assigned and usable for that capability
provider:<type>a cloud provider of that type has an API key
tool:whisper|piper|ffmpegthe on-device tool is installed
credential:<source>the user has signed in to that source
browsera browser is available to drive
const report = await ctx.setup.check();
if (!report.core.ready) {
  // report.core.missing[i] = { key, label, satisfied, setupUrl? }
  showSetup(report.core.missing);   // setupUrl is absolute, ready to open
  return;
}
if (report.features["transcription"]?.available) enableTranscribe();

setup.check()

Evaluate manifest.setup (credentials + requires) against the Gateway.

No parameters.

Returns: Promise<PreflightReport>:

interface PreflightReport {
  core: { ready: boolean; missing: RequirementStatus[] };
  features: Record<string, { available: boolean; missing: RequirementStatus[] }>;
}
// RequirementStatus = { key, label, satisfied, setupUrl? }  // setupUrl set when unmet

See the manifest reference for the setup.requires declaration shape.

Capabilities

Model, media, and browser-extension access. Platform capabilities, not tied to one app.

EggletContext.ai

All model-backed operations live here, grouped by modality: text (chat, embeddings, reranking), image, audio, and video. Every method routes through the Gateway daemon end-to-end: capability resolution, tier gating (per-day quotas based on the user’s plan), credential lookup, and the outbound HTTP call to whichever provider serves the slot all happen there. The browser shell never sees an API key.

A blocked tier returns a structured error the Egglet can surface to the user (e.g., “daily limit reached” or “not available on your plan”). Use EggletContext.tier to check availability before calling.

EggletContext.ai.audio

Speech is organized as intent verbs over one unified slot spine. The Egglet picks the intent; the user’s speech slot (Settings → Models → Speech) picks the engine, location, and voice. There are four capabilities — batch_tts, live_tts, batch_stt, live_stt (plus realtime_voice) — each an ordered fallback chain across browser / Gateway / cloud. When a slot resolves to the browser location, the verb is honored client-side with the platform’s native speechSynthesis / SpeechRecognition; otherwise it routes through the Gateway. The Egglet never names an engine or an API key.

Prefer the verbs:

The lower-level operations remain for batch work: transcribe (batch speech-to-text; accepts an http(s) URL / daemon path or a recorded Blob/File), vad (local voice-activity-detection — speech intervals, useful for chunking long recordings), and listVoices (the provider-tagged voice catalog for a picker). tts is retained; its backend field is legacy — the slot decides location now. recognize is deprecated, kept as an alias of inline transcribe.

Caveat: transcribeStatus currently reports best-effort placeholders; a real /api/ai/transcribe-ready endpoint will replace the stub.

// Speak a finished sentence, and stop it if the user navigates away.
const handle = await ctx.ai.audio.speak("Your export is ready.");
// handle.stop(); await handle.done;

// Listen to the mic with live partials.
const session = ctx.ai.audio
  .listen()
  .on("partial", (t) => setDraft(t))
  .on("final", (t) => submit(t))
  .on("end", () => setDraft(""));
// session.stop();

// Batch transcription of a recording (Blob) or an on-disk file / URL.
const { transcript } = await ctx.ai.audio.transcribe(recordedBlob);
const voices = await ctx.ai.audio.listVoices();

ai.audio.tts(req)

Synthesize speech from text. Uses the local TTS engine when installed, the cloud service otherwise.

ParameterTypeRequiredDefaultDescription
req.textstringyesText to synthesize. Long inputs chunked internally.
req.voicestringnoplatform defaultVoice key from listVoices().
req.backend"auto" \| "local" \| "cloud"no"auto"Force a local or cloud backend. auto prefers local when installed.

Returns: Promise<{ path: string; sizeBytes: number }>. on-disk audio file (typically MP3 or WAV).

Errors: 424 FAILED_DEPENDENCY if backend: "local" and no local voice is installed; 402/429 tier gates on cloud; 502 provider error.

ai.audio.transcribe(audio, modelPref?)

Batch speech-to-text (batch_stt slot). Accepts an on-disk path / http(s):// URL or a recorded Blob/File (base64’d for you).

ParameterTypeRequiredDefaultDescription
audiostring \| BlobyesA local file path or http(s):// URL (downloaded into a temp file first), or a recorded Blob/File captured in the renderer.
modelPrefstringnoslot defaultTranscription-model variant to prefer when the resolved engine is local (e.g. "medium.en").

Returns: Promise<{ transcript: string }>.

Errors: 412 PRECONDITION_FAILED if any local transcription dependency (transcoder, engine, or model) is not installed; 502 BAD_GATEWAY on URL download failure; 500 on subprocess failure.

ai.audio.transcribeStatus()

Report whether the local transcription pipeline is ready (transcoder, engine, model).

No parameters.

Returns: Promise<{ ready: boolean; ffmpeg: boolean; whisper: boolean; model: boolean }>. See caveat above; values are placeholders until /api/ai/transcribe-ready ships.

Errors: none.

ai.audio.recognize(req) deprecated

Deprecated. Kept as an alias of inline transcribe (resolves live_stt) on base64 audio bytes; prefer listen() for mic capture or transcribe(blob) for a recording.

ParameterTypeRequiredDefaultDescription
req.audioBase64stringyesAudio bytes, base64-encoded.
req.encodingstringnoprovider default"LINEAR16", "FLAC", "OGG_OPUS", etc.
req.sampleRateHertznumbernoprovider defaulte.g. 16000.
req.languageCodestringno"en-US"BCP-47 code.
req.modelstringnoprovider defaultProvider-specific model id (e.g. Google "latest_short").

Returns: Promise<{ transcript: string }>.

Errors: 412 PRECONDITION_FAILED when no Speech-to-Text key is configured; 402/429 tier gates; 502 provider error.

ai.audio.vad(req)

Detect speech intervals in an audio file using the local voice-activity-detection engine.

ParameterTypeRequiredDefaultDescription
req.inputPathstringyesDaemon-readable audio file path.
req.thresholdnumberno0.5Voice-probability threshold (0.0–1.0).
req.minSpeechMsnumberno250Drop speech spans shorter than this.
req.minSilenceMsnumberno100Treat silence shorter than this as part of the surrounding speech span.

Returns: Promise<{ speech: { startMs: number; endMs: number }[]; durationMs: number }>.

Errors: 400 on invalid path; 424 FAILED_DEPENDENCY if the VAD model is not installed (install it in Settings).

ai.audio.listVoices()

Enumerate available TTS voices (local + cloud), in catalog order.

No parameters.

Returns: Promise<Voice[]>. { key, displayName, backend: "local" | "cloud", installed }, local voices first.

Errors: none under normal operation.

ai.audio.speak(text, opts?)

Batch text-to-speech (batch_tts slot): synthesize a finished utterance and play it in the page.

ParameterTypeRequiredDefaultDescription
textstringyesText to speak.
opts.voicestringnoslot defaultVoice key, catalog-relative to the resolved engine.
opts.capabilitystringno"batch_tts"Slot to resolve.

Returns: Promise<SpeakHandle>{ stop(): void; done: Promise<void> }. done resolves when playback finishes or is stopped.

Errors: 424 FAILED_DEPENDENCY if the resolved engine needs a local voice that is not installed; tier and provider errors as for tts.

ai.audio.say(text, opts?)

Conversational text-to-speech (live_tts slot): a low-latency reply. Uses the browser voice when the slot resolves to the browser location, else the Gateway.

ParameterTypeRequiredDefaultDescription
textstringyesText to speak.
opts.voicestringnoslot defaultVoice key for the resolved engine.
opts.capabilitystringno"live_tts"Slot to resolve.

Returns: SaySession{ stop(): void } (synchronous; playback starts immediately).

Errors: surfaced on the session; missing local voice falls back down the slot chain.

ai.audio.listen(opts?)

Conversational speech-to-text (live_stt slot): open a mic session with live transcript events.

ParameterTypeRequiredDefaultDescription
opts.capabilitystringno"live_stt"Slot to resolve.

Returns: ListenSession — chain .on("partial", cb) (interim transcript), .on("final", cb) (finalized segment), .on("end", cb), and .stop().

Errors: surfaced on the session; a browser-location slot uses native SpeechRecognition and needs microphone permission.

ai.audio.converse(opts?) not yet available

Realtime duplex voice (realtime_voice slot). Throws until the realtime transport ships.

Returns: never — throws today. Reserved so Egglets can target the capability now.

EggletContext.ai.image

generate calls the configured image-generation model and writes the result to a daemon-known on-disk path. Pair with EggletContext.files to render or serve it. analyze sends the image to the configured vision-capable model with a prompt; pass imagePath for files on disk or imageBase64 for in-memory bytes.

const img = await ctx.ai.image.generate({
  prompt: "a green leaf on a white background, studio lighting",
  aspectRatio: "1:1",
});
const url = ctx.files.urlFor(img.path);

const analysis = await ctx.ai.image.analyze({
  imagePath: img.path,
  prompt: "Describe the lighting and composition.",
});

ai.image.generate(req)

Generate an image from a text prompt. Returns a daemon-known on-disk path.

ParameterTypeRequiredDefaultDescription
req.promptstringyesText prompt. Provider safety filters may reject; surfaces as a 502.
req.aspectRatiostringno"1:1""16:9", "9:16", "1:1", "4:3", etc. Provider-dependent.

Returns: Promise<{ path: string; sizeBytes: number }>. PNG saved into the daemon’s generated-images store. Use ctx.files.urlFor(path) to render.

Errors: 402 PAYMENT_REQUIRED tier block; 429 tier limit; 502 provider error (including safety-filter rejection); 500 on disk write failure.

ai.image.analyze(req)

Send an image to a vision-capable text model with a prompt; returns text.

ParameterTypeRequiredDefaultDescription
req.imagePathstringone of imagePath / imageBase64Daemon-readable on-disk path. Preferred when the image is on disk.
req.imageBase64stringone of imagePath / imageBase64Base64-encoded bytes, no data: prefix.
req.mimeTypestringno"image/png"Only consulted when imageBase64 is set.
req.promptstringyesQuestion or instruction.
req.systemstringnoSystem prompt.

Returns: Promise<{ text: string; model: string; provider: string }>.

Errors: 400 if neither imagePath nor imageBase64 is provided, or the path cannot be read; 402 / 429 tier gates; 502 provider error.

EggletContext.ai.text

Operations whose input or output is plain text against the user’s configured text-model slot. chat.complete and chat.stream require uses.llm in the manifest; the slot is picked by uses.llm.role (default agent-secondary) and uses.llm.capability (default text). chat.stream emits text deltas, tool-call deltas, usage events, and a final done event; the returned promise resolves with the same shape as complete once the stream closes.

const r = await ctx.ai.text.chat.complete({
  system: "You are a concise summarizer.",
  messages: [{ role: "user", content: "Summarize this in one line: ..." }],
  maxTokens: 64,
});

const { embedding, dims, provider } = await ctx.ai.text.embed({ text: "hello world" });

const { scored } = await ctx.ai.text.rerank({
  query: "cats",
  docs: ["a cute kitten", "the weather is sunny", "a feline pet"],
  topN: 2,
});

ai.text.chat.complete(req) uses.llm

Single-turn chat completion against the user's configured text-model slot.

ParameterTypeRequiredDefaultDescription
req.messages{ role: "user" \| "assistant"; content: string }[]yesConversation, in order. Platform supplies provider-specific framing.
req.systemstringnoSystem prompt.
req.maxTokensnumbernoprovider defaultCap on output tokens.

Returns: Promise<ChatResponse>. { text, inputTokens, outputTokens, provider, model }.

Errors: throws if uses.llm is not declared, on tier block (402), tier limit (429), missing provider key (412), or upstream provider error (502).

ai.text.chat.stream(req, onEvent) uses.llm

Streaming chat completion. Same shape as complete plus per-chunk events.

ParameterTypeRequiredDefaultDescription
reqChatRequestyesSame shape as complete.
onEvent(e: ChatStreamEvent) => voidyesCalled for every chunk. Event variants: text, toolCallStart, toolCallDelta, usage, done, error.

Returns: Promise<ChatResponse>. resolves once the stream closes, with accumulated text and totals.

Errors: same gates as complete; an error event during streaming both fires through onEvent and rejects the promise.

ai.text.embed(req)

Generate an embedding vector for a text input. Picks the configured embedding provider.

ParameterTypeRequiredDefaultDescription
req.textstringyesText to embed.
req.provider"google" \| "openai"nopriority orderProvider hint. Default uses the highest-priority configured provider.

Returns: Promise<{ embedding: number[]; provider: string; dims: number }>. Dimensionality varies by provider; always inspect dims, do not hard-code.

Errors: 412 PRECONDITION_FAILED when no embeddings key is configured; 502 on provider error.

ai.text.rerank(req)

Score candidate documents against a query using a local cross-encoder.

ParameterTypeRequiredDefaultDescription
req.querystringyesQuery string.
req.docsstring[]yesCandidate documents. Empty array is allowed and returns empty scored.
req.topNnumbernoallReturn only the top N entries.
req.variantstringnoplatform defaultReranker model variant id.

Returns: Promise<{ scored: { index: number; score: number }[] }>. index refers to the original docs array. Higher score = more relevant.

Errors: 424 FAILED_DEPENDENCY when the reranker model is not installed (install it in Settings).

EggletContext.ai.video

generate calls the configured video-generation model. It needs a public HTTPS seed image URL and accepts a provider-defined duration range. The daemon polls until completion (up to ~3 minutes) and returns the provider-hosted video URL. analyze sends the video to the configured video-understanding model with a prompt; pass videoPath for files on disk or videoBase64 for in-memory bytes.

const video = await ctx.ai.video.generate({
  prompt: "the leaf gently rustles in the breeze",
  imageUrl: "https://...leaf.png",
  durationSeconds: 5,
});
const description = await ctx.ai.video.analyze({
  videoPath: "/path/to/clip.mp4",
  prompt: "Summarize the action in one sentence.",
});

ai.video.generate(req)

Generate a video from a prompt and a public seed-image URL. The daemon polls until completion.

ParameterTypeRequiredDefaultDescription
req.promptstringyesAction / scene description.
req.imageUrlstringyesPublic HTTPS seed image URL the provider can fetch.
req.durationSecondsnumberno5Seconds. The provider enforces its own min / max range.
req.aspectRatiostringnoprovider defaulte.g. "16:9".

Returns: Promise<{ videoUrl: string; requestId: string }>. provider-hosted URL. Daemon polls up to ~3 minutes.

Errors: 412 PRECONDITION_FAILED when no video-generation provider key is configured; 402/429 tier gates; 504 if generation does not finish within the timeout; 502 provider error.

ai.video.analyze(req)

Send a video to the configured video-understanding model with a prompt; returns text.

ParameterTypeRequiredDefaultDescription
req.videoPathstringone of videoPath / videoBase64Daemon-readable on-disk path. Preferred when the video is on disk.
req.videoBase64stringone of videoPath / videoBase64Base64-encoded bytes, no data: prefix.
req.mimeTypestringno"video/mp4"Only consulted when videoBase64 is set.
req.promptstringyesQuestion or instruction.
req.systemstringnoSystem prompt.

Returns: Promise<{ text: string; model: string; provider: string }>.

Errors: 400 if neither videoPath nor videoBase64 is provided, or the path cannot be read; 402/429 tier gates; 502 provider error (the routed model must support video).

EggletContext.media

ffmpeg / ffprobe operations, run by the Gateway against daemon-readable file paths. Every method that writes output returns { outputPath, durationMs }; pair outputPath with ctx.files.urlFor() to render or serve it. Long operations accept an onProgress callback (0–100). Gated by uses.gateway.media.

const { outputPath } = await ctx.media.extractClip({
  inputPath: src, outputPath: "/tmp/clip.mp4", start: "00:00:05", duration: "10",
});
const url = ctx.files.urlFor(outputPath);

EggletContext.extension

Drive the Egg Chrome Extension: capture a URL in the user’s real, logged-in browser (their session and cookies) and read back what it captured — useful for content behind a login the headless fetch can’t reach. Gated by uses.extension.

extension.captureUrl(url, opts?)

Enqueue a capture: the extension opens the URL, extracts the article + metadata, and stores the result. Resolves to a command id — poll getCommand(id) until status is "done". Requires uses.extension.commands.

Returns: Promise<{ id: string }>.

Runtime services

The Gateway’s always-on background machinery.

EggletContext.jobs

The Gateway’s generic deferred-work queue. Domain features (contemplation, media downloads, agent tasks) enqueue here and hand back a Job handle; the lifecycle — wait, subscribe, cancel, children — lives on the handle, not on each domain. The queue enforces concurrency, priority, dedupe, retries, and a token budget. Gated by uses.gateway.jobs.

const job = await ctx.jobs.enqueue("media.download", { url }, { dedupeKey: url, maxRetries: 2 });
const unsub = job.subscribe((e) => { if (e.kind === "progress") setPct(e.progress); });
const result = await job.wait();   // throws if it failed / was cancelled
unsub();

jobs.enqueue(kind, spec?, opts?)

Enqueue a job of a known kind; built-in kinds run their Gateway handler.

ParameterTypeRequiredDefaultDescription
kindstringyes"thinking", "media.download", … a kind the Gateway knows.
specunknownnoKind-specific input.
opts.prioritynumberno0Higher runs first within the budget.
opts.dedupeKeystringnoA second enqueue with the same key returns the existing live job.
opts.maxRetriesnumberno0Bounded auto-retry on failure.
opts.budgetTokensnumbernoToken/compute ceiling.

Returns: Promise<Job<T>>.

The rest of the surface: list(filter?), get(id), cancel(id), and watch(onEvent, opts?) (a live feed of created/updated jobs for an activity view; returns an unsubscribe fn). Filters accept an exact kind or a prefix ("media."), a status, a parentId, or a since timestamp.

A Job<T> handle carries a snapshot (id, kind, status, progress, headline, error, parentId, attempt, createdAt, updatedAt) and methods: wait({ timeoutMs? }) (resolves with T when done, throws on failed/cancelled), subscribe(onEvent) (status/progress/log/domain events; returns unsubscribe), result(), cancel(), and children(). status is one of queued | running | paused | done | cancelled | failed.

EggletContext.thinking

Background contemplation. contemplate() seeds a durable dialectical session (thinker → challenger → judge) and runs it as a thinking job on the queue above; its distilled findings propagate into the agent’s long-term memory by default. Gated by uses.gateway.thinking.

const session = await ctx.thinking.contemplate({
  topic: "What would make onboarding feel instant?",
  sink: "memory",   // "none" keeps findings on the session only
});
const detail = await session.wait();   // { turns, findings, ... }

Egg domains

Data domains a built-in Egg app owns the UX for, exposed here so any Egglet can build on them. They are always present because the Gateway ships them.

EggletContext.feeds

The aggregated feed store the Gateway polls — RSS, social, and scraped sites — and its items. The Feed app is one UI over this domain; any Egglet can read and manage the same data. Egg also emits feeds: channels and the feed host publish the user’s content as open, subscribable RSS.

EggletContext.explorations

Hub-and-spoke exploration graphs — the Ponder / Explore domain. An exploration is a tree of nodes (prompts, responses, images) anchored to a source; any Egglet can create and extend them.

EggletContext.people

The contacts domain — the single write surface for the People tables (entity + profile + work history + education). The People app and the Egg Extension ingest through this same implementation, so a profile captured from a social page and one added by hand land in the same place. Gated to the People Egglet and the agent.

EggletContext.artifacts

Long-lived domain artifacts: code, documents, diagrams. Distinct from files, which surfaces ephemeral generated media. Artifacts are durable, addressable by id, and round-trip through sync.

artifacts.list()

List all artifacts owned by the user, newest first.

No parameters.

Returns: Promise<Artifact[]>. all artifacts owned by the user, newest first.

Errors: none under normal operation.

artifacts.get(id)

Fetch one artifact by id, or null when not found.

ParameterTypeRequiredDefaultDescription
idstringyesArtifact id.

Returns: Promise<Artifact | null>. null when not found.

Errors: none.

artifacts.create(req)

Create a new artifact (code, doc, diagram, or other domain kind).

ParameterTypeRequiredDefaultDescription
req.kindstringyesArtifact kind (e.g. "code", "doc", "diagram").
req.titlestringyesDisplay title.
req.contentstringyesArtifact body.
req.metadataRecord<string, unknown>no{}Free-form domain metadata.

Returns: Promise<Artifact>. full record with assigned id and timestamps.

Errors: 400 on missing required fields.

artifacts.update(id, patch)

Patch fields on an existing artifact. Omitted fields are preserved.

ParameterTypeRequiredDefaultDescription
idstringyesArtifact id.
patchPartial<Artifact>yesFields to overwrite. Omitted fields are preserved.

Returns: Promise<Artifact>. updated record.

Errors: 404 if the id is unknown.

artifacts.remove(id)

Delete an artifact by id.

ParameterTypeRequiredDefaultDescription
idstringyesArtifact id.

Returns: Promise<void>.

Errors: 404 if the id is unknown.

EggletContext.agent

The persistent Egg agent’s own self and state — separate from ctx.ai, which is stateless inference. Where ctx.ai asks a model a question, ctx.agent reads and writes the standing agent: what it remembers, how it spends its time, who it is, and what it can do. It binds to the primary agent, so the agent id is implicit (ctx.agent.id is "agent-primary"). Every call routes through the Gateway, which owns the agent.

Seven sub-surfaces: chat, calendar, personality, memory, workspaces, tools, skills. Data shapes are the Gateway’s own (snake_case), returned verbatim.

// Review and prune what the agent remembers.
const notes = await ctx.agent.memory.list({ category: "preference", limit: 50 });
await ctx.agent.memory.add({ content: "Prefers concise answers.", category: "preference" });
await ctx.agent.memory.remove(notes[0].id);

// Ask the standing agent (full personality + memory + tools), not a bare model.
const reply = await ctx.agent.chat({
  messages: [{ role: "user", content: "What are you working on?" }],
  latest: "What are you working on?",
});

EggletContext.agent.chat

agent.chat(req)

One turn against the full standing agent (its personality, memory, tools, and skills), not a bare model slot.

ParameterTypeRequiredDefaultDescription
req.messages{ role, content }[]yesConversation so far.
req.lateststringyesThe newest user message; drives memory recall on the Gateway side.
req.agentRolestringnoprimaryRole override.

Returns: Promise<{ text: string; input_tokens: number; output_tokens: number }>.

EggletContext.agent.memory

The agent’s long-term memory — the store it extracts on its own from conversations and background thinking, and reads back into its system prompt. This is not the Memorize Egglet’s knowledge base.

agent.memory.list(opts?)

List memories, newest first.

ParameterTypeRequiredDefaultDescription
opts.categorystringnoallFilter to one category (e.g. "semantic", "preference", "note").
opts.limitnumberno200Max rows (1–1000).

Returns: Promise<AgentMemory[]>{ id, category, content, source_context?, entity_id?, created_at, access_count }.

agent.memory.add(req)

Add a hand-authored memory (stored with source_context: "user").

ParameterTypeRequiredDefaultDescription
req.contentstringyesThe fact to remember.
req.categorystringno"note"Category label.

Returns: Promise<AgentMemory> — the created row.

agent.memory.remove(id)

Forget one memory by id.

Returns: Promise<void>.

EggletContext.agent.calendar

The agent’s time allocation and autonomy. Standing reservations materialize into concrete time blocks each day, which the Gateway’s block runner executes during the agent’s private window; the soft plan is its forward-look.

EggletContext.agent.personality

The four markdown documents that shape the agent (identity, motivations, goals, standards). getFile(filename){ content, filename }; setFile(filename, content) writes it back. Changes take effect on the next turn.

EggletContext.agent.workspaces, .tools, .skills

Escape hatch

For endpoints not yet wrapped by a typed surface.

EggletContext.gateway

A deliberate escape hatch: low-level authorized access to the local Gateway, for first-party Egglets that need an endpoint not yet wrapped by a typed ctx.* surface. It is subject to the same authorization as every other ctx call — the Egglet’s per-origin scoped bearer token is attached automatically, and an endpoint that token may not reach still returns 403.

Prefer a typed surface wherever one exists; reach for this only when none does, and expect the call site to migrate to a typed surface later. (This is how the Agent app reached its calendar / personality / tools before ctx.agent existed.)