AI Assistant
The AI Assistant is an admin-side conversational helper that wires GEM's data model into natural-language requests. It opens as a drawer from the admin top bar and stays scoped to the integrator/admin context — it does not control end-user devices on its own (homeowner-facing voice control is a separate, more constrained surface).
You talk to it in plain English. It picks the right tool, asks for confirmation on anything that changes state, and shows you a one-line description of what it's about to do before it runs.
Where to find it
- Click the AI icon in the admin top bar to open the drawer.
- The same assistant is also available on the Admin → Help page.
- The assistant runs against Anthropic's Claude API. The first time you open the drawer it shows AI Assistant — Setup: paste your Claude API key (it starts with
sk-) into the inline field and click Save & Enable. The key is stored encrypted and the assistant turns itself on — you don't have to add it manually under System → Attributes. If a key is already on file but the assistant is switched off, the drawer instead offers Enable with Existing Key (or paste a different key to replace it). Get a key at console.anthropic.com.
Providers and per-chat model selection
The assistant can talk to more than one AI provider. Anthropic (Claude) is the default. You can also add Google Gemini (native generateContent API) and any OpenAI-compatible provider — OpenAI itself, OpenRouter, or a local/self-hosted model server. (Gemini is also available through its OpenAI-compatible endpoint as a separate preset if you prefer that path.)
- Manage providers — click the gear on the provider bar at the top of the assistant drawer. Each provider has a display name, a base URL (pre-filled for the common presets), a default model, an optional set of models to show in the chat picker, and its own API key. Keys are stored encrypted, one secure attribute per provider; leaving the key field blank when editing keeps the existing key.
- Model selection — model fields are dropdowns, not typed ids: once a provider has a key, the editor fetches the live model catalog from the provider's own models API (Anthropic, Gemini, OpenAI, OpenRouter, and local OpenAI-compatible servers all support this). Pick the default model from the list and check which models should appear in the per-chat picker. Pasting a new key loads the catalog immediately, before the first save; Reload models re-fetches. If a catalog can't be loaded (no key yet, or a local server that's offline), the fields fall back to free-text ids so nothing is blocked.
- Default model — the admin assistant runs on Claude Opus 5 (
claude-opus-5) unless theai_modelsystem attribute names another model or the chat picker overrides it for the session. Voice commands keep their own, smaller model (see below). - Pick one per chat — when more than one provider is configured, the provider bar shows a provider dropdown and a model dropdown. The selection applies to the current admin chat session; each request is sent to the chosen provider and model. Start a New Chat to reset context when switching. The same picker appears in the script editor's AI sidebar — the widget code editor, the Script Console, and the driver / web-service / macro-script editors — with its own remembered selection shared across those editors.
- Scope — provider selection is available in the admin assistant chat and the script editor AI sidebar. Voice commands run on their own voice provider + voice model, set in the Voice commands section of the providers editor (typically a small, fast model) — the voice model is likewise picked from the voice provider's fetched catalog; this is independent of the per-chat picker. Blueprint analysis and other vision features run on Anthropic.
- Streaming — responses stream token-by-token on every provider (Claude, Gemini, and OpenAI-compatible alike). The tool-use loop, confirmation gate, and undo work identically across providers.
- Reasoning effort (OpenAI-compatible providers) — an optional level sent as
reasoning_effort: none, minimal, low, medium or high. Leave it on Provider default unless you want to pin one; None or Minimal gives the fastest replies on GPT-5-class models, which otherwise spend seconds thinking before every tool call. GEM talks to these providers over the chat-completions API, and some OpenAI models refuse tool calls there unless reasoning is off — when a model answers that way, GEM re-sends the request with reasoning set to none and remembers the choice for that model, so the first request after a restart may take one extra round trip and the rest go straight through. Models that don't know the field at all (older OpenAI models, Gemini's compatibility endpoint, most local servers) have it dropped the same way.
How it decides what to do
Every skill carries a plain-English description, a defined set of inputs, and — for anything that changes state — a confirmation requirement. When you ask the assistant to do something, it:
- Picks the skill whose description matches your request.
- Resolves IDs (zones, devices, macros, etc.) against the system's live current state, so the values it acts on are always up to date.
- For mutations: shows a one-line preview and waits for your confirmation before writing.
- For atomic bulk operations: pre-validates every entry, writes them, and rolls back the entire batch if any one fails.
If you're not sure what the assistant can do, just ask it: "what can you do?" — it will call its list_skills tool and return the live catalog.
Skill categories
Skills are grouped by intent. The same categories are returned by list_skills.
create — commission new entities
| Skill | One-line summary |
|---|---|
bulk_create_devices | Atomic batch device commissioning with command_set / device_type auto-resolve. |
bulk_create_zones | Atomic batch zone commissioning with subsystem name/id resolve + device/site_space validation. |
map_bacnet_points | Turn a discovered BACnet object list into fully-wired zones — one call instead of ~140 form round-trips for a floor of VAV boxes. Creates (or updates) each zone bound to the BACnet parent device and writes one bacnet_<key>_read / _write attribute per point, normalizing every notation integrators actually use (12345,2,5, 2,5, AV:5, analogValue:5, MSV:2, a bare instance) into the numeric triple the driver parses. Setpoint and mode keys get both halves by default — the missing _write is the classic cause of a setpoint that reads fine but won't change. Validates the device is on the bacnet driver chain and every triple parses before writing anything; rolls back created zones and restores touched attributes on any mid-batch failure. Run it with dry_run first when the point list was pasted. Caps: 60 zones, 40 points each. |
map_modbus_points | The Modbus sibling of map_bacnet_points, for a rack of VFDs or a floor of thermostats. Modbus carries a trap BACnet doesn't: zone.address takes the vendor's 5-/6-digit convention number (40001) while a point attribute holds a type prefix plus a raw 0-based offset (hr_0). Discrete-input, input-register and holding-register points written in convention form (i_10001, ir_30001, hr_40001) are decoded when polled, but coils are never reinterpreted — c_3 is always raw offset 3, because standard coil numbering overlaps raw offsets completely. The skill owns both conventions, decoding convention numbers with the driver's own address bands and accepting hr_5 / 4x:5 / holding 5 / {register_type, address} as raw offsets. It also enforces protocol reality a reviewer misses: discrete inputs (1x) and input registers (3x) are read-only, so the _write half is auto-filled only for coils and holding registers, and an explicit write aimed at 1x/3x is a validation error rather than a silent wire failure. Reports <mode>_setpoint points as dead when the zone has no matching system_modes map. Validates the device is on the modbus driver chain before writing anything; rolls back on any mid-batch failure. Run it with dry_run first when the register map was pasted. Caps: 60 zones, 40 points each. |
map_mqtt_topics | The MQTT sibling of the two above, for a broker full of Zigbee2MQTT sensors, Shellys or Tasmota devices. An MQTT zone is bound by three things that never error when they're wrong: zone.address holds the full topic filter, mqtt_attribute picks which zone attribute the payload lands in and silently defaults to state, and mqtt_transform is a JavaScript expression whose syntax errors surface only in the device's last_error. The skill writes all three per zone, and can also merge the device-level subscriptions (broker → a site or variable attribute) and publications (GEM attribute changes → an outgoing topic template) rules that are hand-edited raw JSON today. It checks every topic against real MQTT wildcard grammar, compile-checks every transform with the same sandbox wrapper the driver uses, rejects duplicate topics, and rejects a subscription rule aimed at zone — the driver skips those by design, so the rule would be silently dead. Validates the device is on the mqtt driver chain before writing anything; rolls back created zones, restored attributes and the device's prior rules JSON on any mid-batch failure. Run it with dry_run first when the topic list was pasted. Caps: 80 zones, 60 subscription rules. |
map_knx_group_addresses | The KNX fourth sibling of the point-mapping family, for an ETS group-address export. KNX gives no error channel back to GEM, so every mis-mapping is silent: a 2-level or raw-integer address makes the driver's parser return null and every command dies as invalid group address; a DPT outside the 1.x / 3.x / 5.x / 9.x families is emitted as a single raw byte; a 3.x zone can send but is never decoded inbound, so its state is frozen; 1/2/3 and 01/02/03 are one address on the bus and two keys in the driver. The skill parses every address with the driver's own bit bounds, normalizes padded spellings, rejects unencodable DPTs, and reports per zone what the mapping will actually reflect onto. It also fixes the dimmer trap — on/off write a 1-bit telegram to the zone's own address while set_level writes 8-bit percent to that same address, but ETS exposes switching and brightness as two group objects, so a 5.001 zone ignores on/off until the entry's switch_address wires on_action/off_action at the switch object. Validates the device is on the knxnet_ip driver chain; rolls back on any mid-batch failure; reloads zones only, never the device (that would drop the tunnel). Run it with dry_run first when the list was pasted. Cap: 100 zones. |
map_opc_ua_points | The OPC UA fifth sibling of the point-mapping family, for a Kepware / Ignition / Siemens / Beckhoff tag export or a pasted browse listing. Every OPC UA mistake is invisible in the admin UI: a malformed NodeId is never rejected by GEM (the server answers BadNodeIdInvalid into a console warning), and a zone with an opc_ua_node but no opc_ua_data_type reads perfectly — so it looks wired — while every write throws, because the driver refuses to guess a type. The skill parses each NodeId against the standard textual grammar (an optional ns=<n>; prefix plus exactly one of i= / s= / g= / b=), rejects a bare integer with the typed form to use, validates DataType names against the driver's own table and stores them in canonical spelling, and treats i=85 and ns=0;i=85 as one node when checking for duplicates. It also fixes the actuation trap: the driver implements no standard zone verb (no on, off, set_level), so a mapped zone streams values in but responds to nothing — mark an entry writable and the skill wires on_action/off_action to write_value with typed on/off values. Validates the device is on the opc_ua driver chain; rolls back on any mid-batch failure; reloads the zones and the device, because zone-mapped nodes are only subscribed on connect. Run it with dry_run first when the listing was pasted. Cap: 100 zones. |
map_zwave_zones | The Z-Wave sixth sibling of the point-mapping family, for nodes already included on a zwave controller. The driver is controllable only through two hand-authored JSON zone attributes — command_map (GEM verb → Z-Wave value ID + value + metadata) and response_map (zone attribute → value ID + optional value map) — and the Z-Wave admin page saves one map entry per round trip, so a single dimmer is five of those loops and a typo surfaces only as command not mapped at the button. The skill emits both maps from a device-class profile — switch (Binary Switch), dimmer / fan / shade (Multilevel Switch, levels clamped at 99 because 255 means "restore last level"), lock (Door Lock with the full lock-state map), sensor (read-only) — always builds the node:endpoint address the driver requires, and attaches the metadata block the driver dereferences unconditionally on dispatch. The shade profile deliberately leaves open/close out of the command map — the driver's native verbs are a momentary relay pulse meant for garage openers — and wires open_action/close_action overrides that re-dispatch level instead. Extra telemetry rides on sensors shorthands (battery, temperature, humidity, motion, contact, leak, smoke, …); Meter power/energy must be given explicitly because its property key differs per device. Rejects duplicate node:endpoint pairs in-batch and against existing zones; warns when a mapped verb has no template='zone' row in the controller's command set. Reload is deliberately different from the other mappers: zones always reload, so commands work immediately, but the controller is not reloaded unless asked (reload_device) because that restarts the Z-Wave serial driver and re-interviews the mesh — attribute feedback stays dark until the controller reconnects, so map the whole batch, then reload the controller once. Atomic with rollback. Run it with dry_run first when profiles were inferred from node names. Cap: 60 zones. |
bulk_create_site_spaces | Atomic batch room/floor/area creation with in-batch parent_name resolution. |
bulk_create_site_modes | Atomic batch site mode creation (Home / Away / Vacation / Party / Sleep / …) — each with enter/exit actions (macro references or inline subsystem commands), at most one default across the batch, auto-assigned sort order, and rollback. |
bulk_create_triggers | Atomic batch attribute_trigger creation with template + targets fan-out. |
bulk_create_schedules | Atomic batch macro_schedule creation with natural-language at/days → cron. |
bulk_create_channels | Atomic batch channel-lineup commissioning under one provider. |
bulk_create_remote_keys | Atomic batch key mapping on one remote — many buttons mapped to macros, device commands, AV sources, or multiview sources in a single call. Every key is validated against the fixed remote vocabulary and every macro/device/command/source is resolved up front (all-or-nothing); existing keys are skipped by default (replace_existing overwrites in place) and the remote is reloaded so mappings take effect immediately. |
bulk_create_av_sources | Atomic batch add of av_sources to an existing av_zone. |
bulk_create_av_zones | Atomic batch creation of many distinct av_zones, each with its own backing zone + per-zone sources. |
bulk_create_commands | Atomic batch add of outbound command rows (name + template + args + arg_options) to an existing command_set, with skip/replace-on-collision and rollback. |
bulk_create_widgets | Atomic batch add of ui_page_widget rows to a ui_page. |
bulk_create_ui_pages | Atomic batch ui_page creation with optional pre-seeded widgets and multi-UI linking. |
create_camera_wall | One-call camera dashboard — resolves camera zones (explicit ids, name substrings, or a pattern like "every zone containing camera"), creates or reuses a ui_page, places a single camera-grid widget covering all of them, and optionally links the page to one or more UIs. Atomic with rollback; capped at 40 cameras per wall. |
create_energy_dashboard | One-call power / energy monitoring page — the energy counterpart to create_camera_wall. Surveys the site's live metric_role tags with the same resolvers the widgets use and auto-selects only the panels that have data (Power Flow, Battery Backup, Self-Sufficiency, Energy Today, Top Circuits), pins them to the single power device or aggregates several, creates or reuses a ui_page, places every panel, and optionally links the page to one or more UIs. Pass an explicit panel list to override the survey; Water Flow and the admin-only Energy & Demand panel are opt-in. Errors instead of building a page of empty tiles when no driver on the site tags power roles. Atomic with rollback. |
create_report | Create a reusable report definition (Unified Activity, Access Activity, Device Health, Request Activity, Alarm Performance, Energy & Demand, or Stale Credentials) and, optionally, its email delivery schedule — cron rule, timezone, period, recipients, and html or csv format. Definitions are declarative and validated by the report registry; the skill cannot author SQL or code. Preview with preview_report first — the schedule is validated before anything is kept, and an invalid schedule removes the definition rather than leaving a half-built report. |
bulk_create_ui_zone_groups | Atomic batch creation of per-subsystem UI zone-group cards — each grouping several zones into one collapsible card, optionally linked to multiple UIs and reassigning existing ui_zone rows into the group. Resolves subsystems / UIs / member zones by name or id, rejects cross-subsystem members, reloads each affected UI, and rolls back every group + link + reassignment on any failure. (Groups already-visible zones — run bulk_assign_zones_to_ui first if a zone isn't on the UI yet.) |
bulk_create_macros | Atomic batch macro creation (each with optional steps) for scene-per-room rollouts. |
bulk_create_monitors | Atomic batch ping/HTTP/TCP monitor commissioning with device-ip auto-resolve and in-batch depends_on chains. |
bulk_create_monitor_tags | Atomic batch creation of many monitor tag rollup groups — each with a health rule (anchor / any / all / threshold), member monitors, and upstream dependency edges — with a shared defaults block, in-batch dependency resolution + topological ordering, cycle rejection, and rollback. Wired through the live monitor cache so rollups arm immediately (no reload). |
bulk_create_variables | Atomic batch automation-variable commissioning (weather / sun_moon / usgs / google_calendar / date_and_time), with driver validation, per-driver prefix-collision checks, and interval resolution — starts each variable's timer on success. User-written variable drivers are selectable too, and their getMetaData() supplies the default interval and produced-variable names the same way a built-in's does. |
bulk_create_users | Atomic batch auth_user creation — each entry with its own roles + profile + sites + credentials, a shared defaults block, in-batch PIN/username collision checks, and full rollback. |
create_av_zone_with_sources | Atomic AV zone + sources + placeholder macros. |
create_scene | Author a multi-zone scene macro from per-zone intent (off / on / level / color_temp), auto-emitting the right command steps. |
snapshot_scene | Capture the current live state of a set of zones into a new scene macro — "save the room the way it is right now". Scoped by zone ids/names, a subsystem (e.g. all lights), or a room keyword; reads each zone's live level / color temp / shade position / lock / arm state and hands the result to create_scene (same validation + rollback). Off/closed zones are captured by default so recall reproduces the exact look (skip_off captures only what's on); climate setpoints and AV are not captured and are reported as skipped. |
bulk_create_scenes | Atomic batch counterpart to create_scene — many scene macros in one call (e.g. the standard Movie / Dinner / Goodnight suite, or bright/dim/off per room), each fanning command steps to its zones, with shared defaults, full pre-validation, and rollback. |
create_climate_schedule | One-call thermostat setback schedule ("70 at 7am, 62 when we leave, 70 at 5pm, 64 at 10pm") — for each time-of-day period it builds a setpoint macro (one setpoint step per thermostat, single or heat/cool dual setpoints) plus the macro_schedule that fires it, all cross-linked, pre-validated (setpoint safe range, time/day parsing, name collisions), and rolled back as a whole batch on any failure. |
create_lighting_schedule | One-call time-of-day lighting schedule ("on at dusk, dim to 30% at 11pm, off at sunrise") — the lighting sibling of create_climate_schedule. For each period it builds a lighting macro (one on/off/level step per zone) plus the macro_schedule that fires it. Accepts clock times and sun-relative tokens (sunset, dusk, sunset-15, sunrise+30) resolved against the built-in sun/moon variable — sun-relative periods fire every day (day masks don't apply to them). Pre-validated and rolled back as a whole batch on any failure. |
create_shade_schedule | One-call daily shade/blind schedule ("open at sunrise, close at sunset", "half closed at noon") — the shades sibling of create_lighting_schedule and create_climate_schedule. For each period it builds a shade macro (one open/close/level step per zone) plus the macro_schedule that fires it, with the same clock-or-sun-relative timing (sunset, dusk, sunset-15, sunrise+30). Two things are shade-specific: position runs the other way from a lighting level — 0 = fully closed, 100 = fully open — with the endpoints dispatched as the explicit close/open verbs so non-positional relay/sequence shades still work; and each period may name its own zone subset, which collapses directional glare control into one call (open everything at sunrise, close only the east shades at 8am, only the west ones at 3pm). Zones that aren't on the shades/blinds subsystem are wired but reported in notes. Pre-validated and rolled back as a whole batch on any failure. Not for continuous sun tracking — that's a trigger on a solar-angle variable, not a schedule. |
create_irrigation_program | One-call sequential sprinkler/irrigation program ("zone 1 for 12 minutes, zone 2 for 8, zone 3 for 15, starting at 5am"). Unlike the lighting and climate schedules — which fire every zone at once — watering runs one valve at a time, so this builds a single program macro of start → wait → stop steps in the order the stations are listed, plus one macro schedule per start time. Every delay is derived from the station's own run time, so the classic hand-built bug (a delay that doesn't match the duration, leaving two valves open) can't happen. Options no other skill reaches: cycles + soak_minutes (cycle-and-soak, so water soaks in instead of running off), seasonal_adjust_percent (scale every run time at once), days: 'odd'/'even' for municipal watering restrictions, and skip_if_rain_chance_over (attaches a run/skip condition on the weather variable's precipitation chance). Warns when two start times are closer together than the program's total runtime. Pre-validated and rolled back as a whole batch. Not for controllers that own their programs in the vendor cloud (Rachio, Hydrawise) — programming GEM on top double-waters. |
create_macro_with_steps | Atomic macro + steps with metadata-validated drivers. Each step may carry the same authoring metadata the step editor writes — label, notes, output, timeout and an on_error policy. |
create_macro | Create one empty macro. |
create_trigger | Create one attribute_trigger. |
create_schedule | Create one macro_schedule. |
create_access_rule | Create a physical access_control rule (reader + users + action). |
bulk_create_access_rules | Atomic batch access_control rule creation with a shared defaults block, natural-language day/hour parsing, and full rollback. |
bulk_create_holiday_calendars | Atomic batch holiday_calendar creation — each with its own dates (single YYYY-MM-DD days and from/to ranges), full date validation, name-collision checks, and rollback. |
bulk_create_schedule_exceptions | Atomic batch schedule exception creation — each window (dates, optional times, yearly recurrence) wired to its affected macro_schedules with a per-link action (skip / substitute / only_during), a shared defaults.schedules list for "same skip list on every holiday" sets, full date/link validation, and rollback of both exceptions and links. The scheduler picks new exceptions up on the next fire — no reload. |
bulk_create_alarms | Atomic batch alarm definition creation — condition + severity + deadband/delays + notify/escalation steps, with a shared defaults block, full pre-validation, rollback, and a single engine reload so new alarms arm immediately. |
create_notification_profile | Create notification_profile + bulk-assign auth_users. |
bulk_create_notification_profiles | Atomic batch creation of many different notification_profiles — each with its own delivery methods / day-hour mask / user assignments, a shared defaults block, in-batch and against-DB name-collision checks, and full rollback. |
create_user_with_roles | Atomic auth_user + roles + profile + sites + credentials. |
create_role | Build a new auth_role from scratch — name, the six allow/deny permission grids (UI, route, API) and the Status / Allow Remote / Elevated / Skip 2FA on Local Network switches — in one call. The roles editor is six independent checklists over a wall of 200+ auto-enumerated API functions, and a mistake is silent in both directions: a missed deny is a permission escalation, a missed allow is a lockout nobody notices until the user calls. The skill composes the rule tokens against the live permission catalog, so UI and function names are validated rather than guessed, and returns a security digest (API scope, elevated, remote, UI scope, route scope, denied count) to read back before you confirm. Optional starting preset — admin, resident, staff (scoped to named UIs, no remote), viewer (read-only: every state-changing function denied by name) — with any field you pass overriding it. Scope by subtraction: it starts from allow api * and carves out with denies, because an allow list built by addition almost always omits the boot-path functions and produces a role that signs in to a blank screen. Admin-only functions passed in the allow list are stripped by default and reported, since naming one explicitly is the documented bypass of the elevated-session gate. Run it with dry_run first when the requested scope is vague. Skip 2FA on Local Network is the one switch no preset sets — it is a security relaxation, so it is always an explicit opt-in and defaults off, and the skill warns when it is combined with Elevated. Use clone_role instead whenever a suitable template role already exists; undo deletes the role, and refuses once it has been assigned to users. |
create_ui_room | Create a UI for a room with auto-discovered controls. |
configure_device | Configure an existing device's network/auth attributes. |
add_device_from_discovery | Add a single device discovered via network scan. |
discover_network | Scan the local network for new devices. |
wire — link existing entities together
| Skill | One-line summary |
|---|---|
bulk_assign_zones_to_ui | Wire many zones to a manual UI (ui_zone rows). |
bulk_assign_macros_to_ui | Wire many macros as scene-buttons on a manual UI (ui_macro rows). |
bulk_assign_pages_to_ui | Link many existing ui_pages to a UI (ui_page_ui rows). |
link_zones_to_site_spaces | Auto-suggest or apply zone → site_space linkage based on name/label matching. |
remap_av_zone_devices | Rebind av_source.device_id across one or many av_zones. |
repoint_device | Move every reference from a replaced device onto its replacement — the hardware-swap / RMA skill. Covers all eight tables that carry a device FK: zone bindings, AV sources, the av_zone volume device, network monitors (probe IP rewritten from the new device), macro steps, trigger conditions and gates, subsystem controllers, and access-control readers. The macro-step and trigger references live inside JSON blobs the admin UI cannot search, so a hand-done swap tends to leave dead device ids behind that fail silently at runtime. Call it with dry_run first to get the full reference inventory ("what still references the old amp?"). Refuses a cross-driver swap unless explicitly allowed, and rolls back every write on any failure. Remote-key (command_macro) and device-script rows key on the device id itself and are reported for manual re-creation rather than moved. |
clone — deep-copy existing entities
| Skill | One-line summary |
|---|---|
clone_ui | Deep-copy a UI with all its zones, controls, macros, and page links. |
clone_ui_page | Deep-copy a single ui_page (and its widgets) with optional re-linking. |
clone_widget | Deep-copy a single dashboard widget (its config intact) to one or many new names, with per-clone config overrides and optional copying of its page placements. |
clone_macro | Clone a macro to one or many new names — the copy keeps the original's run policy and each step's label, notes, output name and error policy. |
clone_trigger | Clone a trigger to one name OR fan out across many targets. |
clone_schedule | Clone a schedule to one or many new names. |
clone_av_zone | Deep-copy an av_zone with all sources (and optionally their macros). |
clone_channel_provider | Deep-copy a channel provider with every child channel row (number + name + call sign + logo + enabled/favorite flags + sort order) to one or many new providers — fast-path for fanning a hand-built lineup across identical hardware (two Rokus with the same apps, three cable boxes with the same channels). The external listings ID is not copied by default, so clones start as manual lineups. |
clone_command_set | Deep-copy a command_set with every child command row (name + template + args + arg_options + description + system flag) to one or many new sets — fast-path for commissioning a vendor whose protocol resembles an existing one. |
clone_zone | Deep-copy a plain zone with every attribute row (and optionally its ui_zone links) to one or many new zones, with per-clone device/address overrides. |
clone_device | Deep-copy a device with every attribute row (and optionally its child zones) to one or many new devices, with per-clone ip/port/credential overrides — secure attributes are skipped by default so credentials aren't silently fanned across clones. |
clone_alarm | Deep-copy an alarm definition — conditions, notify/escalation ladder, actions, deadband/delays/latching/repeat/cooldown — to one or many new names, or "stamp" it across many similar targets with the condition's target rewritten per clone. |
clone_user | Clone an auth_user (roles + sites + notification profile + flags) to one or many new accounts — credentials are never copied. |
clone_role | Clone an auth_role (its full rules array + the is_elevated / allow_remote / enabled / allow_local_2fa_bypass flags) to one or many new roles, with optional per-clone rule patches (append / remove rules) or wholesale rule replacement. Flags are inherited from the source unless overridden, so a clone behaves like the role it came from. |
clone_notification_profile | Clone a notification_profile (delivery methods + day/hour masks) to one or many, optionally copying user assignments. |
clone_monitor | Deep-copy a network monitor — header, check interval, dependency chain, and full config JSON (notification lists, retries, timeouts, type settings) — to one or many new names, or "stamp" it across many IP targets with the ip rewritten per clone. Clones start probing immediately. |
clone_access_rule | Clone an access_control rule (reader + users + days/hours + action) to one or many — name-only, or "stamp" the same rule across many readers and/or many users in one atomic call. |
clone_site_mode | Deep-copy a site mode — label, description, and both enter/exit action lists — to one or many new modes, with per-clone overrides that replace or append actions. The default flag is never copied, so the existing default mode stays intact. |
clone_site_space | Deep-copy a site space subtree — a floor/wing plus every room nested under it — to one or many new spaces ("duplicate floor 2 as floor 3"). Each descendant gets a derived unique name (labels copy verbatim, so the tree still reads Floor 3 → Bedroom 1), and each node's attributes and blueprint floor-plan art come along. Zones are never copied — they're physical hardware; reassign them afterward with link_zones_to_site_spaces. Refuses to clone the root space, and rolls back every row + attribute on any failure. |
clone_ui_zone_group | Clone a UI zone-group card (with its UI links) to one or many new groups — the fan-out counterpart to bulk_create_ui_zone_groups. Member zones either move to the clone (single-clone default, source ends up empty), stay on the source with clones starting empty (copy, multi-clone default), or a chosen subset moves. Per-clone label / sort / UI-link overrides; cross-subsystem clones refuse to move members; reloads each affected UI and rolls back every group + link + reassignment on any failure. |
edit — modify existing rows
| Skill | One-line summary |
|---|---|
update_model | Update fields on one row in any table. |
update_macro_steps | Atomic add / update / remove / renumber for steps in an existing macro. |
add_macro_step | Add a single step to an existing macro, then reload the macro so a running system picks the step up immediately. Covers every step type the server registers, containers and timers included, and takes the step's label, notes, output and on_error policy. |
insert_model | Insert one row into any table. |
batch_insert | Insert multiple rows into one table in a single transaction. |
set_attribute | Set one attribute value. |
build_widget | Create or update a ui_widget definition (with code). |
compose_widget | Create or update a designer (composed) widget from a validated element tree — no code; the result stays visually editable on the UI Widgets page. |
bulk_edit — same change to many rows
| Skill | One-line summary |
|---|---|
bulk_set_attribute | Set the same attribute on many targets (with dry_run + rollback). |
bulk_toggle_enabled | Flip the enabled flag on many rows (with dry_run + rollback). |
bulk_rename | Find/replace the name and/or label columns across many rows (with dry_run + rollback). |
run — execute or undo
| Skill | One-line summary |
|---|---|
run_macro | Run a macro on demand. |
send_command | Dispatch a single command to a device or zone. |
undo_last_action | Revert the last update_model or set_attribute change. |
reload | Reload a single entity from the database into memory. |
discover — read-only lookups
| Skill | One-line summary |
|---|---|
list_skills | Return this catalog grouped by category. |
query_database | Generic read-only lookup against any table (filters plus % wildcards). Credential and session tables (auth_user, auth_role, ui_client, license, and similar) are not readable through it, and secure attribute values are never returned. |
describe_attributes | List attributes on an entity, enriched with registry metadata. |
preview_report | Run a proposed report definition without saving it — returns at most 25 rows plus the summary so the filters and columns can be agreed before create_report. Picture bytes and raw request payloads are never returned to the AI. |
describe_macro_step | Get the data-payload schema for one macro step driver — or several at once via its drivers array, so a multi-step macro needs only one lookup round. |
get_attributes | Get current attribute values for an entity. |
get_zone_states | Read live state attributes for one or many zones. |
get_alarms | Read the alarm center's open events and recent history — severity, acknowledge/shelve state, and the per-recipient notification delivery log. |
alarm_context | Deep-dive on ONE alarm event for a causal brief — the rule that fired and its tripping value, the source entity's live state, the attribute timeline around activation, 30-day recurrence, and correlated alarms / automation runs / user actions in the same window. Behind the Explain with AI button on each alarm row. |
diagnose | Full configuration audit — the same 28-check battery as the Diagnostics page (orphans, duplicates, empty/unused, config smells, naming, connectivity, security, integrity, performance), filterable by category/severity. |
navigate | Open an admin page or entity in the UI. |
search_docs | Search the GEM documentation. |
read_upload | Page through (offset / limit) or search (find) a spreadsheet attached to the current chat, or list every attached file with its sheets and row counts when called with no arguments. Reads only chat uploads — never files on the controller or URLs. See Attaching spreadsheets and images. |
recall_memory_notes | Search persistent per-site memory notes captured in prior sessions. |
notes — capture knowledge
| Skill | One-line summary |
|---|---|
record_attribute_note | Attach a note to a specific attribute. |
update_registry_description | Update the canonical description in the attribute registry. |
record_memory_note | Save site-specific knowledge that future sessions should know. |
automate — multi-artifact one-shots
| Skill | One-line summary |
|---|---|
create_motion_automation | Wire one motion sensor to N target zones with on/off macros and triggers in one call. |
create_scene (build one scene macro that fans out command steps to many zones at known levels / colors) is a closely related one-shot, but list_skills files it under create — you'll find it in that table above.
Workflow patterns
A few recurring shapes the assistant uses well:
Commission then wire. Use bulk_create_zones (or device commissioning) to bring rows into existence, then link_zones_to_site_spaces to bin them into rooms, then bulk_assign_zones_to_ui (manual UIs) — or skip the last step entirely if the install uses space-driven UIs (the 90/10 default).
Suggest then apply. Several skills support a discover/commit split: link_zones_to_site_spaces returns ranked suggestions when called with no mapping, then is called again with the user-confirmed mapping to write. bulk_set_attribute and bulk_toggle_enabled accept dry_run: true for the same purpose. Prefer this two-call shape whenever scope might be broader than expected.
Atomic with rollback. Every bulk_* and clone_* skill validates the whole batch first and rolls back every partial write if any one step fails. This means it's safe to rerun after a failure once you've fixed the offending entry — the state is exactly as it was before the failed call.
Stored data is a public API. The assistant never silently migrates DB rows on a write path. If a stored macro / trigger / attribute predates a feature, the new code paths default to the old behavior; the new shape is opt-in.
Long tasks and "continue". Each request runs within a per-turn tool budget (currently 30 rounds, where one round can carry many parallel lookups). Large multi-device tasks — say a trigger plus a macro touching several devices — normally fit, but if the budget runs out mid-task the assistant stops and reports exactly which tools it ran and whether any changes were already applied. Nothing is lost: the full tool exchange stays in the conversation, so replying "continue" resumes with everything gathered so far instead of starting over.
What query_database cannot read
Tool results are replayed back to the AI provider on every subsequent turn of a conversation, so anything the generic lookup skill returns leaves the building. Two categories are held back from query_database regardless of who is asking:
- Credential and session tables —
auth_user,auth_role,ui_client,licenseand the other credential stores. Asking for one returns a refusal, not rows. Skills that legitimately create or modify users (create_user_with_roles,bulk_create_users,clone_user) still work; they write without reading secrets back. - Secure attribute values — device passwords, integration API keys, resident PINs. Querying the
attributetable returns non-secure rows only, and any row that carries a sensitive name comes back with a blank value even if it was never flagged secure. Secret-bearing columns (hash,salt,pin,token,api_key, …) are blanked on every table. - Query directives that would route around either guard —
_instance(which merges the live in-memory object, secure attributes already decrypted),_include(which joins associated tables, side-stepping the table check) and_strip_secure_attrsare dropped from the assistant's query before it runs, whatever the model asked for. Ordinary filters, wildcards, ordering and limits are unaffected.
Enabling the assistant, generating a schedule from natural language, and the undo actions are all elevated operations — a non-elevated role cannot reach them even with a wildcard API grant.
The voice surface honours the same table reservations. Voice commands are open to any authenticated session — that is the point of a wall panel — so the two skills that read the credential and physical-access tables are restricted to an elevated role:
who_has_access("who has access to the house?", "does the cleaner still have a code?") readsauth_user,access_controlandaccess_group.access_history("did the kids get home?", "who unlocked the front door?", "any denied attempts?") readsaccess_log.
From a non-elevated session — a guest tablet, a lobby PIN panel, a kiosk — both answer "that needs an administrator account" instead of the data. Without this, a panel that is refused the same tables on the data query surface could enumerate the whole credential roster and the door-entry log simply by asking in English. The restriction follows the role, not the session: an elevated role gets the same answer by voice as it does in the admin UI, including from a PIN-minted panel session. Everything else on the voice surface — house state, climate, lighting, scenes, presence — is unchanged.
set_site_mode is restricted the same way, since on a standard configuration a mode change disarms security zones and unlocks entry doors; asking what the current mode is still works from any session.
Confirmation gate
Skills that write to the system trigger a one-line confirmation prompt in the drawer before they execute — anything in create, wire, clone, edit, bulk_edit, notes, or automate, plus run_macro, send_command, add_device_from_discovery, reload and undo_last_action. The prompt shows the resolved description (e.g. "Wire 12 zones to UI 'security_panel'") — not the raw call — so you can sanity-check intent at a glance. Read-only lookups (the discover group) run without prompting.
send_command is the one skill that moves physical hardware, so its prompt names the resolved target and the verb rather than the tool — "Send "unlock" to zone "Front Door" — ELEVATED (lock / gate / alarm target)" — and flags the target when it is one the integrator marked Elevated. Read the target, not just the verb: the assistant's turn can include text that came from a device on your network (an advertised device name, for instance, is written into the attribute table and read back by later tool calls), and the confirmation is what keeps that text from reaching a door strike on its own.
The generic write skills — insert_model, batch_insert and update_model — name the privilege a row would carry, not just its table. They accept any table, so a proposal to create an auth_role arrives the same way one to create a ui_widget does, and "Create new auth_role record" is not enough to approve on:
auth_role— the Elevated switch, Allow Remote, Skip 2FA on Local Network, and any wildcard tokens in the rules, e.g. "Create new auth_role record "svc" — ELEVATED (full admin surface), wildcard grants: allow api *, allow ui *". Wildcard denies are not listed, since those narrow a role rather than widen it.auth_user— the roles the account would hold, resolved to names, and flagged when any of them is elevated (an auth_user has no Elevated switch of its own — it inherits one from every role it holds). Also whether a PIN, RFID or duress PIN is being set, and whether two-factor is being turned off. Credential values are never rendered — the prompt says "pin+duress_pin set" and nothing more.access_control/access_group— the reader the row would open and how many people or groups it would open it for.update_modelon those tables shows the direction a flag moves — "is_elevated: false → true" — rather than the field name alone, which said nothing about which way it went. A row the server does not have cached shows?as the before-value rather than assuming one.
Everything else reads exactly as it did.
Click Allow to run the action or Deny to cancel it. Deny means stop — the assistant cancels that action and ends the turn rather than retrying a different way, so it won't re-prompt you in a loop. Just type your next instruction to continue.
With Auto mode on, configuration writes skip this prompt; the hardware-moving and access-changing actions listed there still stop and ask. A prompt that is still waiting when you interrupt the turn is treated as Deny.
One skill runs without a prompt even though it appears in a write-oriented group: discover_network, a read-only network scan that changes nothing. Everything else in create and run prompts, including add_device_from_discovery, send_command, reload and undo_last_action.
Interrupting and redirecting
A running turn can be stopped at any point, the way you would interrupt a colleague who has misread the request:
- Stop — while the assistant is working, the Send button becomes Stop. Click it (or press Esc in the input) to abort the turn. The provider request is cancelled, any tool that has not started is skipped, and a confirmation prompt that is still waiting is answered no. A tool that is already mid-write is allowed to finish, because a half-applied bulk change is worse than a completed one — the undo log covers whatever did run.
- Type to redirect — the input stays live while a turn runs. Type the correction and press Enter: the drawer stops the current turn, waits for the server to confirm it has unwound, then sends your message. The assistant sees its own cut-off work — the text it had produced and the results of the tools that did run — followed by your correction, so "no, the other Sonos" lands with full context instead of starting over.
Interrupted turns are marked in the chat, and the tools that were skipped are reported as not executed rather than silently dropped.
The script editors' AI sidebar has the same Stop button. Its input is locked while a turn runs, so redirecting there is stop first, then type. Stopping one editor's chat does not touch the admin drawer's turn, or the other way round.
Auto mode
The Auto toggle in the drawer header turns off the per-action confirmation prompt for configuration writes — creating zones, wiring AV sources, adding macro steps, reloading. Each auto-applied action is still listed under the reply, still goes into the undo log, and Undo / Revert All work exactly as they do in confirm mode. The setting is remembered per browser.
Some actions keep asking even in auto mode, because a mistake there is not a configuration fix away:
- anything that moves hardware —
send_commandandrun_macro undo_last_action(reversing visible work)- anything that changes who can get in — creating or cloning users, roles, and access rules, and the generic
insert_model/update_model/batch_inserttools when aimed atauth_user,auth_role,access_control,access_group,licenseorui_client
Auto mode is per request: the drawer sends the flag with each message, so switching it off applies to the very next turn.
Attaching spreadsheets and images
The paperclip button next to the message box attaches files to the next message; you can also drop files onto the chat or paste an image from the clipboard. Up to five attachments go with one message.
- Spreadsheets —
.xlsx,.xlsm,.csv,.tsvand plain-text tables, up to 5 MB. The controller parses the file itself (no third-party parser, nothing leaves the building except what the model is shown) and hands the assistant each sheet as a text grid: the first 300 rows inline, and the rest through theread_uploadskill, which pages through a sheet or searches its rows. So "here are the zones for the new wing, create them" works from a 40-row sheet or a 4,000-row one — the assistant reads what it needs, shows you its column mapping and a sample row, and asks before writing. Old.xlsfiles are refused; save them as.xlsxor.csvfirst. Dates come through as ISO dates, formulas as their last calculated value. - Images — PNG, JPEG, GIF, WebP. A floor plan, a photo of a rack or a device label, a screenshot of a vendor's configuration page. Large images are downscaled in the browser before upload, so a phone photo is fine.
Uploaded spreadsheets stay in memory for the length of the conversation and are dropped with New Chat; they are never written to disk. Text inside an uploaded file is treated as data, not as instructions — the assistant is told to ignore a cell that tries to give it orders and to mention it to you. With Auto mode on, remember that a sheet from a third party can still steer what gets created; review the mapping before you turn it on for a bulk import.
Cost readout
Every reply shows what it cost: a dollar estimate at Anthropic list prices, the number of model rounds, input tokens with the share that came from the prompt cache, and output tokens. The header keeps a running total for the chat. The estimate is computed from the provider's own usage report; models the assistant has no price for show tokens without a dollar figure. The same summary is written to the server log as ai assistant usage: so a site's spend can be checked without opening the drawer.
Two things keep the per-turn number small. The static part of the prompt (instructions plus the tool set) is cached across rounds and turns, so only the conversation itself is billed at full price. And on Anthropic models from Claude 4.5 onward the tool catalog is loaded on demand: a core set of about two dozen tools is always present and the remaining hundred are fetched by the model's own tool search when a task needs them, which takes roughly 150,000 tokens out of every request. Voice commands use the same on-demand loading: the 20 tools a wall panel hears most (lighting, climate, AV, shades, scenes, security, house and room summaries, delays) stay loaded and the other 56 are fetched by name when an utterance needs them, inside the same request, so there is no extra round-trip. The ai_deferred_tools system attribute turns the on-demand loading off for both surfaces if a provider proxy cannot handle it.
Keeping this page current
This page is a written snapshot. The assistant builds its real catalog at runtime from the skills actually installed on your system, and the list_skills skill enumerates that live set and sorts it into the same categories shown here. So the assistant's own answer to "what can you do?" is always accurate for your version — even when this page lags a release or two behind. When in doubt, ask the assistant directly.