Skip to main content

Architecture

Waylo is a Node.js backend that talks to GNOME's compositor over D-Bus and to the browser over WebSocket, with GStreamer doing the video encoding in between.

Why GNOME Mutter (not xdg-desktop-portal)?​

The standard portal approach can't serve unattended access: it requires a human to approve a dialog and pick monitors on every start, and doesn't persist those choices. Waylo instead calls Mutter's own APIs (org.gnome.Mutter.ScreenCast and org.gnome.Mutter.RemoteDesktop) directly: no dialogs, no manual activation, monitor selection persists, and it works immediately across restarts.

The two sessions are linked — the ScreenCast session is created against the RemoteDesktop session's id. That link is what lets Waylo use absolute pointer positioning against the captured streams.

Backend modules​

  • gnomeBackend.js — creates and links the Mutter RemoteDesktop + ScreenCast sessions, records each monitor, injects input, and handles locking/unlocking (including the locked-screen recovery).
  • streamer.js — manages the GStreamer video pipeline (H.264 or MJPEG), emitting encoded frames.
  • audioStreamer.js — Opus desktop-audio pipeline.
  • clipboard.js — bidirectional clipboard + file-offer sync over Mutter selections.
  • inputHandler.js — maps browser input events to evdev keycodes / keysyms and pointer/scroll notifications.
  • auth.js — verifies credentials over SSH and performs unattended-access setup (auto-login, autolock).
  • server.js — the HTTP + WebSocket server, session lifecycle, rate limiting, upload endpoint, and multi-client fan-out.
  • index.js — entry point and CLI parsing.

Frontend​

A Svelte + Vite single-page app, served by the backend from dist/:

  • VideoStream — decodes the stream (WebCodecs for H.264, <img>/canvas for MJPEG) and captures pointer/keyboard/touch input.
  • Login — credentials and per-session settings.
  • Toolbar — monitor switching, fullscreen, audio, lock, settings, disconnect.
  • stores/ — WebSocket connection + auth, audio, upload, download, and stats.

Session lifecycle​

The backend starts capturing only after a client authenticates; the first client's login brings the Mutter/GStreamer backend up, subsequent clients and pop-out windows attach to it, and it is torn down when the last client disconnects. A client that drops reconnects and re-authenticates automatically, resuming its monitor view. See Video Pipeline for the wire format and Security for authentication.