Video Pipeline
WayLo captures the desktop through PipeWire, encodes it with GStreamer, and streams the result over the same WebSocket that carries input.
Capture
Creating the linked Mutter ScreenCast session yields a PipeWire node per recorded monitor. GStreamer reads from that node, so capture stays on the compositor's own path — no virtual display, and hardware buffers where the stack supports it. The cursor is captured out-of-band and drawn by the client, so pointer motion doesn't repaint (and re-encode) the frame.
Encoding: H.264 with MJPEG fallback
WayLo prefers H.264 and falls back to MJPEG automatically:
- H.264 is encoded at native monitor resolution using
vah264enc(VA-API hardware),x264enc(software), oropenh264enc, and sent as Annex-B access units. The browser decodes it with the WebCodecsVideoDecoder. Inter-frame compression means an idle desktop costs almost nothing and scrolled text stays crisp. The server caches the current group-of-pictures (GOP) so a joining client or a monitor switch starts from a keyframe immediately. - MJPEG is used when the browser lacks WebCodecs, no H.264 encoder is installed, or H.264 decoding fails at runtime. Each frame is a complete JPEG.
The choice is automatic and can differ per client (each connection advertises whether it can decode H.264).
Wire format
Frames travel as binary WebSocket messages; control and metadata travel as JSON text messages. Every binary message begins with a one-byte type tag:
| Tag | Meaning |
|---|---|
0 | H.264 delta (inter) frame |
1 | H.264 keyframe |
2 | JPEG (MJPEG) frame |
Audio is a separate Opus stream (see Clipboard, Audio & Files); JSON messages carry authentication, monitor info, clipboard text, download offers, and stream settings.
Multi-client and monitors
The backend records every monitor and fans the encoded stream out to all connected clients. Each client (including a popped-out window) selects which monitor it views; pop-outs attach to the running session by token rather than re-authenticating. Because the server holds the current GOP, switching monitors or attaching a new client renders instantly from a keyframe rather than waiting for the next one.