Skip to main content

Logging

The Logging page provides real-time viewing of system logs, device communication logs, zone state changes, and client logs. It's the primary tool for troubleshooting and monitoring GEM in real-time.

Overview​

Logging capabilities:

  • System Logs: GEM server application logs
  • Device Logs: Device driver communication
  • Zone Logs: Zone state subscription monitoring
  • Client Logs: Browser/client-side logs from remote devices
  • Live Streaming: Real-time log updates
  • Filtering: Search, severity levels, patterns
  • Export: Save logs for analysis

Log Types​

Select log type from dropdown:

System Log​

Server application logs:

  • Startup/shutdown events
  • Errors and warnings
  • Device connections/disconnections
  • Macro executions
  • Trigger activations
  • Database operations
  • Integration events

Severity Levels:

  • ERROR: Critical errors requiring attention
  • WARN: Warnings, potential issues
  • INFO: Informational messages
  • DEBUG: Detailed debugging info (development mode)

Log Flood Protection​

The server rate-limits its own log output so a runaway device (tight reconnect loop, chatty poll, garbled response parser) cannot overwhelm the system journal or fill the disk. Two layers run automatically:

  • Repeat collapse: a message that repeats rapidly (numbers and hex values are ignored when matching, so changing IDs/addresses still count as repeats) is shown a limited number of times per 10-second window, then suppressed. A summary line is emitted at the original severity: 
    [log-guard] suppressed 412 repeat(s) of "reconnecting #..." in last 10s
  • Global rate cap: a flood of distinct lines is capped by a token bucket, summarized as: 
    [log-guard] dropped 1500 line(s) over the global rate cap in last 10s

On Linux/systemd hosts, GEM also installs journald limits at boot (journal disk usage cap plus a per-service rate limit) so output from child processes — which bypasses the in-process guard — is bounded too.

tip

If you need every raw line while debugging, start the service with the environment variable GEM_LOG_GUARD=off to disable the in-process guard. No rebuild is required.

Device Log​

Device driver communication logs:

  • Commands sent to devices
  • Responses received
  • Connection events
  • Communication errors
  • Protocol-level debugging

Device Selection: Choose specific device to monitor

Zone Log​

Zone state change monitoring:

  • Power on/off events
  • Level changes
  • Attribute updates
  • Real-time state subscription

Zone Selection: Choose specific zone to monitor

Client Log​

Browser console logs from connected clients:

  • JavaScript errors on client
  • Client-side warnings
  • UI rendering issues
  • Client application logs

Client Selection: Choose connected client to monitor

Use Cases:

  • Debug UI issues
  • Monitor specific user's experience
  • Troubleshoot browser-specific problems

Log Display​

Log Format​

Each log entry shows:

[HH:MM:SS] SEVERITY - Log message content

Example:

[14:32:15] INFO - device connected: living_room_receiver
[14:32:18] WARN - command timeout: bedroom_lights
[14:32:20] ERROR - macro execution failed: good_morning

Color Coding​

  • ERROR: Red background/text
  • WARN: Yellow/orange background/text
  • INFO: White/normal
  • DEBUG: Gray/subdued

Auto-Scroll​

Toggle to enable/disable:

  • Enabled: Automatically scrolls to newest log entries
  • Disabled: Manual scrolling (useful when reviewing specific entries)

Line Limit​

Maximum lines displayed:

  • Default: 1000 lines
  • Older lines removed when limit reached
  • Prevents memory issues on long monitoring sessions

Text Filter​

Filter Input: Search for specific text

  • Case-insensitive search
  • Matches anywhere in log message
  • Updates in real-time

Examples:

  • Search "error" - shows only errors
  • Search "bedroom" - shows bedroom-related logs
  • Search "192.168" - shows logs with IP addresses

Exclude Filter​

Exclude Input: Hide matching entries

  • Opposite of filter
  • Remove noise from logs

Examples:

  • Exclude "polling" - hide routine polling messages
  • Exclude "keepalive" - hide connection keepalives

Severity Filter​

Log Level Dropdown:

  • All: Show all severities
  • ERROR: Only errors
  • WARN: Warnings and errors
  • INFO: Info, warnings, and errors

Advanced Search:

  • Enable regex mode
  • Use regular expressions for complex patterns
  • Example: (error|warn|fail)

Search Panel:

  • Click to open advanced search
  • Configure:
    • Pattern (regex)
    • Context before/after (lines)
    • Max results
    • Time range
  • Results show matching entries with context

A regex search is given a few seconds to finish. Certain patterns — nested repetition such as (a+)+$ is the classic — take exponentially longer as a matched line grows, and against a log line with a long run of repeated characters one can run effectively forever. Because the controller matches on the same thread it drives doors, alarms and HVAC on, a search that exceeds its budget is abandoned with "search pattern too expensive" rather than allowed to stall the building. Simplify the expression, or use a plain-text search, and narrow the time range.

Since Dropdown (System Log - Production Mode)​

When viewing System Logs in production mode, a Since dropdown filters log history:

  • Real-time (live stream only)
  • 5 minutes ago
  • 15 minutes ago
  • 30 minutes ago
  • 1 hour ago
  • 4 hours ago
  • 8 hours ago
  • 12 hours ago
  • 1 day ago
  • 2 days ago
  • 1 week ago

This retrieves historical logs from the system journal. In development mode, only live console output is available.

Log Monitoring​

Starting Monitoring​

  1. Select log type (System, Device, Zone, Client)
  2. Select specific target if applicable (device, zone, client)
  3. Click Start or Connect
  4. Logs stream in real-time

Stopping Monitoring​

  1. Click Stop or Disconnect
  2. Streaming stops
  3. Current logs remain visible
  4. Can be exported or cleared
note

A stream also stops on its own when the session that started it ends — logging out, a session expiring, or a different user signing in on the same browser tab. The system journal carries command lines, client identifiers, source IPs and occasionally credential material, so it follows the session that was entitled to it rather than the connection. Log back in and click Start again to resume.

Pausing​

Pause Button:

  • Stops display updates
  • Logging continues in background
  • Useful for reviewing specific entries
  • Paused Count: Shows skipped entries

Resume:

  • Click pause again
  • Displays buffered entries
  • Returns to real-time

Log Navigation​

Severity Navigation​

Jump to next error/warning:

Buttons:

  • Next Error: Jump to next ERROR entry
  • Next Warning: Jump to next WARN entry
  • Next Info: Jump to next INFO entry

Keyboard Shortcuts (if implemented):

  • E: Next error
  • W: Next warning
  • I: Next info

Severity Counts​

Display shows:

  • Error count (red badge)
  • Warning count (yellow badge)
  • Info count (blue badge)

Common Use Cases​

Troubleshooting Device Issues​

Monitor device communication:

  1. Select Device Log
  2. Select problematic device
  3. Start monitoring
  4. Send test command
  5. Review communication:
    • Command sent
    • Response received (or timeout)
    • Errors logged

Debugging Macros​

Monitor macro execution:

  1. Select System Log
  2. Filter: macro name
  3. Run macro
  4. Review execution steps:
    • Macro started
    • Each step execution
    • Completion or error
    • Step-by-step timing

Monitoring Zone State​

Watch zone state changes:

  1. Select Zone Log
  2. Select zone
  3. Start monitoring
  4. Control zone from UI
  5. See real-time state updates

Client-Side Debugging​

Troubleshoot UI issues:

  1. Select Client Log
  2. Select connected client (specific tablet/browser)
  3. Start monitoring
  4. User reproduces issue on their device
  5. Review client-side errors in real-time

Network Troubleshooting​

Identify connectivity problems:

  1. Select System Log
  2. Filter: "disconnect" or "timeout"
  3. Review disconnection events
  4. Identify pattern (all devices, specific network, specific time)

Exporting Logs​

Copy: Select text and copy to clipboard

Save: Use browser save/print to PDF

External Tools: Stream logs to external logging systems:

  • Syslog
  • Splunk
  • ELK Stack
  • CloudWatch

(Requires custom configuration)

Best Practices​

  1. Start Specific: Monitor specific device/zone rather than system-wide

  2. Use Filters: Reduce noise with appropriate filters

  3. Pause to Review: Pause when you see the issue to examine closely

  4. Save Important Logs: Screenshot or export critical error sequences

  5. Clear Between Tests: Clear logs between test runs for clarity

  6. Note Timestamps: Correlate logs with user actions by time

  7. Multiple Windows: Open multiple logging windows for correlation

Troubleshooting​

No Logs Appearing​

Check:

  1. Monitoring started (connected)
  2. Selected device/zone is active
  3. Network connection to GEM server
  4. Browser console for client-side errors

Too Much Noise​

Solutions:

  1. Use exclude filter for routine messages
  2. Increase severity filter (ERROR only)
  3. Monitor specific device instead of system
  4. Disable debug logging (if enabled)

Logs Slow or Laggy​

Causes:

  • Too many log messages per second
  • Browser struggling with updates

Solutions:

  1. Increase line limit (less frequent trimming)
  2. Disable auto-scroll
  3. Use filters to reduce volume
  4. Close other browser tabs

Cannot Find Specific Event​

Techniques:

  1. Use search panel with time range
  2. Search for specific keywords
  3. Use regex for pattern matching
  4. Export logs and use external tool (grep, etc.)
  5. If the event was a rapidly repeating message, check for a [log-guard] summary line — the individual repeats may have been collapsed (see Log Flood Protection above)