Performance Logging

The dashboard writes structured JSONL timing events to:

• /data/aurora/products/dashboard/dashboard_perf.jsonl

The log rotates automatically.

aurora-dashboard-perf-summary.timer runs every 15 minutes and writes derived watch files under:

• /data/aurora/products/dashboard/perf_summaries/latest_24h.json
• /data/aurora/products/dashboard/perf_summaries/latest_24h.md
• /data/aurora/products/dashboard/perf_summaries/history_24h.jsonl

The JSON and Markdown files are the latest 24 h view. The history JSONL keeps one summary record per timer run, so we can look back at usage, slow renders, session counts, and frequently changed controls even after the raw timing log rotates. Session summaries include mobile/tablet/desktop/bot counts derived from user agents plus the most common request paths, which helps separate phone-specific loading problems from desktop/browser workflows.

Operations monitoring records whether this log exists, how large it is, and when it was last written. The Operations Dashboard shows that as the Dashboard perf log card so stale browsing telemetry is visible alongside the source-transfer checks. It also reads recent timed events directly and reports the current render-performance summary, including p50, p95, the slowest event, and the maximum live browser-session count seen in the sampled log window. These performance signals are diagnostic only; they do not drive the Operations Dashboard Overall action state or the health report's overall_level.

Environment controls

• AURORA_DASHBOARD_PERF_LOG
• AURORA_DASHBOARD_PERF_LOG_MAX_BYTES
• AURORA_DASHBOARD_PERF_LOG_BACKUP_COUNT
• AURORA_DASHBOARD_PERF_ENABLED
• AURORA_DASHBOARD_SESSION_HEARTBEAT_MS
• AURORA_DASHBOARD_ASSET_PREFIX
• AURORA_RENDER_DEBOUNCE_MS
• AURORA_INTERACTIVE_RENDER_CACHE_SIZE
• AURORA_INTERACTIVE_MAX_TIME_SAMPLES
• AURORA_POWER_INTERACTIVE_MAX_TIME_SAMPLES
• AURORA_MET_INTERACTIVE_MAX_TIME_SAMPLES
• AURORA_RADIATION_INTERACTIVE_MAX_TIME_SAMPLES
• AURORA_OPS_INTERACTIVE_MAX_TIME_SAMPLES
• AURORA_SUMMARY_COARSE_TIME_SAMPLES
• AURORA_POWER_LATEST_CACHE_ROUND_MINUTES
• AURORA_POWER_LATEST_CACHE_TOLERANCE_MINUTES
• AURORA_PREWARM_LATEST_CACHE_TOLERANCE_MINUTES
• AURORA_DASHBOARD_PREP_WORKERS
• AURORA_POWER_PREWARM_WORKERS
• AURORA_POWER_PREWARM_MAX_TIME_SAMPLES
• AURORA_POWER_GENERAL_CACHE_ROUND_MINUTES
• AURORA_POWER_DISPLAY_SUMMARY_FREQ
• AURORA_POWER_DISPLAY_ENERGY_FREQ
• AURORA_MEASURE_PLOT_JSON_BYTES
• POWER_DISPLAY_SUMMARY_ZARR_PATH
• POWER_DISPLAY_ENERGY_ZARR_PATH
• AURORA_INTERACTIVE_PREWARM_DIR
• AURORA_OPS_TREND_CACHE_TTL_MINUTES
• AURORA_OPS_TREND_DAYS
• AURORA_QUICKLOOK_MAX_TIME_SAMPLES

AURORA_DASHBOARD_SESSION_HEARTBEAT_MS defaults to 0, which disables per-session heartbeat logging. That avoids keeping mobile browser sessions alive on the server after the phone has backgrounded or killed the tab.

The dashboard logo and shared stylesheet are served as normal static assets under AURORA_DASHBOARD_ASSET_PREFIX, rather than being embedded in every HTML document. This lets browsers cache them independently of the live dashboard document and prevents the same CSS from being repeated in Panel models.

The native Power API bounds each trace to AURORA_MOBILE_POWER_MAX_POINTS (default 160) using per-bucket extrema, preserving visible peaks and dips without asking Swift Charts to draw thousands of points. The offline Power prewarm writer uses up to AURORA_POWER_PREWARM_WORKERS processes (default 2) only for JSON serialization; it does not share or mutate live Zarr or Panel state.

Browser Power prewarms use AURORA_POWER_PREWARM_MAX_TIME_SAMPLES (default 120, capped at 700) per trace. Sampling retains the local minimum and maximum of each time bucket, so short load and voltage events remain visible while the first browser payload stays compact.

AURORA_DASHBOARD_PREP_WORKERS defaults to 2 and is capped at 4. It is used only for custom browser Power windows that cannot use prewarmed JSON. The executor prepares Zarr/NumPy/Plotly data away from the Panel event loop; the finished figure is committed through a document callback. A new control change cancels the awaiting stale request, so only the newest completed result can update the browser.

Useful commands

tail -f /data/aurora/products/dashboard/dashboard_perf.jsonl
/opt/aurora-cloud-dashboard/venv/bin/python summarize_dashboard_perf.py --hours 24
/opt/aurora-cloud-dashboard/venv/bin/python summarize_dashboard_perf.py --hours 6 --event interactive_view_update
cat /data/aurora/products/dashboard/perf_summaries/latest_24h.md
tail -n 20 /data/aurora/products/dashboard/perf_summaries/history_24h.jsonl
systemctl status aurora-dashboard-perf-summary.timer
journalctl -u aurora-dashboard-perf-summary.service --since '1 hour ago' --no-pager

Logged event families

• base_dataset_open
• dataset_time_bounds
• dataset_time_bounds_cache_hit
• interactive_render_deferred
• interactive_render_cache_hit
• interactive_render_debounced
• interactive_prewarm_load
• interactive_prewarm_load_async
• power_background_prepare_scheduled
• power_background_prepare
• window_open
• power_display_summary_open
• power_display_summary_window
• power_display_section_open
• power_display_energy_open
• power_display_energy_window
• interactive_view_update
• interactive_render_budget_exceeded
• hatpro_render
• stacked_timeseries_render
• science_quicklook_render
• housekeeping_quicklook_render
• wxcam_interactive_render
• wxcam_calendar_options
• wxcam_calendar_day_view
• wxcam_calendar_sync
• operations_dashboard_render
• session_loaded
• session_heartbeat
• session_destroyed
• ui_selection_change
• plot_relayout

The interactive browser also uses a few runtime behaviors that affect how these events should be interpreted:

• per-instrument pane reuse keeps the last rendered view warm while a refresh is queued
• exact interactive views are cached for the most recent instrument/window combinations, so returning to a recent view can repaint immediately
• rapid widget-change bursts are debounced before rendering to avoid duplicate back-to-back Plotly builds
• dataset time bounds and latest timestamps are cached briefly
• the periodic metadata timer invalidates timestamp bounds only; open Zarr handles are reopened by the live-refresh path when they age past the configured data-refresh interval
• stale-render protection drops older queued renders before they can repaint the page
• custom Power preparations run in the bounded background executor; cancelling a superseded request never mutates the active Bokeh document
• the matching cached Science Quicklook is shown first when available, otherwise a loading skeleton is shown for uncached views
• the initial interactive render is deferred until the browser session is loaded, which keeps application startup from blocking on a full Plotly build
• the heavier 2D interactive plots use a coarse-first pass before a full detail pass replaces it
• Power interactive plots use the same display-time preparation and per-trace time downsampling approach as the quicklooks, with display-only sanity limits for impossible APS values
• Power summary traces are read from section-specific one-minute display Zarrs when available: Current Conditions opens only observed variables and Forecast & Planning opens only forecast variables. The legacy combined display-summary and display-energy Zarrs remain compatibility fallbacks. Latest Power, Meteorology, and Radiation interactive figures can be loaded from prewarmed Plotly JSON created by their quicklook generators
• the live Power 24 h window is rounded into 5-minute cache buckets, so a small latest-timestamp change does not force an immediate full rebuild
• fixed-summary instruments use instrument-specific display-time sample caps; this changes only the browser/PNG rendering density, not the ingested Zarr data
• inactive quicklook and operations tabs are lazy-loaded the first time the user opens them
• interactive freshness and availability bars are also delayed until after the page opens, so first paint does not wait for a full timestamp scan
• empty interactive views report the selected UTC window and suggest the next useful check instead of showing a bare blank plot

Session context

When available, events also carry:

• session_id
• live_sessions
• server_sessions
• total_sessions
• session_age_s
• busy

stacked_timeseries_render events include phase timings for the main render path:

• source_open_ms
• combine_ms
• figure_build_ms
• source_window_count
• source_window_time_counts
• source_window_var_counts
• combined_var_count
• plot_trace_count
• plot_points_total
• plot_points_min
• plot_points_max
• optional plot_json_bytes when AURORA_MEASURE_PLOT_JSON_BYTES=1

These fields make it easier to distinguish slow Zarr reads from slow Plotly figure construction and from oversized browser payloads.

These fields are useful for understanding real browser behavior, including multi-user overlap and concurrent browsing.

The wxcam_calendar_* event names are retained for continuity with existing performance logs. In the visible dashboard those events now correspond to the WXcam hourly HDR grid on the Science Quicklooks tab.