Observability
Logs
An app's Logs tab shows its requests and its containers' output, live or from the last week, on every server or one, and stackd can forward it all to Loki or any HTTP endpoint.
Live
The app's Logs tab follows what's picked: its requests, every container, its web replicas, or one process's (a queue worker or Horizon, the scheduler, Reverb), on every server or one. Lines from several replicas are merged by time.
GET /api/logs/{id} streams one container's output (an app replica, a
database member or a router) as Server-Sent Events: its last tail lines,
then new ones as they're written (lines events, JSON arrays of {ts, stream, line}; end when it stops).
From the CLI, stackd logs web -f follows every running replica
(CLI).
Requests
Every request to an app passes through the proxy on the server it reaches, which logs it. Requests in the Logs tab shows the app's, its processes' included (Reverb's, say), one line each:
200 GET shop.example.com/cart?x=1 34 ms 203.0.113.9 "Mozilla/5.0 …"status, method, host and path, time to answer, the visitor's address (the
real one behind Cloudflare too) and user agent. 5xx are shown as errors and
4xx as warnings. GET /api/apps/{id}/requests/{server_id} streams one
server's.
Reading back, up to the last 5000 requests of all apps on a server are searched, so a quiet app beside a busy one may show fewer than asked.
History
The output of an app's containers (web replicas and processes) is kept a
week and searched in the Logs tab → History, or with stackd logs web.
GET /api/apps/{id}/logs takes:
q: lines holding it, case aside;fromanduntil: the last day by default;stream, andprocess(webfor the web replicas);- 200 lines a page (up to 1000), newest first; the next page is
untilthe last line's time.
Limits:
- A replica writing over 200 lines a second has the rest left out, and a line saying how many is kept instead.
- Lines are cut at 8 KiB.
- Release commands, jobs and database containers aren't kept.
Forwarding
Admins add destinations in Settings → System → Log forwarding (GET/POST /api/log-sinks, PUT/DELETE /api/log-sinks/{id}, POST …/test):
- Loki: its push URL; lines go in streams labelled
source=stackd,app,process(webfor the web replicas),server_idandstream. - HTTP: any endpoint taking a JSON array of
{at, app_id, app, deployment_id, server_id, process, stream, line}.
Optionally with an Authorization header, and only for some apps. The URL
and the header are stored encrypted. The lines kept (see History) are sent
every couple of seconds. A destination that's slow or down loses its own
lines once 8 batches wait for it, rather than holding up the others,
and shows its last error.