Logs
Live Cloud Logging query (nothing stored) · master ·
last 1h ·
newest first · page 1 (200/page) · 200 entries in 5559 ms
all
app
neo4j
weaviate
time
relative
absolute
9s ago2026-07-27 00:07:41 UTC DEBUG tusker task.Description has unresolved placeholders, task.Id: f8acd6da-882c-4e74-891a-7e8421a4f4ac
time2026-07-27 00:07:41 UTC
severityDEBUG
servicetusker
sourcejobs/task_template.go:219
clustermaster
versionunknown
podtusker-858b678c65-hsvqd
message
task.Description has unresolved placeholders, task.Id: f8acd6da-882c-4e74-891a-7e8421a4f4ac
raw payload (7 fields)
{
"caller": "jobs/task_template.go:219",
"cluster": "master",
"environment": "DEVELOPMENT",
"message": "task.Description has unresolved placeholders, task.Id: f8acd6da-882c-4e74-891a-7e8421a4f4ac",
"service": "tusker",
"timestamp": "2026-07-27T00:07:41.983Z",
"version": "unknown"
}9s ago2026-07-27 00:07:41 UTC DEBUG tusker task.Description has unresolved placeholders, task.Id: 3de5432c-1afc-45ff-be18-3f54d28858d5
time2026-07-27 00:07:41 UTC
severityDEBUG
servicetusker
sourcejobs/task_template.go:219
clustermaster
versionunknown
podtusker-858b678c65-hsvqd
message
task.Description has unresolved placeholders, task.Id: 3de5432c-1afc-45ff-be18-3f54d28858d5
raw payload (7 fields)
{
"caller": "jobs/task_template.go:219",
"cluster": "master",
"environment": "DEVELOPMENT",
"message": "task.Description has unresolved placeholders, task.Id: 3de5432c-1afc-45ff-be18-3f54d28858d5",
"service": "tusker",
"timestamp": "2026-07-27T00:07:41.982Z",
"version": "unknown"
}9s ago2026-07-27 00:07:41 UTC DEBUG tusker task.Description has unresolved placeholders, task.Id: ebc68bf7-3ee7-4dea-96bc-907a84d4021c
time2026-07-27 00:07:41 UTC
severityDEBUG
servicetusker
sourcejobs/task_template.go:219
clustermaster
versionunknown
podtusker-858b678c65-hsvqd
message
task.Description has unresolved placeholders, task.Id: ebc68bf7-3ee7-4dea-96bc-907a84d4021c
raw payload (7 fields)
{
"caller": "jobs/task_template.go:219",
"cluster": "master",
"environment": "DEVELOPMENT",
"message": "task.Description has unresolved placeholders, task.Id: ebc68bf7-3ee7-4dea-96bc-907a84d4021c",
"service": "tusker",
"timestamp": "2026-07-27T00:07:41.982Z",
"version": "unknown"
}9s ago2026-07-27 00:07:41 UTC DEBUG tusker task.Description has unresolved placeholders, task.Id: 6633c476-d5cc-4553-ae7b-81dc1017f38e
time2026-07-27 00:07:41 UTC
severityDEBUG
servicetusker
sourcejobs/task_template.go:219
clustermaster
versionunknown
podtusker-858b678c65-hsvqd
message
task.Description has unresolved placeholders, task.Id: 6633c476-d5cc-4553-ae7b-81dc1017f38e
raw payload (7 fields)
{
"caller": "jobs/task_template.go:219",
"cluster": "master",
"environment": "DEVELOPMENT",
"message": "task.Description has unresolved placeholders, task.Id: 6633c476-d5cc-4553-ae7b-81dc1017f38e",
"service": "tusker",
"timestamp": "2026-07-27T00:07:41.982Z",
"version": "unknown"
}9s ago2026-07-27 00:07:41 UTC DEBUG tusker task.Description has unresolved placeholders, task.Id: 35bf85fe-58a4-49c9-b8df-2e0c2f05c426
time2026-07-27 00:07:41 UTC
severityDEBUG
servicetusker
sourcejobs/task_template.go:219
clustermaster
versionunknown
podtusker-858b678c65-hsvqd
message
task.Description has unresolved placeholders, task.Id: 35bf85fe-58a4-49c9-b8df-2e0c2f05c426
raw payload (7 fields)
{
"caller": "jobs/task_template.go:219",
"cluster": "master",
"environment": "DEVELOPMENT",
"message": "task.Description has unresolved placeholders, task.Id: 35bf85fe-58a4-49c9-b8df-2e0c2f05c426",
"service": "tusker",
"timestamp": "2026-07-27T00:07:41.982Z",
"version": "unknown"
}9s ago2026-07-27 00:07:41 UTC DEBUG tusker task.Description has unresolved placeholders, task.Id: f6942336-068e-4da0-805a-ab4177d369a8
time2026-07-27 00:07:41 UTC
severityDEBUG
servicetusker
sourcejobs/task_template.go:219
clustermaster
versionunknown
podtusker-858b678c65-hsvqd
message
task.Description has unresolved placeholders, task.Id: f6942336-068e-4da0-805a-ab4177d369a8
raw payload (7 fields)
{
"caller": "jobs/task_template.go:219",
"cluster": "master",
"environment": "DEVELOPMENT",
"message": "task.Description has unresolved placeholders, task.Id: f6942336-068e-4da0-805a-ab4177d369a8",
"service": "tusker",
"timestamp": "2026-07-27T00:07:41.982Z",
"version": "unknown"
}9s ago2026-07-27 00:07:41 UTC DEBUG tusker task.Description has unresolved placeholders, task.Id: 50795cbc-27e1-4b3a-aa89-78557d32c760
time2026-07-27 00:07:41 UTC
severityDEBUG
servicetusker
sourcejobs/task_template.go:219
clustermaster
versionunknown
podtusker-858b678c65-hsvqd
message
task.Description has unresolved placeholders, task.Id: 50795cbc-27e1-4b3a-aa89-78557d32c760
raw payload (7 fields)
{
"caller": "jobs/task_template.go:219",
"cluster": "master",
"environment": "DEVELOPMENT",
"message": "task.Description has unresolved placeholders, task.Id: 50795cbc-27e1-4b3a-aa89-78557d32c760",
"service": "tusker",
"timestamp": "2026-07-27T00:07:41.982Z",
"version": "unknown"
}9s ago2026-07-27 00:07:41 UTC DEBUG tusker task.Description has unresolved placeholders, task.Id: 0631dcb4-726e-4b97-8785-c1bf25597c04
time2026-07-27 00:07:41 UTC
severityDEBUG
servicetusker
sourcejobs/task_template.go:219
clustermaster
versionunknown
podtusker-858b678c65-hsvqd
message
task.Description has unresolved placeholders, task.Id: 0631dcb4-726e-4b97-8785-c1bf25597c04
raw payload (7 fields)
{
"caller": "jobs/task_template.go:219",
"cluster": "master",
"environment": "DEVELOPMENT",
"message": "task.Description has unresolved placeholders, task.Id: 0631dcb4-726e-4b97-8785-c1bf25597c04",
"service": "tusker",
"timestamp": "2026-07-27T00:07:41.982Z",
"version": "unknown"
}9s ago2026-07-27 00:07:41 UTC DEBUG tusker task.Description has unresolved placeholders, task.Id: 2ac14167-44f0-419c-a9b3-39bd0bb6941a
time2026-07-27 00:07:41 UTC
severityDEBUG
servicetusker
sourcejobs/task_template.go:219
clustermaster
versionunknown
podtusker-858b678c65-hsvqd
message
task.Description has unresolved placeholders, task.Id: 2ac14167-44f0-419c-a9b3-39bd0bb6941a
raw payload (7 fields)
{
"caller": "jobs/task_template.go:219",
"cluster": "master",
"environment": "DEVELOPMENT",
"message": "task.Description has unresolved placeholders, task.Id: 2ac14167-44f0-419c-a9b3-39bd0bb6941a",
"service": "tusker",
"timestamp": "2026-07-27T00:07:41.982Z",
"version": "unknown"
}9s ago2026-07-27 00:07:41 UTC DEBUG tusker task.Description has unresolved placeholders, task.Id: b83e8d54-728d-437d-b40a-3ca7b6ec9788
time2026-07-27 00:07:41 UTC
severityDEBUG
servicetusker
sourcejobs/task_template.go:219
clustermaster
versionunknown
podtusker-858b678c65-hsvqd
message
task.Description has unresolved placeholders, task.Id: b83e8d54-728d-437d-b40a-3ca7b6ec9788
raw payload (7 fields)
{
"caller": "jobs/task_template.go:219",
"cluster": "master",
"environment": "DEVELOPMENT",
"message": "task.Description has unresolved placeholders, task.Id: b83e8d54-728d-437d-b40a-3ca7b6ec9788",
"service": "tusker",
"timestamp": "2026-07-27T00:07:41.982Z",
"version": "unknown"
}9s ago2026-07-27 00:07:41 UTC DEBUG tusker task.Description has unresolved placeholders, task.Id: 423f2adf-227c-472d-bb76-188dddbe31b0
time2026-07-27 00:07:41 UTC
severityDEBUG
servicetusker
sourcejobs/task_template.go:219
clustermaster
versionunknown
podtusker-858b678c65-hsvqd
message
task.Description has unresolved placeholders, task.Id: 423f2adf-227c-472d-bb76-188dddbe31b0
raw payload (7 fields)
{
"caller": "jobs/task_template.go:219",
"cluster": "master",
"environment": "DEVELOPMENT",
"message": "task.Description has unresolved placeholders, task.Id: 423f2adf-227c-472d-bb76-188dddbe31b0",
"service": "tusker",
"timestamp": "2026-07-27T00:07:41.982Z",
"version": "unknown"
}9s ago2026-07-27 00:07:41 UTC DEBUG tusker task.Description has unresolved placeholders, task.Id: a4f25183-9847-4d5c-b9a3-54eee9ed89d2
time2026-07-27 00:07:41 UTC
severityDEBUG
servicetusker
sourcejobs/task_template.go:219
clustermaster
versionunknown
podtusker-858b678c65-hsvqd
message
task.Description has unresolved placeholders, task.Id: a4f25183-9847-4d5c-b9a3-54eee9ed89d2
raw payload (7 fields)
{
"caller": "jobs/task_template.go:219",
"cluster": "master",
"environment": "DEVELOPMENT",
"message": "task.Description has unresolved placeholders, task.Id: a4f25183-9847-4d5c-b9a3-54eee9ed89d2",
"service": "tusker",
"timestamp": "2026-07-27T00:07:41.982Z",
"version": "unknown"
}9s ago2026-07-27 00:07:41 UTC DEBUG tusker task.Description has unresolved placeholders, task.Id: 2340cd84-d8e7-489a-bcc3-2a804b50ada9
time2026-07-27 00:07:41 UTC
severityDEBUG
servicetusker
sourcejobs/task_template.go:219
clustermaster
versionunknown
podtusker-858b678c65-hsvqd
message
task.Description has unresolved placeholders, task.Id: 2340cd84-d8e7-489a-bcc3-2a804b50ada9
raw payload (7 fields)
{
"caller": "jobs/task_template.go:219",
"cluster": "master",
"environment": "DEVELOPMENT",
"message": "task.Description has unresolved placeholders, task.Id: 2340cd84-d8e7-489a-bcc3-2a804b50ada9",
"service": "tusker",
"timestamp": "2026-07-27T00:07:41.981Z",
"version": "unknown"
}9s ago2026-07-27 00:07:41 UTC DEBUG tusker task.Description has unresolved placeholders, task.Id: e23ee45a-bd22-4571-8adf-23a2e5d8c910
time2026-07-27 00:07:41 UTC
severityDEBUG
servicetusker
sourcejobs/task_template.go:219
clustermaster
versionunknown
podtusker-858b678c65-hsvqd
message
task.Description has unresolved placeholders, task.Id: e23ee45a-bd22-4571-8adf-23a2e5d8c910
raw payload (7 fields)
{
"caller": "jobs/task_template.go:219",
"cluster": "master",
"environment": "DEVELOPMENT",
"message": "task.Description has unresolved placeholders, task.Id: e23ee45a-bd22-4571-8adf-23a2e5d8c910",
"service": "tusker",
"timestamp": "2026-07-27T00:07:41.981Z",
"version": "unknown"
}9s ago2026-07-27 00:07:41 UTC DEBUG tusker task.Description has unresolved placeholders, task.Id: 0a82139f-3d17-4d4f-a561-0b49aaad692a
time2026-07-27 00:07:41 UTC
severityDEBUG
servicetusker
sourcejobs/task_template.go:219
clustermaster
versionunknown
podtusker-858b678c65-hsvqd
message
task.Description has unresolved placeholders, task.Id: 0a82139f-3d17-4d4f-a561-0b49aaad692a
raw payload (7 fields)
{
"caller": "jobs/task_template.go:219",
"cluster": "master",
"environment": "DEVELOPMENT",
"message": "task.Description has unresolved placeholders, task.Id: 0a82139f-3d17-4d4f-a561-0b49aaad692a",
"service": "tusker",
"timestamp": "2026-07-27T00:07:41.981Z",
"version": "unknown"
}9s ago2026-07-27 00:07:41 UTC DEBUG tusker task.Description has unresolved placeholders, task.Id: ea84c7ac-a01d-4ba5-9765-4762f10ea629
time2026-07-27 00:07:41 UTC
severityDEBUG
servicetusker
sourcejobs/task_template.go:219
clustermaster
versionunknown
podtusker-858b678c65-hsvqd
message
task.Description has unresolved placeholders, task.Id: ea84c7ac-a01d-4ba5-9765-4762f10ea629
raw payload (7 fields)
{
"caller": "jobs/task_template.go:219",
"cluster": "master",
"environment": "DEVELOPMENT",
"message": "task.Description has unresolved placeholders, task.Id: ea84c7ac-a01d-4ba5-9765-4762f10ea629",
"service": "tusker",
"timestamp": "2026-07-27T00:07:41.981Z",
"version": "unknown"
}9s ago2026-07-27 00:07:41 UTC DEBUG tusker task.Description has unresolved placeholders, task.Id: c52918ec-feef-4334-aeb1-e030d42a17f9
time2026-07-27 00:07:41 UTC
severityDEBUG
servicetusker
sourcejobs/task_template.go:219
clustermaster
versionunknown
podtusker-858b678c65-hsvqd
message
task.Description has unresolved placeholders, task.Id: c52918ec-feef-4334-aeb1-e030d42a17f9
raw payload (7 fields)
{
"caller": "jobs/task_template.go:219",
"cluster": "master",
"environment": "DEVELOPMENT",
"message": "task.Description has unresolved placeholders, task.Id: c52918ec-feef-4334-aeb1-e030d42a17f9",
"service": "tusker",
"timestamp": "2026-07-27T00:07:41.981Z",
"version": "unknown"
}9s ago2026-07-27 00:07:41 UTC DEBUG tusker task.Description has unresolved placeholders, task.Id: 0909f0c3-e08d-4ba0-96ef-d73382ec65c4
time2026-07-27 00:07:41 UTC
severityDEBUG
servicetusker
sourcejobs/task_template.go:219
clustermaster
versionunknown
podtusker-858b678c65-hsvqd
message
task.Description has unresolved placeholders, task.Id: 0909f0c3-e08d-4ba0-96ef-d73382ec65c4
raw payload (7 fields)
{
"caller": "jobs/task_template.go:219",
"cluster": "master",
"environment": "DEVELOPMENT",
"message": "task.Description has unresolved placeholders, task.Id: 0909f0c3-e08d-4ba0-96ef-d73382ec65c4",
"service": "tusker",
"timestamp": "2026-07-27T00:07:41.981Z",
"version": "unknown"
}9s ago2026-07-27 00:07:41 UTC DEBUG tusker task.Description has unresolved placeholders, task.Id: 1af5baaf-ae5b-41fc-8599-823e55c740cb
time2026-07-27 00:07:41 UTC
severityDEBUG
servicetusker
sourcejobs/task_template.go:219
clustermaster
versionunknown
podtusker-858b678c65-hsvqd
message
task.Description has unresolved placeholders, task.Id: 1af5baaf-ae5b-41fc-8599-823e55c740cb
raw payload (7 fields)
{
"caller": "jobs/task_template.go:219",
"cluster": "master",
"environment": "DEVELOPMENT",
"message": "task.Description has unresolved placeholders, task.Id: 1af5baaf-ae5b-41fc-8599-823e55c740cb",
"service": "tusker",
"timestamp": "2026-07-27T00:07:41.981Z",
"version": "unknown"
}9s ago2026-07-27 00:07:41 UTC DEBUG tusker task.Description has unresolved placeholders, task.Id: a18f77f5-81cd-4a50-90ec-3eb298df8808
time2026-07-27 00:07:41 UTC
severityDEBUG
servicetusker
sourcejobs/task_template.go:219
clustermaster
versionunknown
podtusker-858b678c65-hsvqd
message
task.Description has unresolved placeholders, task.Id: a18f77f5-81cd-4a50-90ec-3eb298df8808
raw payload (7 fields)
{
"caller": "jobs/task_template.go:219",
"cluster": "master",
"environment": "DEVELOPMENT",
"message": "task.Description has unresolved placeholders, task.Id: a18f77f5-81cd-4a50-90ec-3eb298df8808",
"service": "tusker",
"timestamp": "2026-07-27T00:07:41.981Z",
"version": "unknown"
}9s ago2026-07-27 00:07:41 UTC DEBUG tusker task.Description has unresolved placeholders, task.Id: 23ab1f68-b101-4906-b3a8-1b18092f4664
time2026-07-27 00:07:41 UTC
severityDEBUG
servicetusker
sourcejobs/task_template.go:219
clustermaster
versionunknown
podtusker-858b678c65-hsvqd
message
task.Description has unresolved placeholders, task.Id: 23ab1f68-b101-4906-b3a8-1b18092f4664
raw payload (7 fields)
{
"caller": "jobs/task_template.go:219",
"cluster": "master",
"environment": "DEVELOPMENT",
"message": "task.Description has unresolved placeholders, task.Id: 23ab1f68-b101-4906-b3a8-1b18092f4664",
"service": "tusker",
"timestamp": "2026-07-27T00:07:41.981Z",
"version": "unknown"
}9s ago2026-07-27 00:07:41 UTC DEBUG tusker task.Description has unresolved placeholders, task.Id: 178bccdd-df62-4edd-b6c2-115fcc0f7d60
time2026-07-27 00:07:41 UTC
severityDEBUG
servicetusker
sourcejobs/task_template.go:219
clustermaster
versionunknown
podtusker-858b678c65-hsvqd
message
task.Description has unresolved placeholders, task.Id: 178bccdd-df62-4edd-b6c2-115fcc0f7d60
raw payload (7 fields)
{
"caller": "jobs/task_template.go:219",
"cluster": "master",
"environment": "DEVELOPMENT",
"message": "task.Description has unresolved placeholders, task.Id: 178bccdd-df62-4edd-b6c2-115fcc0f7d60",
"service": "tusker",
"timestamp": "2026-07-27T00:07:41.981Z",
"version": "unknown"
}9s ago2026-07-27 00:07:41 UTC DEBUG tusker task.Description has unresolved placeholders, task.Id: 75c3858f-e811-4ee4-88af-170e665bc524
time2026-07-27 00:07:41 UTC
severityDEBUG
servicetusker
sourcejobs/task_template.go:219
clustermaster
versionunknown
podtusker-858b678c65-hsvqd
message
task.Description has unresolved placeholders, task.Id: 75c3858f-e811-4ee4-88af-170e665bc524
raw payload (7 fields)
{
"caller": "jobs/task_template.go:219",
"cluster": "master",
"environment": "DEVELOPMENT",
"message": "task.Description has unresolved placeholders, task.Id: 75c3858f-e811-4ee4-88af-170e665bc524",
"service": "tusker",
"timestamp": "2026-07-27T00:07:41.981Z",
"version": "unknown"
}9s ago2026-07-27 00:07:41 UTC DEBUG tusker task.Description has unresolved placeholders, task.Id: c6dd12b3-7a8b-43eb-adb9-c6e65da6644f
time2026-07-27 00:07:41 UTC
severityDEBUG
servicetusker
sourcejobs/task_template.go:219
clustermaster
versionunknown
podtusker-858b678c65-hsvqd
message
task.Description has unresolved placeholders, task.Id: c6dd12b3-7a8b-43eb-adb9-c6e65da6644f
raw payload (7 fields)
{
"caller": "jobs/task_template.go:219",
"cluster": "master",
"environment": "DEVELOPMENT",
"message": "task.Description has unresolved placeholders, task.Id: c6dd12b3-7a8b-43eb-adb9-c6e65da6644f",
"service": "tusker",
"timestamp": "2026-07-27T00:07:41.980Z",
"version": "unknown"
}9s ago2026-07-27 00:07:41 UTC DEBUG tusker task.Description has unresolved placeholders, task.Id: 75c436ea-1d1d-4760-9aff-b35a90750229
time2026-07-27 00:07:41 UTC
severityDEBUG
servicetusker
sourcejobs/task_template.go:219
clustermaster
versionunknown
podtusker-858b678c65-hsvqd
message
task.Description has unresolved placeholders, task.Id: 75c436ea-1d1d-4760-9aff-b35a90750229
raw payload (7 fields)
{
"caller": "jobs/task_template.go:219",
"cluster": "master",
"environment": "DEVELOPMENT",
"message": "task.Description has unresolved placeholders, task.Id: 75c436ea-1d1d-4760-9aff-b35a90750229",
"service": "tusker",
"timestamp": "2026-07-27T00:07:41.880Z",
"version": "unknown"
}9s ago2026-07-27 00:07:41 UTC DEBUG tusker task.Description has unresolved placeholders, task.Id: fecf7b3b-2289-44c4-aa40-b8703a1b29f3
time2026-07-27 00:07:41 UTC
severityDEBUG
servicetusker
sourcejobs/task_template.go:219
clustermaster
versionunknown
podtusker-858b678c65-hsvqd
message
task.Description has unresolved placeholders, task.Id: fecf7b3b-2289-44c4-aa40-b8703a1b29f3
raw payload (7 fields)
{
"caller": "jobs/task_template.go:219",
"cluster": "master",
"environment": "DEVELOPMENT",
"message": "task.Description has unresolved placeholders, task.Id: fecf7b3b-2289-44c4-aa40-b8703a1b29f3",
"service": "tusker",
"timestamp": "2026-07-27T00:07:41.880Z",
"version": "unknown"
}9s ago2026-07-27 00:07:41 UTC DEBUG tusker task.Description has unresolved placeholders, task.Id: 294e22d2-9214-434d-a917-6dd6b80ffb53
time2026-07-27 00:07:41 UTC
severityDEBUG
servicetusker
sourcejobs/task_template.go:219
clustermaster
versionunknown
podtusker-858b678c65-hsvqd
message
task.Description has unresolved placeholders, task.Id: 294e22d2-9214-434d-a917-6dd6b80ffb53
raw payload (7 fields)
{
"caller": "jobs/task_template.go:219",
"cluster": "master",
"environment": "DEVELOPMENT",
"message": "task.Description has unresolved placeholders, task.Id: 294e22d2-9214-434d-a917-6dd6b80ffb53",
"service": "tusker",
"timestamp": "2026-07-27T00:07:41.880Z",
"version": "unknown"
}9s ago2026-07-27 00:07:41 UTC DEBUG tusker task.Description has unresolved placeholders, task.Id: 60497984-d5be-4f16-a253-62f20b8c84a8
time2026-07-27 00:07:41 UTC
severityDEBUG
servicetusker
sourcejobs/task_template.go:219
clustermaster
versionunknown
podtusker-858b678c65-hsvqd
message
task.Description has unresolved placeholders, task.Id: 60497984-d5be-4f16-a253-62f20b8c84a8
raw payload (7 fields)
{
"caller": "jobs/task_template.go:219",
"cluster": "master",
"environment": "DEVELOPMENT",
"message": "task.Description has unresolved placeholders, task.Id: 60497984-d5be-4f16-a253-62f20b8c84a8",
"service": "tusker",
"timestamp": "2026-07-27T00:07:41.880Z",
"version": "unknown"
}9s ago2026-07-27 00:07:41 UTC DEBUG tusker task.Description has unresolved placeholders, task.Id: 0b722ebc-8156-44c6-8c1e-1b83e198ee4e
time2026-07-27 00:07:41 UTC
severityDEBUG
servicetusker
sourcejobs/task_template.go:219
clustermaster
versionunknown
podtusker-858b678c65-hsvqd
message
task.Description has unresolved placeholders, task.Id: 0b722ebc-8156-44c6-8c1e-1b83e198ee4e
raw payload (7 fields)
{
"caller": "jobs/task_template.go:219",
"cluster": "master",
"environment": "DEVELOPMENT",
"message": "task.Description has unresolved placeholders, task.Id: 0b722ebc-8156-44c6-8c1e-1b83e198ee4e",
"service": "tusker",
"timestamp": "2026-07-27T00:07:41.862Z",
"version": "unknown"
}9s ago2026-07-27 00:07:41 UTC DEBUG tusker task.Description has unresolved placeholders, task.Id: be4f6fba-887d-447f-b5dd-e718cce87439
time2026-07-27 00:07:41 UTC
severityDEBUG
servicetusker
sourcejobs/task_template.go:219
clustermaster
versionunknown
podtusker-858b678c65-hsvqd
message
task.Description has unresolved placeholders, task.Id: be4f6fba-887d-447f-b5dd-e718cce87439
raw payload (7 fields)
{
"caller": "jobs/task_template.go:219",
"cluster": "master",
"environment": "DEVELOPMENT",
"message": "task.Description has unresolved placeholders, task.Id: be4f6fba-887d-447f-b5dd-e718cce87439",
"service": "tusker",
"timestamp": "2026-07-27T00:07:41.862Z",
"version": "unknown"
}9s ago2026-07-27 00:07:41 UTC DEBUG tusker task.Description has unresolved placeholders, task.Id: 257ad24f-e78b-497c-80eb-649b33f341ff
time2026-07-27 00:07:41 UTC
severityDEBUG
servicetusker
sourcejobs/task_template.go:219
clustermaster
versionunknown
podtusker-858b678c65-hsvqd
message
task.Description has unresolved placeholders, task.Id: 257ad24f-e78b-497c-80eb-649b33f341ff
raw payload (7 fields)
{
"caller": "jobs/task_template.go:219",
"cluster": "master",
"environment": "DEVELOPMENT",
"message": "task.Description has unresolved placeholders, task.Id: 257ad24f-e78b-497c-80eb-649b33f341ff",
"service": "tusker",
"timestamp": "2026-07-27T00:07:41.858Z",
"version": "unknown"
}9s ago2026-07-27 00:07:41 UTC neo4j tusker MATCH (study:Study )-[:MANAGED_BY]-(:Inbox)-[:BELONGS_TO]-(t:Task) WHERE NOT EXISTS ((t)-[:BELONGS_TO]-(:Inbox{type:"User"})) CALL...
time2026-07-27 00:07:41 UTC
severityINFO
servicetusker
clustermaster
podserver-2-dev-0
query
MATCH (study:Study )-[:MANAGED_BY]-(:Inbox)-[:BELONGS_TO]-(t:Task)
WHERE NOT EXISTS ((t)-[:BELONGS_TO]-(:Inbox{type:"User"}))
CALL {
WITH t
MATCH (t)<-[:CREATED_FROM_SPEC]-(ptc:ParticipantTaskConfig)-[:PARTICIPANT_TASKCONFIG]-(:ParticipantTaskList)-[:PARTICIPANT_TASKLIST]-(:ParticipantControl)-[:PARTICIPANT_CONTROL]-(:ParticipantActivity)-[:PARTICIPANT_ACTIVITY]-(pw:ParticipantWorkflow)
RETURN ptc, pw
UNION ALL
WITH t
MATCH (t)<-[:CREATED_FROM_SPEC]-(ptc:ParticipantTaskConfig)-[:PARTICIPANT_TASKCONFIG]-(:ParticipantTaskList)-[:PARTICIPANT_TASKLIST]-(:ParticipantActivity)-[:PARTICIPANT_ACTIVITY]-(pw:ParticipantWorkflow)
RETURN ptc, pw
}
WITH DISTINCT study, t, ptc, pw
MATCH (t)-[:BELONGS_TO]-(:Inbox)-[:MANAGED_BY]-(o:Organization)
MATCH (ptc)-[:BELONGS_TO_TASKCONFIG]-(tc:TaskConfig)
OPTIONAL MATCH (t)-[:BELONGS_TO]-(taskInbox:Inbox{type:"User"})--(taskUser:User)
OPTIONAL MATCH (pw)-[:PARTICIPANTWORKFLOW_TO_SCOPE]-(:StudySiteActor)-[:STUDYSITEACTOR_TO_ACTOR]-(:Actor{delpending:false})-[:FOR_USER]-(user:User)
OPTIONAL MATCH (pw)-[:PARTICIPANTWORKFLOW_TO_SCOPE]-(:DslPlanDefinition)-[:PLAN_DEFINITION_TO_SITE]-(scopeSite:Site)-[:SITE_TO_SMO]-(:Smo)-[:IS_AN_ORG]-(o)
CALL {
WITH study
OPTIONAL MATCH (study)-[:STUDY_TO_ACTOR]-(a:Actor{delpending:false})-[:FOR_USER]-(u:User),
(a)-[:APPOINTED_AS]-(r:Role)
WHERE r.name IN ["principal_investigator", "study_admin", "regulatory_analyst"]
RETURN collect(DISTINCT CASE WHEN r.name = "principal_investigator" THEN u.tildaid END) AS piTildaid,
collect(DISTINCT CASE WHEN r.name = "study_admin" THEN u.tildaid END) AS smoTildaid,
collect(DISTINCT CASE WHEN r.name = "regulatory_analyst" THEN u.tildaid END) AS raTildaid
}
CALL {
WITH o
OPTIONAL MATCH (o)-[:ORGANIZATION_TO_ACTOR]-(a:Actor{delpending:false})-[:FOR_USER]-(u:User),
(a)-[:APPOINTED_AS]-(:Role{name:"org_admin"})
RETURN collect(DISTINCT u.tildaid) AS adminTildaid
}
CALL {
WITH scopeSite
OPTIONAL MATCH (scopeSite)-[:STAFF_TO_ACTOR]-(a:Actor{delpending:false})-[:FOR_USER]-(u:User),
(a)-[:APPOINTED_AS]-(:Role{name:"site_admin"})
RETURN collect(DISTINCT u.tildaid) AS siteAdminTildaid
}
RETURN DISTINCT t AS task, apoc.convert.toJson({userTildaid: user.tildaid, userName: user.name,
PITildaid: piTildaid, studyOwnerTildaid: smoTildaid, regAnalystTildaid: raTildaid,
orgAdminTildaid: adminTildaid, studyName: study.name, siteName: scopeSite.name,
siteAdminTildaid: siteAdminTildaid}) AS details, tc.who AS dsnWho, o.alias AS orgAlias,
taskInbox, taskUser.tildaid AS taskUserId
SKIP 0 LIMIT 400
raw payload (15 fields)
{
"allocatedBytes": 5814768.0,
"cpu": 1879.0,
"elapsedTimeMs": 1880.0,
"executionPlanCacheKeyHash": "A464432C",
"id": "24642121",
"pageFaults": 0.0,
"pageHits": 3753115.0,
"planning": 1.0,
"query": "\n\t\tMATCH (study:Study )-[:MANAGED_BY]-(:Inbox)-[:BELONGS_TO]-(t:Task)\n\t\tWHERE NOT EXISTS ((t)-[:BELONGS_TO]-(:Inbox{type:\"User\"}))\n\t\tCALL {\n\t\t\tWITH t\n\t\t\tMATCH (t)\u003c-[:CREATED_FROM_SPEC]-(ptc:ParticipantTaskConfig)-[:PARTICIPANT_TASKCONFIG]-(:ParticipantTaskList)-[:PARTICIPANT_TASKLIST]-(:ParticipantControl)-[:PARTICIPANT_CONTROL]-(:ParticipantActivity)-[:PARTICIPANT_ACTIVITY]-(pw:ParticipantWorkflow)\n\t\t\tRETURN ptc, pw\n\t\t\tUNION ALL\n\t\t\tWITH t\n\t\t\tMATCH (t)\u003c-[:CREATED_FROM_SPEC]-(ptc:ParticipantTaskConfig)-[:PARTICIPANT_TASKCONFIG]-(:ParticipantTaskList)-[:PARTICIPANT_TASKLIST]-(:ParticipantActivity)-[:PARTICIPANT_ACTIVITY]-(pw:ParticipantWorkflow)\n\t\t\tRETURN ptc, pw\n\t\t}\n\t\tWITH DISTINCT study, t, ptc, pw\n\t\tMATCH (t)-[:BELONGS_TO]-(:Inbox)-[:MANAGED_BY]-(o:Organization)\n\t\tMATCH (ptc)-[:BELONGS_TO_TASKCONFIG]-(tc:TaskConfig)\n\t\tOPTIONAL MATCH (t)-[:BELONGS_TO]-(taskInbox:Inbox{type:\"User\"})--(taskUser:User)\n\t\tOPTIONAL MATCH (pw)-[:PARTICIPANTWORKFLOW_TO_SCOPE]-(:StudySiteActor)-[:STUDYSITEACTOR_TO_ACTOR]-(:Actor{delpending:false})-[:FOR_USER]-(user:User)\n\t\tOPTIONAL MATCH (pw)-[:PARTICIPANTWORKFLOW_TO_SCOPE]-(:DslPlanDefinition)-[:PLAN_DEFINITION_TO_SITE]-(scopeSite:Site)-[:SITE_TO_SMO]-(:Smo)-[:IS_AN_ORG]-(o)\n\t\tCALL {\n\t\t\tWITH study\n\t\t\tOPTIONAL MATCH (study)-[:STUDY_TO_ACTOR]-(a:Actor{delpending:false})-[:FOR_USER]-(u:User),\n\t\t\t\t(a)-[:APPOINTED_AS]-(r:Role)\n\t\t\tWHERE r.name IN [\"principal_investigator\", \"study_admin\", \"regulatory_analyst\"]\n\t\t\tRETURN collect(DISTINCT CASE WHEN r.name = \"principal_investigator\" THEN u.tildaid END) AS piTildaid,\n\t\t\t\tcollect(DISTINCT CASE WHEN r.name = \"study_admin\" THEN u.tildaid END) AS smoTildaid,\n\t\t\t\tcollect(DISTINCT CASE WHEN r.name = \"regulatory_analyst\" THEN u.tildaid END) AS raTildaid\n\t\t}\n\t\tCALL {\n\t\t\tWITH o\n\t\t\tOPTIONAL MATCH (o)-[:ORGANIZATION_TO_ACTOR]-(a:Actor{delpending:false})-[:FOR_USER]-(u:User),\n\t\t\t\t(a)-[:APPOINTED_AS]-(:Role{name:\"org_admin\"})\n\t\t\tRETURN collect(DISTINCT u.tildaid) AS adminTildaid\n\t\t}\n\t\tCALL {\n\t\t\tWITH scopeSite\n\t\t\tOPTIONAL MATCH (scopeSite)-[:STAFF_TO_ACTOR]-(a:Actor{delpending:false})-[:FOR_USER]-(u:User),\n\t\t\t\t(a)-[:APPOINTED_AS]-(:Role{name:\"site_admin\"})\n\t\t\tRETURN collect(DISTINCT u.tildaid) AS siteAdminTildaid\n\t\t}\n\t\tRETURN DISTINCT t AS task, apoc.convert.toJson({userTildaid: user.tildaid, userName: user.name,\n\t\t\tPITildaid: piTildaid, studyOwnerTildaid: smoTildaid, regAnalystTildaid: raTildaid,\n\t\t\torgAdminTildaid: adminTildaid, studyName: study.name, siteName: scopeSite.name,\n\t\t\tsiteAdminTildaid: siteAdminTildaid}) AS details, tc.who AS dsnWho, o.alias AS orgAlias,\n\t\t\ttaskInbox, taskUser.tildaid AS taskUserId\n\t\tSKIP 0 LIMIT 400",
"queryParameters": "{}",
"runtime": "pipelined",
"source": "bolt-session\tbolt\tneo4j-java/5.28.13-24cd0fecc64de123fd5711190246e8b0f1e16fd0\t\tclient/10.173.2.71:36504\tserver/10.173.0.8:7688\u003e",
"timestamp": "2026-07-27T00:07:41.847Z",
"transactionId": 12025360.0,
"waiting": 0.0
}10s ago2026-07-27 00:07:40 UTC INFO
filter by:
severity≥INFO
time2026-07-27 00:07:40 UTC
severityINFO
clustermaster
podlitellm-f54655c99-cbmvt
10s ago2026-07-27 00:07:40 UTC INFO
filter by:
severity≥INFO
time2026-07-27 00:07:40 UTC
severityINFO
clustermaster
podrag-agent-57879dd94f-l9nz7
10s ago2026-07-27 00:07:40 UTC DEBUG tusker TransactionID commit
time2026-07-27 00:07:40 UTC
severityDEBUG
servicetusker
sourcetildadb/api.go:2575
clustermaster
versionunknown
podtusker-858b678c65-hsvqd
message
TransactionID commit
raw payload (8 fields)
{
"caller": "tildadb/api.go:2575",
"cluster": "master",
"environment": "DEVELOPMENT",
"message": "TransactionID commit",
"service": "tusker",
"timestamp": "2026-07-27T00:07:40.182Z",
"transactionID": "FB:kcwQc8qrjECqRtOxBB1HjVmAmsoAPpdlkA==",
"version": "unknown"
}10s ago2026-07-27 00:07:40 UTC INFO tusker taskdist in 1.07µs: tasks: 0 numPriorityChanged 0, numAssigned 0, numReassigned 0, numNotAssigned 0
time2026-07-27 00:07:40 UTC
severityINFO
servicetusker
sourcetaskdist/taskdist.go:367
clustermaster
versionunknown
podtusker-858b678c65-hsvqd
message
taskdist in 1.07µs: tasks: 0 numPriorityChanged 0, numAssigned 0, numReassigned 0, numNotAssigned 0
raw payload (7 fields)
{
"caller": "taskdist/taskdist.go:367",
"cluster": "master",
"environment": "DEVELOPMENT",
"message": "taskdist in 1.07\u00b5s: tasks: 0 numPriorityChanged 0, numAssigned 0, numReassigned 0, numNotAssigned 0",
"service": "tusker",
"timestamp": "2026-07-27T00:07:40.179Z",
"version": "unknown"
}10s ago2026-07-27 00:07:40 UTC DEBUG tusker TransactionID
time2026-07-27 00:07:40 UTC
severityDEBUG
servicetusker
sourcetildadb/api.go:2482
clustermaster
versionunknown
podtusker-858b678c65-hsvqd
message
TransactionID
raw payload (8 fields)
{
"caller": "tildadb/api.go:2482",
"cluster": "master",
"environment": "DEVELOPMENT",
"message": "TransactionID",
"service": "tusker",
"timestamp": "2026-07-27T00:07:40.179Z",
"transactionID": "",
"version": "unknown"
}10s ago2026-07-27 00:07:40 UTC neo4j tusker match (task:Task{assignment:'automatic'}) where (task.state = "ready" or task.state = "draft") and not exists...
time2026-07-27 00:07:40 UTC
severityINFO
servicetusker
clustermaster
podserver-3-dev-0
query
match (task:Task{assignment:'automatic'})
where (task.state = "ready" or task.state = "draft")
and not exists ((task)-[:BELONGS_TO]-(:Inbox{type:'User'}))
and coalesce(task.dist_deferred, false) = false
and not exists (
(task)-[:CREATED_FROM_SPEC]-(:ParticipantTaskConfig)-[:PARTICIPANT_TASKCONFIG]-(:ParticipantTaskList)-[:PARTICIPANT_TASKLIST]-(:ParticipantControl)
)
and not exists (
(task)<-[:CREATED_FROM_SPEC]-(:ParticipantDslSpecNode)<-[:PARTICIPANT_DSLTASK_TO_ALL_PARTICIPANT_DSLSPECNODE]-(:ParticipantDslTask)<-[:PARTICIPANT_DSLTASK]-(:ParticipantControl)
)
match (task)-[:WITH_CONTEXT]->()
optional match (task)-[:BELONGS_TO]->(userinbox:Inbox{type:"User"})-[:OWNED_BY]->(user:User)
optional match (user)-[:USER_TO_PATIENT]-(patient:Patient)
optional match (task)-[:BELONGS_TO]->(:Inbox{type:"Category"})-[:MANAGED_BY]->(org:Organization)
optional match (task)-[:BELONGS_TO]->(:Inbox{type:"Study"})-[:MANAGED_BY]->(study:Study)
optional match (task)-[:BELONGS_TO]->(:Inbox{type:"Site"})-[:MANAGED_BY]->(site:Site)
return distinct task, userinbox, org, study.name, site.name, patient.tildaid skip 0 limit 1000
raw payload (15 fields)
{
"allocatedBytes": 48924.0,
"cpu": 106.0,
"elapsedTimeMs": 107.0,
"executionPlanCacheKeyHash": "2897E389",
"id": "13356405",
"pageFaults": 0.0,
"pageHits": 72087.0,
"planning": 1.0,
"query": "\n\t\tmatch (task:Task{assignment:\u0027automatic\u0027})\n\t\twhere (task.state = \"ready\" or task.state = \"draft\") \n\t\tand not exists ((task)-[:BELONGS_TO]-(:Inbox{type:\u0027User\u0027}))\n\t\tand coalesce(task.dist_deferred, false) = false\n\t\tand not exists (\n\t\t\t(task)-[:CREATED_FROM_SPEC]-(:ParticipantTaskConfig)-[:PARTICIPANT_TASKCONFIG]-(:ParticipantTaskList)-[:PARTICIPANT_TASKLIST]-(:ParticipantControl)\n\t\t)\n\t\tand not exists (\n\t\t\t(task)\u003c-[:CREATED_FROM_SPEC]-(:ParticipantDslSpecNode)\u003c-[:PARTICIPANT_DSLTASK_TO_ALL_PARTICIPANT_DSLSPECNODE]-(:ParticipantDslTask)\u003c-[:PARTICIPANT_DSLTASK]-(:ParticipantControl)\n\t\t)\n match (task)-[:WITH_CONTEXT]-\u003e()\n\t\toptional match (task)-[:BELONGS_TO]-\u003e(userinbox:Inbox{type:\"User\"})-[:OWNED_BY]-\u003e(user:User)\n\t\toptional match (user)-[:USER_TO_PATIENT]-(patient:Patient)\n\t\toptional match (task)-[:BELONGS_TO]-\u003e(:Inbox{type:\"Category\"})-[:MANAGED_BY]-\u003e(org:Organization)\n\t\toptional match (task)-[:BELONGS_TO]-\u003e(:Inbox{type:\"Study\"})-[:MANAGED_BY]-\u003e(study:Study)\n\t\toptional match (task)-[:BELONGS_TO]-\u003e(:Inbox{type:\"Site\"})-[:MANAGED_BY]-\u003e(site:Site)\n\t\treturn distinct task, userinbox, org, study.name, site.name, patient.tildaid skip 0 limit 1000",
"queryParameters": "{}",
"runtime": "pipelined",
"source": "bolt-session\tbolt\tGo Driver/5.20.0\t\tclient/10.129.0.236:36350\tserver/10.173.0.201:7687\u003e",
"timestamp": "2026-07-27T00:07:40.161Z",
"transactionId": 1750734.0,
"waiting": 0.0
}10s ago2026-07-27 00:07:40 UTC neo4j tusker MATCH (org:Organization) WITH org MATCH (task:Task)-[:BELONGS_TO]->(inbox:Inbox{type:'Category'})-[:MANAGED_BY]->(org) WHERE task.task_number...
time2026-07-27 00:07:40 UTC
severityINFO
servicetusker
clustermaster
podserver-2-dev-0
query
MATCH (org:Organization)
WITH org
MATCH (task:Task)-[:BELONGS_TO]->(inbox:Inbox{type:'Category'})-[:MANAGED_BY]->(org)
WHERE task.task_number IS NULL OR task.task_number = 0
WITH org, task
ORDER BY org.alias, task.createdat
WITH org, collect(task) as tasks
CALL {
WITH org
MATCH (t:Task)-[:BELONGS_TO]->(:Inbox{type:'Category'})-[:MANAGED_BY]->(org)
WHERE t.task_number IS NOT NULL AND t.task_number > 0
RETURN toInteger(max(t.task_number)) as maxTaskNumber
}
WITH org, tasks, COALESCE(maxTaskNumber, 0) as maxTaskNumber
UNWIND range(0, size(tasks)-1) as i
WITH tasks[i] as task, org, i, maxTaskNumber
SET task.task_number = maxTaskNumber + i + 1
RETURN count(task) as updated
raw payload (15 fields)
{
"allocatedBytes": 6560.0,
"cpu": 78.0,
"elapsedTimeMs": 78.0,
"executionPlanCacheKeyHash": "86A7FEEC",
"id": "24642122",
"pageFaults": 0.0,
"pageHits": 45081.0,
"planning": 0.0,
"query": "\n\tMATCH (org:Organization)\n\tWITH org\n\tMATCH (task:Task)-[:BELONGS_TO]-\u003e(inbox:Inbox{type:\u0027Category\u0027})-[:MANAGED_BY]-\u003e(org)\n\tWHERE task.task_number IS NULL OR task.task_number = 0\n\tWITH org, task\n\tORDER BY org.alias, task.createdat\n\tWITH org, collect(task) as tasks\n\tCALL {\n\t\tWITH org\n\t\tMATCH (t:Task)-[:BELONGS_TO]-\u003e(:Inbox{type:\u0027Category\u0027})-[:MANAGED_BY]-\u003e(org)\n\t\tWHERE t.task_number IS NOT NULL AND t.task_number \u003e 0\n\t\tRETURN toInteger(max(t.task_number)) as maxTaskNumber\n\t}\n\tWITH org, tasks, COALESCE(maxTaskNumber, 0) as maxTaskNumber\n\tUNWIND range(0, size(tasks)-1) as i\n\tWITH tasks[i] as task, org, i, maxTaskNumber\n\tSET task.task_number = maxTaskNumber + i + 1\n\tRETURN count(task) as updated",
"queryParameters": "{}",
"runtime": "pipelined",
"source": "bolt-session\tbolt\tneo4j-java/5.28.13-24cd0fecc64de123fd5711190246e8b0f1e16fd0\t\tclient/10.173.0.201:39990\tserver/10.173.0.8:7688\u003e",
"timestamp": "2026-07-27T00:07:40.045Z",
"transactionId": 12025361.0,
"waiting": 0.0
}11s ago2026-07-27 00:07:39 UTC DEBUG tusker TransactionID
time2026-07-27 00:07:39 UTC
severityDEBUG
servicetusker
sourcetildadb/api.go:2482
clustermaster
versionunknown
podtusker-858b678c65-hsvqd
message
TransactionID
raw payload (8 fields)
{
"caller": "tildadb/api.go:2482",
"cluster": "master",
"environment": "DEVELOPMENT",
"message": "TransactionID",
"service": "tusker",
"timestamp": "2026-07-27T00:07:39.964Z",
"transactionID": "",
"version": "unknown"
}11s ago2026-07-27 00:07:39 UTC DEBUG tusker TransactionID commit
time2026-07-27 00:07:39 UTC
severityDEBUG
servicetusker
sourcetildadb/api.go:2575
clustermaster
versionunknown
podtusker-858b678c65-hsvqd
message
TransactionID commit
raw payload (8 fields)
{
"caller": "tildadb/api.go:2575",
"cluster": "master",
"environment": "DEVELOPMENT",
"message": "TransactionID commit",
"service": "tusker",
"timestamp": "2026-07-27T00:07:39.960Z",
"transactionID": "FB:kcwQc8qrjECqRtOxBB1HjVmAmsoAPpdlkA==",
"version": "unknown"
}11s ago2026-07-27 00:07:39 UTC neo4j tusker MATCH (study:Study )-[:MANAGED_BY]-(:Inbox)-[:BELONGS_TO]-(t:Task) WHERE NOT EXISTS ((t)-[:BELONGS_TO]-(:Inbox{type:"User"})) CALL...
time2026-07-27 00:07:39 UTC
severityINFO
servicetusker
clustermaster
podserver-2-dev-0
query
MATCH (study:Study )-[:MANAGED_BY]-(:Inbox)-[:BELONGS_TO]-(t:Task)
WHERE NOT EXISTS ((t)-[:BELONGS_TO]-(:Inbox{type:"User"}))
CALL {
WITH t
MATCH (t)<-[:CREATED_FROM_SPEC]-(ptc:ParticipantTaskConfig)-[:PARTICIPANT_TASKCONFIG]-(:ParticipantTaskList)-[:PARTICIPANT_TASKLIST]-(:ParticipantControl)-[:PARTICIPANT_CONTROL]-(:ParticipantActivity)-[:PARTICIPANT_ACTIVITY]-(pw:ParticipantWorkflow)
RETURN ptc, pw
UNION ALL
WITH t
MATCH (t)<-[:CREATED_FROM_SPEC]-(ptc:ParticipantTaskConfig)-[:PARTICIPANT_TASKCONFIG]-(:ParticipantTaskList)-[:PARTICIPANT_TASKLIST]-(:ParticipantActivity)-[:PARTICIPANT_ACTIVITY]-(pw:ParticipantWorkflow)
RETURN ptc, pw
}
WITH DISTINCT study, t, ptc, pw
MATCH (t)-[:BELONGS_TO]-(:Inbox)-[:MANAGED_BY]-(o:Organization)
MATCH (ptc)-[:BELONGS_TO_TASKCONFIG]-(tc:TaskConfig)
OPTIONAL MATCH (t)-[:BELONGS_TO]-(taskInbox:Inbox{type:"User"})--(taskUser:User)
OPTIONAL MATCH (pw)-[:PARTICIPANTWORKFLOW_TO_SCOPE]-(:StudySiteActor)-[:STUDYSITEACTOR_TO_ACTOR]-(:Actor{delpending:false})-[:FOR_USER]-(user:User)
OPTIONAL MATCH (pw)-[:PARTICIPANTWORKFLOW_TO_SCOPE]-(:DslPlanDefinition)-[:PLAN_DEFINITION_TO_SITE]-(scopeSite:Site)-[:SITE_TO_SMO]-(:Smo)-[:IS_AN_ORG]-(o)
CALL {
WITH study
OPTIONAL MATCH (study)-[:STUDY_TO_ACTOR]-(a:Actor{delpending:false})-[:FOR_USER]-(u:User),
(a)-[:APPOINTED_AS]-(r:Role)
WHERE r.name IN ["principal_investigator", "study_admin", "regulatory_analyst"]
RETURN collect(DISTINCT CASE WHEN r.name = "principal_investigator" THEN u.tildaid END) AS piTildaid,
collect(DISTINCT CASE WHEN r.name = "study_admin" THEN u.tildaid END) AS smoTildaid,
collect(DISTINCT CASE WHEN r.name = "regulatory_analyst" THEN u.tildaid END) AS raTildaid
}
CALL {
WITH o
OPTIONAL MATCH (o)-[:ORGANIZATION_TO_ACTOR]-(a:Actor{delpending:false})-[:FOR_USER]-(u:User),
(a)-[:APPOINTED_AS]-(:Role{name:"org_admin"})
RETURN collect(DISTINCT u.tildaid) AS adminTildaid
}
CALL {
WITH scopeSite
OPTIONAL MATCH (scopeSite)-[:STAFF_TO_ACTOR]-(a:Actor{delpending:false})-[:FOR_USER]-(u:User),
(a)-[:APPOINTED_AS]-(:Role{name:"site_admin"})
RETURN collect(DISTINCT u.tildaid) AS siteAdminTildaid
}
RETURN DISTINCT t AS task, apoc.convert.toJson({userTildaid: user.tildaid, userName: user.name,
PITildaid: piTildaid, studyOwnerTildaid: smoTildaid, regAnalystTildaid: raTildaid,
orgAdminTildaid: adminTildaid, studyName: study.name, siteName: scopeSite.name,
siteAdminTildaid: siteAdminTildaid}) AS details, tc.who AS dsnWho, o.alias AS orgAlias,
taskInbox, taskUser.tildaid AS taskUserId
SKIP 1200 LIMIT 400
raw payload (15 fields)
{
"allocatedBytes": 12001120.0,
"cpu": 36152.0,
"elapsedTimeMs": 36237.0,
"executionPlanCacheKeyHash": "29FFC5DC",
"id": "24640301",
"pageFaults": 0.0,
"pageHits": 76161217.0,
"planning": 60.0,
"query": "\n\t\tMATCH (study:Study )-[:MANAGED_BY]-(:Inbox)-[:BELONGS_TO]-(t:Task)\n\t\tWHERE NOT EXISTS ((t)-[:BELONGS_TO]-(:Inbox{type:\"User\"}))\n\t\tCALL {\n\t\t\tWITH t\n\t\t\tMATCH (t)\u003c-[:CREATED_FROM_SPEC]-(ptc:ParticipantTaskConfig)-[:PARTICIPANT_TASKCONFIG]-(:ParticipantTaskList)-[:PARTICIPANT_TASKLIST]-(:ParticipantControl)-[:PARTICIPANT_CONTROL]-(:ParticipantActivity)-[:PARTICIPANT_ACTIVITY]-(pw:ParticipantWorkflow)\n\t\t\tRETURN ptc, pw\n\t\t\tUNION ALL\n\t\t\tWITH t\n\t\t\tMATCH (t)\u003c-[:CREATED_FROM_SPEC]-(ptc:ParticipantTaskConfig)-[:PARTICIPANT_TASKCONFIG]-(:ParticipantTaskList)-[:PARTICIPANT_TASKLIST]-(:ParticipantActivity)-[:PARTICIPANT_ACTIVITY]-(pw:ParticipantWorkflow)\n\t\t\tRETURN ptc, pw\n\t\t}\n\t\tWITH DISTINCT study, t, ptc, pw\n\t\tMATCH (t)-[:BELONGS_TO]-(:Inbox)-[:MANAGED_BY]-(o:Organization)\n\t\tMATCH (ptc)-[:BELONGS_TO_TASKCONFIG]-(tc:TaskConfig)\n\t\tOPTIONAL MATCH (t)-[:BELONGS_TO]-(taskInbox:Inbox{type:\"User\"})--(taskUser:User)\n\t\tOPTIONAL MATCH (pw)-[:PARTICIPANTWORKFLOW_TO_SCOPE]-(:StudySiteActor)-[:STUDYSITEACTOR_TO_ACTOR]-(:Actor{delpending:false})-[:FOR_USER]-(user:User)\n\t\tOPTIONAL MATCH (pw)-[:PARTICIPANTWORKFLOW_TO_SCOPE]-(:DslPlanDefinition)-[:PLAN_DEFINITION_TO_SITE]-(scopeSite:Site)-[:SITE_TO_SMO]-(:Smo)-[:IS_AN_ORG]-(o)\n\t\tCALL {\n\t\t\tWITH study\n\t\t\tOPTIONAL MATCH (study)-[:STUDY_TO_ACTOR]-(a:Actor{delpending:false})-[:FOR_USER]-(u:User),\n\t\t\t\t(a)-[:APPOINTED_AS]-(r:Role)\n\t\t\tWHERE r.name IN [\"principal_investigator\", \"study_admin\", \"regulatory_analyst\"]\n\t\t\tRETURN collect(DISTINCT CASE WHEN r.name = \"principal_investigator\" THEN u.tildaid END) AS piTildaid,\n\t\t\t\tcollect(DISTINCT CASE WHEN r.name = \"study_admin\" THEN u.tildaid END) AS smoTildaid,\n\t\t\t\tcollect(DISTINCT CASE WHEN r.name = \"regulatory_analyst\" THEN u.tildaid END) AS raTildaid\n\t\t}\n\t\tCALL {\n\t\t\tWITH o\n\t\t\tOPTIONAL MATCH (o)-[:ORGANIZATION_TO_ACTOR]-(a:Actor{delpending:false})-[:FOR_USER]-(u:User),\n\t\t\t\t(a)-[:APPOINTED_AS]-(:Role{name:\"org_admin\"})\n\t\t\tRETURN collect(DISTINCT u.tildaid) AS adminTildaid\n\t\t}\n\t\tCALL {\n\t\t\tWITH scopeSite\n\t\t\tOPTIONAL MATCH (scopeSite)-[:STAFF_TO_ACTOR]-(a:Actor{delpending:false})-[:FOR_USER]-(u:User),\n\t\t\t\t(a)-[:APPOINTED_AS]-(:Role{name:\"site_admin\"})\n\t\t\tRETURN collect(DISTINCT u.tildaid) AS siteAdminTildaid\n\t\t}\n\t\tRETURN DISTINCT t AS task, apoc.convert.toJson({userTildaid: user.tildaid, userName: user.name,\n\t\t\tPITildaid: piTildaid, studyOwnerTildaid: smoTildaid, regAnalystTildaid: raTildaid,\n\t\t\torgAdminTildaid: adminTildaid, studyName: study.name, siteName: scopeSite.name,\n\t\t\tsiteAdminTildaid: siteAdminTildaid}) AS details, tc.who AS dsnWho, o.alias AS orgAlias,\n\t\t\ttaskInbox, taskUser.tildaid AS taskUserId\n\t\tSKIP 1200 LIMIT 400",
"queryParameters": "{}",
"runtime": "pipelined",
"source": "bolt-session\tbolt\tneo4j-java/5.28.13-24cd0fecc64de123fd5711190246e8b0f1e16fd0\t\tclient/10.173.2.71:36504\tserver/10.173.0.8:7688\u003e",
"timestamp": "2026-07-27T00:07:39.953Z",
"transactionId": 12022065.0,
"waiting": 0.0
}11s ago2026-07-27 00:07:39 UTC INFO
filter by:
severity≥INFO
time2026-07-27 00:07:39 UTC
severityINFO
clustermaster
podlitellm-f54655c99-tt2pg
11s ago2026-07-27 00:07:39 UTC INFO
filter by:
severity≥INFO
time2026-07-27 00:07:39 UTC
severityINFO
clustermaster
podrag-agent-57879dd94f-l9nz7
14s ago2026-07-27 00:07:36 UTC neo4j advpsvc match(a:AdvRateRule{deleted:true})-[r]-(b) with distinct a return a.portalorgid, a.tildaid, a.ruletype limit 10
time2026-07-27 00:07:36 UTC
severityINFO
serviceadvpsvc
clustermaster
podserver-1-dev-0
query
match(a:AdvRateRule{deleted:true})-[r]-(b)
with distinct a
return a.portalorgid, a.tildaid, a.ruletype
limit 10
raw payload (15 fields)
{
"allocatedBytes": 328.0,
"cpu": 2.0,
"elapsedTimeMs": 2.0,
"executionPlanCacheKeyHash": "AFAFBFFB",
"id": "17372849",
"pageFaults": 0.0,
"pageHits": 1146.0,
"planning": 0.0,
"query": "match(a:AdvRateRule{deleted:true})-[r]-(b) with distinct a return a.portalorgid, a.tildaid, a.ruletype limit 10",
"queryParameters": "{}",
"runtime": "pipelined",
"source": "bolt-session\tbolt\tGo Driver/5.20.0\t\tclient/10.129.3.19:37624\tserver/10.173.2.71:7687\u003e",
"timestamp": "2026-07-27T00:07:36.982Z",
"transactionId": 1771019.0,
"waiting": 0.0
}15s ago2026-07-27 00:07:35 UTC DEBUG virtual-assistant [mem] background_reclaimer executed
time2026-07-27 00:07:35 UTC
severityDEBUG
servicevirtual-assistant
sourcesrc/main.py:540
clustermaster
versionunknown
podvirtual-assistant-65b975d66b-l8wpw
message
[mem] background_reclaimer executed
raw payload (7 fields)
{
"caller": "src/main.py:540",
"cluster": "master",
"environment": "DEVELOPMENT",
"message": "[mem] background_reclaimer executed",
"service": "virtual-assistant",
"timestamp": "2026-07-27T00:07:35.753Z",
"version": "unknown"
}15s ago2026-07-27 00:07:35 UTC DEBUG virtual-assistant aggressive_memory_release: GC skipped based on should_run_gc() decision
time2026-07-27 00:07:35 UTC
severityDEBUG
servicevirtual-assistant
sourcecommon/mem_release.py:79
clustermaster
versionunknown
podvirtual-assistant-65b975d66b-l8wpw
message
aggressive_memory_release: GC skipped based on should_run_gc() decision
raw payload (7 fields)
{
"caller": "common/mem_release.py:79",
"cluster": "master",
"environment": "DEVELOPMENT",
"message": "aggressive_memory_release: GC skipped based on should_run_gc() decision",
"service": "virtual-assistant",
"timestamp": "2026-07-27T00:07:35.753Z",
"version": "unknown"
}15s ago2026-07-27 00:07:35 UTC DEBUG virtual-assistant should_run_gc: mem usage: 22.9% (458.4MB/2000.0MB), GC gen0 counts=315
time2026-07-27 00:07:35 UTC
severityDEBUG
servicevirtual-assistant
sourcereg/memory_monitor.py:464
clustermaster
versionunknown
podvirtual-assistant-65b975d66b-l8wpw
message
should_run_gc: mem usage: 22.9% (458.4MB/2000.0MB), GC gen0 counts=315
raw payload (7 fields)
{
"caller": "reg/memory_monitor.py:464",
"cluster": "master",
"environment": "DEVELOPMENT",
"message": "should_run_gc: mem usage: 22.9% (458.4MB/2000.0MB), GC gen0 counts=315",
"service": "virtual-assistant",
"timestamp": "2026-07-27T00:07:35.753Z",
"version": "unknown"
}15s ago2026-07-27 00:07:35 UTC INFO
filter by:
severity≥INFO
time2026-07-27 00:07:35 UTC
severityINFO
clustermaster
podlitellm-f54655c99-cbmvt
15s ago2026-07-27 00:07:35 UTC INFO
filter by:
severity≥INFO
time2026-07-27 00:07:35 UTC
severityINFO
clustermaster
podrag-agent-57879dd94f-l9nz7
16s ago2026-07-27 00:07:34 UTC DEBUG execsvc Memory statistics
time2026-07-27 00:07:34 UTC
severityDEBUG
serviceexecsvc
sourcemetrics/memory.go:146
clustermaster
versionv1
podexecsvc-5c986994bc-psst2
message
Memory statistics
raw payload (12 fields)
{
"caller": "metrics/memory.go:146",
"cluster": "master",
"environment": "DEVELOPMENT",
"goroutines": 232.0,
"heap_alloc_mb": 9.0,
"message": "Memory statistics",
"msg": "mem_stats",
"num_gc": 408.0,
"service": "execsvc",
"sys_mb": 34.0,
"timestamp": "2026-07-27T00:07:34.611Z",
"version": "v1"
}16s ago2026-07-27 00:07:34 UTC neo4j virtual-assistant RETURN 1 as health_check
time2026-07-27 00:07:34 UTC
severityINFO
servicevirtual-assistant
clustermaster
trace85a85285030da1f0712a89c55bf698d3
span80f1c544c5f4be3a
podserver-2-dev-0
query
RETURN 1 as health_check
raw payload (15 fields)
{
"allocatedBytes": 312.0,
"cpu": 0.0,
"elapsedTimeMs": 0.0,
"executionPlanCacheKeyHash": "745CE4BF",
"id": "24641314",
"pageFaults": 0.0,
"pageHits": 0.0,
"planning": 0.0,
"query": "RETURN 1 as health_check",
"queryParameters": "{}",
"runtime": "pipelined",
"source": "bolt-session\tbolt\tneo4j-python/5.28.0 Python/3.12.13-final-0 (linux)\t\tclient/10.129.4.156:57418\tserver/10.173.0.8:7687\u003e",
"timestamp": "2026-07-27T00:07:34.229Z",
"transactionId": 12025357.0,
"waiting": 0.0
}16s ago2026-07-27 00:07:34 UTC neo4j virtual-assistant RETURN 1
time2026-07-27 00:07:34 UTC
severityINFO
servicevirtual-assistant
clustermaster
trace85a85285030da1f0712a89c55bf698d3
span80f1c544c5f4be3a
podserver-2-dev-0
query
RETURN 1
raw payload (15 fields)
{
"allocatedBytes": 312.0,
"cpu": 0.0,
"elapsedTimeMs": 0.0,
"executionPlanCacheKeyHash": "4CADEF45",
"id": "24641313",
"pageFaults": 0.0,
"pageHits": 0.0,
"planning": 0.0,
"query": "RETURN 1",
"queryParameters": "{}",
"runtime": "pipelined",
"source": "bolt-session\tbolt\tneo4j-python/5.28.0 Python/3.12.13-final-0 (linux)\t\tclient/10.129.4.156:57418\tserver/10.173.0.8:7687\u003e",
"timestamp": "2026-07-27T00:07:34.225Z",
"transactionId": 12025356.0,
"waiting": 0.0
}16s ago2026-07-27 00:07:34 UTC DEBUG virtual-assistant Session attributes: ['_auto_result', '_bookmark_manager', '_bookmarks', '_check_state', '_closed', '_config', '_connect', '_connection',...
time2026-07-27 00:07:34 UTC
severityDEBUG
servicevirtual-assistant
sourcesrc/neo4j_client.py:850
clustermaster
versionunknown
trace85a85285030da1f0712a89c55bf698d3
span80f1c544c5f4be3a
podvirtual-assistant-65b975d66b-l8wpw
message
Session attributes: ['_auto_result', '_bookmark_manager', '_bookmarks', '_check_state', '_closed', '_config', '_connect', '_connection', '_connection_access_mode', '_disconnect', '_get_bookmarks', '_get_routing_target_database', '_get_server_info', '_handle_cancellation', '_initial_bookmarks', '_initialize_bookmarks', '_last_cache_key', '_last_from_bookmark_manager', '_make_db_resolution_callback', '_non_concurrent_iter', '_non_concurrent_method', '_open_transaction', '_pinned_database', '_pipelined_begin', '_pool', '_resolve_session_auth', '_result_closed', '_result_error', '_run_transaction', '_set_pinned_database', '_state_failed', '_transaction', '_transaction_cancel_handler', '_transaction_closed_handler', '_transaction_error_handler', '_update_bookmark', '_update_bookmarks', '_verify_authentication', 'begin_transaction', 'close', 'closed', 'execute_read', 'execute_write', 'last_bookmark', 'last_bookmarks', 'read_transaction', 'run', 'write_transaction']
raw payload (9 fields)
{
"caller": "src/neo4j_client.py:850",
"cluster": "master",
"environment": "DEVELOPMENT",
"message": "Session attributes: [\u0027_auto_result\u0027, \u0027_bookmark_manager\u0027, \u0027_bookmarks\u0027, \u0027_check_state\u0027, \u0027_closed\u0027, \u0027_config\u0027, \u0027_connect\u0027, \u0027_connection\u0027, \u0027_connection_access_mode\u0027, \u0027_disconnect\u0027, \u0027_get_bookmarks\u0027, \u0027_get_routing_target_database\u0027, \u0027_get_server_info\u0027, \u0027_handle_cancellation\u0027, \u0027_initial_bookmarks\u0027, \u0027_initialize_bookmarks\u0027, \u0027_last_cache_key\u0027, \u0027_last_from_bookmark_manager\u0027, \u0027_make_db_resolution_callback\u0027, \u0027_non_concurrent_iter\u0027, \u0027_non_concurrent_method\u0027, \u0027_open_transaction\u0027, \u0027_pinned_database\u0027, \u0027_pipelined_begin\u0027, \u0027_pool\u0027, \u0027_resolve_session_auth\u0027, \u0027_result_closed\u0027, \u0027_result_error\u0027, \u0027_run_transaction\u0027, \u0027_set_pinned_database\u0027, \u0027_state_failed\u0027, \u0027_transaction\u0027, \u0027_transaction_cancel_handler\u0027, \u0027_transaction_closed_handler\u0027, \u0027_transaction_error_handler\u0027, \u0027_update_bookmark\u0027, \u0027_update_bookmarks\u0027, \u0027_verify_authentication\u0027, \u0027begin_transaction\u0027, \u0027close\u0027, \u0027closed\u0027, \u0027execute_read\u0027, \u0027execute_write\u0027, \u0027last_bookmark\u0027, \u0027last_bookmarks\u0027, \u0027read_transaction\u0027, \u0027run\u0027, \u0027write_transaction\u0027]",
"method": "GET",
"path": "/health",
"service": "virtual-assistant",
"timestamp": "2026-07-27T00:07:34.219Z",
"version": "unknown"
}16s ago2026-07-27 00:07:34 UTC DEBUG virtual-assistant Pool attributes: ['_abc_impl', '_acquire', '_acquire_from_pool', '_acquire_from_pool_checked', '_acquire_new_later', '_add_connections',...
time2026-07-27 00:07:34 UTC
severityDEBUG
servicevirtual-assistant
sourcesrc/neo4j_client.py:796
clustermaster
versionunknown
trace85a85285030da1f0712a89c55bf698d3
span80f1c544c5f4be3a
podvirtual-assistant-65b975d66b-l8wpw
message
Pool attributes: ['_abc_impl', '_acquire', '_acquire_from_pool', '_acquire_from_pool_checked', '_acquire_new_later', '_add_connections', '_close_connections', '_re_auth_connection', '_remove_connection', '_remove_connections', '_select_address', '_ssr_feature_tracker', '_update_routing_table_from', 'acquire', 'address', 'close', 'cond', 'connections', 'connections_reservations', 'deactivate', 'ensure_routing_table_is_fresh', 'fetch_routing_info', 'fetch_routing_table', 'get_auth', 'get_or_create_routing_table', 'get_routing_table', 'home_db_cache', 'in_use_connection_count', 'is_direct_pool', 'kill_and_release', 'lock', 'mark_all_stale', 'on_neo4j_error', 'on_write_failure', 'open', 'opener', 'pool_config', 'refresh_lock', 'release', 'routing_tables', 'ssr_enabled', 'update_connection_pool', 'update_routing_table', 'workspace_config']
raw payload (9 fields)
{
"caller": "src/neo4j_client.py:796",
"cluster": "master",
"environment": "DEVELOPMENT",
"message": "Pool attributes: [\u0027_abc_impl\u0027, \u0027_acquire\u0027, \u0027_acquire_from_pool\u0027, \u0027_acquire_from_pool_checked\u0027, \u0027_acquire_new_later\u0027, \u0027_add_connections\u0027, \u0027_close_connections\u0027, \u0027_re_auth_connection\u0027, \u0027_remove_connection\u0027, \u0027_remove_connections\u0027, \u0027_select_address\u0027, \u0027_ssr_feature_tracker\u0027, \u0027_update_routing_table_from\u0027, \u0027acquire\u0027, \u0027address\u0027, \u0027close\u0027, \u0027cond\u0027, \u0027connections\u0027, \u0027connections_reservations\u0027, \u0027deactivate\u0027, \u0027ensure_routing_table_is_fresh\u0027, \u0027fetch_routing_info\u0027, \u0027fetch_routing_table\u0027, \u0027get_auth\u0027, \u0027get_or_create_routing_table\u0027, \u0027get_routing_table\u0027, \u0027home_db_cache\u0027, \u0027in_use_connection_count\u0027, \u0027is_direct_pool\u0027, \u0027kill_and_release\u0027, \u0027lock\u0027, \u0027mark_all_stale\u0027, \u0027on_neo4j_error\u0027, \u0027on_write_failure\u0027, \u0027open\u0027, \u0027opener\u0027, \u0027pool_config\u0027, \u0027refresh_lock\u0027, \u0027release\u0027, \u0027routing_tables\u0027, \u0027ssr_enabled\u0027, \u0027update_connection_pool\u0027, \u0027update_routing_table\u0027, \u0027workspace_config\u0027]",
"method": "GET",
"path": "/health",
"service": "virtual-assistant",
"timestamp": "2026-07-27T00:07:34.218Z",
"version": "unknown"
}16s ago2026-07-27 00:07:34 UTC DEBUG virtual-assistant Driver attributes: ['_check_state', '_closed', '_default_workspace_config', '_get_server_info', '_initial_addresses', '_pool',...
time2026-07-27 00:07:34 UTC
severityDEBUG
servicevirtual-assistant
sourcesrc/neo4j_client.py:787
clustermaster
versionunknown
trace85a85285030da1f0712a89c55bf698d3
span80f1c544c5f4be3a
podvirtual-assistant-65b975d66b-l8wpw
message
Driver attributes: ['_check_state', '_closed', '_default_workspace_config', '_get_server_info', '_initial_addresses', '_pool', '_prepare_session_config', '_query_bookmark_manager', '_read_session_config', '_session', 'close', 'default_host', 'default_port', 'default_targets', 'encrypted', 'execute_query', 'execute_query_bookmark_manager', 'get_server_info', 'initial_addresses', 'open', 'parse_targets', 'session', 'supports_multi_db', 'supports_session_auth', 'verify_authentication', 'verify_connectivity']
raw payload (9 fields)
{
"caller": "src/neo4j_client.py:787",
"cluster": "master",
"environment": "DEVELOPMENT",
"message": "Driver attributes: [\u0027_check_state\u0027, \u0027_closed\u0027, \u0027_default_workspace_config\u0027, \u0027_get_server_info\u0027, \u0027_initial_addresses\u0027, \u0027_pool\u0027, \u0027_prepare_session_config\u0027, \u0027_query_bookmark_manager\u0027, \u0027_read_session_config\u0027, \u0027_session\u0027, \u0027close\u0027, \u0027default_host\u0027, \u0027default_port\u0027, \u0027default_targets\u0027, \u0027encrypted\u0027, \u0027execute_query\u0027, \u0027execute_query_bookmark_manager\u0027, \u0027get_server_info\u0027, \u0027initial_addresses\u0027, \u0027open\u0027, \u0027parse_targets\u0027, \u0027session\u0027, \u0027supports_multi_db\u0027, \u0027supports_session_auth\u0027, \u0027verify_authentication\u0027, \u0027verify_connectivity\u0027]",
"method": "GET",
"path": "/health",
"service": "virtual-assistant",
"timestamp": "2026-07-27T00:07:34.218Z",
"version": "unknown"
}16s ago2026-07-27 00:07:34 UTC DEBUG virtual-assistant 🔍 AUTH_DEBUG: Path in UNPROTECTED_ENDPOINTS: True
time2026-07-27 00:07:34 UTC
severityDEBUG
servicevirtual-assistant
sourceapi/api.py:491
clustermaster
versionunknown
trace85a85285030da1f0712a89c55bf698d3
span80f1c544c5f4be3a
podvirtual-assistant-65b975d66b-l8wpw
message
🔍 AUTH_DEBUG: Path in UNPROTECTED_ENDPOINTS: True
raw payload (9 fields)
{
"caller": "api/api.py:491",
"cluster": "master",
"environment": "DEVELOPMENT",
"message": "\ud83d\udd0d AUTH_DEBUG: Path in UNPROTECTED_ENDPOINTS: True",
"method": "GET",
"path": "/health",
"service": "virtual-assistant",
"timestamp": "2026-07-27T00:07:34.214Z",
"version": "unknown"
}16s ago2026-07-27 00:07:34 UTC DEBUG virtual-assistant 🔍 AUTH_DEBUG: Method: GET
time2026-07-27 00:07:34 UTC
severityDEBUG
servicevirtual-assistant
sourceapi/api.py:489
clustermaster
versionunknown
trace85a85285030da1f0712a89c55bf698d3
span80f1c544c5f4be3a
podvirtual-assistant-65b975d66b-l8wpw
message
🔍 AUTH_DEBUG: Method: GET
raw payload (9 fields)
{
"caller": "api/api.py:489",
"cluster": "master",
"environment": "DEVELOPMENT",
"message": "\ud83d\udd0d AUTH_DEBUG: Method: GET",
"method": "GET",
"path": "/health",
"service": "virtual-assistant",
"timestamp": "2026-07-27T00:07:34.213Z",
"version": "unknown"
}17s ago2026-07-27 00:07:33 UTC ERROR observability: level watcher could not stat config file /etc/config/observability.yaml: stat /etc/config/observability.yaml: no such file or directory
filter by:
severity≥ERROR
time2026-07-27 00:07:33 UTC
severityERROR
clustermaster
podadvpsvc-66c87987bc-6vk8q
message
observability: level watcher could not stat config file /etc/config/observability.yaml: stat /etc/config/observability.yaml: no such file or directory
18s ago2026-07-27 00:07:32 UTC ERROR observability: level watcher could not stat config file /etc/config/observability.yaml: stat /etc/config/observability.yaml: no such file or directory
filter by:
severity≥ERROR
time2026-07-27 00:07:32 UTC
severityERROR
clustermaster
podexecsvc-5c986994bc-psst2
message
observability: level watcher could not stat config file /etc/config/observability.yaml: stat /etc/config/observability.yaml: no such file or directory
19s ago2026-07-27 00:07:31 UTC neo4j advpsvc match(a:AdvRateRule{deleted:true})-[r]-(b) with distinct a return a.portalorgid, a.tildaid, a.ruletype limit 10
time2026-07-27 00:07:31 UTC
severityINFO
serviceadvpsvc
clustermaster
podserver-1-dev-0
query
match(a:AdvRateRule{deleted:true})-[r]-(b)
with distinct a
return a.portalorgid, a.tildaid, a.ruletype
limit 10
raw payload (15 fields)
{
"allocatedBytes": 328.0,
"cpu": 2.0,
"elapsedTimeMs": 2.0,
"executionPlanCacheKeyHash": "AFAFBFFB",
"id": "17372835",
"pageFaults": 0.0,
"pageHits": 1146.0,
"planning": 0.0,
"query": "match(a:AdvRateRule{deleted:true})-[r]-(b) with distinct a return a.portalorgid, a.tildaid, a.ruletype limit 10",
"queryParameters": "{}",
"runtime": "pipelined",
"source": "bolt-session\tbolt\tGo Driver/5.20.0\t\tclient/10.129.3.19:37624\tserver/10.173.2.71:7687\u003e",
"timestamp": "2026-07-27T00:07:31.973Z",
"transactionId": 1771018.0,
"waiting": 0.0
}19s ago2026-07-27 00:07:31 UTC DEBUG docproc Reported pending workload
time2026-07-27 00:07:31 UTC
severityDEBUG
servicedocproc
sourcejobs/hpa_pending_workload.py:176
clustermaster
version0.1.0
poddocproc-7bc76669-ljh8h
message
Reported pending workload
raw payload (15 fields)
{
"backlog_capped": 0.0,
"caller": "jobs/hpa_pending_workload.py:176",
"cluster": "master",
"component": "hpa_pending_workload",
"environment": "DEVELOPMENT",
"in_progress_count": 0.0,
"message": "Reported pending workload",
"num_replicas": 1.0,
"recovery_workload": 0.0,
"service": "docproc",
"signal": 0.0,
"timestamp": "2026-07-27T00:07:31.714Z",
"total_workload": 0.0,
"unacked": 0.0,
"version": "0.1.0"
}19s ago2026-07-27 00:07:31 UTC DEBUG docproc Pub/Sub num_unacked result: project_id=tilda-trial-automation subscription_id_in_filter=tilda-document-master-docproc-sub num_series=1 total=0
time2026-07-27 00:07:31 UTC
severityDEBUG
servicedocproc
sourcejobs/cloud_monitoring_metrics.py:296
clustermaster
version0.1.0
poddocproc-7bc76669-ljh8h
message
Pub/Sub num_unacked result: project_id=tilda-trial-automation subscription_id_in_filter=tilda-document-master-docproc-sub num_series=1 total=0
raw payload (7 fields)
{
"caller": "jobs/cloud_monitoring_metrics.py:296",
"cluster": "master",
"environment": "DEVELOPMENT",
"message": "Pub/Sub num_unacked result: project_id=tilda-trial-automation subscription_id_in_filter=tilda-document-master-docproc-sub num_series=1 total=0",
"service": "docproc",
"timestamp": "2026-07-27T00:07:31.641Z",
"version": "0.1.0"
}19s ago2026-07-27 00:07:31 UTC neo4j docproc MATCH (o:Organization{orgType:"CRO"})-[:DOC_TO_ORG]-(d:Document)-[:INCLUDES]-(f:Folder) WHERE f.name IN ['Staging', 'Pre-Staging'] ...
time2026-07-27 00:07:31 UTC
severityINFO
servicedocproc
clustermaster
podserver-2-dev-0
query
MATCH (o:Organization{orgType:"CRO"})-[:DOC_TO_ORG]-(d:Document)-[:INCLUDES]-(f:Folder)
WHERE f.name IN ['Staging', 'Pre-Staging']
AND d.delpending = false AND d.aiEnabled = true AND d.studyId <> ""
AND d.primaryVersionId IS NOT NULL AND d.primaryVersionId <> ""
OPTIONAL MATCH (d)<-[:AI_RUN_TO_DOCUMENT_LATEST]-(ai:AIRun)
WITH o, d, ai
ORDER BY d.tildaid, COALESCE(ai.updatedat, ai.createdat) DESC
WITH o, d, collect(ai)[0] AS latest_ai
WHERE (
latest_ai IS NULL
OR latest_ai.status = "aborted"
OR (
latest_ai.status = "failed"
AND COALESCE(latest_ai.auto_retry_count, 0) < $max_retries
AND COALESCE(latest_ai.updatedat, latest_ai.createdat) < $threshold_ms
)
)
WITH d
RETURN count(d) AS total
raw payload (15 fields)
{
"allocatedBytes": 7666360.0,
"cpu": 163.0,
"elapsedTimeMs": 163.0,
"executionPlanCacheKeyHash": "590286C5",
"id": "24641309",
"pageFaults": 0.0,
"pageHits": 156362.0,
"planning": 0.0,
"query": "\n MATCH (o:Organization{orgType:\"CRO\"})-[:DOC_TO_ORG]-(d:Document)-[:INCLUDES]-(f:Folder)\n WHERE f.name IN [\u0027Staging\u0027, \u0027Pre-Staging\u0027]\n AND d.delpending = false AND d.aiEnabled = true AND d.studyId \u003c\u003e \"\"\n AND d.primaryVersionId IS NOT NULL AND d.primaryVersionId \u003c\u003e \"\"\n OPTIONAL MATCH (d)\u003c-[:AI_RUN_TO_DOCUMENT_LATEST]-(ai:AIRun)\n WITH o, d, ai\n ORDER BY d.tildaid, COALESCE(ai.updatedat, ai.createdat) DESC\n WITH o, d, collect(ai)[0] AS latest_ai\n WHERE (\n latest_ai IS NULL\n OR latest_ai.status = \"aborted\"\n OR (\n latest_ai.status = \"failed\"\n AND COALESCE(latest_ai.auto_retry_count, 0) \u003c $max_retries\n AND COALESCE(latest_ai.updatedat, latest_ai.createdat) \u003c $threshold_ms\n )\n )\n WITH d\n RETURN count(d) AS total\n ",
"queryParameters": "{max_retries: 5, threshold_ms: 1785109951236}",
"runtime": "pipelined",
"source": "bolt-session\tbolt\tneo4j-python/6.2.0 Python/3.12.13-final-0 (linux)\t\tclient/10.129.3.15:59818\tserver/10.173.0.8:7687\u003e",
"timestamp": "2026-07-27T00:07:31.401Z",
"transactionId": 12025355.0,
"waiting": 0.0
}19s ago2026-07-27 00:07:31 UTC neo4j docproc MATCH (ai:AIRun)-[:AI_RUN_TO_DOCUMENT_LATEST]->(d:Document) WHERE d.aiEnabled = true AND ( (ai.status = "in_progress" AND...
time2026-07-27 00:07:31 UTC
severityINFO
servicedocproc
clustermaster
podserver-2-dev-0
query
MATCH (ai:AIRun)-[:AI_RUN_TO_DOCUMENT_LATEST]->(d:Document)
WHERE d.aiEnabled = true
AND (
(ai.status = "in_progress" AND ($cutoff_ms <= 0 OR COALESCE(ai.updatedat, ai.createdat) < $cutoff_ms))
OR ai.status = "aborted"
)
RETURN count(ai) AS total
raw payload (15 fields)
{
"allocatedBytes": 488.0,
"cpu": 63.0,
"elapsedTimeMs": 63.0,
"executionPlanCacheKeyHash": "E4907905",
"id": "24641308",
"pageFaults": 0.0,
"pageHits": 66825.0,
"planning": 0.0,
"query": "\n MATCH (ai:AIRun)-[:AI_RUN_TO_DOCUMENT_LATEST]-\u003e(d:Document)\n WHERE d.aiEnabled = true\n AND (\n (ai.status = \"in_progress\" AND ($cutoff_ms \u003c= 0 OR COALESCE(ai.updatedat, ai.createdat) \u003c $cutoff_ms))\n OR ai.status = \"aborted\"\n )\n RETURN count(ai) AS total\n ",
"queryParameters": "{cutoff_ms: 0}",
"runtime": "pipelined",
"source": "bolt-session\tbolt\tneo4j-python/6.2.0 Python/3.12.13-final-0 (linux)\t\tclient/10.129.3.15:59818\tserver/10.173.0.8:7687\u003e",
"timestamp": "2026-07-27T00:07:31.235Z",
"transactionId": 12025354.0,
"waiting": 0.0
}20s ago2026-07-27 00:07:30 UTC ERROR observability: level watcher could not stat config file /etc/config/observability.yaml: stat /etc/config/observability.yaml: no such file or directory
filter by:
severity≥ERROR
time2026-07-27 00:07:30 UTC
severityERROR
clustermaster
podtusker-858b678c65-hsvqd
message
observability: level watcher could not stat config file /etc/config/observability.yaml: stat /etc/config/observability.yaml: no such file or directory
20s ago2026-07-27 00:07:30 UTC ERROR observability: level watcher could not stat config file /etc/config/observability.yaml: stat /etc/config/observability.yaml: no such file or directory
filter by:
severity≥ERROR
time2026-07-27 00:07:30 UTC
severityERROR
clustermaster
podnotifsvc-8c4bdf758-8kx6b
message
observability: level watcher could not stat config file /etc/config/observability.yaml: stat /etc/config/observability.yaml: no such file or directory
20s ago2026-07-27 00:07:30 UTC INFO
filter by:
severity≥INFO
time2026-07-27 00:07:30 UTC
severityINFO
clustermaster
podlitellm-f54655c99-tt2pg
20s ago2026-07-27 00:07:30 UTC INFO
filter by:
severity≥INFO
time2026-07-27 00:07:30 UTC
severityINFO
clustermaster
podrag-agent-57879dd94f-l9nz7
20s ago2026-07-27 00:07:30 UTC ERROR observability: level watcher could not stat config file /etc/config/observability.yaml: stat /etc/config/observability.yaml: no such file or directory
filter by:
severity≥ERROR
time2026-07-27 00:07:30 UTC
severityERROR
clustermaster
podfinsvc-7cb7dbf559-dsqxb
message
observability: level watcher could not stat config file /etc/config/observability.yaml: stat /etc/config/observability.yaml: no such file or directory
21s ago2026-07-27 00:07:29 UTC INFO
filter by:
severity≥INFO
time2026-07-27 00:07:29 UTC
severityINFO
clustermaster
podlitellm-f54655c99-cbmvt
21s ago2026-07-27 00:07:29 UTC INFO
filter by:
severity≥INFO
time2026-07-27 00:07:29 UTC
severityINFO
clustermaster
podrag-agent-57879dd94f-l9nz7
24s ago2026-07-27 00:07:26 UTC neo4j advpsvc match(a:AdvRateRule{deleted:true})-[r]-(b) with distinct a return a.portalorgid, a.tildaid, a.ruletype limit 10
time2026-07-27 00:07:26 UTC
severityINFO
serviceadvpsvc
clustermaster
podserver-1-dev-0
query
match(a:AdvRateRule{deleted:true})-[r]-(b)
with distinct a
return a.portalorgid, a.tildaid, a.ruletype
limit 10
raw payload (15 fields)
{
"allocatedBytes": 328.0,
"cpu": 2.0,
"elapsedTimeMs": 2.0,
"executionPlanCacheKeyHash": "AFAFBFFB",
"id": "17372819",
"pageFaults": 0.0,
"pageHits": 1146.0,
"planning": 0.0,
"query": "match(a:AdvRateRule{deleted:true})-[r]-(b) with distinct a return a.portalorgid, a.tildaid, a.ruletype limit 10",
"queryParameters": "{}",
"runtime": "pipelined",
"source": "bolt-session\tbolt\tGo Driver/5.20.0\t\tclient/10.129.3.19:37624\tserver/10.173.2.71:7687\u003e",
"timestamp": "2026-07-27T00:07:26.964Z",
"transactionId": 1771017.0,
"waiting": 0.0
}24s ago2026-07-27 00:07:26 UTC ERROR observability: level watcher could not stat config file /etc/config/observability.yaml: stat /etc/config/observability.yaml: no such file or directory
filter by:
severity≥ERROR
time2026-07-27 00:07:26 UTC
severityERROR
clustermaster
podonboardingsvc-f7f5bf697-xrcmm
message
observability: level watcher could not stat config file /etc/config/observability.yaml: stat /etc/config/observability.yaml: no such file or directory
25s ago2026-07-27 00:07:25 UTC INFO
filter by:
severity≥INFO
time2026-07-27 00:07:25 UTC
severityINFO
clustermaster
podlitellm-f54655c99-tt2pg
25s ago2026-07-27 00:07:25 UTC INFO
filter by:
severity≥INFO
time2026-07-27 00:07:25 UTC
severityINFO
clustermaster
podrag-agent-57879dd94f-l9nz7
25s ago2026-07-27 00:07:25 UTC ERROR observability: level watcher could not stat config file /etc/config/observability.yaml: stat /etc/config/observability.yaml: no such file or directory
filter by:
severity≥ERROR
time2026-07-27 00:07:25 UTC
severityERROR
clustermaster
poddashrsvc-5d5d9b97d8-69bw8
message
observability: level watcher could not stat config file /etc/config/observability.yaml: stat /etc/config/observability.yaml: no such file or directory
26s ago2026-07-27 00:07:24 UTC DEBUG docproc Pulling messages
time2026-07-27 00:07:24 UTC
severityDEBUG
servicedocproc
sourcehandlers/pubsub_handler.py:240
clustermaster
version0.1.0
poddocproc-7bc76669-ljh8h
message
Pulling messages
raw payload (11 fields)
{
"caller": "handlers/pubsub_handler.py:240",
"cluster": "master",
"component": "pubsub",
"environment": "DEVELOPMENT",
"max_messages": 1.0,
"message": "Pulling messages",
"pull_timeout_s": 60.0,
"service": "docproc",
"subscription_name": "tilda-document-master-docproc-sub",
"timestamp": "2026-07-27T00:07:24.813Z",
"version": "0.1.0"
}26s ago2026-07-27 00:07:24 UTC neo4j virtual-assistant RETURN 1 as health_check
time2026-07-27 00:07:24 UTC
severityINFO
servicevirtual-assistant
clustermaster
trace8733bfed94f1216dc718d89fcf15c491
spanfcd9b5c5de7bb98b
podserver-2-dev-0
query
RETURN 1 as health_check
raw payload (15 fields)
{
"allocatedBytes": 312.0,
"cpu": 0.0,
"elapsedTimeMs": 0.0,
"executionPlanCacheKeyHash": "745CE4BF",
"id": "24641297",
"pageFaults": 0.0,
"pageHits": 0.0,
"planning": 0.0,
"query": "RETURN 1 as health_check",
"queryParameters": "{}",
"runtime": "pipelined",
"source": "bolt-session\tbolt\tneo4j-python/5.28.0 Python/3.12.13-final-0 (linux)\t\tclient/10.129.4.156:57418\tserver/10.173.0.8:7687\u003e",
"timestamp": "2026-07-27T00:07:24.226Z",
"transactionId": 12025350.0,
"waiting": 0.0
}26s ago2026-07-27 00:07:24 UTC neo4j virtual-assistant RETURN 1
time2026-07-27 00:07:24 UTC
severityINFO
servicevirtual-assistant
clustermaster
trace8733bfed94f1216dc718d89fcf15c491
spanfcd9b5c5de7bb98b
podserver-2-dev-0
query
RETURN 1
raw payload (15 fields)
{
"allocatedBytes": 312.0,
"cpu": 0.0,
"elapsedTimeMs": 0.0,
"executionPlanCacheKeyHash": "4CADEF45",
"id": "24641296",
"pageFaults": 0.0,
"pageHits": 0.0,
"planning": 0.0,
"query": "RETURN 1",
"queryParameters": "{}",
"runtime": "pipelined",
"source": "bolt-session\tbolt\tneo4j-python/5.28.0 Python/3.12.13-final-0 (linux)\t\tclient/10.129.4.156:57418\tserver/10.173.0.8:7687\u003e",
"timestamp": "2026-07-27T00:07:24.223Z",
"transactionId": 12025349.0,
"waiting": 0.0
}26s ago2026-07-27 00:07:24 UTC DEBUG virtual-assistant Session attributes: ['_auto_result', '_bookmark_manager', '_bookmarks', '_check_state', '_closed', '_config', '_connect', '_connection',...
time2026-07-27 00:07:24 UTC
severityDEBUG
servicevirtual-assistant
sourcesrc/neo4j_client.py:850
clustermaster
versionunknown
trace8733bfed94f1216dc718d89fcf15c491
spanfcd9b5c5de7bb98b
podvirtual-assistant-65b975d66b-l8wpw
message
Session attributes: ['_auto_result', '_bookmark_manager', '_bookmarks', '_check_state', '_closed', '_config', '_connect', '_connection', '_connection_access_mode', '_disconnect', '_get_bookmarks', '_get_routing_target_database', '_get_server_info', '_handle_cancellation', '_initial_bookmarks', '_initialize_bookmarks', '_last_cache_key', '_last_from_bookmark_manager', '_make_db_resolution_callback', '_non_concurrent_iter', '_non_concurrent_method', '_open_transaction', '_pinned_database', '_pipelined_begin', '_pool', '_resolve_session_auth', '_result_closed', '_result_error', '_run_transaction', '_set_pinned_database', '_state_failed', '_transaction', '_transaction_cancel_handler', '_transaction_closed_handler', '_transaction_error_handler', '_update_bookmark', '_update_bookmarks', '_verify_authentication', 'begin_transaction', 'close', 'closed', 'execute_read', 'execute_write', 'last_bookmark', 'last_bookmarks', 'read_transaction', 'run', 'write_transaction']
raw payload (9 fields)
{
"caller": "src/neo4j_client.py:850",
"cluster": "master",
"environment": "DEVELOPMENT",
"message": "Session attributes: [\u0027_auto_result\u0027, \u0027_bookmark_manager\u0027, \u0027_bookmarks\u0027, \u0027_check_state\u0027, \u0027_closed\u0027, \u0027_config\u0027, \u0027_connect\u0027, \u0027_connection\u0027, \u0027_connection_access_mode\u0027, \u0027_disconnect\u0027, \u0027_get_bookmarks\u0027, \u0027_get_routing_target_database\u0027, \u0027_get_server_info\u0027, \u0027_handle_cancellation\u0027, \u0027_initial_bookmarks\u0027, \u0027_initialize_bookmarks\u0027, \u0027_last_cache_key\u0027, \u0027_last_from_bookmark_manager\u0027, \u0027_make_db_resolution_callback\u0027, \u0027_non_concurrent_iter\u0027, \u0027_non_concurrent_method\u0027, \u0027_open_transaction\u0027, \u0027_pinned_database\u0027, \u0027_pipelined_begin\u0027, \u0027_pool\u0027, \u0027_resolve_session_auth\u0027, \u0027_result_closed\u0027, \u0027_result_error\u0027, \u0027_run_transaction\u0027, \u0027_set_pinned_database\u0027, \u0027_state_failed\u0027, \u0027_transaction\u0027, \u0027_transaction_cancel_handler\u0027, \u0027_transaction_closed_handler\u0027, \u0027_transaction_error_handler\u0027, \u0027_update_bookmark\u0027, \u0027_update_bookmarks\u0027, \u0027_verify_authentication\u0027, \u0027begin_transaction\u0027, \u0027close\u0027, \u0027closed\u0027, \u0027execute_read\u0027, \u0027execute_write\u0027, \u0027last_bookmark\u0027, \u0027last_bookmarks\u0027, \u0027read_transaction\u0027, \u0027run\u0027, \u0027write_transaction\u0027]",
"method": "GET",
"path": "/health",
"service": "virtual-assistant",
"timestamp": "2026-07-27T00:07:24.218Z",
"version": "unknown"
}26s ago2026-07-27 00:07:24 UTC DEBUG virtual-assistant Pool attributes: ['_abc_impl', '_acquire', '_acquire_from_pool', '_acquire_from_pool_checked', '_acquire_new_later', '_add_connections',...
time2026-07-27 00:07:24 UTC
severityDEBUG
servicevirtual-assistant
sourcesrc/neo4j_client.py:796
clustermaster
versionunknown
trace8733bfed94f1216dc718d89fcf15c491
spanfcd9b5c5de7bb98b
podvirtual-assistant-65b975d66b-l8wpw
message
Pool attributes: ['_abc_impl', '_acquire', '_acquire_from_pool', '_acquire_from_pool_checked', '_acquire_new_later', '_add_connections', '_close_connections', '_re_auth_connection', '_remove_connection', '_remove_connections', '_select_address', '_ssr_feature_tracker', '_update_routing_table_from', 'acquire', 'address', 'close', 'cond', 'connections', 'connections_reservations', 'deactivate', 'ensure_routing_table_is_fresh', 'fetch_routing_info', 'fetch_routing_table', 'get_auth', 'get_or_create_routing_table', 'get_routing_table', 'home_db_cache', 'in_use_connection_count', 'is_direct_pool', 'kill_and_release', 'lock', 'mark_all_stale', 'on_neo4j_error', 'on_write_failure', 'open', 'opener', 'pool_config', 'refresh_lock', 'release', 'routing_tables', 'ssr_enabled', 'update_connection_pool', 'update_routing_table', 'workspace_config']
raw payload (9 fields)
{
"caller": "src/neo4j_client.py:796",
"cluster": "master",
"environment": "DEVELOPMENT",
"message": "Pool attributes: [\u0027_abc_impl\u0027, \u0027_acquire\u0027, \u0027_acquire_from_pool\u0027, \u0027_acquire_from_pool_checked\u0027, \u0027_acquire_new_later\u0027, \u0027_add_connections\u0027, \u0027_close_connections\u0027, \u0027_re_auth_connection\u0027, \u0027_remove_connection\u0027, \u0027_remove_connections\u0027, \u0027_select_address\u0027, \u0027_ssr_feature_tracker\u0027, \u0027_update_routing_table_from\u0027, \u0027acquire\u0027, \u0027address\u0027, \u0027close\u0027, \u0027cond\u0027, \u0027connections\u0027, \u0027connections_reservations\u0027, \u0027deactivate\u0027, \u0027ensure_routing_table_is_fresh\u0027, \u0027fetch_routing_info\u0027, \u0027fetch_routing_table\u0027, \u0027get_auth\u0027, \u0027get_or_create_routing_table\u0027, \u0027get_routing_table\u0027, \u0027home_db_cache\u0027, \u0027in_use_connection_count\u0027, \u0027is_direct_pool\u0027, \u0027kill_and_release\u0027, \u0027lock\u0027, \u0027mark_all_stale\u0027, \u0027on_neo4j_error\u0027, \u0027on_write_failure\u0027, \u0027open\u0027, \u0027opener\u0027, \u0027pool_config\u0027, \u0027refresh_lock\u0027, \u0027release\u0027, \u0027routing_tables\u0027, \u0027ssr_enabled\u0027, \u0027update_connection_pool\u0027, \u0027update_routing_table\u0027, \u0027workspace_config\u0027]",
"method": "GET",
"path": "/health",
"service": "virtual-assistant",
"timestamp": "2026-07-27T00:07:24.218Z",
"version": "unknown"
}26s ago2026-07-27 00:07:24 UTC DEBUG virtual-assistant Driver attributes: ['_check_state', '_closed', '_default_workspace_config', '_get_server_info', '_initial_addresses', '_pool',...
time2026-07-27 00:07:24 UTC
severityDEBUG
servicevirtual-assistant
sourcesrc/neo4j_client.py:787
clustermaster
versionunknown
trace8733bfed94f1216dc718d89fcf15c491
spanfcd9b5c5de7bb98b
podvirtual-assistant-65b975d66b-l8wpw
message
Driver attributes: ['_check_state', '_closed', '_default_workspace_config', '_get_server_info', '_initial_addresses', '_pool', '_prepare_session_config', '_query_bookmark_manager', '_read_session_config', '_session', 'close', 'default_host', 'default_port', 'default_targets', 'encrypted', 'execute_query', 'execute_query_bookmark_manager', 'get_server_info', 'initial_addresses', 'open', 'parse_targets', 'session', 'supports_multi_db', 'supports_session_auth', 'verify_authentication', 'verify_connectivity']
raw payload (9 fields)
{
"caller": "src/neo4j_client.py:787",
"cluster": "master",
"environment": "DEVELOPMENT",
"message": "Driver attributes: [\u0027_check_state\u0027, \u0027_closed\u0027, \u0027_default_workspace_config\u0027, \u0027_get_server_info\u0027, \u0027_initial_addresses\u0027, \u0027_pool\u0027, \u0027_prepare_session_config\u0027, \u0027_query_bookmark_manager\u0027, \u0027_read_session_config\u0027, \u0027_session\u0027, \u0027close\u0027, \u0027default_host\u0027, \u0027default_port\u0027, \u0027default_targets\u0027, \u0027encrypted\u0027, \u0027execute_query\u0027, \u0027execute_query_bookmark_manager\u0027, \u0027get_server_info\u0027, \u0027initial_addresses\u0027, \u0027open\u0027, \u0027parse_targets\u0027, \u0027session\u0027, \u0027supports_multi_db\u0027, \u0027supports_session_auth\u0027, \u0027verify_authentication\u0027, \u0027verify_connectivity\u0027]",
"method": "GET",
"path": "/health",
"service": "virtual-assistant",
"timestamp": "2026-07-27T00:07:24.217Z",
"version": "unknown"
}26s ago2026-07-27 00:07:24 UTC DEBUG virtual-assistant 🔍 AUTH_DEBUG: Path in UNPROTECTED_ENDPOINTS: True
time2026-07-27 00:07:24 UTC
severityDEBUG
servicevirtual-assistant
sourceapi/api.py:491
clustermaster
versionunknown
trace8733bfed94f1216dc718d89fcf15c491
spanfcd9b5c5de7bb98b
podvirtual-assistant-65b975d66b-l8wpw
message
🔍 AUTH_DEBUG: Path in UNPROTECTED_ENDPOINTS: True
raw payload (9 fields)
{
"caller": "api/api.py:491",
"cluster": "master",
"environment": "DEVELOPMENT",
"message": "\ud83d\udd0d AUTH_DEBUG: Path in UNPROTECTED_ENDPOINTS: True",
"method": "GET",
"path": "/health",
"service": "virtual-assistant",
"timestamp": "2026-07-27T00:07:24.213Z",
"version": "unknown"
}26s ago2026-07-27 00:07:24 UTC DEBUG virtual-assistant 🔍 AUTH_DEBUG: Method: GET
time2026-07-27 00:07:24 UTC
severityDEBUG
servicevirtual-assistant
sourceapi/api.py:489
clustermaster
versionunknown
trace8733bfed94f1216dc718d89fcf15c491
spanfcd9b5c5de7bb98b
podvirtual-assistant-65b975d66b-l8wpw
message
🔍 AUTH_DEBUG: Method: GET
raw payload (9 fields)
{
"caller": "api/api.py:489",
"cluster": "master",
"environment": "DEVELOPMENT",
"message": "\ud83d\udd0d AUTH_DEBUG: Method: GET",
"method": "GET",
"path": "/health",
"service": "virtual-assistant",
"timestamp": "2026-07-27T00:07:24.213Z",
"version": "unknown"
}26s ago2026-07-27 00:07:24 UTC ERROR observability: level watcher could not stat config file /etc/config/observability.yaml: stat /etc/config/observability.yaml: no such file or directory
filter by:
severity≥ERROR
time2026-07-27 00:07:24 UTC
severityERROR
clustermaster
podpatientsvc-9975d5469-pl5lt
message
observability: level watcher could not stat config file /etc/config/observability.yaml: stat /etc/config/observability.yaml: no such file or directory
26s ago2026-07-27 00:07:24 UTC neo4j virtual-assistant RETURN 1 as health_check
time2026-07-27 00:07:24 UTC
severityINFO
servicevirtual-assistant
clustermaster
tracee2d4772e7991720190f4ce563ac98d70
span36364824ffac4f46
podserver-2-dev-0
query
RETURN 1 as health_check
raw payload (15 fields)
{
"allocatedBytes": 312.0,
"cpu": 0.0,
"elapsedTimeMs": 0.0,
"executionPlanCacheKeyHash": "745CE4BF",
"id": "24641295",
"pageFaults": 0.0,
"pageHits": 0.0,
"planning": 0.0,
"query": "RETURN 1 as health_check",
"queryParameters": "{}",
"runtime": "pipelined",
"source": "bolt-session\tbolt\tneo4j-python/5.28.0 Python/3.12.13-final-0 (linux)\t\tclient/10.129.4.156:57418\tserver/10.173.0.8:7687\u003e",
"timestamp": "2026-07-27T00:07:24.127Z",
"transactionId": 12025348.0,
"waiting": 0.0
}26s ago2026-07-27 00:07:24 UTC neo4j virtual-assistant RETURN 1
time2026-07-27 00:07:24 UTC
severityINFO
servicevirtual-assistant
clustermaster
tracee2d4772e7991720190f4ce563ac98d70
span36364824ffac4f46
podserver-2-dev-0
query
RETURN 1
raw payload (15 fields)
{
"allocatedBytes": 312.0,
"cpu": 0.0,
"elapsedTimeMs": 0.0,
"executionPlanCacheKeyHash": "4CADEF45",
"id": "24641294",
"pageFaults": 0.0,
"pageHits": 0.0,
"planning": 0.0,
"query": "RETURN 1",
"queryParameters": "{}",
"runtime": "pipelined",
"source": "bolt-session\tbolt\tneo4j-python/5.28.0 Python/3.12.13-final-0 (linux)\t\tclient/10.129.4.156:57418\tserver/10.173.0.8:7687\u003e",
"timestamp": "2026-07-27T00:07:24.123Z",
"transactionId": 12025347.0,
"waiting": 0.0
}26s ago2026-07-27 00:07:24 UTC DEBUG virtual-assistant Session attributes: ['_auto_result', '_bookmark_manager', '_bookmarks', '_check_state', '_closed', '_config', '_connect', '_connection',...
time2026-07-27 00:07:24 UTC
severityDEBUG
servicevirtual-assistant
sourcesrc/neo4j_client.py:850
clustermaster
versionunknown
tracee2d4772e7991720190f4ce563ac98d70
span36364824ffac4f46
podvirtual-assistant-65b975d66b-l8wpw
message
Session attributes: ['_auto_result', '_bookmark_manager', '_bookmarks', '_check_state', '_closed', '_config', '_connect', '_connection', '_connection_access_mode', '_disconnect', '_get_bookmarks', '_get_routing_target_database', '_get_server_info', '_handle_cancellation', '_initial_bookmarks', '_initialize_bookmarks', '_last_cache_key', '_last_from_bookmark_manager', '_make_db_resolution_callback', '_non_concurrent_iter', '_non_concurrent_method', '_open_transaction', '_pinned_database', '_pipelined_begin', '_pool', '_resolve_session_auth', '_result_closed', '_result_error', '_run_transaction', '_set_pinned_database', '_state_failed', '_transaction', '_transaction_cancel_handler', '_transaction_closed_handler', '_transaction_error_handler', '_update_bookmark', '_update_bookmarks', '_verify_authentication', 'begin_transaction', 'close', 'closed', 'execute_read', 'execute_write', 'last_bookmark', 'last_bookmarks', 'read_transaction', 'run', 'write_transaction']
raw payload (9 fields)
{
"caller": "src/neo4j_client.py:850",
"cluster": "master",
"environment": "DEVELOPMENT",
"message": "Session attributes: [\u0027_auto_result\u0027, \u0027_bookmark_manager\u0027, \u0027_bookmarks\u0027, \u0027_check_state\u0027, \u0027_closed\u0027, \u0027_config\u0027, \u0027_connect\u0027, \u0027_connection\u0027, \u0027_connection_access_mode\u0027, \u0027_disconnect\u0027, \u0027_get_bookmarks\u0027, \u0027_get_routing_target_database\u0027, \u0027_get_server_info\u0027, \u0027_handle_cancellation\u0027, \u0027_initial_bookmarks\u0027, \u0027_initialize_bookmarks\u0027, \u0027_last_cache_key\u0027, \u0027_last_from_bookmark_manager\u0027, \u0027_make_db_resolution_callback\u0027, \u0027_non_concurrent_iter\u0027, \u0027_non_concurrent_method\u0027, \u0027_open_transaction\u0027, \u0027_pinned_database\u0027, \u0027_pipelined_begin\u0027, \u0027_pool\u0027, \u0027_resolve_session_auth\u0027, \u0027_result_closed\u0027, \u0027_result_error\u0027, \u0027_run_transaction\u0027, \u0027_set_pinned_database\u0027, \u0027_state_failed\u0027, \u0027_transaction\u0027, \u0027_transaction_cancel_handler\u0027, \u0027_transaction_closed_handler\u0027, \u0027_transaction_error_handler\u0027, \u0027_update_bookmark\u0027, \u0027_update_bookmarks\u0027, \u0027_verify_authentication\u0027, \u0027begin_transaction\u0027, \u0027close\u0027, \u0027closed\u0027, \u0027execute_read\u0027, \u0027execute_write\u0027, \u0027last_bookmark\u0027, \u0027last_bookmarks\u0027, \u0027read_transaction\u0027, \u0027run\u0027, \u0027write_transaction\u0027]",
"method": "GET",
"path": "/health",
"service": "virtual-assistant",
"timestamp": "2026-07-27T00:07:24.117Z",
"version": "unknown"
}26s ago2026-07-27 00:07:24 UTC DEBUG virtual-assistant Pool attributes: ['_abc_impl', '_acquire', '_acquire_from_pool', '_acquire_from_pool_checked', '_acquire_new_later', '_add_connections',...
time2026-07-27 00:07:24 UTC
severityDEBUG
servicevirtual-assistant
sourcesrc/neo4j_client.py:796
clustermaster
versionunknown
tracee2d4772e7991720190f4ce563ac98d70
span36364824ffac4f46
podvirtual-assistant-65b975d66b-l8wpw
message
Pool attributes: ['_abc_impl', '_acquire', '_acquire_from_pool', '_acquire_from_pool_checked', '_acquire_new_later', '_add_connections', '_close_connections', '_re_auth_connection', '_remove_connection', '_remove_connections', '_select_address', '_ssr_feature_tracker', '_update_routing_table_from', 'acquire', 'address', 'close', 'cond', 'connections', 'connections_reservations', 'deactivate', 'ensure_routing_table_is_fresh', 'fetch_routing_info', 'fetch_routing_table', 'get_auth', 'get_or_create_routing_table', 'get_routing_table', 'home_db_cache', 'in_use_connection_count', 'is_direct_pool', 'kill_and_release', 'lock', 'mark_all_stale', 'on_neo4j_error', 'on_write_failure', 'open', 'opener', 'pool_config', 'refresh_lock', 'release', 'routing_tables', 'ssr_enabled', 'update_connection_pool', 'update_routing_table', 'workspace_config']
raw payload (9 fields)
{
"caller": "src/neo4j_client.py:796",
"cluster": "master",
"environment": "DEVELOPMENT",
"message": "Pool attributes: [\u0027_abc_impl\u0027, \u0027_acquire\u0027, \u0027_acquire_from_pool\u0027, \u0027_acquire_from_pool_checked\u0027, \u0027_acquire_new_later\u0027, \u0027_add_connections\u0027, \u0027_close_connections\u0027, \u0027_re_auth_connection\u0027, \u0027_remove_connection\u0027, \u0027_remove_connections\u0027, \u0027_select_address\u0027, \u0027_ssr_feature_tracker\u0027, \u0027_update_routing_table_from\u0027, \u0027acquire\u0027, \u0027address\u0027, \u0027close\u0027, \u0027cond\u0027, \u0027connections\u0027, \u0027connections_reservations\u0027, \u0027deactivate\u0027, \u0027ensure_routing_table_is_fresh\u0027, \u0027fetch_routing_info\u0027, \u0027fetch_routing_table\u0027, \u0027get_auth\u0027, \u0027get_or_create_routing_table\u0027, \u0027get_routing_table\u0027, \u0027home_db_cache\u0027, \u0027in_use_connection_count\u0027, \u0027is_direct_pool\u0027, \u0027kill_and_release\u0027, \u0027lock\u0027, \u0027mark_all_stale\u0027, \u0027on_neo4j_error\u0027, \u0027on_write_failure\u0027, \u0027open\u0027, \u0027opener\u0027, \u0027pool_config\u0027, \u0027refresh_lock\u0027, \u0027release\u0027, \u0027routing_tables\u0027, \u0027ssr_enabled\u0027, \u0027update_connection_pool\u0027, \u0027update_routing_table\u0027, \u0027workspace_config\u0027]",
"method": "GET",
"path": "/health",
"service": "virtual-assistant",
"timestamp": "2026-07-27T00:07:24.116Z",
"version": "unknown"
}26s ago2026-07-27 00:07:24 UTC DEBUG virtual-assistant Driver attributes: ['_check_state', '_closed', '_default_workspace_config', '_get_server_info', '_initial_addresses', '_pool',...
time2026-07-27 00:07:24 UTC
severityDEBUG
servicevirtual-assistant
sourcesrc/neo4j_client.py:787
clustermaster
versionunknown
tracee2d4772e7991720190f4ce563ac98d70
span36364824ffac4f46
podvirtual-assistant-65b975d66b-l8wpw
message
Driver attributes: ['_check_state', '_closed', '_default_workspace_config', '_get_server_info', '_initial_addresses', '_pool', '_prepare_session_config', '_query_bookmark_manager', '_read_session_config', '_session', 'close', 'default_host', 'default_port', 'default_targets', 'encrypted', 'execute_query', 'execute_query_bookmark_manager', 'get_server_info', 'initial_addresses', 'open', 'parse_targets', 'session', 'supports_multi_db', 'supports_session_auth', 'verify_authentication', 'verify_connectivity']
raw payload (9 fields)
{
"caller": "src/neo4j_client.py:787",
"cluster": "master",
"environment": "DEVELOPMENT",
"message": "Driver attributes: [\u0027_check_state\u0027, \u0027_closed\u0027, \u0027_default_workspace_config\u0027, \u0027_get_server_info\u0027, \u0027_initial_addresses\u0027, \u0027_pool\u0027, \u0027_prepare_session_config\u0027, \u0027_query_bookmark_manager\u0027, \u0027_read_session_config\u0027, \u0027_session\u0027, \u0027close\u0027, \u0027default_host\u0027, \u0027default_port\u0027, \u0027default_targets\u0027, \u0027encrypted\u0027, \u0027execute_query\u0027, \u0027execute_query_bookmark_manager\u0027, \u0027get_server_info\u0027, \u0027initial_addresses\u0027, \u0027open\u0027, \u0027parse_targets\u0027, \u0027session\u0027, \u0027supports_multi_db\u0027, \u0027supports_session_auth\u0027, \u0027verify_authentication\u0027, \u0027verify_connectivity\u0027]",
"method": "GET",
"path": "/health",
"service": "virtual-assistant",
"timestamp": "2026-07-27T00:07:24.116Z",
"version": "unknown"
}26s ago2026-07-27 00:07:24 UTC DEBUG virtual-assistant 🔍 AUTH_DEBUG: Path in UNPROTECTED_ENDPOINTS: True
time2026-07-27 00:07:24 UTC
severityDEBUG
servicevirtual-assistant
sourceapi/api.py:491
clustermaster
versionunknown
tracee2d4772e7991720190f4ce563ac98d70
span36364824ffac4f46
podvirtual-assistant-65b975d66b-l8wpw
message
🔍 AUTH_DEBUG: Path in UNPROTECTED_ENDPOINTS: True
raw payload (9 fields)
{
"caller": "api/api.py:491",
"cluster": "master",
"environment": "DEVELOPMENT",
"message": "\ud83d\udd0d AUTH_DEBUG: Path in UNPROTECTED_ENDPOINTS: True",
"method": "GET",
"path": "/health",
"service": "virtual-assistant",
"timestamp": "2026-07-27T00:07:24.112Z",
"version": "unknown"
}26s ago2026-07-27 00:07:24 UTC DEBUG virtual-assistant 🔍 AUTH_DEBUG: Method: GET
time2026-07-27 00:07:24 UTC
severityDEBUG
servicevirtual-assistant
sourceapi/api.py:489
clustermaster
versionunknown
tracee2d4772e7991720190f4ce563ac98d70
span36364824ffac4f46
podvirtual-assistant-65b975d66b-l8wpw
message
🔍 AUTH_DEBUG: Method: GET
raw payload (9 fields)
{
"caller": "api/api.py:489",
"cluster": "master",
"environment": "DEVELOPMENT",
"message": "\ud83d\udd0d AUTH_DEBUG: Method: GET",
"method": "GET",
"path": "/health",
"service": "virtual-assistant",
"timestamp": "2026-07-27T00:07:24.111Z",
"version": "unknown"
}27s ago2026-07-27 00:07:23 UTC DEBUG docproc Pull returned 0 messages
time2026-07-27 00:07:23 UTC
severityDEBUG
servicedocproc
sourcehandlers/pubsub_handler.py:284
clustermaster
version0.1.0
poddocproc-7bc76669-ljh8h
message
Pull returned 0 messages
raw payload (9 fields)
{
"caller": "handlers/pubsub_handler.py:284",
"cluster": "master",
"component": "pubsub",
"environment": "DEVELOPMENT",
"message": "Pull returned 0 messages",
"service": "docproc",
"subscription_name": "tilda-document-master-docproc-sub",
"timestamp": "2026-07-27T00:07:23.808Z",
"version": "0.1.0"
}27s ago2026-07-27 00:07:23 UTC DEBUG docproc Recovery fill
time2026-07-27 00:07:23 UTC
severityDEBUG
servicedocproc
sourcehandlers/pubsub_handler.py:429
clustermaster
version0.1.0
poddocproc-7bc76669-ljh8h
message
Recovery fill
raw payload (13 fields)
{
"adopt_list_len": 0.0,
"adopt_started": 0.0,
"backlog_list_len": 0.0,
"backlog_started": 0.0,
"caller": "handlers/pubsub_handler.py:429",
"cluster": "master",
"component": "pubsub",
"environment": "DEVELOPMENT",
"message": "Recovery fill",
"remaining_slots": 10.0,
"service": "docproc",
"timestamp": "2026-07-27T00:07:23.808Z",
"version": "0.1.0"
}27s ago2026-07-27 00:07:23 UTC neo4j docproc MATCH (o:Organization{orgType:"CRO"})-[:DOC_TO_ORG]-(d:Document)-[:INCLUDES]-(f:Folder) WHERE f.name IN ['Staging', 'Pre-Staging'] ...
time2026-07-27 00:07:23 UTC
severityINFO
servicedocproc
clustermaster
podserver-2-dev-0
query
MATCH (o:Organization{orgType:"CRO"})-[:DOC_TO_ORG]-(d:Document)-[:INCLUDES]-(f:Folder)
WHERE f.name IN ['Staging', 'Pre-Staging']
AND d.delpending = false AND d.aiEnabled = true AND d.studyId <> ""
AND d.primaryVersionId IS NOT NULL AND d.primaryVersionId <> ""
OPTIONAL MATCH (d)<-[:AI_RUN_TO_DOCUMENT_LATEST]-(ai:AIRun)
WITH o, d, ai
ORDER BY d.tildaid, COALESCE(ai.updatedat, ai.createdat) DESC
WITH o, d, collect(ai)[0] AS latest_ai
WHERE (
latest_ai IS NULL
OR latest_ai.status = "aborted"
OR (
latest_ai.status = "failed"
AND COALESCE(latest_ai.auto_retry_count, 0) < $max_retries
AND COALESCE(latest_ai.updatedat, latest_ai.createdat) < $threshold_ms
)
)
WITH d.tildaid AS doc_id, o.alias AS org_alias, d.primaryVersionId AS version_id,
d.studyId AS study_id, COALESCE(latest_ai.auto_retry_count, 0) AS current_retry_count
RETURN doc_id, org_alias, version_id, study_id, current_retry_count
ORDER BY doc_id
LIMIT $limit
raw payload (15 fields)
{
"allocatedBytes": 9610664.0,
"cpu": 163.0,
"elapsedTimeMs": 163.0,
"executionPlanCacheKeyHash": "7644B06C",
"id": "24641293",
"pageFaults": 0.0,
"pageHits": 156362.0,
"planning": 0.0,
"query": "\n MATCH (o:Organization{orgType:\"CRO\"})-[:DOC_TO_ORG]-(d:Document)-[:INCLUDES]-(f:Folder)\n WHERE f.name IN [\u0027Staging\u0027, \u0027Pre-Staging\u0027]\n AND d.delpending = false AND d.aiEnabled = true AND d.studyId \u003c\u003e \"\"\n AND d.primaryVersionId IS NOT NULL AND d.primaryVersionId \u003c\u003e \"\"\n OPTIONAL MATCH (d)\u003c-[:AI_RUN_TO_DOCUMENT_LATEST]-(ai:AIRun)\n WITH o, d, ai\n ORDER BY d.tildaid, COALESCE(ai.updatedat, ai.createdat) DESC\n WITH o, d, collect(ai)[0] AS latest_ai\n WHERE (\n latest_ai IS NULL\n OR latest_ai.status = \"aborted\"\n OR (\n latest_ai.status = \"failed\"\n AND COALESCE(latest_ai.auto_retry_count, 0) \u003c $max_retries\n AND COALESCE(latest_ai.updatedat, latest_ai.createdat) \u003c $threshold_ms\n )\n )\n WITH d.tildaid AS doc_id, o.alias AS org_alias, d.primaryVersionId AS version_id,\n d.studyId AS study_id, COALESCE(latest_ai.auto_retry_count, 0) AS current_retry_count\n RETURN doc_id, org_alias, version_id, study_id, current_retry_count\n ORDER BY doc_id\n LIMIT $limit\n ",
"queryParameters": "{limit: 10, max_retries: 5, threshold_ms: 1785109943640}",
"runtime": "pipelined",
"source": "bolt-session\tbolt\tneo4j-python/6.2.0 Python/3.12.13-final-0 (linux)\t\tclient/10.129.3.15:59818\tserver/10.173.0.8:7687\u003e",
"timestamp": "2026-07-27T00:07:23.806Z",
"transactionId": 12025346.0,
"waiting": 0.0
}27s ago2026-07-27 00:07:23 UTC neo4j docproc MATCH (ai:AIRun)-[:AI_RUN_TO_DOCUMENT_LATEST]->(d:Document) WHERE d.aiEnabled = true AND ( (ai.status = "in_progress" AND...
time2026-07-27 00:07:23 UTC
severityINFO
servicedocproc
clustermaster
podserver-2-dev-0
query
MATCH (ai:AIRun)-[:AI_RUN_TO_DOCUMENT_LATEST]->(d:Document)
WHERE d.aiEnabled = true
AND (
(ai.status = "in_progress" AND ($cutoff_ms <= 0 OR COALESCE(ai.updatedat, ai.createdat) < $cutoff_ms))
OR ai.status = "aborted"
)
MATCH (d)-[:DOC_TO_ORG]->(o:Organization)
RETURN d.tildaid AS document_id, d.primaryVersionId AS version_id,
ai.tildaid AS ai_run_id, d.studyId AS study_id, ai.run_type AS run_type,
o.tildaid AS org_id, o.alias AS org_alias
ORDER BY COALESCE(ai.updatedat, ai.createdat) ASC
LIMIT $limit
raw payload (15 fields)
{
"allocatedBytes": 736.0,
"cpu": 88.0,
"elapsedTimeMs": 88.0,
"executionPlanCacheKeyHash": "E5FA14C6",
"id": "24641292",
"pageFaults": 0.0,
"pageHits": 89927.0,
"planning": 0.0,
"query": "\n MATCH (ai:AIRun)-[:AI_RUN_TO_DOCUMENT_LATEST]-\u003e(d:Document)\n WHERE d.aiEnabled = true\n AND (\n (ai.status = \"in_progress\" AND ($cutoff_ms \u003c= 0 OR COALESCE(ai.updatedat, ai.createdat) \u003c $cutoff_ms))\n OR ai.status = \"aborted\"\n )\n MATCH (d)-[:DOC_TO_ORG]-\u003e(o:Organization)\n RETURN d.tildaid AS document_id, d.primaryVersionId AS version_id,\n ai.tildaid AS ai_run_id, d.studyId AS study_id, ai.run_type AS run_type,\n o.tildaid AS org_id, o.alias AS org_alias\n ORDER BY COALESCE(ai.updatedat, ai.createdat) ASC\n LIMIT $limit\n ",
"queryParameters": "{limit: 30, cutoff_ms: 1785110243547}",
"runtime": "pipelined",
"source": "bolt-session\tbolt\tneo4j-python/6.2.0 Python/3.12.13-final-0 (linux)\t\tclient/10.129.3.15:59818\tserver/10.173.0.8:7687\u003e",
"timestamp": "2026-07-27T00:07:23.638Z",
"transactionId": 12025345.0,
"waiting": 0.0
}27s ago2026-07-27 00:07:23 UTC ERROR observability: level watcher could not stat config file /etc/config/observability.yaml: stat /etc/config/observability.yaml: no such file or directory
filter by:
severity≥ERROR
time2026-07-27 00:07:23 UTC
severityERROR
clustermaster
podswaggersvc-86f5455f58-zqm46
message
observability: level watcher could not stat config file /etc/config/observability.yaml: stat /etc/config/observability.yaml: no such file or directory
29s ago2026-07-27 00:07:21 UTC neo4j advpsvc match(a:AdvRateRule{deleted:true})-[r]-(b) with distinct a return a.portalorgid, a.tildaid, a.ruletype limit 10
time2026-07-27 00:07:21 UTC
severityINFO
serviceadvpsvc
clustermaster
podserver-1-dev-0
query
match(a:AdvRateRule{deleted:true})-[r]-(b)
with distinct a
return a.portalorgid, a.tildaid, a.ruletype
limit 10
raw payload (15 fields)
{
"allocatedBytes": 328.0,
"cpu": 2.0,
"elapsedTimeMs": 2.0,
"executionPlanCacheKeyHash": "AFAFBFFB",
"id": "17372810",
"pageFaults": 0.0,
"pageHits": 1146.0,
"planning": 0.0,
"query": "match(a:AdvRateRule{deleted:true})-[r]-(b) with distinct a return a.portalorgid, a.tildaid, a.ruletype limit 10",
"queryParameters": "{}",
"runtime": "pipelined",
"source": "bolt-session\tbolt\tGo Driver/5.20.0\t\tclient/10.129.3.19:37624\tserver/10.173.2.71:7687\u003e",
"timestamp": "2026-07-27T00:07:21.955Z",
"transactionId": 1771016.0,
"waiting": 0.0
}30s ago2026-07-27 00:07:20 UTC DEBUG virtual-assistant [mem] background_reclaimer executed
time2026-07-27 00:07:20 UTC
severityDEBUG
servicevirtual-assistant
sourcesrc/main.py:540
clustermaster
versionunknown
podvirtual-assistant-65b975d66b-l8wpw
message
[mem] background_reclaimer executed
raw payload (7 fields)
{
"caller": "src/main.py:540",
"cluster": "master",
"environment": "DEVELOPMENT",
"message": "[mem] background_reclaimer executed",
"service": "virtual-assistant",
"timestamp": "2026-07-27T00:07:20.751Z",
"version": "unknown"
}30s ago2026-07-27 00:07:20 UTC DEBUG virtual-assistant aggressive_memory_release: GC skipped based on should_run_gc() decision
time2026-07-27 00:07:20 UTC
severityDEBUG
servicevirtual-assistant
sourcecommon/mem_release.py:79
clustermaster
versionunknown
podvirtual-assistant-65b975d66b-l8wpw
message
aggressive_memory_release: GC skipped based on should_run_gc() decision
raw payload (7 fields)
{
"caller": "common/mem_release.py:79",
"cluster": "master",
"environment": "DEVELOPMENT",
"message": "aggressive_memory_release: GC skipped based on should_run_gc() decision",
"service": "virtual-assistant",
"timestamp": "2026-07-27T00:07:20.751Z",
"version": "unknown"
}30s ago2026-07-27 00:07:20 UTC DEBUG virtual-assistant should_run_gc: mem usage: 22.9% (458.4MB/2000.0MB), GC gen0 counts=2
time2026-07-27 00:07:20 UTC
severityDEBUG
servicevirtual-assistant
sourcereg/memory_monitor.py:464
clustermaster
versionunknown
podvirtual-assistant-65b975d66b-l8wpw
message
should_run_gc: mem usage: 22.9% (458.4MB/2000.0MB), GC gen0 counts=2
raw payload (7 fields)
{
"caller": "reg/memory_monitor.py:464",
"cluster": "master",
"environment": "DEVELOPMENT",
"message": "should_run_gc: mem usage: 22.9% (458.4MB/2000.0MB), GC gen0 counts=2",
"service": "virtual-assistant",
"timestamp": "2026-07-27T00:07:20.751Z",
"version": "unknown"
}30s ago2026-07-27 00:07:20 UTC INFO
filter by:
severity≥INFO
time2026-07-27 00:07:20 UTC
severityINFO
clustermaster
podrag-agent-57879dd94f-l9nz7
30s ago2026-07-27 00:07:20 UTC INFO
filter by:
severity≥INFO
time2026-07-27 00:07:20 UTC
severityINFO
clustermaster
podlitellm-f54655c99-tt2pg
30s ago2026-07-27 00:07:20 UTC ERROR observability: level watcher could not stat config file /etc/config/observability.yaml: stat /etc/config/observability.yaml: no such file or directory
filter by:
severity≥ERROR
time2026-07-27 00:07:20 UTC
severityERROR
clustermaster
poddashql-77f7d55ffc-97fsq
message
observability: level watcher could not stat config file /etc/config/observability.yaml: stat /etc/config/observability.yaml: no such file or directory
30s ago2026-07-27 00:07:20 UTC ERROR observability: level watcher could not stat config file /etc/config/observability.yaml: stat /etc/config/observability.yaml: no such file or directory
filter by:
severity≥ERROR
time2026-07-27 00:07:20 UTC
severityERROR
clustermaster
podsynthea-8b797c55c-79hm4
message
observability: level watcher could not stat config file /etc/config/observability.yaml: stat /etc/config/observability.yaml: no such file or directory
31s ago2026-07-27 00:07:19 UTC INFO
time2026-07-27 00:07:19 UTC
severityINFO
clustermaster
tracea5adfa70c0f43a0692f43846f490dc75
spanc30eda3b3b2c132f
podrag-agent-57879dd94f-l9nz7
31s ago2026-07-27 00:07:19 UTC INFO
filter by:
severity≥INFO
time2026-07-27 00:07:19 UTC
severityINFO
clustermaster
podlitellm-f54655c99-cbmvt
31s ago2026-07-27 00:07:19 UTC ERROR observability: level watcher could not stat config file /etc/config/observability.yaml: stat /etc/config/observability.yaml: no such file or directory
filter by:
severity≥ERROR
time2026-07-27 00:07:19 UTC
severityERROR
clustermaster
poddocsvc-5698bb997f-pc9ql
message
observability: level watcher could not stat config file /etc/config/observability.yaml: stat /etc/config/observability.yaml: no such file or directory
32s ago2026-07-27 00:07:18 UTC DEBUG workplansvc TransactionID commit
time2026-07-27 00:07:18 UTC
severityDEBUG
serviceworkplansvc
sourcetildadb/api.go:2575
clustermaster
versionunknown
podworkplansvc-75f7d8dcb6-jxnsg
message
TransactionID commit
raw payload (8 fields)
{
"caller": "tildadb/api.go:2575",
"cluster": "master",
"environment": "DEVELOPMENT",
"message": "TransactionID commit",
"service": "workplansvc",
"timestamp": "2026-07-27T00:07:18.682Z",
"transactionID": "FB:kcwQc8qrjECqRtOxBB1HjVmAmsoAPpdlkA==",
"version": "unknown"
}32s ago2026-07-27 00:07:18 UTC DEBUG workplansvc TransactionID commit
time2026-07-27 00:07:18 UTC
severityDEBUG
serviceworkplansvc
sourcetildadb/api.go:2575
clustermaster
versionunknown
podworkplansvc-75f7d8dcb6-jxnsg
message
TransactionID commit
raw payload (8 fields)
{
"caller": "tildadb/api.go:2575",
"cluster": "master",
"environment": "DEVELOPMENT",
"message": "TransactionID commit",
"service": "workplansvc",
"timestamp": "2026-07-27T00:07:18.672Z",
"transactionID": "FB:kcwQc8qrjECqRtOxBB1HjVmAmsoAPpdkkA==",
"version": "unknown"
}32s ago2026-07-27 00:07:18 UTC neo4j workplansvc CREATE (a:WorkplanInstanceEvent{deltimestamp:0}) SET a.createdat = timestamp(), a.updatedat = timestamp(), a.tildaid = randomUUID(),...
time2026-07-27 00:07:18 UTC
severityINFO
serviceworkplansvc
clustermaster
podserver-2-dev-0
query
CREATE (a:WorkplanInstanceEvent{deltimestamp:0})
SET a.createdat = timestamp(), a.updatedat = timestamp(), a.tildaid = randomUUID(), a.holdbitmap=0,a.holdbitmap=0,a.delpending=false,a.instanceId="4f4a2dc3-117b-4b69-a1c1-8944eb009099",a.orgId="a5fb9821-522a-4061-823f-b0a66173a204",a.eventType="sweeper_send_blocked",a.requestId="",a.summary="Sweeper skipped send: 1 unlinked work order(s)",a.payloadJson='{"reason":"unlinked_work_orders","unlinkedWorkOrderCount":1}'
RETURN a.tildaid, a.createdat
raw payload (15 fields)
{
"allocatedBytes": 328.0,
"cpu": 0.0,
"elapsedTimeMs": 0.0,
"executionPlanCacheKeyHash": "C23BE7B4",
"id": "24641285",
"pageFaults": 0.0,
"pageHits": 0.0,
"planning": 0.0,
"query": "CREATE (a:WorkplanInstanceEvent{deltimestamp:0}) SET a.createdat = timestamp(), a.updatedat = timestamp(), a.tildaid = randomUUID(), a.holdbitmap=0,a.holdbitmap=0,a.delpending=false,a.instanceId=\"4f4a2dc3-117b-4b69-a1c1-8944eb009099\",a.orgId=\"a5fb9821-522a-4061-823f-b0a66173a204\",a.eventType=\"sweeper_send_blocked\",a.requestId=\"\",a.summary=\"Sweeper skipped send: 1 unlinked work order(s)\",a.payloadJson=\u0027{\"reason\":\"unlinked_work_orders\",\"unlinkedWorkOrderCount\":1}\u0027 RETURN a.tildaid, a.createdat",
"queryParameters": "{}",
"runtime": "pipelined",
"source": "bolt-session\tbolt\tGo Driver/5.20.0\t\tclient/10.129.4.159:55948\tserver/10.173.0.8:7687\u003e",
"timestamp": "2026-07-27T00:07:18.672Z",
"transactionId": 12025342.0,
"waiting": 0.0
}32s ago2026-07-27 00:07:18 UTC neo4j workplansvc MATCH (org:Organization)-[:ORGANIZATION_TO_WORKPLAN_INSTANCE]->(wp:WorkPlanInstance{tildaid:"4f4a2dc3-117b-4b69-a1c1-8944eb009099"}) RETURN...
time2026-07-27 00:07:18 UTC
severityINFO
serviceworkplansvc
clustermaster
podserver-2-dev-0
query
MATCH (org:Organization)-[:ORGANIZATION_TO_WORKPLAN_INSTANCE]->(wp:WorkPlanInstance{tildaid:"4f4a2dc3-117b-4b69-a1c1-8944eb009099"})
RETURN org.tildaid as oid
raw payload (15 fields)
{
"allocatedBytes": 328.0,
"cpu": 1.0,
"elapsedTimeMs": 1.0,
"executionPlanCacheKeyHash": "CB2C51F3",
"id": "24641284",
"pageFaults": 0.0,
"pageHits": 414.0,
"planning": 0.0,
"query": "\n\t\tMATCH (org:Organization)-[:ORGANIZATION_TO_WORKPLAN_INSTANCE]-\u003e(wp:WorkPlanInstance{tildaid:\"4f4a2dc3-117b-4b69-a1c1-8944eb009099\"})\n\t\tRETURN org.tildaid as oid\n\t",
"queryParameters": "{}",
"runtime": "pipelined",
"source": "bolt-session\tbolt\tGo Driver/5.20.0\t\tclient/10.129.4.159:55948\tserver/10.173.0.8:7687\u003e",
"timestamp": "2026-07-27T00:07:18.669Z",
"transactionId": 12025342.0,
"waiting": 0.0
}32s ago2026-07-27 00:07:18 UTC DEBUG workplansvc TransactionID
time2026-07-27 00:07:18 UTC
severityDEBUG
serviceworkplansvc
sourcetildadb/api.go:2482
clustermaster
versionunknown
podworkplansvc-75f7d8dcb6-jxnsg
message
TransactionID
raw payload (8 fields)
{
"caller": "tildadb/api.go:2482",
"cluster": "master",
"environment": "DEVELOPMENT",
"message": "TransactionID",
"service": "workplansvc",
"timestamp": "2026-07-27T00:07:18.667Z",
"transactionID": "",
"version": "unknown"
}32s ago2026-07-27 00:07:18 UTC neo4j workplansvc CREATE (a:WorkplanInstanceEvent{deltimestamp:0}) SET a.createdat = timestamp(), a.updatedat = timestamp(), a.tildaid = randomUUID(),...
time2026-07-27 00:07:18 UTC
severityINFO
serviceworkplansvc
clustermaster
podserver-2-dev-0
query
CREATE (a:WorkplanInstanceEvent{deltimestamp:0})
SET a.createdat = timestamp(), a.updatedat = timestamp(), a.tildaid = randomUUID(), a.holdbitmap=0,a.holdbitmap=0,a.delpending=false,a.instanceId="8cb2d38c-046c-4910-bb36-a02d9b856938",a.orgId="aa59151d-6c89-4c13-811b-0dffb1ed1693",a.eventType="sweeper_send_blocked",a.requestId="",a.summary="Sweeper skipped send: 1 unlinked work order(s)",a.payloadJson='{"reason":"unlinked_work_orders","unlinkedWorkOrderCount":1}'
RETURN a.tildaid, a.createdat
raw payload (15 fields)
{
"allocatedBytes": 328.0,
"cpu": 0.0,
"elapsedTimeMs": 1.0,
"executionPlanCacheKeyHash": "C23BE7B4",
"id": "24641283",
"pageFaults": 0.0,
"pageHits": 0.0,
"planning": 0.0,
"query": "CREATE (a:WorkplanInstanceEvent{deltimestamp:0}) SET a.createdat = timestamp(), a.updatedat = timestamp(), a.tildaid = randomUUID(), a.holdbitmap=0,a.holdbitmap=0,a.delpending=false,a.instanceId=\"8cb2d38c-046c-4910-bb36-a02d9b856938\",a.orgId=\"aa59151d-6c89-4c13-811b-0dffb1ed1693\",a.eventType=\"sweeper_send_blocked\",a.requestId=\"\",a.summary=\"Sweeper skipped send: 1 unlinked work order(s)\",a.payloadJson=\u0027{\"reason\":\"unlinked_work_orders\",\"unlinkedWorkOrderCount\":1}\u0027 RETURN a.tildaid, a.createdat",
"queryParameters": "{}",
"runtime": "pipelined",
"source": "bolt-session\tbolt\tneo4j-java/5.28.13-24cd0fecc64de123fd5711190246e8b0f1e16fd0\t\tclient/10.173.0.201:39990\tserver/10.173.0.8:7688\u003e",
"timestamp": "2026-07-27T00:07:18.652Z",
"transactionId": 12025341.0,
"waiting": 0.0
}32s ago2026-07-27 00:07:18 UTC neo4j workplansvc MATCH (org:Organization)-[:ORGANIZATION_TO_WORKPLAN_INSTANCE]->(wp:WorkPlanInstance{tildaid:"8cb2d38c-046c-4910-bb36-a02d9b856938"}) RETURN...
time2026-07-27 00:07:18 UTC
severityINFO
serviceworkplansvc
clustermaster
podserver-2-dev-0
query
MATCH (org:Organization)-[:ORGANIZATION_TO_WORKPLAN_INSTANCE]->(wp:WorkPlanInstance{tildaid:"8cb2d38c-046c-4910-bb36-a02d9b856938"})
RETURN org.tildaid as oid
raw payload (15 fields)
{
"allocatedBytes": 328.0,
"cpu": 1.0,
"elapsedTimeMs": 1.0,
"executionPlanCacheKeyHash": "CB2C51F3",
"id": "24641282",
"pageFaults": 0.0,
"pageHits": 432.0,
"planning": 0.0,
"query": "\n\t\tMATCH (org:Organization)-[:ORGANIZATION_TO_WORKPLAN_INSTANCE]-\u003e(wp:WorkPlanInstance{tildaid:\"8cb2d38c-046c-4910-bb36-a02d9b856938\"})\n\t\tRETURN org.tildaid as oid\n\t",
"queryParameters": "{}",
"runtime": "pipelined",
"source": "bolt-session\tbolt\tneo4j-java/5.28.13-24cd0fecc64de123fd5711190246e8b0f1e16fd0\t\tclient/10.173.0.201:39990\tserver/10.173.0.8:7688\u003e",
"timestamp": "2026-07-27T00:07:18.639Z",
"transactionId": 12025341.0,
"waiting": 0.0
}32s ago2026-07-27 00:07:18 UTC DEBUG workplansvc TransactionID
time2026-07-27 00:07:18 UTC
severityDEBUG
serviceworkplansvc
sourcetildadb/api.go:2482
clustermaster
versionunknown
podworkplansvc-75f7d8dcb6-jxnsg
message
TransactionID
raw payload (8 fields)
{
"caller": "tildadb/api.go:2482",
"cluster": "master",
"environment": "DEVELOPMENT",
"message": "TransactionID",
"service": "workplansvc",
"timestamp": "2026-07-27T00:07:18.632Z",
"transactionID": "",
"version": "unknown"
}32s ago2026-07-27 00:07:18 UTC DEBUG workplansvc workplan periodic sweep
time2026-07-27 00:07:18 UTC
severityDEBUG
serviceworkplansvc
sourceworkplan/summarization_sweep_ticker.go:90
clustermaster
versionunknown
podworkplansvc-75f7d8dcb6-jxnsg
message
workplan periodic sweep
raw payload (15 fields)
{
"activates_published": 0.0,
"autosend_catchups": 0.0,
"caller": "workplan/summarization_sweep_ticker.go:90",
"cluster": "master",
"environment": "DEVELOPMENT",
"message": "workplan periodic sweep",
"progress_summarized": 0.0,
"ready_det_repaired_desync": 0.0,
"ready_det_send_published": 0.0,
"ready_det_skipped_blocked": 2.0,
"recipient_retries": 0.0,
"service": "workplansvc",
"timestamp": "2026-07-27T00:07:18.629Z",
"trigger": "tick",
"version": "unknown"
}32s ago2026-07-27 00:07:18 UTC DEBUG workplansvc ready determination sweep: skip pending (unlinked work orders)
time2026-07-27 00:07:18 UTC
severityDEBUG
serviceworkplansvc
sourcepubsub/instance_ready_determination.go:352
clustermaster
versionunknown
podworkplansvc-75f7d8dcb6-jxnsg
message
ready determination sweep: skip pending (unlinked work orders)
raw payload (9 fields)
{
"caller": "pubsub/instance_ready_determination.go:352",
"cluster": "master",
"environment": "DEVELOPMENT",
"instance_id": "8cb2d38c-046c-4910-bb36-a02d9b856938",
"message": "ready determination sweep: skip pending (unlinked work orders)",
"service": "workplansvc",
"timestamp": "2026-07-27T00:07:18.629Z",
"unlinked_work_order_count": 1.0,
"version": "unknown"
}32s ago2026-07-27 00:07:18 UTC neo4j workplansvc MATCH...
time2026-07-27 00:07:18 UTC
severityINFO
serviceworkplansvc
clustermaster
podserver-3-dev-0
query
MATCH (wp:WorkPlanInstance{tildaid:"8cb2d38c-046c-4910-bb36-a02d9b856938"})-[:WORKPLAN_INSTANCE_TO_MILESTONE]->(:WorkPlanMilestone)-[:REL_WORKPLAN_INSTANCE_MILESTONE_TO_ITEM]->(item:WorkplanInstanceMilestoneItem)
WHERE item.itemType = "Workorder"
AND coalesce(item.skippedAt, 0) = 0
AND coalesce(item.notApplicableAt, 0) = 0
AND coalesce(item.workOrderCompletedAt, 0) = 0
AND coalesce(item.workOrderId, "") = ""
RETURN count(item) AS n
raw payload (15 fields)
{
"allocatedBytes": 488.0,
"cpu": 1.0,
"elapsedTimeMs": 1.0,
"executionPlanCacheKeyHash": "0CD31179",
"id": "13356372",
"pageFaults": 0.0,
"pageHits": 420.0,
"planning": 0.0,
"query": "\n\t\tMATCH (wp:WorkPlanInstance{tildaid:\"8cb2d38c-046c-4910-bb36-a02d9b856938\"})-[:WORKPLAN_INSTANCE_TO_MILESTONE]-\u003e(:WorkPlanMilestone)-[:REL_WORKPLAN_INSTANCE_MILESTONE_TO_ITEM]-\u003e(item:WorkplanInstanceMilestoneItem)\n\t\tWHERE item.itemType = \"Workorder\"\n\t\t AND coalesce(item.skippedAt, 0) = 0\n\t\t AND coalesce(item.notApplicableAt, 0) = 0\n\t\t AND coalesce(item.workOrderCompletedAt, 0) = 0\n\t\t AND coalesce(item.workOrderId, \"\") = \"\"\n\t\tRETURN count(item) AS n\n\t",
"queryParameters": "{}",
"runtime": "pipelined",
"source": "bolt-session\tbolt\tGo Driver/5.20.0\t\tclient/10.129.4.159:37136\tserver/10.173.0.201:7687\u003e",
"timestamp": "2026-07-27T00:07:18.626Z",
"transactionId": 1750733.0,
"waiting": 0.0
}32s ago2026-07-27 00:07:18 UTC DEBUG workplansvc ready determination sweep: skip pending (unlinked work orders)
time2026-07-27 00:07:18 UTC
severityDEBUG
serviceworkplansvc
sourcepubsub/instance_ready_determination.go:352
clustermaster
versionunknown
podworkplansvc-75f7d8dcb6-jxnsg
message
ready determination sweep: skip pending (unlinked work orders)
raw payload (9 fields)
{
"caller": "pubsub/instance_ready_determination.go:352",
"cluster": "master",
"environment": "DEVELOPMENT",
"instance_id": "4f4a2dc3-117b-4b69-a1c1-8944eb009099",
"message": "ready determination sweep: skip pending (unlinked work orders)",
"service": "workplansvc",
"timestamp": "2026-07-27T00:07:18.622Z",
"unlinked_work_order_count": 1.0,
"version": "unknown"
}32s ago2026-07-27 00:07:18 UTC neo4j workplansvc MATCH...
time2026-07-27 00:07:18 UTC
severityINFO
serviceworkplansvc
clustermaster
podserver-3-dev-0
query
MATCH (wp:WorkPlanInstance{tildaid:"4f4a2dc3-117b-4b69-a1c1-8944eb009099"})-[:WORKPLAN_INSTANCE_TO_MILESTONE]->(:WorkPlanMilestone)-[:REL_WORKPLAN_INSTANCE_MILESTONE_TO_ITEM]->(item:WorkplanInstanceMilestoneItem)
WHERE item.itemType = "Workorder"
AND coalesce(item.skippedAt, 0) = 0
AND coalesce(item.notApplicableAt, 0) = 0
AND coalesce(item.workOrderCompletedAt, 0) = 0
AND coalesce(item.workOrderId, "") = ""
RETURN count(item) AS n
raw payload (15 fields)
{
"allocatedBytes": 488.0,
"cpu": 10.0,
"elapsedTimeMs": 10.0,
"executionPlanCacheKeyHash": "0CD31179",
"id": "13356371",
"pageFaults": 0.0,
"pageHits": 452.0,
"planning": 9.0,
"query": "\n\t\tMATCH (wp:WorkPlanInstance{tildaid:\"4f4a2dc3-117b-4b69-a1c1-8944eb009099\"})-[:WORKPLAN_INSTANCE_TO_MILESTONE]-\u003e(:WorkPlanMilestone)-[:REL_WORKPLAN_INSTANCE_MILESTONE_TO_ITEM]-\u003e(item:WorkplanInstanceMilestoneItem)\n\t\tWHERE item.itemType = \"Workorder\"\n\t\t AND coalesce(item.skippedAt, 0) = 0\n\t\t AND coalesce(item.notApplicableAt, 0) = 0\n\t\t AND coalesce(item.workOrderCompletedAt, 0) = 0\n\t\t AND coalesce(item.workOrderId, \"\") = \"\"\n\t\tRETURN count(item) AS n\n\t",
"queryParameters": "{}",
"runtime": "pipelined",
"source": "bolt-session\tbolt\tGo Driver/5.20.0\t\tclient/10.129.4.159:37136\tserver/10.173.0.201:7687\u003e",
"timestamp": "2026-07-27T00:07:18.620Z",
"transactionId": 1750732.0,
"waiting": 0.0
}32s ago2026-07-27 00:07:18 UTC ERROR observability: level watcher could not stat config file /etc/config/observability.yaml: stat /etc/config/observability.yaml: no such file or directory
filter by:
severity≥ERROR
time2026-07-27 00:07:18 UTC
severityERROR
clustermaster
podenrollsvc-5fc97ccf8b-sc8q5
message
observability: level watcher could not stat config file /etc/config/observability.yaml: stat /etc/config/observability.yaml: no such file or directory
32s ago2026-07-27 00:07:18 UTC DEBUG workplansvc TransactionID commit
time2026-07-27 00:07:18 UTC
severityDEBUG
serviceworkplansvc
sourcetildadb/api.go:2575
clustermaster
versionunknown
podworkplansvc-75f7d8dcb6-jxnsg
message
TransactionID commit
raw payload (8 fields)
{
"caller": "tildadb/api.go:2575",
"cluster": "master",
"environment": "DEVELOPMENT",
"message": "TransactionID commit",
"service": "workplansvc",
"timestamp": "2026-07-27T00:07:18.606Z",
"transactionID": "FB:kcwQc8qrjECqRtOxBB1HjVmAmsoAPpdjkA==",
"version": "unknown"
}32s ago2026-07-27 00:07:18 UTC neo4j workplansvc MATCH (wp:WorkPlanInstance) WHERE wp.instanceReadyDeterminationState = "pending" AND wp.state <> "archived" RETURN wp.tildaid AS...
time2026-07-27 00:07:18 UTC
severityINFO
serviceworkplansvc
clustermaster
podserver-2-dev-0
query
MATCH (wp:WorkPlanInstance) WHERE wp.instanceReadyDeterminationState = "pending" AND wp.state <> "archived" RETURN wp.tildaid AS id ORDER BY coalesce(wp.instanceReadyDeterminationUpdatedAt, 0) ASC LIMIT 50
raw payload (15 fields)
{
"allocatedBytes": 832.0,
"cpu": 1.0,
"elapsedTimeMs": 1.0,
"executionPlanCacheKeyHash": "81C085B2",
"id": "24641281",
"pageFaults": 0.0,
"pageHits": 427.0,
"planning": 0.0,
"query": "\n\t\tMATCH (wp:WorkPlanInstance)\n\t\tWHERE wp.instanceReadyDeterminationState = \"pending\"\n\t\t AND wp.state \u003c\u003e \"archived\"\n\t\tRETURN wp.tildaid AS id\n\t\tORDER BY coalesce(wp.instanceReadyDeterminationUpdatedAt, 0) ASC\n\t\tLIMIT 50\n\t",
"queryParameters": "{}",
"runtime": "pipelined",
"source": "bolt-session\tbolt\tneo4j-java/5.28.13-24cd0fecc64de123fd5711190246e8b0f1e16fd0\t\tclient/10.173.0.201:39990\tserver/10.173.0.8:7688\u003e",
"timestamp": "2026-07-27T00:07:18.601Z",
"transactionId": 12025340.0,
"waiting": 0.0
}32s ago2026-07-27 00:07:18 UTC neo4j workplansvc MATCH (wp:WorkPlanInstance) WHERE wp.instanceReadyDeterminationState IN ["in_progress", "in_progress_and_pending"] AND wp.state <>...
time2026-07-27 00:07:18 UTC
severityINFO
serviceworkplansvc
clustermaster
podserver-2-dev-0
query
MATCH (wp:WorkPlanInstance) WHERE wp.instanceReadyDeterminationState IN ["in_progress", "in_progress_and_pending"] AND wp.state <> "archived" OPTIONAL MATCH (wp)<-[:WORKPLAN_JOB_TO_OBJ]-(job:WorkplanJob) WITH wp, count(job) AS jobN WHERE jobN = 0 RETURN wp.tildaid AS id LIMIT 50
raw payload (15 fields)
{
"allocatedBytes": 1888.0,
"cpu": 1.0,
"elapsedTimeMs": 1.0,
"executionPlanCacheKeyHash": "9B9F9D65",
"id": "24641280",
"pageFaults": 0.0,
"pageHits": 441.0,
"planning": 0.0,
"query": "\n\t\tMATCH (wp:WorkPlanInstance)\n\t\tWHERE wp.instanceReadyDeterminationState IN [\"in_progress\", \"in_progress_and_pending\"]\n\t\t AND wp.state \u003c\u003e \"archived\"\n\t\tOPTIONAL MATCH (wp)\u003c-[:WORKPLAN_JOB_TO_OBJ]-(job:WorkplanJob)\n\t\tWITH wp, count(job) AS jobN\n\t\tWHERE jobN = 0\n\t\tRETURN wp.tildaid AS id\n\t\tLIMIT 50\n\t",
"queryParameters": "{}",
"runtime": "pipelined",
"source": "bolt-session\tbolt\tneo4j-java/5.28.13-24cd0fecc64de123fd5711190246e8b0f1e16fd0\t\tclient/10.173.0.201:39990\tserver/10.173.0.8:7688\u003e",
"timestamp": "2026-07-27T00:07:18.590Z",
"transactionId": 12025340.0,
"waiting": 0.0
}32s ago2026-07-27 00:07:18 UTC DEBUG workplansvc TransactionID
time2026-07-27 00:07:18 UTC
severityDEBUG
serviceworkplansvc
sourcetildadb/api.go:2482
clustermaster
versionunknown
podworkplansvc-75f7d8dcb6-jxnsg
message
TransactionID
raw payload (8 fields)
{
"caller": "tildadb/api.go:2482",
"cluster": "master",
"environment": "DEVELOPMENT",
"message": "TransactionID",
"service": "workplansvc",
"timestamp": "2026-07-27T00:07:18.579Z",
"transactionID": "",
"version": "unknown"
}32s ago2026-07-27 00:07:18 UTC neo4j workplansvc UNWIND $site_ids AS sid OPTIONAL MATCH (s:Site {tildaid: sid, delpending: false}) OPTIONAL MATCH...
time2026-07-27 00:07:18 UTC
severityINFO
serviceworkplansvc
clustermaster
podserver-3-dev-0
query
UNWIND $site_ids AS sid
OPTIONAL MATCH (s:Site {tildaid: sid, delpending: false})
OPTIONAL MATCH (s)<-[:PLAN_DEFINITION_TO_SITE]-(pd:DslPlanDefinition {delpending: false})-[:PLAN_DEFINITION_TO_STUDY]->(st:Study {tildaid: $study_id, delpending: false})
OPTIONAL MATCH (pd)-[pdc:PLAN_DEFINITION_TO_CONTACT]->(actor:Actor {delpending: false})-[:FOR_USER]->(u:User)
OPTIONAL MATCH (pd)-[copdc:PLAN_DEFINITION_TO_CLINOP_CONTACT]->(co:ClinOp {deltimestamp: 0})-[:FOR_USER]->(cou:User)
WITH sid, s,
collect(DISTINCT CASE
WHEN pdc IS NOT NULL AND actor IS NOT NULL AND (u IS NULL OR u.status <> $deleted_status) THEN {
t: pdc.type,
p: coalesce(pdc.isPrimary, false),
e: coalesce(u.email, "")
} END) AS actorContactRows,
collect(DISTINCT CASE
WHEN copdc IS NOT NULL AND co IS NOT NULL AND (cou IS NULL OR cou.status <> $deleted_status) THEN {
t: copdc.type,
p: coalesce(copdc.isPrimary, false),
e: coalesce(cou.email, "")
} END) AS clinopContactRows
WITH sid, s, actorContactRows + clinopContactRows AS pdContactRowsRaw
WITH sid, s,
[row IN pdContactRowsRaw WHERE row IS NOT NULL AND row.t IS NOT NULL AND row.e IS NOT NULL AND row.e <> ""] AS pd_rows
RETURN sid AS site_id, coalesce(s.name, "") AS site_name, pd_rows
raw payload (15 fields)
{
"allocatedBytes": 19512.0,
"cpu": 1.0,
"elapsedTimeMs": 1.0,
"executionPlanCacheKeyHash": "3832EC22",
"id": "13356368",
"pageFaults": 0.0,
"pageHits": 17.0,
"planning": 0.0,
"query": "\nUNWIND $site_ids AS sid\nOPTIONAL MATCH (s:Site {tildaid: sid, delpending: false})\nOPTIONAL MATCH (s)\u003c-[:PLAN_DEFINITION_TO_SITE]-(pd:DslPlanDefinition {delpending: false})-[:PLAN_DEFINITION_TO_STUDY]-\u003e(st:Study {tildaid: $study_id, delpending: false})\nOPTIONAL MATCH (pd)-[pdc:PLAN_DEFINITION_TO_CONTACT]-\u003e(actor:Actor {delpending: false})-[:FOR_USER]-\u003e(u:User)\nOPTIONAL MATCH (pd)-[copdc:PLAN_DEFINITION_TO_CLINOP_CONTACT]-\u003e(co:ClinOp {deltimestamp: 0})-[:FOR_USER]-\u003e(cou:User)\nWITH sid, s,\n collect(DISTINCT CASE\n WHEN pdc IS NOT NULL AND actor IS NOT NULL AND (u IS NULL OR u.status \u003c\u003e $deleted_status) THEN {\n t: pdc.type,\n p: coalesce(pdc.isPrimary, false),\n e: coalesce(u.email, \"\")\n } END) AS actorContactRows,\n collect(DISTINCT CASE\n WHEN copdc IS NOT NULL AND co IS NOT NULL AND (cou IS NULL OR cou.status \u003c\u003e $deleted_status) THEN {\n t: copdc.type,\n p: coalesce(copdc.isPrimary, false),\n e: coalesce(cou.email, \"\")\n } END) AS clinopContactRows\nWITH sid, s, actorContactRows + clinopContactRows AS pdContactRowsRaw\nWITH sid, s,\n [row IN pdContactRowsRaw WHERE row IS NOT NULL AND row.t IS NOT NULL AND row.e IS NOT NULL AND row.e \u003c\u003e \"\"] AS pd_rows\nRETURN sid AS site_id, coalesce(s.name, \"\") AS site_name, pd_rows\n",
"queryParameters": "{deleted_status: \u0027Deleted\u0027, site_ids: [\u00276ef91479-dbf6-4a3b-82b9-4c07b66b8eb4\u0027], study_id: \u002716ffa942-94ca-4e1c-88f7-e0d87180dfe8\u0027}",
"runtime": "pipelined",
"source": "bolt-session\tbolt\tGo Driver/5.20.0\t\tclient/10.129.4.159:37136\tserver/10.173.0.201:7687\u003e",
"timestamp": "2026-07-27T00:07:18.573Z",
"transactionId": 1750731.0,
"waiting": 0.0
}32s ago2026-07-27 00:07:18 UTC neo4j workplansvc MATCH (instance:WorkPlanInstance{tildaid:"a1492aa6-ba07-49a9-aa99-0758bbdd18d3"})-[:WORKPLAN_TO_SITE]->(site:Site) RETURN site.tildaid as siteId
time2026-07-27 00:07:18 UTC
severityINFO
serviceworkplansvc
clustermaster
podserver-3-dev-0
query
MATCH (instance:WorkPlanInstance{tildaid:"a1492aa6-ba07-49a9-aa99-0758bbdd18d3"})-[:WORKPLAN_TO_SITE]->(site:Site)
RETURN site.tildaid as siteId
raw payload (15 fields)
{
"allocatedBytes": 328.0,
"cpu": 5.0,
"elapsedTimeMs": 5.0,
"executionPlanCacheKeyHash": "9B774BFC",
"id": "13356367",
"pageFaults": 0.0,
"pageHits": 414.0,
"planning": 4.0,
"query": "\n\t\tMATCH (instance:WorkPlanInstance{tildaid:\"a1492aa6-ba07-49a9-aa99-0758bbdd18d3\"})-[:WORKPLAN_TO_SITE]-\u003e(site:Site)\n\t\tRETURN site.tildaid as siteId\n\t",
"queryParameters": "{}",
"runtime": "pipelined",
"source": "bolt-session\tbolt\tGo Driver/5.20.0\t\tclient/10.129.4.159:37136\tserver/10.173.0.201:7687\u003e",
"timestamp": "2026-07-27T00:07:18.567Z",
"transactionId": 1750730.0,
"waiting": 0.0
}32s ago2026-07-27 00:07:18 UTC neo4j workplansvc MATCH (cfg:WorkplanCfg{tildaid:"92f49070-1232-4531-bb13-73157a75b154"})-[:WORKPLAN_CFG_TO_STUDY]->(study:Study) RETURN study.tildaid as studyId
time2026-07-27 00:07:18 UTC
severityINFO
serviceworkplansvc
clustermaster
podserver-3-dev-0
query
MATCH (cfg:WorkplanCfg{tildaid:"92f49070-1232-4531-bb13-73157a75b154"})-[:WORKPLAN_CFG_TO_STUDY]->(study:Study)
RETURN study.tildaid as studyId
raw payload (15 fields)
{
"allocatedBytes": 328.0,
"cpu": 1.0,
"elapsedTimeMs": 1.0,
"executionPlanCacheKeyHash": "682A7B96",
"id": "13356366",
"pageFaults": 0.0,
"pageHits": 329.0,
"planning": 0.0,
"query": "\n\t\tMATCH (cfg:WorkplanCfg{tildaid:\"92f49070-1232-4531-bb13-73157a75b154\"})-[:WORKPLAN_CFG_TO_STUDY]-\u003e(study:Study)\n\t\tRETURN study.tildaid as studyId\n\t",
"queryParameters": "{}",
"runtime": "pipelined",
"source": "bolt-session\tbolt\tGo Driver/5.20.0\t\tclient/10.129.4.159:37136\tserver/10.173.0.201:7687\u003e",
"timestamp": "2026-07-27T00:07:18.556Z",
"transactionId": 1750729.0,
"waiting": 0.0
}32s ago2026-07-27 00:07:18 UTC neo4j workplansvc UNWIND $site_ids AS sid OPTIONAL MATCH (s:Site {tildaid: sid, delpending: false}) OPTIONAL MATCH...
time2026-07-27 00:07:18 UTC
severityINFO
serviceworkplansvc
clustermaster
podserver-3-dev-0
query
UNWIND $site_ids AS sid
OPTIONAL MATCH (s:Site {tildaid: sid, delpending: false})
OPTIONAL MATCH (s)<-[:PLAN_DEFINITION_TO_SITE]-(pd:DslPlanDefinition {delpending: false})-[:PLAN_DEFINITION_TO_STUDY]->(st:Study {tildaid: $study_id, delpending: false})
OPTIONAL MATCH (pd)-[pdc:PLAN_DEFINITION_TO_CONTACT]->(actor:Actor {delpending: false})-[:FOR_USER]->(u:User)
OPTIONAL MATCH (pd)-[copdc:PLAN_DEFINITION_TO_CLINOP_CONTACT]->(co:ClinOp {deltimestamp: 0})-[:FOR_USER]->(cou:User)
WITH sid, s,
collect(DISTINCT CASE
WHEN pdc IS NOT NULL AND actor IS NOT NULL AND (u IS NULL OR u.status <> $deleted_status) THEN {
t: pdc.type,
p: coalesce(pdc.isPrimary, false),
e: coalesce(u.email, "")
} END) AS actorContactRows,
collect(DISTINCT CASE
WHEN copdc IS NOT NULL AND co IS NOT NULL AND (cou IS NULL OR cou.status <> $deleted_status) THEN {
t: copdc.type,
p: coalesce(copdc.isPrimary, false),
e: coalesce(cou.email, "")
} END) AS clinopContactRows
WITH sid, s, actorContactRows + clinopContactRows AS pdContactRowsRaw
WITH sid, s,
[row IN pdContactRowsRaw WHERE row IS NOT NULL AND row.t IS NOT NULL AND row.e IS NOT NULL AND row.e <> ""] AS pd_rows
RETURN sid AS site_id, coalesce(s.name, "") AS site_name, pd_rows
raw payload (15 fields)
{
"allocatedBytes": 19512.0,
"cpu": 1.0,
"elapsedTimeMs": 1.0,
"executionPlanCacheKeyHash": "3832EC22",
"id": "13356365",
"pageFaults": 0.0,
"pageHits": 142.0,
"planning": 1.0,
"query": "\nUNWIND $site_ids AS sid\nOPTIONAL MATCH (s:Site {tildaid: sid, delpending: false})\nOPTIONAL MATCH (s)\u003c-[:PLAN_DEFINITION_TO_SITE]-(pd:DslPlanDefinition {delpending: false})-[:PLAN_DEFINITION_TO_STUDY]-\u003e(st:Study {tildaid: $study_id, delpending: false})\nOPTIONAL MATCH (pd)-[pdc:PLAN_DEFINITION_TO_CONTACT]-\u003e(actor:Actor {delpending: false})-[:FOR_USER]-\u003e(u:User)\nOPTIONAL MATCH (pd)-[copdc:PLAN_DEFINITION_TO_CLINOP_CONTACT]-\u003e(co:ClinOp {deltimestamp: 0})-[:FOR_USER]-\u003e(cou:User)\nWITH sid, s,\n collect(DISTINCT CASE\n WHEN pdc IS NOT NULL AND actor IS NOT NULL AND (u IS NULL OR u.status \u003c\u003e $deleted_status) THEN {\n t: pdc.type,\n p: coalesce(pdc.isPrimary, false),\n e: coalesce(u.email, \"\")\n } END) AS actorContactRows,\n collect(DISTINCT CASE\n WHEN copdc IS NOT NULL AND co IS NOT NULL AND (cou IS NULL OR cou.status \u003c\u003e $deleted_status) THEN {\n t: copdc.type,\n p: coalesce(copdc.isPrimary, false),\n e: coalesce(cou.email, \"\")\n } END) AS clinopContactRows\nWITH sid, s, actorContactRows + clinopContactRows AS pdContactRowsRaw\nWITH sid, s,\n [row IN pdContactRowsRaw WHERE row IS NOT NULL AND row.t IS NOT NULL AND row.e IS NOT NULL AND row.e \u003c\u003e \"\"] AS pd_rows\nRETURN sid AS site_id, coalesce(s.name, \"\") AS site_name, pd_rows\n",
"queryParameters": "{deleted_status: \u0027Deleted\u0027, site_ids: [\u0027f17cfa1f-97b5-4f9f-9b7b-a7d3cb0420a4\u0027], study_id: \u0027e47593bd-8d16-45f9-bda7-6875606eecc7\u0027}",
"runtime": "pipelined",
"source": "bolt-session\tbolt\tGo Driver/5.20.0\t\tclient/10.129.4.159:37136\tserver/10.173.0.201:7687\u003e",
"timestamp": "2026-07-27T00:07:18.550Z",
"transactionId": 1750728.0,
"waiting": 0.0
}32s ago2026-07-27 00:07:18 UTC neo4j workplansvc MATCH (instance:WorkPlanInstance{tildaid:"a947c13c-168f-4975-a246-376dd9402562"})-[:WORKPLAN_TO_SITE]->(site:Site) RETURN site.tildaid as siteId
time2026-07-27 00:07:18 UTC
severityINFO
serviceworkplansvc
clustermaster
podserver-3-dev-0
query
MATCH (instance:WorkPlanInstance{tildaid:"a947c13c-168f-4975-a246-376dd9402562"})-[:WORKPLAN_TO_SITE]->(site:Site)
RETURN site.tildaid as siteId
raw payload (15 fields)
{
"allocatedBytes": 328.0,
"cpu": 1.0,
"elapsedTimeMs": 1.0,
"executionPlanCacheKeyHash": "9B774BFC",
"id": "13356364",
"pageFaults": 0.0,
"pageHits": 414.0,
"planning": 0.0,
"query": "\n\t\tMATCH (instance:WorkPlanInstance{tildaid:\"a947c13c-168f-4975-a246-376dd9402562\"})-[:WORKPLAN_TO_SITE]-\u003e(site:Site)\n\t\tRETURN site.tildaid as siteId\n\t",
"queryParameters": "{}",
"runtime": "pipelined",
"source": "bolt-session\tbolt\tGo Driver/5.20.0\t\tclient/10.129.4.159:37136\tserver/10.173.0.201:7687\u003e",
"timestamp": "2026-07-27T00:07:18.543Z",
"transactionId": 1750727.0,
"waiting": 0.0
}32s ago2026-07-27 00:07:18 UTC neo4j workplansvc MATCH (cfg:WorkplanCfg{tildaid:"b63053ca-d6b2-47b2-9dbb-effaf732dd75"})-[:WORKPLAN_CFG_TO_STUDY]->(study:Study) RETURN study.tildaid as studyId
time2026-07-27 00:07:18 UTC
severityINFO
serviceworkplansvc
clustermaster
podserver-3-dev-0
query
MATCH (cfg:WorkplanCfg{tildaid:"b63053ca-d6b2-47b2-9dbb-effaf732dd75"})-[:WORKPLAN_CFG_TO_STUDY]->(study:Study)
RETURN study.tildaid as studyId
raw payload (15 fields)
{
"allocatedBytes": 328.0,
"cpu": 4.0,
"elapsedTimeMs": 5.0,
"executionPlanCacheKeyHash": "682A7B96",
"id": "13356363",
"pageFaults": 0.0,
"pageHits": 329.0,
"planning": 4.0,
"query": "\n\t\tMATCH (cfg:WorkplanCfg{tildaid:\"b63053ca-d6b2-47b2-9dbb-effaf732dd75\"})-[:WORKPLAN_CFG_TO_STUDY]-\u003e(study:Study)\n\t\tRETURN study.tildaid as studyId\n\t",
"queryParameters": "{}",
"runtime": "pipelined",
"source": "bolt-session\tbolt\tGo Driver/5.20.0\t\tclient/10.129.4.159:37136\tserver/10.173.0.201:7687\u003e",
"timestamp": "2026-07-27T00:07:18.536Z",
"transactionId": 1750726.0,
"waiting": 0.0
}32s ago2026-07-27 00:07:18 UTC neo4j workplansvc UNWIND $site_ids AS sid OPTIONAL MATCH (s:Site {tildaid: sid, delpending: false}) OPTIONAL MATCH...
time2026-07-27 00:07:18 UTC
severityINFO
serviceworkplansvc
clustermaster
podserver-3-dev-0
query
UNWIND $site_ids AS sid
OPTIONAL MATCH (s:Site {tildaid: sid, delpending: false})
OPTIONAL MATCH (s)<-[:PLAN_DEFINITION_TO_SITE]-(pd:DslPlanDefinition {delpending: false})-[:PLAN_DEFINITION_TO_STUDY]->(st:Study {tildaid: $study_id, delpending: false})
OPTIONAL MATCH (pd)-[pdc:PLAN_DEFINITION_TO_CONTACT]->(actor:Actor {delpending: false})-[:FOR_USER]->(u:User)
OPTIONAL MATCH (pd)-[copdc:PLAN_DEFINITION_TO_CLINOP_CONTACT]->(co:ClinOp {deltimestamp: 0})-[:FOR_USER]->(cou:User)
WITH sid, s,
collect(DISTINCT CASE
WHEN pdc IS NOT NULL AND actor IS NOT NULL AND (u IS NULL OR u.status <> $deleted_status) THEN {
t: pdc.type,
p: coalesce(pdc.isPrimary, false),
e: coalesce(u.email, "")
} END) AS actorContactRows,
collect(DISTINCT CASE
WHEN copdc IS NOT NULL AND co IS NOT NULL AND (cou IS NULL OR cou.status <> $deleted_status) THEN {
t: copdc.type,
p: coalesce(copdc.isPrimary, false),
e: coalesce(cou.email, "")
} END) AS clinopContactRows
WITH sid, s, actorContactRows + clinopContactRows AS pdContactRowsRaw
WITH sid, s,
[row IN pdContactRowsRaw WHERE row IS NOT NULL AND row.t IS NOT NULL AND row.e IS NOT NULL AND row.e <> ""] AS pd_rows
RETURN sid AS site_id, coalesce(s.name, "") AS site_name, pd_rows
raw payload (15 fields)
{
"allocatedBytes": 19512.0,
"cpu": 1.0,
"elapsedTimeMs": 1.0,
"executionPlanCacheKeyHash": "3832EC22",
"id": "13356362",
"pageFaults": 0.0,
"pageHits": 29.0,
"planning": 0.0,
"query": "\nUNWIND $site_ids AS sid\nOPTIONAL MATCH (s:Site {tildaid: sid, delpending: false})\nOPTIONAL MATCH (s)\u003c-[:PLAN_DEFINITION_TO_SITE]-(pd:DslPlanDefinition {delpending: false})-[:PLAN_DEFINITION_TO_STUDY]-\u003e(st:Study {tildaid: $study_id, delpending: false})\nOPTIONAL MATCH (pd)-[pdc:PLAN_DEFINITION_TO_CONTACT]-\u003e(actor:Actor {delpending: false})-[:FOR_USER]-\u003e(u:User)\nOPTIONAL MATCH (pd)-[copdc:PLAN_DEFINITION_TO_CLINOP_CONTACT]-\u003e(co:ClinOp {deltimestamp: 0})-[:FOR_USER]-\u003e(cou:User)\nWITH sid, s,\n collect(DISTINCT CASE\n WHEN pdc IS NOT NULL AND actor IS NOT NULL AND (u IS NULL OR u.status \u003c\u003e $deleted_status) THEN {\n t: pdc.type,\n p: coalesce(pdc.isPrimary, false),\n e: coalesce(u.email, \"\")\n } END) AS actorContactRows,\n collect(DISTINCT CASE\n WHEN copdc IS NOT NULL AND co IS NOT NULL AND (cou IS NULL OR cou.status \u003c\u003e $deleted_status) THEN {\n t: copdc.type,\n p: coalesce(copdc.isPrimary, false),\n e: coalesce(cou.email, \"\")\n } END) AS clinopContactRows\nWITH sid, s, actorContactRows + clinopContactRows AS pdContactRowsRaw\nWITH sid, s,\n [row IN pdContactRowsRaw WHERE row IS NOT NULL AND row.t IS NOT NULL AND row.e IS NOT NULL AND row.e \u003c\u003e \"\"] AS pd_rows\nRETURN sid AS site_id, coalesce(s.name, \"\") AS site_name, pd_rows\n",
"queryParameters": "{deleted_status: \u0027Deleted\u0027, site_ids: [\u0027a6259678-9d0f-4311-8051-d1dca2333297\u0027], study_id: \u002716ffa942-94ca-4e1c-88f7-e0d87180dfe8\u0027}",
"runtime": "pipelined",
"source": "bolt-session\tbolt\tGo Driver/5.20.0\t\tclient/10.129.4.159:37136\tserver/10.173.0.201:7687\u003e",
"timestamp": "2026-07-27T00:07:18.526Z",
"transactionId": 1750725.0,
"waiting": 0.0
}32s ago2026-07-27 00:07:18 UTC neo4j workplansvc MATCH (instance:WorkPlanInstance{tildaid:"344bb43b-c114-414a-9ede-3024085e25ec"})-[:WORKPLAN_TO_SITE]->(site:Site) RETURN site.tildaid as siteId
time2026-07-27 00:07:18 UTC
severityINFO
serviceworkplansvc
clustermaster
podserver-3-dev-0
query
MATCH (instance:WorkPlanInstance{tildaid:"344bb43b-c114-414a-9ede-3024085e25ec"})-[:WORKPLAN_TO_SITE]->(site:Site)
RETURN site.tildaid as siteId
raw payload (15 fields)
{
"allocatedBytes": 328.0,
"cpu": 5.0,
"elapsedTimeMs": 5.0,
"executionPlanCacheKeyHash": "9B774BFC",
"id": "13356361",
"pageFaults": 0.0,
"pageHits": 414.0,
"planning": 4.0,
"query": "\n\t\tMATCH (instance:WorkPlanInstance{tildaid:\"344bb43b-c114-414a-9ede-3024085e25ec\"})-[:WORKPLAN_TO_SITE]-\u003e(site:Site)\n\t\tRETURN site.tildaid as siteId\n\t",
"queryParameters": "{}",
"runtime": "pipelined",
"source": "bolt-session\tbolt\tGo Driver/5.20.0\t\tclient/10.129.4.159:37136\tserver/10.173.0.201:7687\u003e",
"timestamp": "2026-07-27T00:07:18.519Z",
"transactionId": 1750724.0,
"waiting": 0.0
}32s ago2026-07-27 00:07:18 UTC neo4j workplansvc MATCH (cfg:WorkplanCfg{tildaid:"aa651aca-e3b1-437b-91c0-1a782428010b"})-[:WORKPLAN_CFG_TO_STUDY]->(study:Study) RETURN study.tildaid as studyId
time2026-07-27 00:07:18 UTC
severityINFO
serviceworkplansvc
clustermaster
podserver-3-dev-0
query
MATCH (cfg:WorkplanCfg{tildaid:"aa651aca-e3b1-437b-91c0-1a782428010b"})-[:WORKPLAN_CFG_TO_STUDY]->(study:Study)
RETURN study.tildaid as studyId
raw payload (15 fields)
{
"allocatedBytes": 328.0,
"cpu": 1.0,
"elapsedTimeMs": 1.0,
"executionPlanCacheKeyHash": "682A7B96",
"id": "13356360",
"pageFaults": 0.0,
"pageHits": 347.0,
"planning": 0.0,
"query": "\n\t\tMATCH (cfg:WorkplanCfg{tildaid:\"aa651aca-e3b1-437b-91c0-1a782428010b\"})-[:WORKPLAN_CFG_TO_STUDY]-\u003e(study:Study)\n\t\tRETURN study.tildaid as studyId\n\t",
"queryParameters": "{}",
"runtime": "pipelined",
"source": "bolt-session\tbolt\tGo Driver/5.20.0\t\tclient/10.129.4.159:37136\tserver/10.173.0.201:7687\u003e",
"timestamp": "2026-07-27T00:07:18.509Z",
"transactionId": 1750723.0,
"waiting": 0.0
}32s ago2026-07-27 00:07:18 UTC neo4j workplansvc UNWIND $site_ids AS sid OPTIONAL MATCH (s:Site {tildaid: sid, delpending: false}) OPTIONAL MATCH...
time2026-07-27 00:07:18 UTC
severityINFO
serviceworkplansvc
clustermaster
podserver-3-dev-0
query
UNWIND $site_ids AS sid
OPTIONAL MATCH (s:Site {tildaid: sid, delpending: false})
OPTIONAL MATCH (s)<-[:PLAN_DEFINITION_TO_SITE]-(pd:DslPlanDefinition {delpending: false})-[:PLAN_DEFINITION_TO_STUDY]->(st:Study {tildaid: $study_id, delpending: false})
OPTIONAL MATCH (pd)-[pdc:PLAN_DEFINITION_TO_CONTACT]->(actor:Actor {delpending: false})-[:FOR_USER]->(u:User)
OPTIONAL MATCH (pd)-[copdc:PLAN_DEFINITION_TO_CLINOP_CONTACT]->(co:ClinOp {deltimestamp: 0})-[:FOR_USER]->(cou:User)
WITH sid, s,
collect(DISTINCT CASE
WHEN pdc IS NOT NULL AND actor IS NOT NULL AND (u IS NULL OR u.status <> $deleted_status) THEN {
t: pdc.type,
p: coalesce(pdc.isPrimary, false),
e: coalesce(u.email, "")
} END) AS actorContactRows,
collect(DISTINCT CASE
WHEN copdc IS NOT NULL AND co IS NOT NULL AND (cou IS NULL OR cou.status <> $deleted_status) THEN {
t: copdc.type,
p: coalesce(copdc.isPrimary, false),
e: coalesce(cou.email, "")
} END) AS clinopContactRows
WITH sid, s, actorContactRows + clinopContactRows AS pdContactRowsRaw
WITH sid, s,
[row IN pdContactRowsRaw WHERE row IS NOT NULL AND row.t IS NOT NULL AND row.e IS NOT NULL AND row.e <> ""] AS pd_rows
RETURN sid AS site_id, coalesce(s.name, "") AS site_name, pd_rows
raw payload (15 fields)
{
"allocatedBytes": 19512.0,
"cpu": 1.0,
"elapsedTimeMs": 1.0,
"executionPlanCacheKeyHash": "3832EC22",
"id": "13356359",
"pageFaults": 0.0,
"pageHits": 28.0,
"planning": 0.0,
"query": "\nUNWIND $site_ids AS sid\nOPTIONAL MATCH (s:Site {tildaid: sid, delpending: false})\nOPTIONAL MATCH (s)\u003c-[:PLAN_DEFINITION_TO_SITE]-(pd:DslPlanDefinition {delpending: false})-[:PLAN_DEFINITION_TO_STUDY]-\u003e(st:Study {tildaid: $study_id, delpending: false})\nOPTIONAL MATCH (pd)-[pdc:PLAN_DEFINITION_TO_CONTACT]-\u003e(actor:Actor {delpending: false})-[:FOR_USER]-\u003e(u:User)\nOPTIONAL MATCH (pd)-[copdc:PLAN_DEFINITION_TO_CLINOP_CONTACT]-\u003e(co:ClinOp {deltimestamp: 0})-[:FOR_USER]-\u003e(cou:User)\nWITH sid, s,\n collect(DISTINCT CASE\n WHEN pdc IS NOT NULL AND actor IS NOT NULL AND (u IS NULL OR u.status \u003c\u003e $deleted_status) THEN {\n t: pdc.type,\n p: coalesce(pdc.isPrimary, false),\n e: coalesce(u.email, \"\")\n } END) AS actorContactRows,\n collect(DISTINCT CASE\n WHEN copdc IS NOT NULL AND co IS NOT NULL AND (cou IS NULL OR cou.status \u003c\u003e $deleted_status) THEN {\n t: copdc.type,\n p: coalesce(copdc.isPrimary, false),\n e: coalesce(cou.email, \"\")\n } END) AS clinopContactRows\nWITH sid, s, actorContactRows + clinopContactRows AS pdContactRowsRaw\nWITH sid, s,\n [row IN pdContactRowsRaw WHERE row IS NOT NULL AND row.t IS NOT NULL AND row.e IS NOT NULL AND row.e \u003c\u003e \"\"] AS pd_rows\nRETURN sid AS site_id, coalesce(s.name, \"\") AS site_name, pd_rows\n",
"queryParameters": "{deleted_status: \u0027Deleted\u0027, site_ids: [\u0027a6259678-9d0f-4311-8051-d1dca2333297\u0027], study_id: \u00270a62f219-a803-482f-8973-ab67b1a3baf7\u0027}",
"runtime": "pipelined",
"source": "bolt-session\tbolt\tGo Driver/5.20.0\t\tclient/10.129.4.159:37136\tserver/10.173.0.201:7687\u003e",
"timestamp": "2026-07-27T00:07:18.499Z",
"transactionId": 1750722.0,
"waiting": 0.0
}32s ago2026-07-27 00:07:18 UTC neo4j workplansvc MATCH (instance:WorkPlanInstance{tildaid:"272b7c49-648b-454b-b9e4-34bbc9c082cc"})-[:WORKPLAN_TO_SITE]->(site:Site) RETURN site.tildaid as siteId
time2026-07-27 00:07:18 UTC
severityINFO
serviceworkplansvc
clustermaster
podserver-3-dev-0
query
MATCH (instance:WorkPlanInstance{tildaid:"272b7c49-648b-454b-b9e4-34bbc9c082cc"})-[:WORKPLAN_TO_SITE]->(site:Site)
RETURN site.tildaid as siteId
raw payload (15 fields)
{
"allocatedBytes": 328.0,
"cpu": 1.0,
"elapsedTimeMs": 1.0,
"executionPlanCacheKeyHash": "9B774BFC",
"id": "13356358",
"pageFaults": 0.0,
"pageHits": 416.0,
"planning": 0.0,
"query": "\n\t\tMATCH (instance:WorkPlanInstance{tildaid:\"272b7c49-648b-454b-b9e4-34bbc9c082cc\"})-[:WORKPLAN_TO_SITE]-\u003e(site:Site)\n\t\tRETURN site.tildaid as siteId\n\t",
"queryParameters": "{}",
"runtime": "pipelined",
"source": "bolt-session\tbolt\tGo Driver/5.20.0\t\tclient/10.129.4.159:37136\tserver/10.173.0.201:7687\u003e",
"timestamp": "2026-07-27T00:07:18.493Z",
"transactionId": 1750721.0,
"waiting": 0.0
}32s ago2026-07-27 00:07:18 UTC neo4j workplansvc MATCH (cfg:WorkplanCfg{tildaid:"a546762b-b73b-495c-b06f-543822164be1"})-[:WORKPLAN_CFG_TO_STUDY]->(study:Study) RETURN study.tildaid as studyId
time2026-07-27 00:07:18 UTC
severityINFO
serviceworkplansvc
clustermaster
podserver-3-dev-0
query
MATCH (cfg:WorkplanCfg{tildaid:"a546762b-b73b-495c-b06f-543822164be1"})-[:WORKPLAN_CFG_TO_STUDY]->(study:Study)
RETURN study.tildaid as studyId
raw payload (15 fields)
{
"allocatedBytes": 328.0,
"cpu": 4.0,
"elapsedTimeMs": 5.0,
"executionPlanCacheKeyHash": "682A7B96",
"id": "13356357",
"pageFaults": 0.0,
"pageHits": 329.0,
"planning": 4.0,
"query": "\n\t\tMATCH (cfg:WorkplanCfg{tildaid:\"a546762b-b73b-495c-b06f-543822164be1\"})-[:WORKPLAN_CFG_TO_STUDY]-\u003e(study:Study)\n\t\tRETURN study.tildaid as studyId\n\t",
"queryParameters": "{}",
"runtime": "pipelined",
"source": "bolt-session\tbolt\tGo Driver/5.20.0\t\tclient/10.129.4.159:37136\tserver/10.173.0.201:7687\u003e",
"timestamp": "2026-07-27T00:07:18.486Z",
"transactionId": 1750720.0,
"waiting": 0.0
}32s ago2026-07-27 00:07:18 UTC neo4j workplansvc UNWIND $site_ids AS sid OPTIONAL MATCH (s:Site {tildaid: sid, delpending: false}) OPTIONAL MATCH...
time2026-07-27 00:07:18 UTC
severityINFO
serviceworkplansvc
clustermaster
podserver-3-dev-0
query
UNWIND $site_ids AS sid
OPTIONAL MATCH (s:Site {tildaid: sid, delpending: false})
OPTIONAL MATCH (s)<-[:PLAN_DEFINITION_TO_SITE]-(pd:DslPlanDefinition {delpending: false})-[:PLAN_DEFINITION_TO_STUDY]->(st:Study {tildaid: $study_id, delpending: false})
OPTIONAL MATCH (pd)-[pdc:PLAN_DEFINITION_TO_CONTACT]->(actor:Actor {delpending: false})-[:FOR_USER]->(u:User)
OPTIONAL MATCH (pd)-[copdc:PLAN_DEFINITION_TO_CLINOP_CONTACT]->(co:ClinOp {deltimestamp: 0})-[:FOR_USER]->(cou:User)
WITH sid, s,
collect(DISTINCT CASE
WHEN pdc IS NOT NULL AND actor IS NOT NULL AND (u IS NULL OR u.status <> $deleted_status) THEN {
t: pdc.type,
p: coalesce(pdc.isPrimary, false),
e: coalesce(u.email, "")
} END) AS actorContactRows,
collect(DISTINCT CASE
WHEN copdc IS NOT NULL AND co IS NOT NULL AND (cou IS NULL OR cou.status <> $deleted_status) THEN {
t: copdc.type,
p: coalesce(copdc.isPrimary, false),
e: coalesce(cou.email, "")
} END) AS clinopContactRows
WITH sid, s, actorContactRows + clinopContactRows AS pdContactRowsRaw
WITH sid, s,
[row IN pdContactRowsRaw WHERE row IS NOT NULL AND row.t IS NOT NULL AND row.e IS NOT NULL AND row.e <> ""] AS pd_rows
RETURN sid AS site_id, coalesce(s.name, "") AS site_name, pd_rows
raw payload (15 fields)
{
"allocatedBytes": 19512.0,
"cpu": 1.0,
"elapsedTimeMs": 1.0,
"executionPlanCacheKeyHash": "3832EC22",
"id": "13356356",
"pageFaults": 0.0,
"pageHits": 22.0,
"planning": 0.0,
"query": "\nUNWIND $site_ids AS sid\nOPTIONAL MATCH (s:Site {tildaid: sid, delpending: false})\nOPTIONAL MATCH (s)\u003c-[:PLAN_DEFINITION_TO_SITE]-(pd:DslPlanDefinition {delpending: false})-[:PLAN_DEFINITION_TO_STUDY]-\u003e(st:Study {tildaid: $study_id, delpending: false})\nOPTIONAL MATCH (pd)-[pdc:PLAN_DEFINITION_TO_CONTACT]-\u003e(actor:Actor {delpending: false})-[:FOR_USER]-\u003e(u:User)\nOPTIONAL MATCH (pd)-[copdc:PLAN_DEFINITION_TO_CLINOP_CONTACT]-\u003e(co:ClinOp {deltimestamp: 0})-[:FOR_USER]-\u003e(cou:User)\nWITH sid, s,\n collect(DISTINCT CASE\n WHEN pdc IS NOT NULL AND actor IS NOT NULL AND (u IS NULL OR u.status \u003c\u003e $deleted_status) THEN {\n t: pdc.type,\n p: coalesce(pdc.isPrimary, false),\n e: coalesce(u.email, \"\")\n } END) AS actorContactRows,\n collect(DISTINCT CASE\n WHEN copdc IS NOT NULL AND co IS NOT NULL AND (cou IS NULL OR cou.status \u003c\u003e $deleted_status) THEN {\n t: copdc.type,\n p: coalesce(copdc.isPrimary, false),\n e: coalesce(cou.email, \"\")\n } END) AS clinopContactRows\nWITH sid, s, actorContactRows + clinopContactRows AS pdContactRowsRaw\nWITH sid, s,\n [row IN pdContactRowsRaw WHERE row IS NOT NULL AND row.t IS NOT NULL AND row.e IS NOT NULL AND row.e \u003c\u003e \"\"] AS pd_rows\nRETURN sid AS site_id, coalesce(s.name, \"\") AS site_name, pd_rows\n",
"queryParameters": "{deleted_status: \u0027Deleted\u0027, site_ids: [\u0027cec6713e-0087-4942-91db-316352bbca2c\u0027], study_id: \u00275b971f19-ded8-4c5d-8649-5b2a96841d67\u0027}",
"runtime": "pipelined",
"source": "bolt-session\tbolt\tGo Driver/5.20.0\t\tclient/10.129.4.159:37136\tserver/10.173.0.201:7687\u003e",
"timestamp": "2026-07-27T00:07:18.476Z",
"transactionId": 1750719.0,
"waiting": 0.0
}32s ago2026-07-27 00:07:18 UTC neo4j workplansvc MATCH (instance:WorkPlanInstance{tildaid:"bad745ed-c7c6-496e-bd2b-1cbc61040198"})-[:WORKPLAN_TO_SITE]->(site:Site) RETURN site.tildaid as siteId
time2026-07-27 00:07:18 UTC
severityINFO
serviceworkplansvc
clustermaster
podserver-3-dev-0
query
MATCH (instance:WorkPlanInstance{tildaid:"bad745ed-c7c6-496e-bd2b-1cbc61040198"})-[:WORKPLAN_TO_SITE]->(site:Site)
RETURN site.tildaid as siteId
raw payload (15 fields)
{
"allocatedBytes": 328.0,
"cpu": 5.0,
"elapsedTimeMs": 5.0,
"executionPlanCacheKeyHash": "9B774BFC",
"id": "13356354",
"pageFaults": 0.0,
"pageHits": 415.0,
"planning": 4.0,
"query": "\n\t\tMATCH (instance:WorkPlanInstance{tildaid:\"bad745ed-c7c6-496e-bd2b-1cbc61040198\"})-[:WORKPLAN_TO_SITE]-\u003e(site:Site)\n\t\tRETURN site.tildaid as siteId\n\t",
"queryParameters": "{}",
"runtime": "pipelined",
"source": "bolt-session\tbolt\tGo Driver/5.20.0\t\tclient/10.129.4.159:37136\tserver/10.173.0.201:7687\u003e",
"timestamp": "2026-07-27T00:07:18.470Z",
"transactionId": 1750718.0,
"waiting": 0.0
}32s ago2026-07-27 00:07:18 UTC neo4j workplansvc MATCH (cfg:WorkplanCfg{tildaid:"cb82212d-9ee3-4739-9c4f-bec323411efb"})-[:WORKPLAN_CFG_TO_STUDY]->(study:Study) RETURN study.tildaid as studyId
time2026-07-27 00:07:18 UTC
severityINFO
serviceworkplansvc
clustermaster
podserver-3-dev-0
query
MATCH (cfg:WorkplanCfg{tildaid:"cb82212d-9ee3-4739-9c4f-bec323411efb"})-[:WORKPLAN_CFG_TO_STUDY]->(study:Study)
RETURN study.tildaid as studyId
raw payload (15 fields)
{
"allocatedBytes": 328.0,
"cpu": 1.0,
"elapsedTimeMs": 1.0,
"executionPlanCacheKeyHash": "682A7B96",
"id": "13356353",
"pageFaults": 0.0,
"pageHits": 329.0,
"planning": 0.0,
"query": "\n\t\tMATCH (cfg:WorkplanCfg{tildaid:\"cb82212d-9ee3-4739-9c4f-bec323411efb\"})-[:WORKPLAN_CFG_TO_STUDY]-\u003e(study:Study)\n\t\tRETURN study.tildaid as studyId\n\t",
"queryParameters": "{}",
"runtime": "pipelined",
"source": "bolt-session\tbolt\tGo Driver/5.20.0\t\tclient/10.129.4.159:37136\tserver/10.173.0.201:7687\u003e",
"timestamp": "2026-07-27T00:07:18.460Z",
"transactionId": 1750717.0,
"waiting": 0.0
}32s ago2026-07-27 00:07:18 UTC neo4j workplansvc UNWIND $site_ids AS sid OPTIONAL MATCH (s:Site {tildaid: sid, delpending: false}) OPTIONAL MATCH...
time2026-07-27 00:07:18 UTC
severityINFO
serviceworkplansvc
clustermaster
podserver-3-dev-0
query
UNWIND $site_ids AS sid
OPTIONAL MATCH (s:Site {tildaid: sid, delpending: false})
OPTIONAL MATCH (s)<-[:PLAN_DEFINITION_TO_SITE]-(pd:DslPlanDefinition {delpending: false})-[:PLAN_DEFINITION_TO_STUDY]->(st:Study {tildaid: $study_id, delpending: false})
OPTIONAL MATCH (pd)-[pdc:PLAN_DEFINITION_TO_CONTACT]->(actor:Actor {delpending: false})-[:FOR_USER]->(u:User)
OPTIONAL MATCH (pd)-[copdc:PLAN_DEFINITION_TO_CLINOP_CONTACT]->(co:ClinOp {deltimestamp: 0})-[:FOR_USER]->(cou:User)
WITH sid, s,
collect(DISTINCT CASE
WHEN pdc IS NOT NULL AND actor IS NOT NULL AND (u IS NULL OR u.status <> $deleted_status) THEN {
t: pdc.type,
p: coalesce(pdc.isPrimary, false),
e: coalesce(u.email, "")
} END) AS actorContactRows,
collect(DISTINCT CASE
WHEN copdc IS NOT NULL AND co IS NOT NULL AND (cou IS NULL OR cou.status <> $deleted_status) THEN {
t: copdc.type,
p: coalesce(copdc.isPrimary, false),
e: coalesce(cou.email, "")
} END) AS clinopContactRows
WITH sid, s, actorContactRows + clinopContactRows AS pdContactRowsRaw
WITH sid, s,
[row IN pdContactRowsRaw WHERE row IS NOT NULL AND row.t IS NOT NULL AND row.e IS NOT NULL AND row.e <> ""] AS pd_rows
RETURN sid AS site_id, coalesce(s.name, "") AS site_name, pd_rows
raw payload (15 fields)
{
"allocatedBytes": 19512.0,
"cpu": 1.0,
"elapsedTimeMs": 1.0,
"executionPlanCacheKeyHash": "3832EC22",
"id": "13356352",
"pageFaults": 0.0,
"pageHits": 28.0,
"planning": 0.0,
"query": "\nUNWIND $site_ids AS sid\nOPTIONAL MATCH (s:Site {tildaid: sid, delpending: false})\nOPTIONAL MATCH (s)\u003c-[:PLAN_DEFINITION_TO_SITE]-(pd:DslPlanDefinition {delpending: false})-[:PLAN_DEFINITION_TO_STUDY]-\u003e(st:Study {tildaid: $study_id, delpending: false})\nOPTIONAL MATCH (pd)-[pdc:PLAN_DEFINITION_TO_CONTACT]-\u003e(actor:Actor {delpending: false})-[:FOR_USER]-\u003e(u:User)\nOPTIONAL MATCH (pd)-[copdc:PLAN_DEFINITION_TO_CLINOP_CONTACT]-\u003e(co:ClinOp {deltimestamp: 0})-[:FOR_USER]-\u003e(cou:User)\nWITH sid, s,\n collect(DISTINCT CASE\n WHEN pdc IS NOT NULL AND actor IS NOT NULL AND (u IS NULL OR u.status \u003c\u003e $deleted_status) THEN {\n t: pdc.type,\n p: coalesce(pdc.isPrimary, false),\n e: coalesce(u.email, \"\")\n } END) AS actorContactRows,\n collect(DISTINCT CASE\n WHEN copdc IS NOT NULL AND co IS NOT NULL AND (cou IS NULL OR cou.status \u003c\u003e $deleted_status) THEN {\n t: copdc.type,\n p: coalesce(copdc.isPrimary, false),\n e: coalesce(cou.email, \"\")\n } END) AS clinopContactRows\nWITH sid, s, actorContactRows + clinopContactRows AS pdContactRowsRaw\nWITH sid, s,\n [row IN pdContactRowsRaw WHERE row IS NOT NULL AND row.t IS NOT NULL AND row.e IS NOT NULL AND row.e \u003c\u003e \"\"] AS pd_rows\nRETURN sid AS site_id, coalesce(s.name, \"\") AS site_name, pd_rows\n",
"queryParameters": "{deleted_status: \u0027Deleted\u0027, site_ids: [\u0027a6259678-9d0f-4311-8051-d1dca2333297\u0027], study_id: \u00270a62f219-a803-482f-8973-ab67b1a3baf7\u0027}",
"runtime": "pipelined",
"source": "bolt-session\tbolt\tGo Driver/5.20.0\t\tclient/10.129.4.159:37136\tserver/10.173.0.201:7687\u003e",
"timestamp": "2026-07-27T00:07:18.454Z",
"transactionId": 1750716.0,
"waiting": 0.0
}32s ago2026-07-27 00:07:18 UTC neo4j workplansvc MATCH (instance:WorkPlanInstance{tildaid:"5ca8bb00-38a6-4d01-b31b-20cbe6b1a878"})-[:WORKPLAN_TO_SITE]->(site:Site) RETURN site.tildaid as siteId
time2026-07-27 00:07:18 UTC
severityINFO
serviceworkplansvc
clustermaster
podserver-3-dev-0
query
MATCH (instance:WorkPlanInstance{tildaid:"5ca8bb00-38a6-4d01-b31b-20cbe6b1a878"})-[:WORKPLAN_TO_SITE]->(site:Site)
RETURN site.tildaid as siteId
raw payload (15 fields)
{
"allocatedBytes": 328.0,
"cpu": 1.0,
"elapsedTimeMs": 1.0,
"executionPlanCacheKeyHash": "9B774BFC",
"id": "13356351",
"pageFaults": 0.0,
"pageHits": 414.0,
"planning": 0.0,
"query": "\n\t\tMATCH (instance:WorkPlanInstance{tildaid:\"5ca8bb00-38a6-4d01-b31b-20cbe6b1a878\"})-[:WORKPLAN_TO_SITE]-\u003e(site:Site)\n\t\tRETURN site.tildaid as siteId\n\t",
"queryParameters": "{}",
"runtime": "pipelined",
"source": "bolt-session\tbolt\tGo Driver/5.20.0\t\tclient/10.129.4.159:37136\tserver/10.173.0.201:7687\u003e",
"timestamp": "2026-07-27T00:07:18.447Z",
"transactionId": 1750715.0,
"waiting": 0.0
}32s ago2026-07-27 00:07:18 UTC neo4j workplansvc MATCH (cfg:WorkplanCfg{tildaid:"37e5546c-d37f-4d97-a4ac-f5ebcc616248"})-[:WORKPLAN_CFG_TO_STUDY]->(study:Study) RETURN study.tildaid as studyId
time2026-07-27 00:07:18 UTC
severityINFO
serviceworkplansvc
clustermaster
podserver-3-dev-0
query
MATCH (cfg:WorkplanCfg{tildaid:"37e5546c-d37f-4d97-a4ac-f5ebcc616248"})-[:WORKPLAN_CFG_TO_STUDY]->(study:Study)
RETURN study.tildaid as studyId
raw payload (15 fields)
{
"allocatedBytes": 328.0,
"cpu": 1.0,
"elapsedTimeMs": 1.0,
"executionPlanCacheKeyHash": "682A7B96",
"id": "13356350",
"pageFaults": 0.0,
"pageHits": 329.0,
"planning": 0.0,
"query": "\n\t\tMATCH (cfg:WorkplanCfg{tildaid:\"37e5546c-d37f-4d97-a4ac-f5ebcc616248\"})-[:WORKPLAN_CFG_TO_STUDY]-\u003e(study:Study)\n\t\tRETURN study.tildaid as studyId\n\t",
"queryParameters": "{}",
"runtime": "pipelined",
"source": "bolt-session\tbolt\tGo Driver/5.20.0\t\tclient/10.129.4.159:37136\tserver/10.173.0.201:7687\u003e",
"timestamp": "2026-07-27T00:07:18.440Z",
"transactionId": 1750714.0,
"waiting": 0.0
}32s ago2026-07-27 00:07:18 UTC neo4j workplansvc UNWIND $site_ids AS sid OPTIONAL MATCH (s:Site {tildaid: sid, delpending: false}) OPTIONAL MATCH...
time2026-07-27 00:07:18 UTC
severityINFO
serviceworkplansvc
clustermaster
podserver-3-dev-0
query
UNWIND $site_ids AS sid
OPTIONAL MATCH (s:Site {tildaid: sid, delpending: false})
OPTIONAL MATCH (s)<-[:PLAN_DEFINITION_TO_SITE]-(pd:DslPlanDefinition {delpending: false})-[:PLAN_DEFINITION_TO_STUDY]->(st:Study {tildaid: $study_id, delpending: false})
OPTIONAL MATCH (pd)-[pdc:PLAN_DEFINITION_TO_CONTACT]->(actor:Actor {delpending: false})-[:FOR_USER]->(u:User)
OPTIONAL MATCH (pd)-[copdc:PLAN_DEFINITION_TO_CLINOP_CONTACT]->(co:ClinOp {deltimestamp: 0})-[:FOR_USER]->(cou:User)
WITH sid, s,
collect(DISTINCT CASE
WHEN pdc IS NOT NULL AND actor IS NOT NULL AND (u IS NULL OR u.status <> $deleted_status) THEN {
t: pdc.type,
p: coalesce(pdc.isPrimary, false),
e: coalesce(u.email, "")
} END) AS actorContactRows,
collect(DISTINCT CASE
WHEN copdc IS NOT NULL AND co IS NOT NULL AND (cou IS NULL OR cou.status <> $deleted_status) THEN {
t: copdc.type,
p: coalesce(copdc.isPrimary, false),
e: coalesce(cou.email, "")
} END) AS clinopContactRows
WITH sid, s, actorContactRows + clinopContactRows AS pdContactRowsRaw
WITH sid, s,
[row IN pdContactRowsRaw WHERE row IS NOT NULL AND row.t IS NOT NULL AND row.e IS NOT NULL AND row.e <> ""] AS pd_rows
RETURN sid AS site_id, coalesce(s.name, "") AS site_name, pd_rows
raw payload (15 fields)
{
"allocatedBytes": 19512.0,
"cpu": 1.0,
"elapsedTimeMs": 1.0,
"executionPlanCacheKeyHash": "3832EC22",
"id": "13356349",
"pageFaults": 0.0,
"pageHits": 36.0,
"planning": 0.0,
"query": "\nUNWIND $site_ids AS sid\nOPTIONAL MATCH (s:Site {tildaid: sid, delpending: false})\nOPTIONAL MATCH (s)\u003c-[:PLAN_DEFINITION_TO_SITE]-(pd:DslPlanDefinition {delpending: false})-[:PLAN_DEFINITION_TO_STUDY]-\u003e(st:Study {tildaid: $study_id, delpending: false})\nOPTIONAL MATCH (pd)-[pdc:PLAN_DEFINITION_TO_CONTACT]-\u003e(actor:Actor {delpending: false})-[:FOR_USER]-\u003e(u:User)\nOPTIONAL MATCH (pd)-[copdc:PLAN_DEFINITION_TO_CLINOP_CONTACT]-\u003e(co:ClinOp {deltimestamp: 0})-[:FOR_USER]-\u003e(cou:User)\nWITH sid, s,\n collect(DISTINCT CASE\n WHEN pdc IS NOT NULL AND actor IS NOT NULL AND (u IS NULL OR u.status \u003c\u003e $deleted_status) THEN {\n t: pdc.type,\n p: coalesce(pdc.isPrimary, false),\n e: coalesce(u.email, \"\")\n } END) AS actorContactRows,\n collect(DISTINCT CASE\n WHEN copdc IS NOT NULL AND co IS NOT NULL AND (cou IS NULL OR cou.status \u003c\u003e $deleted_status) THEN {\n t: copdc.type,\n p: coalesce(copdc.isPrimary, false),\n e: coalesce(cou.email, \"\")\n } END) AS clinopContactRows\nWITH sid, s, actorContactRows + clinopContactRows AS pdContactRowsRaw\nWITH sid, s,\n [row IN pdContactRowsRaw WHERE row IS NOT NULL AND row.t IS NOT NULL AND row.e IS NOT NULL AND row.e \u003c\u003e \"\"] AS pd_rows\nRETURN sid AS site_id, coalesce(s.name, \"\") AS site_name, pd_rows\n",
"queryParameters": "{deleted_status: \u0027Deleted\u0027, site_ids: [\u00279c081cd2-085b-4c6e-851e-abf432f301b4\u0027], study_id: \u00270a62f219-a803-482f-8973-ab67b1a3baf7\u0027}",
"runtime": "pipelined",
"source": "bolt-session\tbolt\tGo Driver/5.20.0\t\tclient/10.129.4.159:37136\tserver/10.173.0.201:7687\u003e",
"timestamp": "2026-07-27T00:07:18.434Z",
"transactionId": 1750713.0,
"waiting": 0.0
}32s ago2026-07-27 00:07:18 UTC neo4j workplansvc MATCH (instance:WorkPlanInstance{tildaid:"e6c6652d-3cd8-4b9e-ac36-9df330acb222"})-[:WORKPLAN_TO_SITE]->(site:Site) RETURN site.tildaid as siteId
time2026-07-27 00:07:18 UTC
severityINFO
serviceworkplansvc
clustermaster
podserver-3-dev-0
query
MATCH (instance:WorkPlanInstance{tildaid:"e6c6652d-3cd8-4b9e-ac36-9df330acb222"})-[:WORKPLAN_TO_SITE]->(site:Site)
RETURN site.tildaid as siteId
raw payload (15 fields)
{
"allocatedBytes": 328.0,
"cpu": 5.0,
"elapsedTimeMs": 5.0,
"executionPlanCacheKeyHash": "9B774BFC",
"id": "13356348",
"pageFaults": 0.0,
"pageHits": 432.0,
"planning": 4.0,
"query": "\n\t\tMATCH (instance:WorkPlanInstance{tildaid:\"e6c6652d-3cd8-4b9e-ac36-9df330acb222\"})-[:WORKPLAN_TO_SITE]-\u003e(site:Site)\n\t\tRETURN site.tildaid as siteId\n\t",
"queryParameters": "{}",
"runtime": "pipelined",
"source": "bolt-session\tbolt\tGo Driver/5.20.0\t\tclient/10.129.4.159:37136\tserver/10.173.0.201:7687\u003e",
"timestamp": "2026-07-27T00:07:18.428Z",
"transactionId": 1750712.0,
"waiting": 0.0
}32s ago2026-07-27 00:07:18 UTC neo4j workplansvc MATCH (cfg:WorkplanCfg{tildaid:"37e5546c-d37f-4d97-a4ac-f5ebcc616248"})-[:WORKPLAN_CFG_TO_STUDY]->(study:Study) RETURN study.tildaid as studyId
time2026-07-27 00:07:18 UTC
severityINFO
serviceworkplansvc
clustermaster
podserver-3-dev-0
query
MATCH (cfg:WorkplanCfg{tildaid:"37e5546c-d37f-4d97-a4ac-f5ebcc616248"})-[:WORKPLAN_CFG_TO_STUDY]->(study:Study)
RETURN study.tildaid as studyId
raw payload (15 fields)
{
"allocatedBytes": 328.0,
"cpu": 1.0,
"elapsedTimeMs": 1.0,
"executionPlanCacheKeyHash": "682A7B96",
"id": "13356347",
"pageFaults": 0.0,
"pageHits": 329.0,
"planning": 0.0,
"query": "\n\t\tMATCH (cfg:WorkplanCfg{tildaid:\"37e5546c-d37f-4d97-a4ac-f5ebcc616248\"})-[:WORKPLAN_CFG_TO_STUDY]-\u003e(study:Study)\n\t\tRETURN study.tildaid as studyId\n\t",
"queryParameters": "{}",
"runtime": "pipelined",
"source": "bolt-session\tbolt\tGo Driver/5.20.0\t\tclient/10.129.4.159:37136\tserver/10.173.0.201:7687\u003e",
"timestamp": "2026-07-27T00:07:18.418Z",
"transactionId": 1750711.0,
"waiting": 0.0
}32s ago2026-07-27 00:07:18 UTC neo4j workplansvc UNWIND $site_ids AS sid OPTIONAL MATCH (s:Site {tildaid: sid, delpending: false}) OPTIONAL MATCH...
time2026-07-27 00:07:18 UTC
severityINFO
serviceworkplansvc
clustermaster
podserver-3-dev-0
query
UNWIND $site_ids AS sid
OPTIONAL MATCH (s:Site {tildaid: sid, delpending: false})
OPTIONAL MATCH (s)<-[:PLAN_DEFINITION_TO_SITE]-(pd:DslPlanDefinition {delpending: false})-[:PLAN_DEFINITION_TO_STUDY]->(st:Study {tildaid: $study_id, delpending: false})
OPTIONAL MATCH (pd)-[pdc:PLAN_DEFINITION_TO_CONTACT]->(actor:Actor {delpending: false})-[:FOR_USER]->(u:User)
OPTIONAL MATCH (pd)-[copdc:PLAN_DEFINITION_TO_CLINOP_CONTACT]->(co:ClinOp {deltimestamp: 0})-[:FOR_USER]->(cou:User)
WITH sid, s,
collect(DISTINCT CASE
WHEN pdc IS NOT NULL AND actor IS NOT NULL AND (u IS NULL OR u.status <> $deleted_status) THEN {
t: pdc.type,
p: coalesce(pdc.isPrimary, false),
e: coalesce(u.email, "")
} END) AS actorContactRows,
collect(DISTINCT CASE
WHEN copdc IS NOT NULL AND co IS NOT NULL AND (cou IS NULL OR cou.status <> $deleted_status) THEN {
t: copdc.type,
p: coalesce(copdc.isPrimary, false),
e: coalesce(cou.email, "")
} END) AS clinopContactRows
WITH sid, s, actorContactRows + clinopContactRows AS pdContactRowsRaw
WITH sid, s,
[row IN pdContactRowsRaw WHERE row IS NOT NULL AND row.t IS NOT NULL AND row.e IS NOT NULL AND row.e <> ""] AS pd_rows
RETURN sid AS site_id, coalesce(s.name, "") AS site_name, pd_rows
raw payload (15 fields)
{
"allocatedBytes": 19512.0,
"cpu": 1.0,
"elapsedTimeMs": 1.0,
"executionPlanCacheKeyHash": "3832EC22",
"id": "13356346",
"pageFaults": 0.0,
"pageHits": 28.0,
"planning": 0.0,
"query": "\nUNWIND $site_ids AS sid\nOPTIONAL MATCH (s:Site {tildaid: sid, delpending: false})\nOPTIONAL MATCH (s)\u003c-[:PLAN_DEFINITION_TO_SITE]-(pd:DslPlanDefinition {delpending: false})-[:PLAN_DEFINITION_TO_STUDY]-\u003e(st:Study {tildaid: $study_id, delpending: false})\nOPTIONAL MATCH (pd)-[pdc:PLAN_DEFINITION_TO_CONTACT]-\u003e(actor:Actor {delpending: false})-[:FOR_USER]-\u003e(u:User)\nOPTIONAL MATCH (pd)-[copdc:PLAN_DEFINITION_TO_CLINOP_CONTACT]-\u003e(co:ClinOp {deltimestamp: 0})-[:FOR_USER]-\u003e(cou:User)\nWITH sid, s,\n collect(DISTINCT CASE\n WHEN pdc IS NOT NULL AND actor IS NOT NULL AND (u IS NULL OR u.status \u003c\u003e $deleted_status) THEN {\n t: pdc.type,\n p: coalesce(pdc.isPrimary, false),\n e: coalesce(u.email, \"\")\n } END) AS actorContactRows,\n collect(DISTINCT CASE\n WHEN copdc IS NOT NULL AND co IS NOT NULL AND (cou IS NULL OR cou.status \u003c\u003e $deleted_status) THEN {\n t: copdc.type,\n p: coalesce(copdc.isPrimary, false),\n e: coalesce(cou.email, \"\")\n } END) AS clinopContactRows\nWITH sid, s, actorContactRows + clinopContactRows AS pdContactRowsRaw\nWITH sid, s,\n [row IN pdContactRowsRaw WHERE row IS NOT NULL AND row.t IS NOT NULL AND row.e IS NOT NULL AND row.e \u003c\u003e \"\"] AS pd_rows\nRETURN sid AS site_id, coalesce(s.name, \"\") AS site_name, pd_rows\n",
"queryParameters": "{deleted_status: \u0027Deleted\u0027, site_ids: [\u0027a6259678-9d0f-4311-8051-d1dca2333297\u0027], study_id: \u00270a62f219-a803-482f-8973-ab67b1a3baf7\u0027}",
"runtime": "pipelined",
"source": "bolt-session\tbolt\tGo Driver/5.20.0\t\tclient/10.129.4.159:37136\tserver/10.173.0.201:7687\u003e",
"timestamp": "2026-07-27T00:07:18.412Z",
"transactionId": 1750710.0,
"waiting": 0.0
}32s ago2026-07-27 00:07:18 UTC neo4j workplansvc MATCH (instance:WorkPlanInstance{tildaid:"3dde6deb-a932-4d03-8c71-ce3a8e26bef7"})-[:WORKPLAN_TO_SITE]->(site:Site) RETURN site.tildaid as siteId
time2026-07-27 00:07:18 UTC
severityINFO
serviceworkplansvc
clustermaster
podserver-3-dev-0
query
MATCH (instance:WorkPlanInstance{tildaid:"3dde6deb-a932-4d03-8c71-ce3a8e26bef7"})-[:WORKPLAN_TO_SITE]->(site:Site)
RETURN site.tildaid as siteId
raw payload (15 fields)
{
"allocatedBytes": 328.0,
"cpu": 1.0,
"elapsedTimeMs": 1.0,
"executionPlanCacheKeyHash": "9B774BFC",
"id": "13356345",
"pageFaults": 0.0,
"pageHits": 414.0,
"planning": 0.0,
"query": "\n\t\tMATCH (instance:WorkPlanInstance{tildaid:\"3dde6deb-a932-4d03-8c71-ce3a8e26bef7\"})-[:WORKPLAN_TO_SITE]-\u003e(site:Site)\n\t\tRETURN site.tildaid as siteId\n\t",
"queryParameters": "{}",
"runtime": "pipelined",
"source": "bolt-session\tbolt\tGo Driver/5.20.0\t\tclient/10.129.4.159:37136\tserver/10.173.0.201:7687\u003e",
"timestamp": "2026-07-27T00:07:18.406Z",
"transactionId": 1750709.0,
"waiting": 0.0
}32s ago2026-07-27 00:07:18 UTC neo4j workplansvc MATCH (cfg:WorkplanCfg{tildaid:"a9f727b2-f56d-4267-b30f-8859577d46a9"})-[:WORKPLAN_CFG_TO_STUDY]->(study:Study) RETURN study.tildaid as studyId
time2026-07-27 00:07:18 UTC
severityINFO
serviceworkplansvc
clustermaster
podserver-3-dev-0
query
MATCH (cfg:WorkplanCfg{tildaid:"a9f727b2-f56d-4267-b30f-8859577d46a9"})-[:WORKPLAN_CFG_TO_STUDY]->(study:Study)
RETURN study.tildaid as studyId
raw payload (15 fields)
{
"allocatedBytes": 328.0,
"cpu": 1.0,
"elapsedTimeMs": 1.0,
"executionPlanCacheKeyHash": "682A7B96",
"id": "13356344",
"pageFaults": 0.0,
"pageHits": 329.0,
"planning": 0.0,
"query": "\n\t\tMATCH (cfg:WorkplanCfg{tildaid:\"a9f727b2-f56d-4267-b30f-8859577d46a9\"})-[:WORKPLAN_CFG_TO_STUDY]-\u003e(study:Study)\n\t\tRETURN study.tildaid as studyId\n\t",
"queryParameters": "{}",
"runtime": "pipelined",
"source": "bolt-session\tbolt\tGo Driver/5.20.0\t\tclient/10.129.4.159:37136\tserver/10.173.0.201:7687\u003e",
"timestamp": "2026-07-27T00:07:18.399Z",
"transactionId": 1750708.0,
"waiting": 0.0
}32s ago2026-07-27 00:07:18 UTC neo4j workplansvc UNWIND $site_ids AS sid OPTIONAL MATCH (s:Site {tildaid: sid, delpending: false}) OPTIONAL MATCH...
time2026-07-27 00:07:18 UTC
severityINFO
serviceworkplansvc
clustermaster
podserver-3-dev-0
query
UNWIND $site_ids AS sid
OPTIONAL MATCH (s:Site {tildaid: sid, delpending: false})
OPTIONAL MATCH (s)<-[:PLAN_DEFINITION_TO_SITE]-(pd:DslPlanDefinition {delpending: false})-[:PLAN_DEFINITION_TO_STUDY]->(st:Study {tildaid: $study_id, delpending: false})
OPTIONAL MATCH (pd)-[pdc:PLAN_DEFINITION_TO_CONTACT]->(actor:Actor {delpending: false})-[:FOR_USER]->(u:User)
OPTIONAL MATCH (pd)-[copdc:PLAN_DEFINITION_TO_CLINOP_CONTACT]->(co:ClinOp {deltimestamp: 0})-[:FOR_USER]->(cou:User)
WITH sid, s,
collect(DISTINCT CASE
WHEN pdc IS NOT NULL AND actor IS NOT NULL AND (u IS NULL OR u.status <> $deleted_status) THEN {
t: pdc.type,
p: coalesce(pdc.isPrimary, false),
e: coalesce(u.email, "")
} END) AS actorContactRows,
collect(DISTINCT CASE
WHEN copdc IS NOT NULL AND co IS NOT NULL AND (cou IS NULL OR cou.status <> $deleted_status) THEN {
t: copdc.type,
p: coalesce(copdc.isPrimary, false),
e: coalesce(cou.email, "")
} END) AS clinopContactRows
WITH sid, s, actorContactRows + clinopContactRows AS pdContactRowsRaw
WITH sid, s,
[row IN pdContactRowsRaw WHERE row IS NOT NULL AND row.t IS NOT NULL AND row.e IS NOT NULL AND row.e <> ""] AS pd_rows
RETURN sid AS site_id, coalesce(s.name, "") AS site_name, pd_rows
raw payload (15 fields)
{
"allocatedBytes": 19512.0,
"cpu": 1.0,
"elapsedTimeMs": 1.0,
"executionPlanCacheKeyHash": "3832EC22",
"id": "13356343",
"pageFaults": 0.0,
"pageHits": 36.0,
"planning": 0.0,
"query": "\nUNWIND $site_ids AS sid\nOPTIONAL MATCH (s:Site {tildaid: sid, delpending: false})\nOPTIONAL MATCH (s)\u003c-[:PLAN_DEFINITION_TO_SITE]-(pd:DslPlanDefinition {delpending: false})-[:PLAN_DEFINITION_TO_STUDY]-\u003e(st:Study {tildaid: $study_id, delpending: false})\nOPTIONAL MATCH (pd)-[pdc:PLAN_DEFINITION_TO_CONTACT]-\u003e(actor:Actor {delpending: false})-[:FOR_USER]-\u003e(u:User)\nOPTIONAL MATCH (pd)-[copdc:PLAN_DEFINITION_TO_CLINOP_CONTACT]-\u003e(co:ClinOp {deltimestamp: 0})-[:FOR_USER]-\u003e(cou:User)\nWITH sid, s,\n collect(DISTINCT CASE\n WHEN pdc IS NOT NULL AND actor IS NOT NULL AND (u IS NULL OR u.status \u003c\u003e $deleted_status) THEN {\n t: pdc.type,\n p: coalesce(pdc.isPrimary, false),\n e: coalesce(u.email, \"\")\n } END) AS actorContactRows,\n collect(DISTINCT CASE\n WHEN copdc IS NOT NULL AND co IS NOT NULL AND (cou IS NULL OR cou.status \u003c\u003e $deleted_status) THEN {\n t: copdc.type,\n p: coalesce(copdc.isPrimary, false),\n e: coalesce(cou.email, \"\")\n } END) AS clinopContactRows\nWITH sid, s, actorContactRows + clinopContactRows AS pdContactRowsRaw\nWITH sid, s,\n [row IN pdContactRowsRaw WHERE row IS NOT NULL AND row.t IS NOT NULL AND row.e IS NOT NULL AND row.e \u003c\u003e \"\"] AS pd_rows\nRETURN sid AS site_id, coalesce(s.name, \"\") AS site_name, pd_rows\n",
"queryParameters": "{deleted_status: \u0027Deleted\u0027, site_ids: [\u00279c081cd2-085b-4c6e-851e-abf432f301b4\u0027], study_id: \u00270a62f219-a803-482f-8973-ab67b1a3baf7\u0027}",
"runtime": "pipelined",
"source": "bolt-session\tbolt\tGo Driver/5.20.0\t\tclient/10.129.4.159:37136\tserver/10.173.0.201:7687\u003e",
"timestamp": "2026-07-27T00:07:18.392Z",
"transactionId": 1750707.0,
"waiting": 0.0
}32s ago2026-07-27 00:07:18 UTC neo4j workplansvc MATCH (instance:WorkPlanInstance{tildaid:"4102804a-71a2-48ef-8acf-e1d9a4321996"})-[:WORKPLAN_TO_SITE]->(site:Site) RETURN site.tildaid as siteId
time2026-07-27 00:07:18 UTC
severityINFO
serviceworkplansvc
clustermaster
podserver-3-dev-0
query
MATCH (instance:WorkPlanInstance{tildaid:"4102804a-71a2-48ef-8acf-e1d9a4321996"})-[:WORKPLAN_TO_SITE]->(site:Site)
RETURN site.tildaid as siteId
raw payload (15 fields)
{
"allocatedBytes": 328.0,
"cpu": 5.0,
"elapsedTimeMs": 5.0,
"executionPlanCacheKeyHash": "9B774BFC",
"id": "13356342",
"pageFaults": 0.0,
"pageHits": 416.0,
"planning": 4.0,
"query": "\n\t\tMATCH (instance:WorkPlanInstance{tildaid:\"4102804a-71a2-48ef-8acf-e1d9a4321996\"})-[:WORKPLAN_TO_SITE]-\u003e(site:Site)\n\t\tRETURN site.tildaid as siteId\n\t",
"queryParameters": "{}",
"runtime": "pipelined",
"source": "bolt-session\tbolt\tGo Driver/5.20.0\t\tclient/10.129.4.159:37136\tserver/10.173.0.201:7687\u003e",
"timestamp": "2026-07-27T00:07:18.386Z",
"transactionId": 1750706.0,
"waiting": 0.0
}32s ago2026-07-27 00:07:18 UTC neo4j workplansvc MATCH (cfg:WorkplanCfg{tildaid:"a9f727b2-f56d-4267-b30f-8859577d46a9"})-[:WORKPLAN_CFG_TO_STUDY]->(study:Study) RETURN study.tildaid as studyId
time2026-07-27 00:07:18 UTC
severityINFO
serviceworkplansvc
clustermaster
podserver-3-dev-0
query
MATCH (cfg:WorkplanCfg{tildaid:"a9f727b2-f56d-4267-b30f-8859577d46a9"})-[:WORKPLAN_CFG_TO_STUDY]->(study:Study)
RETURN study.tildaid as studyId
raw payload (15 fields)
{
"allocatedBytes": 328.0,
"cpu": 1.0,
"elapsedTimeMs": 1.0,
"executionPlanCacheKeyHash": "682A7B96",
"id": "13356341",
"pageFaults": 0.0,
"pageHits": 329.0,
"planning": 0.0,
"query": "\n\t\tMATCH (cfg:WorkplanCfg{tildaid:\"a9f727b2-f56d-4267-b30f-8859577d46a9\"})-[:WORKPLAN_CFG_TO_STUDY]-\u003e(study:Study)\n\t\tRETURN study.tildaid as studyId\n\t",
"queryParameters": "{}",
"runtime": "pipelined",
"source": "bolt-session\tbolt\tGo Driver/5.20.0\t\tclient/10.129.4.159:37136\tserver/10.173.0.201:7687\u003e",
"timestamp": "2026-07-27T00:07:18.375Z",
"transactionId": 1750705.0,
"waiting": 0.0
}32s ago2026-07-27 00:07:18 UTC neo4j workplansvc UNWIND $site_ids AS sid OPTIONAL MATCH (s:Site {tildaid: sid, delpending: false}) OPTIONAL MATCH...
time2026-07-27 00:07:18 UTC
severityINFO
serviceworkplansvc
clustermaster
podserver-3-dev-0
query
UNWIND $site_ids AS sid
OPTIONAL MATCH (s:Site {tildaid: sid, delpending: false})
OPTIONAL MATCH (s)<-[:PLAN_DEFINITION_TO_SITE]-(pd:DslPlanDefinition {delpending: false})-[:PLAN_DEFINITION_TO_STUDY]->(st:Study {tildaid: $study_id, delpending: false})
OPTIONAL MATCH (pd)-[pdc:PLAN_DEFINITION_TO_CONTACT]->(actor:Actor {delpending: false})-[:FOR_USER]->(u:User)
OPTIONAL MATCH (pd)-[copdc:PLAN_DEFINITION_TO_CLINOP_CONTACT]->(co:ClinOp {deltimestamp: 0})-[:FOR_USER]->(cou:User)
WITH sid, s,
collect(DISTINCT CASE
WHEN pdc IS NOT NULL AND actor IS NOT NULL AND (u IS NULL OR u.status <> $deleted_status) THEN {
t: pdc.type,
p: coalesce(pdc.isPrimary, false),
e: coalesce(u.email, "")
} END) AS actorContactRows,
collect(DISTINCT CASE
WHEN copdc IS NOT NULL AND co IS NOT NULL AND (cou IS NULL OR cou.status <> $deleted_status) THEN {
t: copdc.type,
p: coalesce(copdc.isPrimary, false),
e: coalesce(cou.email, "")
} END) AS clinopContactRows
WITH sid, s, actorContactRows + clinopContactRows AS pdContactRowsRaw
WITH sid, s,
[row IN pdContactRowsRaw WHERE row IS NOT NULL AND row.t IS NOT NULL AND row.e IS NOT NULL AND row.e <> ""] AS pd_rows
RETURN sid AS site_id, coalesce(s.name, "") AS site_name, pd_rows
raw payload (15 fields)
{
"allocatedBytes": 19512.0,
"cpu": 1.0,
"elapsedTimeMs": 1.0,
"executionPlanCacheKeyHash": "3832EC22",
"id": "13356340",
"pageFaults": 0.0,
"pageHits": 22.0,
"planning": 0.0,
"query": "\nUNWIND $site_ids AS sid\nOPTIONAL MATCH (s:Site {tildaid: sid, delpending: false})\nOPTIONAL MATCH (s)\u003c-[:PLAN_DEFINITION_TO_SITE]-(pd:DslPlanDefinition {delpending: false})-[:PLAN_DEFINITION_TO_STUDY]-\u003e(st:Study {tildaid: $study_id, delpending: false})\nOPTIONAL MATCH (pd)-[pdc:PLAN_DEFINITION_TO_CONTACT]-\u003e(actor:Actor {delpending: false})-[:FOR_USER]-\u003e(u:User)\nOPTIONAL MATCH (pd)-[copdc:PLAN_DEFINITION_TO_CLINOP_CONTACT]-\u003e(co:ClinOp {deltimestamp: 0})-[:FOR_USER]-\u003e(cou:User)\nWITH sid, s,\n collect(DISTINCT CASE\n WHEN pdc IS NOT NULL AND actor IS NOT NULL AND (u IS NULL OR u.status \u003c\u003e $deleted_status) THEN {\n t: pdc.type,\n p: coalesce(pdc.isPrimary, false),\n e: coalesce(u.email, \"\")\n } END) AS actorContactRows,\n collect(DISTINCT CASE\n WHEN copdc IS NOT NULL AND co IS NOT NULL AND (cou IS NULL OR cou.status \u003c\u003e $deleted_status) THEN {\n t: copdc.type,\n p: coalesce(copdc.isPrimary, false),\n e: coalesce(cou.email, \"\")\n } END) AS clinopContactRows\nWITH sid, s, actorContactRows + clinopContactRows AS pdContactRowsRaw\nWITH sid, s,\n [row IN pdContactRowsRaw WHERE row IS NOT NULL AND row.t IS NOT NULL AND row.e IS NOT NULL AND row.e \u003c\u003e \"\"] AS pd_rows\nRETURN sid AS site_id, coalesce(s.name, \"\") AS site_name, pd_rows\n",
"queryParameters": "{deleted_status: \u0027Deleted\u0027, site_ids: [\u0027cec6713e-0087-4942-91db-316352bbca2c\u0027], study_id: \u00275b971f19-ded8-4c5d-8649-5b2a96841d67\u0027}",
"runtime": "pipelined",
"source": "bolt-session\tbolt\tGo Driver/5.20.0\t\tclient/10.129.4.159:37136\tserver/10.173.0.201:7687\u003e",
"timestamp": "2026-07-27T00:07:18.369Z",
"transactionId": 1750704.0,
"waiting": 0.0
}32s ago2026-07-27 00:07:18 UTC neo4j workplansvc MATCH (instance:WorkPlanInstance{tildaid:"4104335e-49f9-4396-a635-7a4724b2c2cf"})-[:WORKPLAN_TO_SITE]->(site:Site) RETURN site.tildaid as siteId
time2026-07-27 00:07:18 UTC
severityINFO
serviceworkplansvc
clustermaster
podserver-3-dev-0
query
MATCH (instance:WorkPlanInstance{tildaid:"4104335e-49f9-4396-a635-7a4724b2c2cf"})-[:WORKPLAN_TO_SITE]->(site:Site)
RETURN site.tildaid as siteId
raw payload (15 fields)
{
"allocatedBytes": 328.0,
"cpu": 1.0,
"elapsedTimeMs": 1.0,
"executionPlanCacheKeyHash": "9B774BFC",
"id": "13356339",
"pageFaults": 0.0,
"pageHits": 415.0,
"planning": 0.0,
"query": "\n\t\tMATCH (instance:WorkPlanInstance{tildaid:\"4104335e-49f9-4396-a635-7a4724b2c2cf\"})-[:WORKPLAN_TO_SITE]-\u003e(site:Site)\n\t\tRETURN site.tildaid as siteId\n\t",
"queryParameters": "{}",
"runtime": "pipelined",
"source": "bolt-session\tbolt\tGo Driver/5.20.0\t\tclient/10.129.4.159:37136\tserver/10.173.0.201:7687\u003e",
"timestamp": "2026-07-27T00:07:18.362Z",
"transactionId": 1750703.0,
"waiting": 0.0
}32s ago2026-07-27 00:07:18 UTC neo4j workplansvc MATCH (cfg:WorkplanCfg{tildaid:"0590e24b-65f6-4c37-b293-8899478c233a"})-[:WORKPLAN_CFG_TO_STUDY]->(study:Study) RETURN study.tildaid as studyId
time2026-07-27 00:07:18 UTC
severityINFO
serviceworkplansvc
clustermaster
podserver-3-dev-0
query
MATCH (cfg:WorkplanCfg{tildaid:"0590e24b-65f6-4c37-b293-8899478c233a"})-[:WORKPLAN_CFG_TO_STUDY]->(study:Study)
RETURN study.tildaid as studyId
raw payload (15 fields)
{
"allocatedBytes": 328.0,
"cpu": 4.0,
"elapsedTimeMs": 5.0,
"executionPlanCacheKeyHash": "682A7B96",
"id": "13356338",
"pageFaults": 0.0,
"pageHits": 329.0,
"planning": 4.0,
"query": "\n\t\tMATCH (cfg:WorkplanCfg{tildaid:\"0590e24b-65f6-4c37-b293-8899478c233a\"})-[:WORKPLAN_CFG_TO_STUDY]-\u003e(study:Study)\n\t\tRETURN study.tildaid as studyId\n\t",
"queryParameters": "{}",
"runtime": "pipelined",
"source": "bolt-session\tbolt\tGo Driver/5.20.0\t\tclient/10.129.4.159:37136\tserver/10.173.0.201:7687\u003e",
"timestamp": "2026-07-27T00:07:18.356Z",
"transactionId": 1750702.0,
"waiting": 0.0
}32s ago2026-07-27 00:07:18 UTC neo4j workplansvc MATCH...
time2026-07-27 00:07:18 UTC
severityINFO
serviceworkplansvc
clustermaster
podserver-3-dev-0
query
MATCH (cfg:WorkplanCfg)-[:WORKPLAN_CFG_TO_INSTANCE]->(wp:WorkPlanInstance)-[:WORKPLAN_INSTANCE_TO_MILESTONE]->(m:WorkPlanMilestone)-[:REL_WORKPLAN_INSTANCE_MILESTONE_TO_ITEM]->(item:WorkplanInstanceMilestoneItem) WHERE item.itemType = "Workorder" AND coalesce(item.workOrderActivationFailedAt, 0) > 0 AND coalesce(item.workOrderActivationErrorCode, "") = "NO_RECIPIENTS_FOR_SITE_CONTACT_CATEGORY" AND coalesce(item.workOrderCompletedAt, 0) = 0 AND coalesce(item.workOrderResolvedAt, 0) = 0 AND coalesce(item.skippedAt, 0) = 0 AND coalesce(item.notApplicableAt, 0) = 0 AND coalesce(item.workOrderId, "") <> "" RETURN cfg.tildaid AS cfgId, wp.tildaid AS instanceId, item.tildaid AS itemId, item.workOrderId AS workOrderId, coalesce(item.itemConfig, "") AS itemConfig
raw payload (15 fields)
{
"allocatedBytes": 336.0,
"cpu": 6.0,
"elapsedTimeMs": 7.0,
"executionPlanCacheKeyHash": "C9113EA6",
"id": "13356337",
"pageFaults": 0.0,
"pageHits": 7311.0,
"planning": 0.0,
"query": "\n\t\tMATCH (cfg:WorkplanCfg)-[:WORKPLAN_CFG_TO_INSTANCE]-\u003e(wp:WorkPlanInstance)-[:WORKPLAN_INSTANCE_TO_MILESTONE]-\u003e(m:WorkPlanMilestone)-[:REL_WORKPLAN_INSTANCE_MILESTONE_TO_ITEM]-\u003e(item:WorkplanInstanceMilestoneItem)\n\t\tWHERE item.itemType = \"Workorder\"\n\t\t AND coalesce(item.workOrderActivationFailedAt, 0) \u003e 0\n\t\t AND coalesce(item.workOrderActivationErrorCode, \"\") = \"NO_RECIPIENTS_FOR_SITE_CONTACT_CATEGORY\"\n\t\t AND coalesce(item.workOrderCompletedAt, 0) = 0\n\t\t AND coalesce(item.workOrderResolvedAt, 0) = 0\n\t\t AND coalesce(item.skippedAt, 0) = 0 AND coalesce(item.notApplicableAt, 0) = 0\n\t\t AND coalesce(item.workOrderId, \"\") \u003c\u003e \"\"\n\t\tRETURN cfg.tildaid AS cfgId, wp.tildaid AS instanceId, item.tildaid AS itemId,\n\t\t item.workOrderId AS workOrderId, coalesce(item.itemConfig, \"\") AS itemConfig\n\t",
"queryParameters": "{}",
"runtime": "pipelined",
"source": "bolt-session\tbolt\tGo Driver/5.20.0\t\tclient/10.129.4.159:37136\tserver/10.173.0.201:7687\u003e",
"timestamp": "2026-07-27T00:07:18.345Z",
"transactionId": 1750701.0,
"waiting": 0.0
}32s ago2026-07-27 00:07:18 UTC neo4j workplansvc MATCH...
time2026-07-27 00:07:18 UTC
severityINFO
serviceworkplansvc
clustermaster
podserver-3-dev-0
query
MATCH (wp:WorkPlanInstance)-[:WORKPLAN_INSTANCE_TO_MILESTONE]->(:WorkPlanMilestone)-[:REL_WORKPLAN_INSTANCE_MILESTONE_TO_ITEM]->(item:WorkplanInstanceMilestoneItem) WHERE item.itemType = "Workorder" AND wp.state <> "archived" AND coalesce(item.workOrderId, "") <> "" AND coalesce(item.activationPending, false) = true AND coalesce(item.activationEligible, false) = true AND coalesce(item.activateRequestedAt, 0) = 0 RETURN item.tildaid AS id LIMIT 80
raw payload (15 fields)
{
"allocatedBytes": 336.0,
"cpu": 16.0,
"elapsedTimeMs": 16.0,
"executionPlanCacheKeyHash": "32028411",
"id": "13356336",
"pageFaults": 0.0,
"pageHits": 7580.0,
"planning": 9.0,
"query": "\n\t\tMATCH (wp:WorkPlanInstance)-[:WORKPLAN_INSTANCE_TO_MILESTONE]-\u003e(:WorkPlanMilestone)-[:REL_WORKPLAN_INSTANCE_MILESTONE_TO_ITEM]-\u003e(item:WorkplanInstanceMilestoneItem)\n\t\tWHERE item.itemType = \"Workorder\"\n\t\t AND wp.state \u003c\u003e \"archived\"\n\t\t AND coalesce(item.workOrderId, \"\") \u003c\u003e \"\"\n\t\t AND coalesce(item.activationPending, false) = true\n\t\t AND coalesce(item.activationEligible, false) = true\n\t\t AND coalesce(item.activateRequestedAt, 0) = 0\n\t\tRETURN item.tildaid AS id\n\t\tLIMIT 80\n\t",
"queryParameters": "{}",
"runtime": "pipelined",
"source": "bolt-session\tbolt\tGo Driver/5.20.0\t\tclient/10.129.4.159:37136\tserver/10.173.0.201:7687\u003e",
"timestamp": "2026-07-27T00:07:18.332Z",
"transactionId": 1750700.0,
"waiting": 0.0
}32s ago2026-07-27 00:07:18 UTC DEBUG workplansvc TransactionID commit
time2026-07-27 00:07:18 UTC
severityDEBUG
serviceworkplansvc
sourcetildadb/api.go:2575
clustermaster
versionunknown
podworkplansvc-75f7d8dcb6-jxnsg
message
TransactionID commit
raw payload (8 fields)
{
"caller": "tildadb/api.go:2575",
"cluster": "master",
"environment": "DEVELOPMENT",
"message": "TransactionID commit",
"service": "workplansvc",
"timestamp": "2026-07-27T00:07:18.313Z",
"transactionID": "FB:kcwQc8qrjECqRtOxBB1HjVmAmsoAPpdjkA==",
"version": "unknown"
}32s ago2026-07-27 00:07:18 UTC neo4j workplansvc RETURN EXISTS { MATCH (inst:WorkPlanInstance) WHERE coalesce(inst.progressSummariesPending, false) = true } AS progressPending, ...
time2026-07-27 00:07:18 UTC
severityINFO
serviceworkplansvc
clustermaster
podserver-2-dev-0
query
RETURN EXISTS {
MATCH (inst:WorkPlanInstance)
WHERE coalesce(inst.progressSummariesPending, false) = true
} AS progressPending,
EXISTS {
MATCH (item:WorkplanInstanceMilestoneItem)
WHERE item.itemType = "Workorder"
AND coalesce(item.workOrderId, "") <> ""
AND coalesce(item.activationPending, false) = true
AND coalesce(item.activationEligible, false) = true
AND coalesce(item.activateRequestedAt, 0) = 0
} AS activationPending,
EXISTS {
MATCH (cfg:WorkplanCfg)
WHERE coalesce(cfg.workOrderEmailAutoSendTransitionPending, false) = true
} AS autosendTransition,
EXISTS {
MATCH (cfg:WorkplanCfg)-[:WORKPLAN_CFG_TO_INSTANCE]->(wp:WorkPlanInstance)-[:WORKPLAN_INSTANCE_TO_MILESTONE]->(:WorkPlanMilestone)-[:REL_WORKPLAN_INSTANCE_MILESTONE_TO_ITEM]->(item:WorkplanInstanceMilestoneItem)
WHERE item.itemType = "Workorder"
AND coalesce(item.workOrderActivationFailedAt, 0) > 0
AND coalesce(item.workOrderActivationErrorCode, "") = "NO_RECIPIENTS_FOR_SITE_CONTACT_CATEGORY"
AND coalesce(item.workOrderCompletedAt, 0) = 0
AND coalesce(item.workOrderResolvedAt, 0) = 0
AND coalesce(item.skippedAt, 0) = 0
AND coalesce(item.notApplicableAt, 0) = 0
AND coalesce(item.workOrderId, "") <> ""
} AS recipientRetry,
EXISTS {
MATCH (wp2:WorkPlanInstance)
WHERE wp2.instanceReadyDeterminationState = "pending"
} AS readyDetermination
raw payload (15 fields)
{
"allocatedBytes": 102656.0,
"cpu": 4.0,
"elapsedTimeMs": 5.0,
"executionPlanCacheKeyHash": "5CC92BFF",
"id": "24641277",
"pageFaults": 0.0,
"pageHits": 2298.0,
"planning": 0.0,
"query": "\n\t\tRETURN EXISTS {\n\t\t\tMATCH (inst:WorkPlanInstance)\n\t\t\tWHERE coalesce(inst.progressSummariesPending, false) = true\n\t\t} AS progressPending,\n\t\t EXISTS {\n\t\t MATCH (item:WorkplanInstanceMilestoneItem)\n\t\t WHERE item.itemType = \"Workorder\"\n\t\t AND coalesce(item.workOrderId, \"\") \u003c\u003e \"\"\n\t\t AND coalesce(item.activationPending, false) = true\n\t\t AND coalesce(item.activationEligible, false) = true\n\t\t AND coalesce(item.activateRequestedAt, 0) = 0\n\t\t } AS activationPending,\n\t\t EXISTS {\n\t\t MATCH (cfg:WorkplanCfg)\n\t\t WHERE coalesce(cfg.workOrderEmailAutoSendTransitionPending, false) = true\n\t\t } AS autosendTransition,\n\t\t EXISTS {\n\t\t MATCH (cfg:WorkplanCfg)-[:WORKPLAN_CFG_TO_INSTANCE]-\u003e(wp:WorkPlanInstance)-[:WORKPLAN_INSTANCE_TO_MILESTONE]-\u003e(:WorkPlanMilestone)-[:REL_WORKPLAN_INSTANCE_MILESTONE_TO_ITEM]-\u003e(item:WorkplanInstanceMilestoneItem)\n\t\t WHERE item.itemType = \"Workorder\"\n\t\t AND coalesce(item.workOrderActivationFailedAt, 0) \u003e 0\n\t\t AND coalesce(item.workOrderActivationErrorCode, \"\") = \"NO_RECIPIENTS_FOR_SITE_CONTACT_CATEGORY\"\n\t\t AND coalesce(item.workOrderCompletedAt, 0) = 0\n\t\t AND coalesce(item.workOrderResolvedAt, 0) = 0\n\t\t AND coalesce(item.skippedAt, 0) = 0\n\t\t AND coalesce(item.notApplicableAt, 0) = 0\n\t\t AND coalesce(item.workOrderId, \"\") \u003c\u003e \"\"\n\t\t } AS recipientRetry,\n\t\t EXISTS {\n\t\t MATCH (wp2:WorkPlanInstance)\n\t\t WHERE wp2.instanceReadyDeterminationState = \"pending\"\n\t\t } AS readyDetermination\n\t",
"queryParameters": "{}",
"runtime": "pipelined",
"source": "bolt-session\tbolt\tneo4j-java/5.28.13-24cd0fecc64de123fd5711190246e8b0f1e16fd0\t\tclient/10.173.0.201:39990\tserver/10.173.0.8:7688\u003e",
"timestamp": "2026-07-27T00:07:18.308Z",
"transactionId": 12025339.0,
"waiting": 0.0
}32s ago2026-07-27 00:07:18 UTC DEBUG workplansvc TransactionID
time2026-07-27 00:07:18 UTC
severityDEBUG
serviceworkplansvc
sourcetildadb/api.go:2482
clustermaster
versionunknown
podworkplansvc-75f7d8dcb6-jxnsg
message
TransactionID
raw payload (8 fields)
{
"caller": "tildadb/api.go:2482",
"cluster": "master",
"environment": "DEVELOPMENT",
"message": "TransactionID",
"service": "workplansvc",
"timestamp": "2026-07-27T00:07:18.301Z",
"transactionID": "",
"version": "unknown"
}33s ago2026-07-27 00:07:17 UTC INFO [litellm-bootstrap] ensured vkey svc-workplan-agent
filter by:
severity≥INFO
time2026-07-27 00:07:17 UTC
severityINFO
clustermaster
podlitellm-f54655c99-cbmvt
message
[litellm-bootstrap] ensured vkey svc-workplan-agent
33s ago2026-07-27 00:07:17 UTC INFO [litellm-bootstrap] ensured vkey svc-studioaisvc
filter by:
severity≥INFO
time2026-07-27 00:07:17 UTC
severityINFO
clustermaster
podlitellm-f54655c99-cbmvt
message
[litellm-bootstrap] ensured vkey svc-studioaisvc
33s ago2026-07-27 00:07:17 UTC INFO [litellm-bootstrap] ensured vkey svc-virtual-assistant
filter by:
severity≥INFO
time2026-07-27 00:07:17 UTC
severityINFO
clustermaster
podlitellm-f54655c99-cbmvt
message
[litellm-bootstrap] ensured vkey svc-virtual-assistant
33s ago2026-07-27 00:07:17 UTC INFO [litellm-bootstrap] ensured vkey svc-rag-agent
filter by:
severity≥INFO
time2026-07-27 00:07:17 UTC
severityINFO
clustermaster
podlitellm-f54655c99-cbmvt
message
[litellm-bootstrap] ensured vkey svc-rag-agent
33s ago2026-07-27 00:07:17 UTC INFO [litellm-bootstrap] ensured vkey svc-docproc
filter by:
severity≥INFO
time2026-07-27 00:07:17 UTC
severityINFO
clustermaster
podlitellm-f54655c99-cbmvt
message
[litellm-bootstrap] ensured vkey svc-docproc
33s ago2026-07-27 00:07:17 UTC INFO [litellm-bootstrap] ensured vkey svc-ontologysvc
filter by:
severity≥INFO
time2026-07-27 00:07:17 UTC
severityINFO
clustermaster
podlitellm-f54655c99-cbmvt
message
[litellm-bootstrap] ensured vkey svc-ontologysvc
33s ago2026-07-27 00:07:17 UTC INFO [litellm-bootstrap] ensured vkey svc-chat-agent
filter by:
severity≥INFO
time2026-07-27 00:07:17 UTC
severityINFO
clustermaster
podlitellm-f54655c99-cbmvt
message
[litellm-bootstrap] ensured vkey svc-chat-agent
34s ago2026-07-27 00:07:16 UTC neo4j advpsvc match(a:AdvRateRule{deleted:true})-[r]-(b) with distinct a return a.portalorgid, a.tildaid, a.ruletype limit 10
time2026-07-27 00:07:16 UTC
severityINFO
serviceadvpsvc
clustermaster
podserver-1-dev-0
query
match(a:AdvRateRule{deleted:true})-[r]-(b)
with distinct a
return a.portalorgid, a.tildaid, a.ruletype
limit 10
raw payload (15 fields)
{
"allocatedBytes": 328.0,
"cpu": 2.0,
"elapsedTimeMs": 2.0,
"executionPlanCacheKeyHash": "AFAFBFFB",
"id": "17372789",
"pageFaults": 0.0,
"pageHits": 1146.0,
"planning": 0.0,
"query": "match(a:AdvRateRule{deleted:true})-[r]-(b) with distinct a return a.portalorgid, a.tildaid, a.ruletype limit 10",
"queryParameters": "{}",
"runtime": "pipelined",
"source": "bolt-session\tbolt\tGo Driver/5.20.0\t\tclient/10.129.3.19:37624\tserver/10.173.2.71:7687\u003e",
"timestamp": "2026-07-27T00:07:16.947Z",
"transactionId": 1771015.0,
"waiting": 0.0
}34s ago2026-07-27 00:07:16 UTC ERROR observability: level watcher could not stat config file /etc/config/observability.yaml: stat /etc/config/observability.yaml: no such file or directory
filter by:
severity≥ERROR
time2026-07-27 00:07:16 UTC
severityERROR
clustermaster
podworkplansvc-75f7d8dcb6-jxnsg
message
observability: level watcher could not stat config file /etc/config/observability.yaml: stat /etc/config/observability.yaml: no such file or directory
34s ago2026-07-27 00:07:16 UTC ERROR observability: level watcher could not stat config file /etc/config/observability.yaml: stat /etc/config/observability.yaml: no such file or directory
filter by:
severity≥ERROR
time2026-07-27 00:07:16 UTC
severityERROR
clustermaster
podstudiosvc-7649ffd596-dwzsc
message
observability: level watcher could not stat config file /etc/config/observability.yaml: stat /etc/config/observability.yaml: no such file or directory
34s ago2026-07-27 00:07:16 UTC DEBUG docproc Reported pending workload
time2026-07-27 00:07:16 UTC
severityDEBUG
servicedocproc
sourcejobs/hpa_pending_workload.py:176
clustermaster
version0.1.0
poddocproc-7bc76669-ljh8h
message
Reported pending workload
raw payload (15 fields)
{
"backlog_capped": 0.0,
"caller": "jobs/hpa_pending_workload.py:176",
"cluster": "master",
"component": "hpa_pending_workload",
"environment": "DEVELOPMENT",
"in_progress_count": 0.0,
"message": "Reported pending workload",
"num_replicas": 1.0,
"recovery_workload": 0.0,
"service": "docproc",
"signal": 0.0,
"timestamp": "2026-07-27T00:07:16.163Z",
"total_workload": 0.0,
"unacked": 0.0,
"version": "0.1.0"
}34s ago2026-07-27 00:07:16 UTC DEBUG docproc Pub/Sub num_unacked result: project_id=tilda-trial-automation subscription_id_in_filter=tilda-document-master-docproc-sub num_series=1 total=0
time2026-07-27 00:07:16 UTC
severityDEBUG
servicedocproc
sourcejobs/cloud_monitoring_metrics.py:296
clustermaster
version0.1.0
poddocproc-7bc76669-ljh8h
message
Pub/Sub num_unacked result: project_id=tilda-trial-automation subscription_id_in_filter=tilda-document-master-docproc-sub num_series=1 total=0
raw payload (7 fields)
{
"caller": "jobs/cloud_monitoring_metrics.py:296",
"cluster": "master",
"environment": "DEVELOPMENT",
"message": "Pub/Sub num_unacked result: project_id=tilda-trial-automation subscription_id_in_filter=tilda-document-master-docproc-sub num_series=1 total=0",
"service": "docproc",
"timestamp": "2026-07-27T00:07:16.097Z",
"version": "0.1.0"
}35s ago2026-07-27 00:07:15 UTC neo4j docproc MATCH (o:Organization{orgType:"CRO"})-[:DOC_TO_ORG]-(d:Document)-[:INCLUDES]-(f:Folder) WHERE f.name IN ['Staging', 'Pre-Staging'] ...
time2026-07-27 00:07:15 UTC
severityINFO
servicedocproc
clustermaster
podserver-2-dev-0
query
MATCH (o:Organization{orgType:"CRO"})-[:DOC_TO_ORG]-(d:Document)-[:INCLUDES]-(f:Folder)
WHERE f.name IN ['Staging', 'Pre-Staging']
AND d.delpending = false AND d.aiEnabled = true AND d.studyId <> ""
AND d.primaryVersionId IS NOT NULL AND d.primaryVersionId <> ""
OPTIONAL MATCH (d)<-[:AI_RUN_TO_DOCUMENT_LATEST]-(ai:AIRun)
WITH o, d, ai
ORDER BY d.tildaid, COALESCE(ai.updatedat, ai.createdat) DESC
WITH o, d, collect(ai)[0] AS latest_ai
WHERE (
latest_ai IS NULL
OR latest_ai.status = "aborted"
OR (
latest_ai.status = "failed"
AND COALESCE(latest_ai.auto_retry_count, 0) < $max_retries
AND COALESCE(latest_ai.updatedat, latest_ai.createdat) < $threshold_ms
)
)
WITH d
RETURN count(d) AS total
raw payload (15 fields)
{
"allocatedBytes": 7666360.0,
"cpu": 171.0,
"elapsedTimeMs": 171.0,
"executionPlanCacheKeyHash": "590286C5",
"id": "24641275",
"pageFaults": 0.0,
"pageHits": 156362.0,
"planning": 0.0,
"query": "\n MATCH (o:Organization{orgType:\"CRO\"})-[:DOC_TO_ORG]-(d:Document)-[:INCLUDES]-(f:Folder)\n WHERE f.name IN [\u0027Staging\u0027, \u0027Pre-Staging\u0027]\n AND d.delpending = false AND d.aiEnabled = true AND d.studyId \u003c\u003e \"\"\n AND d.primaryVersionId IS NOT NULL AND d.primaryVersionId \u003c\u003e \"\"\n OPTIONAL MATCH (d)\u003c-[:AI_RUN_TO_DOCUMENT_LATEST]-(ai:AIRun)\n WITH o, d, ai\n ORDER BY d.tildaid, COALESCE(ai.updatedat, ai.createdat) DESC\n WITH o, d, collect(ai)[0] AS latest_ai\n WHERE (\n latest_ai IS NULL\n OR latest_ai.status = \"aborted\"\n OR (\n latest_ai.status = \"failed\"\n AND COALESCE(latest_ai.auto_retry_count, 0) \u003c $max_retries\n AND COALESCE(latest_ai.updatedat, latest_ai.createdat) \u003c $threshold_ms\n )\n )\n WITH d\n RETURN count(d) AS total\n ",
"queryParameters": "{max_retries: 5, threshold_ms: 1785109935708}",
"runtime": "pipelined",
"source": "bolt-session\tbolt\tneo4j-python/6.2.0 Python/3.12.13-final-0 (linux)\t\tclient/10.129.3.15:59818\tserver/10.173.0.8:7687\u003e",
"timestamp": "2026-07-27T00:07:15.882Z",
"transactionId": 12025338.0,
"waiting": 0.0
}35s ago2026-07-27 00:07:15 UTC neo4j docproc MATCH (ai:AIRun)-[:AI_RUN_TO_DOCUMENT_LATEST]->(d:Document) WHERE d.aiEnabled = true AND ( (ai.status = "in_progress" AND...
time2026-07-27 00:07:15 UTC
severityINFO
servicedocproc
clustermaster
podserver-2-dev-0
query
MATCH (ai:AIRun)-[:AI_RUN_TO_DOCUMENT_LATEST]->(d:Document)
WHERE d.aiEnabled = true
AND (
(ai.status = "in_progress" AND ($cutoff_ms <= 0 OR COALESCE(ai.updatedat, ai.createdat) < $cutoff_ms))
OR ai.status = "aborted"
)
RETURN count(ai) AS total
raw payload (15 fields)
{
"allocatedBytes": 488.0,
"cpu": 64.0,
"elapsedTimeMs": 64.0,
"executionPlanCacheKeyHash": "E4907905",
"id": "24641274",
"pageFaults": 0.0,
"pageHits": 66825.0,
"planning": 0.0,
"query": "\n MATCH (ai:AIRun)-[:AI_RUN_TO_DOCUMENT_LATEST]-\u003e(d:Document)\n WHERE d.aiEnabled = true\n AND (\n (ai.status = \"in_progress\" AND ($cutoff_ms \u003c= 0 OR COALESCE(ai.updatedat, ai.createdat) \u003c $cutoff_ms))\n OR ai.status = \"aborted\"\n )\n RETURN count(ai) AS total\n ",
"queryParameters": "{cutoff_ms: 0}",
"runtime": "pipelined",
"source": "bolt-session\tbolt\tneo4j-python/6.2.0 Python/3.12.13-final-0 (linux)\t\tclient/10.129.3.15:59818\tserver/10.173.0.8:7687\u003e",
"timestamp": "2026-07-27T00:07:15.706Z",
"transactionId": 12025337.0,
"waiting": 0.0
}35s ago2026-07-27 00:07:15 UTC INFO
filter by:
severity≥INFO
time2026-07-27 00:07:15 UTC
severityINFO
clustermaster
podlitellm-f54655c99-tt2pg
35s ago2026-07-27 00:07:15 UTC INFO
filter by:
severity≥INFO
time2026-07-27 00:07:15 UTC
severityINFO
clustermaster
podrag-agent-57879dd94f-l9nz7
36s ago2026-07-27 00:07:14 UTC ERROR observability: level watcher could not stat config file /etc/config/observability.yaml: stat /etc/config/observability.yaml: no such file or directory
filter by:
severity≥ERROR
time2026-07-27 00:07:14 UTC
severityERROR
clustermaster
podschedsvc-786465b756-5smng
message
observability: level watcher could not stat config file /etc/config/observability.yaml: stat /etc/config/observability.yaml: no such file or directory
36s ago2026-07-27 00:07:14 UTC neo4j virtual-assistant RETURN 1 as health_check
time2026-07-27 00:07:14 UTC
severityINFO
servicevirtual-assistant
clustermaster
trace5d1bd1d520cac81d93cd7eb8f4cd7485
spanb5e565c8c705097a
podserver-2-dev-0
query
RETURN 1 as health_check
raw payload (15 fields)
{
"allocatedBytes": 312.0,
"cpu": 0.0,
"elapsedTimeMs": 0.0,
"executionPlanCacheKeyHash": "745CE4BF",
"id": "24641271",
"pageFaults": 0.0,
"pageHits": 0.0,
"planning": 0.0,
"query": "RETURN 1 as health_check",
"queryParameters": "{}",
"runtime": "pipelined",
"source": "bolt-session\tbolt\tneo4j-python/5.28.0 Python/3.12.13-final-0 (linux)\t\tclient/10.129.4.156:57418\tserver/10.173.0.8:7687\u003e",
"timestamp": "2026-07-27T00:07:14.227Z",
"transactionId": 12025335.0,
"waiting": 0.0
}36s ago2026-07-27 00:07:14 UTC neo4j virtual-assistant RETURN 1
time2026-07-27 00:07:14 UTC
severityINFO
servicevirtual-assistant
clustermaster
trace5d1bd1d520cac81d93cd7eb8f4cd7485
spanb5e565c8c705097a
podserver-2-dev-0
query
RETURN 1
raw payload (15 fields)
{
"allocatedBytes": 312.0,
"cpu": 0.0,
"elapsedTimeMs": 0.0,
"executionPlanCacheKeyHash": "4CADEF45",
"id": "24641270",
"pageFaults": 0.0,
"pageHits": 0.0,
"planning": 0.0,
"query": "RETURN 1",
"queryParameters": "{}",
"runtime": "pipelined",
"source": "bolt-session\tbolt\tneo4j-python/5.28.0 Python/3.12.13-final-0 (linux)\t\tclient/10.129.4.156:57418\tserver/10.173.0.8:7687\u003e",
"timestamp": "2026-07-27T00:07:14.223Z",
"transactionId": 12025334.0,
"waiting": 0.0
}36s ago2026-07-27 00:07:14 UTC DEBUG virtual-assistant Session attributes: ['_auto_result', '_bookmark_manager', '_bookmarks', '_check_state', '_closed', '_config', '_connect', '_connection',...
time2026-07-27 00:07:14 UTC
severityDEBUG
servicevirtual-assistant
sourcesrc/neo4j_client.py:850
clustermaster
versionunknown
trace5d1bd1d520cac81d93cd7eb8f4cd7485
spanb5e565c8c705097a
podvirtual-assistant-65b975d66b-l8wpw
message
Session attributes: ['_auto_result', '_bookmark_manager', '_bookmarks', '_check_state', '_closed', '_config', '_connect', '_connection', '_connection_access_mode', '_disconnect', '_get_bookmarks', '_get_routing_target_database', '_get_server_info', '_handle_cancellation', '_initial_bookmarks', '_initialize_bookmarks', '_last_cache_key', '_last_from_bookmark_manager', '_make_db_resolution_callback', '_non_concurrent_iter', '_non_concurrent_method', '_open_transaction', '_pinned_database', '_pipelined_begin', '_pool', '_resolve_session_auth', '_result_closed', '_result_error', '_run_transaction', '_set_pinned_database', '_state_failed', '_transaction', '_transaction_cancel_handler', '_transaction_closed_handler', '_transaction_error_handler', '_update_bookmark', '_update_bookmarks', '_verify_authentication', 'begin_transaction', 'close', 'closed', 'execute_read', 'execute_write', 'last_bookmark', 'last_bookmarks', 'read_transaction', 'run', 'write_transaction']
raw payload (9 fields)
{
"caller": "src/neo4j_client.py:850",
"cluster": "master",
"environment": "DEVELOPMENT",
"message": "Session attributes: [\u0027_auto_result\u0027, \u0027_bookmark_manager\u0027, \u0027_bookmarks\u0027, \u0027_check_state\u0027, \u0027_closed\u0027, \u0027_config\u0027, \u0027_connect\u0027, \u0027_connection\u0027, \u0027_connection_access_mode\u0027, \u0027_disconnect\u0027, \u0027_get_bookmarks\u0027, \u0027_get_routing_target_database\u0027, \u0027_get_server_info\u0027, \u0027_handle_cancellation\u0027, \u0027_initial_bookmarks\u0027, \u0027_initialize_bookmarks\u0027, \u0027_last_cache_key\u0027, \u0027_last_from_bookmark_manager\u0027, \u0027_make_db_resolution_callback\u0027, \u0027_non_concurrent_iter\u0027, \u0027_non_concurrent_method\u0027, \u0027_open_transaction\u0027, \u0027_pinned_database\u0027, \u0027_pipelined_begin\u0027, \u0027_pool\u0027, \u0027_resolve_session_auth\u0027, \u0027_result_closed\u0027, \u0027_result_error\u0027, \u0027_run_transaction\u0027, \u0027_set_pinned_database\u0027, \u0027_state_failed\u0027, \u0027_transaction\u0027, \u0027_transaction_cancel_handler\u0027, \u0027_transaction_closed_handler\u0027, \u0027_transaction_error_handler\u0027, \u0027_update_bookmark\u0027, \u0027_update_bookmarks\u0027, \u0027_verify_authentication\u0027, \u0027begin_transaction\u0027, \u0027close\u0027, \u0027closed\u0027, \u0027execute_read\u0027, \u0027execute_write\u0027, \u0027last_bookmark\u0027, \u0027last_bookmarks\u0027, \u0027read_transaction\u0027, \u0027run\u0027, \u0027write_transaction\u0027]",
"method": "GET",
"path": "/health",
"service": "virtual-assistant",
"timestamp": "2026-07-27T00:07:14.218Z",
"version": "unknown"
}36s ago2026-07-27 00:07:14 UTC DEBUG virtual-assistant Pool attributes: ['_abc_impl', '_acquire', '_acquire_from_pool', '_acquire_from_pool_checked', '_acquire_new_later', '_add_connections',...
time2026-07-27 00:07:14 UTC
severityDEBUG
servicevirtual-assistant
sourcesrc/neo4j_client.py:796
clustermaster
versionunknown
trace5d1bd1d520cac81d93cd7eb8f4cd7485
spanb5e565c8c705097a
podvirtual-assistant-65b975d66b-l8wpw
message
Pool attributes: ['_abc_impl', '_acquire', '_acquire_from_pool', '_acquire_from_pool_checked', '_acquire_new_later', '_add_connections', '_close_connections', '_re_auth_connection', '_remove_connection', '_remove_connections', '_select_address', '_ssr_feature_tracker', '_update_routing_table_from', 'acquire', 'address', 'close', 'cond', 'connections', 'connections_reservations', 'deactivate', 'ensure_routing_table_is_fresh', 'fetch_routing_info', 'fetch_routing_table', 'get_auth', 'get_or_create_routing_table', 'get_routing_table', 'home_db_cache', 'in_use_connection_count', 'is_direct_pool', 'kill_and_release', 'lock', 'mark_all_stale', 'on_neo4j_error', 'on_write_failure', 'open', 'opener', 'pool_config', 'refresh_lock', 'release', 'routing_tables', 'ssr_enabled', 'update_connection_pool', 'update_routing_table', 'workspace_config']
raw payload (9 fields)
{
"caller": "src/neo4j_client.py:796",
"cluster": "master",
"environment": "DEVELOPMENT",
"message": "Pool attributes: [\u0027_abc_impl\u0027, \u0027_acquire\u0027, \u0027_acquire_from_pool\u0027, \u0027_acquire_from_pool_checked\u0027, \u0027_acquire_new_later\u0027, \u0027_add_connections\u0027, \u0027_close_connections\u0027, \u0027_re_auth_connection\u0027, \u0027_remove_connection\u0027, \u0027_remove_connections\u0027, \u0027_select_address\u0027, \u0027_ssr_feature_tracker\u0027, \u0027_update_routing_table_from\u0027, \u0027acquire\u0027, \u0027address\u0027, \u0027close\u0027, \u0027cond\u0027, \u0027connections\u0027, \u0027connections_reservations\u0027, \u0027deactivate\u0027, \u0027ensure_routing_table_is_fresh\u0027, \u0027fetch_routing_info\u0027, \u0027fetch_routing_table\u0027, \u0027get_auth\u0027, \u0027get_or_create_routing_table\u0027, \u0027get_routing_table\u0027, \u0027home_db_cache\u0027, \u0027in_use_connection_count\u0027, \u0027is_direct_pool\u0027, \u0027kill_and_release\u0027, \u0027lock\u0027, \u0027mark_all_stale\u0027, \u0027on_neo4j_error\u0027, \u0027on_write_failure\u0027, \u0027open\u0027, \u0027opener\u0027, \u0027pool_config\u0027, \u0027refresh_lock\u0027, \u0027release\u0027, \u0027routing_tables\u0027, \u0027ssr_enabled\u0027, \u0027update_connection_pool\u0027, \u0027update_routing_table\u0027, \u0027workspace_config\u0027]",
"method": "GET",
"path": "/health",
"service": "virtual-assistant",
"timestamp": "2026-07-27T00:07:14.218Z",
"version": "unknown"
}36s ago2026-07-27 00:07:14 UTC DEBUG virtual-assistant Driver attributes: ['_check_state', '_closed', '_default_workspace_config', '_get_server_info', '_initial_addresses', '_pool',...
time2026-07-27 00:07:14 UTC
severityDEBUG
servicevirtual-assistant
sourcesrc/neo4j_client.py:787
clustermaster
versionunknown
trace5d1bd1d520cac81d93cd7eb8f4cd7485
spanb5e565c8c705097a
podvirtual-assistant-65b975d66b-l8wpw
message
Driver attributes: ['_check_state', '_closed', '_default_workspace_config', '_get_server_info', '_initial_addresses', '_pool', '_prepare_session_config', '_query_bookmark_manager', '_read_session_config', '_session', 'close', 'default_host', 'default_port', 'default_targets', 'encrypted', 'execute_query', 'execute_query_bookmark_manager', 'get_server_info', 'initial_addresses', 'open', 'parse_targets', 'session', 'supports_multi_db', 'supports_session_auth', 'verify_authentication', 'verify_connectivity']
raw payload (9 fields)
{
"caller": "src/neo4j_client.py:787",
"cluster": "master",
"environment": "DEVELOPMENT",
"message": "Driver attributes: [\u0027_check_state\u0027, \u0027_closed\u0027, \u0027_default_workspace_config\u0027, \u0027_get_server_info\u0027, \u0027_initial_addresses\u0027, \u0027_pool\u0027, \u0027_prepare_session_config\u0027, \u0027_query_bookmark_manager\u0027, \u0027_read_session_config\u0027, \u0027_session\u0027, \u0027close\u0027, \u0027default_host\u0027, \u0027default_port\u0027, \u0027default_targets\u0027, \u0027encrypted\u0027, \u0027execute_query\u0027, \u0027execute_query_bookmark_manager\u0027, \u0027get_server_info\u0027, \u0027initial_addresses\u0027, \u0027open\u0027, \u0027parse_targets\u0027, \u0027session\u0027, \u0027supports_multi_db\u0027, \u0027supports_session_auth\u0027, \u0027verify_authentication\u0027, \u0027verify_connectivity\u0027]",
"method": "GET",
"path": "/health",
"service": "virtual-assistant",
"timestamp": "2026-07-27T00:07:14.217Z",
"version": "unknown"
}36s ago2026-07-27 00:07:14 UTC DEBUG virtual-assistant 🔍 AUTH_DEBUG: Path in UNPROTECTED_ENDPOINTS: True
time2026-07-27 00:07:14 UTC
severityDEBUG
servicevirtual-assistant
sourceapi/api.py:491
clustermaster
versionunknown
trace5d1bd1d520cac81d93cd7eb8f4cd7485
spanb5e565c8c705097a
podvirtual-assistant-65b975d66b-l8wpw
message
🔍 AUTH_DEBUG: Path in UNPROTECTED_ENDPOINTS: True
raw payload (9 fields)
{
"caller": "api/api.py:491",
"cluster": "master",
"environment": "DEVELOPMENT",
"message": "\ud83d\udd0d AUTH_DEBUG: Path in UNPROTECTED_ENDPOINTS: True",
"method": "GET",
"path": "/health",
"service": "virtual-assistant",
"timestamp": "2026-07-27T00:07:14.214Z",
"version": "unknown"
}36s ago2026-07-27 00:07:14 UTC DEBUG virtual-assistant 🔍 AUTH_DEBUG: Method: GET
time2026-07-27 00:07:14 UTC
severityDEBUG
servicevirtual-assistant
sourceapi/api.py:489
clustermaster
versionunknown
trace5d1bd1d520cac81d93cd7eb8f4cd7485
spanb5e565c8c705097a
podvirtual-assistant-65b975d66b-l8wpw
message
🔍 AUTH_DEBUG: Method: GET
raw payload (9 fields)
{
"caller": "api/api.py:489",
"cluster": "master",
"environment": "DEVELOPMENT",
"message": "\ud83d\udd0d AUTH_DEBUG: Method: GET",
"method": "GET",
"path": "/health",
"service": "virtual-assistant",
"timestamp": "2026-07-27T00:07:14.214Z",
"version": "unknown"
}38s ago2026-07-27 00:07:12 UTC ERROR observability: level watcher could not stat config file /etc/config/observability.yaml: stat /etc/config/observability.yaml: no such file or directory
filter by:
severity≥ERROR
time2026-07-27 00:07:12 UTC
severityERROR
clustermaster
podregsvc-766cff9b49-cfs28
message
observability: level watcher could not stat config file /etc/config/observability.yaml: stat /etc/config/observability.yaml: no such file or directory
39s ago2026-07-27 00:07:11 UTC neo4j advpsvc match(a:AdvRateRule{deleted:true})-[r]-(b) with distinct a return a.portalorgid, a.tildaid, a.ruletype limit 10
time2026-07-27 00:07:11 UTC
severityINFO
serviceadvpsvc
clustermaster
podserver-1-dev-0
query
match(a:AdvRateRule{deleted:true})-[r]-(b)
with distinct a
return a.portalorgid, a.tildaid, a.ruletype
limit 10
raw payload (15 fields)
{
"allocatedBytes": 328.0,
"cpu": 2.0,
"elapsedTimeMs": 2.0,
"executionPlanCacheKeyHash": "AFAFBFFB",
"id": "17372772",
"pageFaults": 0.0,
"pageHits": 1146.0,
"planning": 0.0,
"query": "match(a:AdvRateRule{deleted:true})-[r]-(b) with distinct a return a.portalorgid, a.tildaid, a.ruletype limit 10",
"queryParameters": "{}",
"runtime": "pipelined",
"source": "bolt-session\tbolt\tGo Driver/5.20.0\t\tclient/10.129.3.19:37624\tserver/10.173.2.71:7687\u003e",
"timestamp": "2026-07-27T00:07:11.938Z",
"transactionId": 1771014.0,
"waiting": 0.0
}40s ago2026-07-27 00:07:10 UTC INFO
filter by:
severity≥INFO
time2026-07-27 00:07:10 UTC
severityINFO
clustermaster
podrag-agent-57879dd94f-l9nz7
40s ago2026-07-27 00:07:10 UTC INFO
filter by:
severity≥INFO
time2026-07-27 00:07:10 UTC
severityINFO
clustermaster
podlitellm-f54655c99-cbmvt
40s ago2026-07-27 00:07:10 UTC INFO virtual-assistant 🏁 Daily check complete: orgs=6, work_orders=409, reminders_sent=0, moved_to_need_attention=0
time2026-07-27 00:07:10 UTC
severityINFO
servicevirtual-assistant
sourcejobs/daily_work_order_cron.py:393
clustermaster
versionunknown
podvirtual-assistant-65b975d66b-l8wpw
message
🏁 Daily check complete: orgs=6, work_orders=409, reminders_sent=0, moved_to_need_attention=0
raw payload (7 fields)
{
"caller": "jobs/daily_work_order_cron.py:393",
"cluster": "master",
"environment": "DEVELOPMENT",
"message": "\ud83c\udfc1 Daily check complete: orgs=6, work_orders=409, reminders_sent=0, moved_to_need_attention=0",
"service": "virtual-assistant",
"timestamp": "2026-07-27T00:07:10.278Z",
"version": "unknown"
}40s ago2026-07-27 00:07:10 UTC INFO virtual-assistant No work order locks currently held
time2026-07-27 00:07:10 UTC
severityINFO
servicevirtual-assistant
sourceshared/work_order_locks.py:147
clustermaster
versionunknown
podvirtual-assistant-65b975d66b-l8wpw
message
No work order locks currently held
raw payload (7 fields)
{
"caller": "shared/work_order_locks.py:147",
"cluster": "master",
"environment": "DEVELOPMENT",
"message": "No work order locks currently held",
"service": "virtual-assistant",
"timestamp": "2026-07-27T00:07:10.278Z",
"version": "unknown"
}40s ago2026-07-27 00:07:10 UTC DEBUG virtual-assistant 🔍 Final lock status after daily check:
time2026-07-27 00:07:10 UTC
severityDEBUG
servicevirtual-assistant
sourcejobs/daily_work_order_cron.py:385
clustermaster
versionunknown
podvirtual-assistant-65b975d66b-l8wpw
message
🔍 Final lock status after daily check:
raw payload (7 fields)
{
"caller": "jobs/daily_work_order_cron.py:385",
"cluster": "master",
"environment": "DEVELOPMENT",
"message": "\ud83d\udd0d Final lock status after daily check:",
"service": "virtual-assistant",
"timestamp": "2026-07-27T00:07:10.278Z",
"version": "unknown"
}40s ago2026-07-27 00:07:10 UTC DEBUG virtual-assistant ✅ Org 4510cbf5-af86-40c7-ba81-1e7aa0df8c2e summary: processed=4, reminders_sent=0, moved_to_need_attention=0
filter by:
service=virtual-assistant
org=4510cbf5-af86-40c7-ba81-1e7aa0df8c2e
severity≥DEBUG
trace=030b3d02596a5fe6…
open trace flow ›
time2026-07-27 00:07:10 UTC
severityDEBUG
servicevirtual-assistant
sourcejobs/daily_work_order_cron.py:377
clustermaster
org4510cbf5-af86-40c7-ba81-1e7aa0df8c2e
versionunknown
trace030b3d02596a5fe630d1335ca674a257
spane4e1fa1fc62bdafe
podvirtual-assistant-65b975d66b-l8wpw
message
✅ Org 4510cbf5-af86-40c7-ba81-1e7aa0df8c2e summary: processed=4, reminders_sent=0, moved_to_need_attention=0
raw payload (8 fields)
{
"caller": "jobs/daily_work_order_cron.py:377",
"cluster": "master",
"environment": "DEVELOPMENT",
"message": "\u2705 Org 4510cbf5-af86-40c7-ba81-1e7aa0df8c2e summary: processed=4, reminders_sent=0, moved_to_need_attention=0",
"org_alias": "4510cbf5-af86-40c7-ba81-1e7aa0df8c2e",
"service": "virtual-assistant",
"timestamp": "2026-07-27T00:07:10.277Z",
"version": "unknown"
}40s ago2026-07-27 00:07:10 UTC DEBUG virtual-assistant [reminder] wo=5fda6743-4617-464f-9598-5ffebf341fa6 mode=RECURRING should_send=False escalate=False override=none reason='RECURRING not due:...
filter by:
service=virtual-assistant
org=4510cbf5-af86-40c7-ba81-1e7aa0df8c2e
severity≥DEBUG
trace=030b3d02596a5fe6…
open trace flow ›
time2026-07-27 00:07:10 UTC
severityDEBUG
servicevirtual-assistant
sourcejobs/daily_work_order_cron.py:244
clustermaster
org4510cbf5-af86-40c7-ba81-1e7aa0df8c2e
versionunknown
trace030b3d02596a5fe630d1335ca674a257
spane4e1fa1fc62bdafe
podvirtual-assistant-65b975d66b-l8wpw
message
[reminder] wo=5fda6743-4617-464f-9598-5ffebf341fa6 mode=RECURRING should_send=False escalate=False override=none reason='RECURRING not due: now-base=827m < step=1440m'
raw payload (8 fields)
{
"caller": "jobs/daily_work_order_cron.py:244",
"cluster": "master",
"environment": "DEVELOPMENT",
"message": "[reminder] wo=5fda6743-4617-464f-9598-5ffebf341fa6 mode=RECURRING should_send=False escalate=False override=none reason=\u0027RECURRING not due: now-base=827m \u003c step=1440m\u0027",
"org_alias": "4510cbf5-af86-40c7-ba81-1e7aa0df8c2e",
"service": "virtual-assistant",
"timestamp": "2026-07-27T00:07:10.277Z",
"version": "unknown"
}40s ago2026-07-27 00:07:10 UTC neo4j virtual-assistant MATCH (w:WorkOrder {tildaid: $work_order_id})<-[:FACT_TO_WORK_ORDER]-(f:Fact) WHERE f.delpending = false ...
filter by:
service=virtual-assistant
org=4510cbf5-af86-40c7-ba81-1e7aa0df8c2e
severity≥INFO
trace=030b3d02596a5fe6…
open trace flow ›
time2026-07-27 00:07:10 UTC
severityINFO
servicevirtual-assistant
clustermaster
org4510cbf5-af86-40c7-ba81-1e7aa0df8c2e
trace030b3d02596a5fe630d1335ca674a257
spane4e1fa1fc62bdafe
podserver-2-dev-0
query
MATCH (w:WorkOrder {tildaid: $work_order_id})<-[:FACT_TO_WORK_ORDER]-(f:Fact)
WHERE f.delpending = false
RETURN f
ORDER BY f.createdat ASC
raw payload (15 fields)
{
"allocatedBytes": 724.0,
"cpu": 0.0,
"elapsedTimeMs": 0.0,
"executionPlanCacheKeyHash": "23A43F51",
"id": "24641263",
"pageFaults": 0.0,
"pageHits": 6.0,
"planning": 0.0,
"query": "\n MATCH (w:WorkOrder {tildaid: $work_order_id})\u003c-[:FACT_TO_WORK_ORDER]-(f:Fact)\n WHERE f.delpending = false\n RETURN f\n ORDER BY f.createdat ASC\n ",
"queryParameters": "{work_order_id: \u00275fda6743-4617-464f-9598-5ffebf341fa6\u0027}",
"runtime": "pipelined",
"source": "bolt-session\tbolt\tneo4j-python/5.28.0 Python/3.12.13-final-0 (linux)\t\tclient/10.129.4.156:38312\tserver/10.173.0.8:7687\u003e",
"timestamp": "2026-07-27T00:07:10.274Z",
"transactionId": 12025332.0,
"waiting": 0.0
}40s ago2026-07-27 00:07:10 UTC neo4j virtual-assistant MATCH (w:WorkOrder {tildaid: $work_order_id})-[:WORK_ORDER_TO_ACTIVITY]->(a:WorkOrderActivity) WHERE 1=1 AND a.activity_type IN...
filter by:
service=virtual-assistant
org=4510cbf5-af86-40c7-ba81-1e7aa0df8c2e
severity≥INFO
trace=030b3d02596a5fe6…
open trace flow ›
time2026-07-27 00:07:10 UTC
severityINFO
servicevirtual-assistant
clustermaster
org4510cbf5-af86-40c7-ba81-1e7aa0df8c2e
trace030b3d02596a5fe630d1335ca674a257
spane4e1fa1fc62bdafe
podserver-2-dev-0
query
MATCH (w:WorkOrder {tildaid: $work_order_id})-[:WORK_ORDER_TO_ACTIVITY]->(a:WorkOrderActivity)
WHERE 1=1 AND a.activity_type IN $activity_types AND a.status IN $statuses
OPTIONAL MATCH (d:Document)
WHERE d.tildaid IN COALESCE(a.docIds, [])
WITH w, a, COLLECT(DISTINCT {tildaid: d.tildaid, fileName: d.fileName, contentType: d.contentType}) as documents
// Compute latest AI_MSG draft id per work order for gating REGENERATE
OPTIONAL MATCH (w)-[:WORK_ORDER_TO_ACTIVITY]->(ai:WorkOrderActivity)
WHERE ai.activity_type = 'AI_MSG' // any status: a newer SENT/QUEUED must supersede an older FindingsChanged draft, else REGENERATE stays stuck
WITH w, a, documents,
max(coalesce(ai.updatedat, ai.createdat)) AS latest_any_ts,
collect({id: ai.tildaid, ts: coalesce(ai.updatedat, ai.createdat), status: ai.status, ignoreType: ai.ignoreType}) AS ai_msgs
WITH a, documents,
CASE
WHEN size(ai_msgs) = 0 THEN false
ELSE any(x IN ai_msgs WHERE x.id = a.tildaid AND x.ts = latest_any_ts)
END AS is_latest_draft
RETURN a,
CASE WHEN size(documents) > 0 AND documents[0].tildaid IS NOT NULL THEN documents ELSE [] END AS documents,
is_latest_draft
ORDER BY a.createdat DESC
SKIP $skip
LIMIT $limit
raw payload (15 fields)
{
"allocatedBytes": 3272.0,
"cpu": 0.0,
"elapsedTimeMs": 0.0,
"executionPlanCacheKeyHash": "2A13D7A9",
"id": "24641262",
"pageFaults": 0.0,
"pageHits": 12.0,
"planning": 0.0,
"query": "\n MATCH (w:WorkOrder {tildaid: $work_order_id})-[:WORK_ORDER_TO_ACTIVITY]-\u003e(a:WorkOrderActivity)\n WHERE 1=1 AND a.activity_type IN $activity_types AND a.status IN $statuses \n OPTIONAL MATCH (d:Document)\n WHERE d.tildaid IN COALESCE(a.docIds, [])\n WITH w, a, COLLECT(DISTINCT {tildaid: d.tildaid, fileName: d.fileName, contentType: d.contentType}) as documents\n // Compute latest AI_MSG draft id per work order for gating REGENERATE\n OPTIONAL MATCH (w)-[:WORK_ORDER_TO_ACTIVITY]-\u003e(ai:WorkOrderActivity)\n WHERE ai.activity_type = \u0027AI_MSG\u0027 // any status: a newer SENT/QUEUED must supersede an older FindingsChanged draft, else REGENERATE stays stuck\n WITH w, a, documents,\n max(coalesce(ai.updatedat, ai.createdat)) AS latest_any_ts,\n collect({id: ai.tildaid, ts: coalesce(ai.updatedat, ai.createdat), status: ai.status, ignoreType: ai.ignoreType}) AS ai_msgs\n WITH a, documents,\n CASE\n WHEN size(ai_msgs) = 0 THEN false\n ELSE any(x IN ai_msgs WHERE x.id = a.tildaid AND x.ts = latest_any_ts)\n END AS is_latest_draft\n RETURN a,\n CASE WHEN size(documents) \u003e 0 AND documents[0].tildaid IS NOT NULL THEN documents ELSE [] END AS documents,\n is_latest_draft\n ORDER BY a.createdat DESC\n SKIP $skip\n LIMIT $limit\n ",
"queryParameters": "{limit: 1, statuses: [\u0027DRAFT\u0027], skip: 0, work_order_id: \u00275fda6743-4617-464f-9598-5ffebf341fa6\u0027, activity_types: [\u0027AI_MSG\u0027]}",
"runtime": "pipelined",
"source": "bolt-session\tbolt\tneo4j-python/5.28.0 Python/3.12.13-final-0 (linux)\t\tclient/10.129.4.156:38312\tserver/10.173.0.8:7687\u003e",
"timestamp": "2026-07-27T00:07:10.266Z",
"transactionId": 12025331.0,
"waiting": 0.0
}40s ago2026-07-27 00:07:10 UTC neo4j virtual-assistant MATCH (w:WorkOrder {tildaid: $work_order_id})-[:WORK_ORDER_TO_ACTIVITY]->(a:WorkOrderActivity) WHERE 1=1 AND a.activity_type IN...
filter by:
service=virtual-assistant
org=4510cbf5-af86-40c7-ba81-1e7aa0df8c2e
severity≥INFO
trace=030b3d02596a5fe6…
open trace flow ›
time2026-07-27 00:07:10 UTC
severityINFO
servicevirtual-assistant
clustermaster
org4510cbf5-af86-40c7-ba81-1e7aa0df8c2e
trace030b3d02596a5fe630d1335ca674a257
spane4e1fa1fc62bdafe
podserver-2-dev-0
query
MATCH (w:WorkOrder {tildaid: $work_order_id})-[:WORK_ORDER_TO_ACTIVITY]->(a:WorkOrderActivity)
WHERE 1=1 AND a.activity_type IN $activity_types AND a.status IN $statuses
RETURN count(a) as total
raw payload (15 fields)
{
"allocatedBytes": 488.0,
"cpu": 0.0,
"elapsedTimeMs": 0.0,
"executionPlanCacheKeyHash": "FA66F084",
"id": "24641261",
"pageFaults": 0.0,
"pageHits": 13.0,
"planning": 0.0,
"query": "\n MATCH (w:WorkOrder {tildaid: $work_order_id})-[:WORK_ORDER_TO_ACTIVITY]-\u003e(a:WorkOrderActivity)\n WHERE 1=1 AND a.activity_type IN $activity_types AND a.status IN $statuses \n RETURN count(a) as total\n ",
"queryParameters": "{statuses: [\u0027DRAFT\u0027], work_order_id: \u00275fda6743-4617-464f-9598-5ffebf341fa6\u0027, activity_types: [\u0027AI_MSG\u0027]}",
"runtime": "pipelined",
"source": "bolt-session\tbolt\tneo4j-python/5.28.0 Python/3.12.13-final-0 (linux)\t\tclient/10.129.4.156:38312\tserver/10.173.0.8:7687\u003e",
"timestamp": "2026-07-27T00:07:10.263Z",
"transactionId": 12025331.0,
"waiting": 0.0
}40s ago2026-07-27 00:07:10 UTC neo4j virtual-assistant MATCH (wo:WorkOrder {tildaid: $wo_id}) OPTIONAL MATCH (wo)-[:WORK_ORDER_TO_ACTIVITY]->(a:WorkOrderActivity {activity_type:...
filter by:
service=virtual-assistant
org=4510cbf5-af86-40c7-ba81-1e7aa0df8c2e
severity≥INFO
trace=030b3d02596a5fe6…
open trace flow ›
time2026-07-27 00:07:10 UTC
severityINFO
servicevirtual-assistant
clustermaster
org4510cbf5-af86-40c7-ba81-1e7aa0df8c2e
trace030b3d02596a5fe630d1335ca674a257
spane4e1fa1fc62bdafe
podserver-2-dev-0
query
MATCH (wo:WorkOrder {tildaid: $wo_id})
OPTIONAL MATCH (wo)-[:WORK_ORDER_TO_ACTIVITY]->(a:WorkOrderActivity {activity_type: 'U_REPLY'})
RETURN wo.createdat AS created_at, max(a.timestamp) AS latest_reply_ts
raw payload (15 fields)
{
"allocatedBytes": 928.0,
"cpu": 0.0,
"elapsedTimeMs": 0.0,
"executionPlanCacheKeyHash": "4C51129A",
"id": "24641260",
"pageFaults": 0.0,
"pageHits": 15.0,
"planning": 0.0,
"query": "\n MATCH (wo:WorkOrder {tildaid: $wo_id})\n OPTIONAL MATCH (wo)-[:WORK_ORDER_TO_ACTIVITY]-\u003e(a:WorkOrderActivity {activity_type: \u0027U_REPLY\u0027})\n RETURN wo.createdat AS created_at, max(a.timestamp) AS latest_reply_ts\n ",
"queryParameters": "{wo_id: \u00275fda6743-4617-464f-9598-5ffebf341fa6\u0027}",
"runtime": "pipelined",
"source": "bolt-session\tbolt\tneo4j-python/5.28.0 Python/3.12.13-final-0 (linux)\t\tclient/10.129.4.156:38312\tserver/10.173.0.8:7687\u003e",
"timestamp": "2026-07-27T00:07:10.256Z",
"transactionId": 12025330.0,
"waiting": 0.0
}40s ago2026-07-27 00:07:10 UTC neo4j virtual-assistant MATCH (n:WorkOrder {tildaid: $node_id}) RETURN n
filter by:
service=virtual-assistant
org=4510cbf5-af86-40c7-ba81-1e7aa0df8c2e
severity≥INFO
trace=030b3d02596a5fe6…
open trace flow ›
time2026-07-27 00:07:10 UTC
severityINFO
servicevirtual-assistant
clustermaster
org4510cbf5-af86-40c7-ba81-1e7aa0df8c2e
trace030b3d02596a5fe630d1335ca674a257
spane4e1fa1fc62bdafe
podserver-2-dev-0
query
MATCH (n:WorkOrder {tildaid: $node_id})
RETURN n
raw payload (15 fields)
{
"allocatedBytes": 312.0,
"cpu": 0.0,
"elapsedTimeMs": 0.0,
"executionPlanCacheKeyHash": "E0146DD7",
"id": "24641259",
"pageFaults": 0.0,
"pageHits": 8.0,
"planning": 0.0,
"query": "MATCH (n:WorkOrder {tildaid: $node_id}) RETURN n",
"queryParameters": "{node_id: \u00275fda6743-4617-464f-9598-5ffebf341fa6\u0027}",
"runtime": "pipelined",
"source": "bolt-session\tbolt\tneo4j-python/5.28.0 Python/3.12.13-final-0 (linux)\t\tclient/10.129.4.156:38312\tserver/10.173.0.8:7687\u003e",
"timestamp": "2026-07-27T00:07:10.247Z",
"transactionId": 12025329.0,
"waiting": 0.0
}Paginated with Cloud Logging page cursors, so you can walk the whole 1h window — not just the newest page. Click any filter by value to scope the view (resets to page 1). Paste a trace id to see every log line across all services for that one request.