From Sentry Replays to Org IDs: Closing the Gap With Automation
When product development outpaces internal tooling, a quick prompt to Kognitos and a clear SOP out of those prompts/chats can turn session data into actionable, deterministic customer data—updated every day.
The problem
As we were building the product, we kept hitting the same friction: there was no fast way to tie a user to their organization ID. That matters when something goes wrong for a specific org—we want to identify who to contact and act quickly, not chase breadcrumbs across tools.
We needed a way to proactively reach customers when org-related issues arise, with enough context to be helpful in the first message.
Where we started: a prompt to Kognitos
The first slice was intentionally small. We asked Kognitos to use the Sentry API (HTTP requests with Bearer token authentication) to:
- Fetch the last 30 days of session replays from the organization’s replays endpoint (paginated, with a sensible limit and stats period).
- Exclude anyone whose email contained our internal domain (
kognitos.com), so we only saw real customers. - Update a Notion page with what we learned.
That single prompt was enough to prove the idea: session replays carry URLs and user context we could operationalize.
What we turned it into: a repeatable SOP
After a few rounds of iteration, Kognitos created a standard operating procedure from the chat prompts to keep the workflow consistent and auditable.
Overview
The pipeline pulls recent Sentry session replays, filters out internal users, and produces a table that maps each organization to account identifiers and the server hostname where those sessions were recorded.
Execution steps
- Fetch replays from Sentry. Call the Replays API for a rolling window (for example, up to 90 days and up to 1,000 results), scoped to the relevant project, authenticated with a Bearer token—matching how we already explore replays in the Sentry UI.
- Filter and extract org–user mappings. Drop internal addresses (e.g.
kognitos.comor internal mail aliases). For each remaining replay, scan visited URLs for paths like/organizations/{id}/:- org_id — the segment right after
/organizations/ - server — the hostname from the absolute URL (e.g.
app.us-1.kognitos.com) - account_name — derived from the user’s email: typically the domain stem for work email (e.g.
al@wip.com→wip), or the full address for consumer providers like Gmail
/organizations/, and empty org IDs. - org_id — the segment right after
- Build
org_users_table. Columns:account_name·server·org_id. - Sync new orgs into Notion. Read existing rows from the target Notion database, compare org IDs to today’s run, and append a row for each org not yet present—same three columns as above.
The SOP becomes the living source of “who maps to which org, on which server,” without manual copy-paste from Sentry to Notion Table
Running it on a schedule
Kognitos now runs this flow every day. New customer orgs surface automatically; the table stays aligned with what we’re actually seeing in session replays.
Example output (illustrative)
Shape of the data—not production values in the notion:
| account_name | server | org_id |
|---|---|---|
| user@gmail.com | app.us-1.kognitos.com | AKp0… |
| user2@gmail.com | app.us-1.kognitos.com | BaCp… |
| abc-interiors | app.us-1.kognitos.com | CGwi… |
| maryXYZ@gmail.com | app.us-1.kognitos.com | Y0UU… |
| mihirZBC@gmail.com | app.us-1.kognitos.com | ZvJz… |
Why it’s worth it
This isn’t glamour work—it’s glue. But the glue that connects observability (Sentry), customer context (email and org IDs from real sessions), and team workflow (Notion) is what lets support and engineering move in sync. One prompt started it; a clear SOP and a daily schedule keep it honest.
No comments:
Post a Comment