# Create or replace a knowledge file
Source: https://docs.traversal.com/api-reference/knowledge-files/create-or-replace-a-knowledge-file
/api/openapi.yaml put /v1/knowledge/files/{file_path}
Sends the file bytes directly as the request body. Do not wrap the
content in JSON or multipart form data. The extension and `Content-Type`
must describe the same supported file format. Send
`application/octet-stream` to infer the type from the extension.
The request body can contain at most 10 MiB (10,485,760 bytes).
A new path returns `201 Created`; replacing an existing path returns
`200 OK`. Use `If-Match` with the current metadata ETag to prevent a
concurrent update from being overwritten.
# Delete a knowledge file
Source: https://docs.traversal.com/api-reference/knowledge-files/delete-a-knowledge-file
/api/openapi.yaml delete /v1/knowledge/files/{file_path}
Deletes a customer-visible file under `user_generated/` or `skills/`.
A successful delete returns `204 No Content`. Missing or hidden paths
return `404 Not Found`.
Use `If-Match` with the current metadata ETag to delete only that
version. An unconditional delete retries one concurrent race, then
returns `409 Conflict` if another request still wins.
# Download a knowledge file
Source: https://docs.traversal.com/api-reference/knowledge-files/download-a-knowledge-file
/api/openapi.yaml get /v1/knowledge/files/{file_path}/content
Streams the original file bytes.
# Get knowledge file metadata
Source: https://docs.traversal.com/api-reference/knowledge-files/get-knowledge-file-metadata
/api/openapi.yaml get /v1/knowledge/files/{file_path}
Returns metadata for one customer-visible knowledge file.
# List knowledge files
Source: https://docs.traversal.com/api-reference/knowledge-files/list-knowledge-files
/api/openapi.yaml get /v1/knowledge/files
Lists customer-visible files under `user_generated/` and `skills/`,
ordered by path. Traversal-generated knowledge and memories are not
exposed.
# Create a session
Source: https://docs.traversal.com/api-reference/sessions/create-a-session
/api/openapi.yaml post /v1/sessions
Starts a new investigation. The request returns immediately while
Traversal investigates in the background — poll `GET /v1/sessions/{session_id}`
until the session reaches the `idle` state to retrieve the result.
Each organization is limited to **15 concurrent running sessions**
(a running investigation or an in-flight follow-up both count toward
this limit). Exceeding the limit returns `429 Too Many Requests` with
`retry_after`.
# Get a session
Source: https://docs.traversal.com/api-reference/sessions/get-a-session
/api/openapi.yaml get /v1/sessions/{session_id}
Retrieves a single session, including its full conversation history.
This is the only endpoint that populates the `messages` array — all
other endpoints return `messages: null`.
# List sessions
Source: https://docs.traversal.com/api-reference/sessions/list-sessions
/api/openapi.yaml get /v1/sessions
Returns sessions created via the V1 API, scoped to the authenticated
user's organization. Sessions created in the web app are not included.
**Requires the `admin` role.** API keys created by members can call
every other Sessions endpoint, but this one is admin-only because it
returns sessions across all users in the organization. Calls from a
member-role key return `403 Forbidden`.
If `page` and `limit` are both omitted, all sessions are returned in a
single response.
# Send a follow-up message
Source: https://docs.traversal.com/api-reference/sessions/send-a-follow-up-message
/api/openapi.yaml post /v1/sessions/{session_id}/messages
Sends a follow-up question on an existing session. Traversal uses the
full conversation history as context.
The session must be in the `idle` status. If it is `running` or
`follow_up_running`, the API returns `409 Conflict` with `retry_after`.
Poll `GET /v1/sessions/{session_id}` until the session status returns
to `idle` to retrieve the assistant response.
Follow-ups count toward the same **15 concurrent running sessions**
per-organization limit as `POST /v1/sessions`. If the organization is
already at the limit, the request returns `429 Too Many Requests` with
`retry_after`.
# Stream session status
Source: https://docs.traversal.com/api-reference/sessions/stream-session-status
/api/openapi.yaml get /v1/sessions/{session_id}/events
Opens a Server-Sent Events (SSE) stream for a session. The stream sends
a `status` event immediately after connecting and whenever the session
status changes. Each event's `data` field contains a session snapshot
with `messages: null`.
While the session remains `running` or `follow_up_running`, the server
sends a `: keepalive` comment every 15 seconds when no status change
occurs. SSE clients ignore comments automatically.
The stream closes after sending an `idle`, `failed`, or `cancelled`
status. After receiving a terminal status, retrieve the final
conversation with `GET /v1/sessions/{session_id}`.
If the connection closes before a terminal status arrives, reconnect
to this endpoint. The first event always reports the current status, so
clients do not need an event cursor or `Last-Event-ID` header.
# Authentication
Source: https://docs.traversal.com/api/authentication
Create and manage API keys to authenticate with the Traversal API.
The Traversal API is available to all users. Authenticate your requests with an API key, which you can create yourself from the Traversal web app.
## Create an API key
Go to **[Settings > API Keys](https://app.traversal.com/settings/api-keys)** in the Traversal web app.
Click **Add API Key**, give it a descriptive name (e.g., `ci-pipeline` or `local-dev`), and confirm.
Traversal displays the key in a modal. **This is your only chance to copy it** — once you dismiss the modal, the key cannot be retrieved again.
Store it in a secure secret manager (1Password, AWS Secrets Manager, Vault, etc.) before closing the dialog.
API keys are shown **once**. If you lose a key, you'll need to create a new one and revoke the old.
## Use the key
Send the key as a bearer token in the `Authorization` header:
```http theme={null}
Authorization: Bearer trv_ak_your_api_key_here
```
Each key is bound to the user who created it and their organization, and inherits that user's role. Most endpoints require the `member` role, but a few (e.g., `GET /v1/sessions`) require `admin` — see the per-endpoint **Required roles** table in [Sessions API](/api/sessions#required-roles).
| Condition | Response |
| --------------------------------------------------------------------- | ------------------ |
| Missing, invalid, or revoked token | `401 Unauthorized` |
| Insufficient role, ownership, permission, or endpoint-specific access | `403 Forbidden` |
## Key hygiene
* **Name keys descriptively** so you can identify which system uses them.
* **Use separate keys per environment or service** (e.g., one for CI, one for a local script). This limits blast radius and makes revocation surgical.
* **Rotate regularly** and whenever someone with access leaves the team.
* **Never commit keys to source control.** Use environment variables or a secret manager.
## Revoke a key
If a key is lost, leaked, or no longer needed, revoke it from **[Settings > API Keys](https://app.traversal.com/settings/api-keys)**. Revocation takes effect immediately — subsequent requests using the key return `401 Unauthorized`.
# Knowledge Files API
Source: https://docs.traversal.com/api/knowledge-files
Upload runbooks, reference documents, and agent skills to Traversal with the V1 Knowledge Files API.
The Knowledge Files API lets you manage the customer-authored files that Traversal agents use during investigations. You can upload runbooks and reference documents, define reusable agent skills, replace those files, or delete them.
All endpoints require authentication — see [Authentication](/api/authentication) to create an API key. For the base URL and error envelope, see the [API overview](/api/overview).
## Endpoints
See the **Endpoints** section in the sidebar for the full API reference, generated from the OpenAPI spec.
API keys inherit the role of the user who created them. Every Knowledge Files endpoint requires the `member` role or higher. Requests use the organization bound to the API key — you cannot read or modify another organization's files.
## File organization
Every path must start with one of these roots:
| Root | Use it for |
| ----------------- | ----------------------------------------------------------------------------------------------------- |
| `user_generated/` | Runbooks, architecture notes, service catalogs, and other reference material. |
| `skills/` | Agent skills. Each skill uses a directory containing a `SKILL.md` file and optional supporting files. |
The API does not expose Traversal-generated knowledge or agent memories.
Paths are case-sensitive and must:
* Use `/` separators
* Be relative, not absolute
* Exclude `.` and `..` segments
* Exclude control characters
* Use at most 900 UTF-8 bytes in total and 255 bytes per segment
### Supported file formats
Each upload request can contain at most **10 MiB (10,485,760 bytes)**. The file extension and `Content-Type` must describe the same format.
| Extensions | Accepted `Content-Type` |
| ------------------ | ----------------------------------------------------- |
| `.md`, `.markdown` | `text/markdown` |
| `.txt` | `text/plain` |
| `.csv` | `text/csv` |
| `.tsv` | `text/tab-separated-values`, `text/tsv` |
| `.json` | `application/json` |
| `.yaml`, `.yml` | `application/yaml`, `application/x-yaml`, `text/yaml` |
| `.pdf` | `application/pdf` |
You can send `application/octet-stream` to infer the stored content type from the extension.
## Upload a file
Send the file bytes directly as the `PUT` request body. Do not wrap the content in JSON or multipart form data.
```bash theme={null}
curl -X PUT https://api.traversal.com/v1/knowledge/files/user_generated/runbooks/checkout.md \
-H "Authorization: Bearer $TRAVERSAL_API_KEY" \
-H "Content-Type: text/markdown" \
-H "If-None-Match: *" \
--data-binary @checkout.md
```
`If-None-Match: *` prevents an existing file from being overwritten. A successful create returns `201 Created` with file metadata, `ETag`, and `Location`.
```json theme={null}
{
"path": "user_generated/runbooks/checkout.md",
"content_type": "text/markdown",
"size_bytes": 2841,
"sha256": "e89873d4d7f2b6d71e414f184425ca95410cb3dfd41a497b3d9f9e12a891c617",
"etag": "\"d50c991679d03cc0b4b29afec405e581c89fca80d3a7c31ab2e8ab7d34f3d78d\"",
"created_at": "2026-08-25T15:30:00Z",
"updated_at": "2026-08-25T15:30:00Z",
"created": true
}
```
Without a conditional header, `PUT` creates a missing file or completely replaces an existing file. An unconditional replacement returns `200 OK`.
## Write files safely
### Create only if the path is unused
Use `If-None-Match: *`, as shown in the upload example, whenever you intend to create a new file. Traversal returns `412 Precondition Failed` if the path already exists or another request creates it concurrently.
### Replace only the version you retrieved
First retrieve the current metadata and save its `etag` value:
```bash theme={null}
ETAG=$(curl --silent --show-error \
"https://api.traversal.com/v1/knowledge/files/user_generated/runbooks/checkout.md" \
-H "Authorization: Bearer $TRAVERSAL_API_KEY" \
| jq -r '.etag')
```
Then send the saved value in `If-Match`:
```bash theme={null}
curl -X PUT https://api.traversal.com/v1/knowledge/files/user_generated/runbooks/checkout.md \
-H "Authorization: Bearer $TRAVERSAL_API_KEY" \
-H "Content-Type: text/markdown" \
-H "If-Match: $ETAG" \
--data-binary @checkout.md
```
If the file changed or moved after you retrieved it, Traversal returns `412 Precondition Failed`. Retrieve the latest metadata before deciding whether to retry.
You can also send `If-Match: *` to replace the file only if the path already exists.
An unconditional `PUT` retries one concurrent replacement, then returns `409 Conflict` if the file still changed underneath the request. Retry the write after fetching the latest metadata.
## Delete a file
```bash theme={null}
curl -X DELETE https://api.traversal.com/v1/knowledge/files/user_generated/runbooks/checkout.md \
-H "Authorization: Bearer $TRAVERSAL_API_KEY"
```
A successful delete returns `204 No Content`. A missing or hidden path returns `404 Not Found`.
To delete only the version you retrieved, send its `etag` in `If-Match`:
```bash theme={null}
curl -X DELETE https://api.traversal.com/v1/knowledge/files/user_generated/runbooks/checkout.md \
-H "Authorization: Bearer $TRAVERSAL_API_KEY" \
-H "If-Match: $ETAG"
```
A stale `If-Match` returns `412 Precondition Failed`. An unconditional `DELETE` retries one concurrent delete, then returns `409 Conflict` if another request still wins the race.
## Create an agent skill
An agent skill is a directory under `skills/` with a `SKILL.md` file. The file starts with YAML frontmatter containing a unique `name` and a `description`, followed by the instructions:
```markdown theme={null}
---
name: checkout-triage
description: Investigate failures and latency in the checkout service.
---
1. Read `references/services.md` for service ownership.
2. Check checkout request errors and latency.
3. Compare the first failing deployment with the incident start time.
```
Upload the main skill file:
```bash theme={null}
curl -X PUT https://api.traversal.com/v1/knowledge/files/skills/checkout-triage/SKILL.md \
-H "Authorization: Bearer $TRAVERSAL_API_KEY" \
-H "Content-Type: text/markdown" \
-H "If-None-Match: *" \
--data-binary @SKILL.md
```
Upload supporting files to the same directory with separate requests:
```bash theme={null}
curl -X PUT https://api.traversal.com/v1/knowledge/files/skills/checkout-triage/references/services.md \
-H "Authorization: Bearer $TRAVERSAL_API_KEY" \
-H "Content-Type: text/markdown" \
-H "If-None-Match: *" \
--data-binary @references/services.md
```
Traversal validates `SKILL.md` when you upload it. Invalid frontmatter or a duplicate skill name returns `400 Bad Request`.
## List files
List files under both customer-managed roots:
```bash theme={null}
curl \
"https://api.traversal.com/v1/knowledge/files?limit=100&page=1" \
-H "Authorization: Bearer $TRAVERSAL_API_KEY"
```
Use `prefix` to list one subtree:
```bash theme={null}
curl \
--get \
"https://api.traversal.com/v1/knowledge/files" \
-H "Authorization: Bearer $TRAVERSAL_API_KEY" \
--data-urlencode "prefix=skills/checkout-triage" \
--data-urlencode "limit=100" \
--data-urlencode "page=1"
```
Files are ordered by path. If you omit pagination parameters, `page` defaults to `1` and `limit` defaults to `50`; the maximum limit is `100`. The response includes `count`, `total`, `prev`, and `next`.
## Download a file
```bash theme={null}
curl \
"https://api.traversal.com/v1/knowledge/files/user_generated/runbooks/checkout.md/content" \
-H "Authorization: Bearer $TRAVERSAL_API_KEY" \
--output checkout.md
```
Downloads stream the original bytes and return the stored `Content-Type`, `Content-Length`, `Content-Disposition`, and `ETag`. Responses also include `Cache-Control: private, no-cache, must-revalidate` and `X-Content-Type-Options: nosniff`.
To avoid downloading an unchanged file, send its previous `ETag` in `If-None-Match`:
```bash theme={null}
curl \
"https://api.traversal.com/v1/knowledge/files/user_generated/runbooks/checkout.md/content" \
-H "Authorization: Bearer $TRAVERSAL_API_KEY" \
-H 'If-None-Match: "d50c991679d03cc0b4b29afec405e581c89fca80d3a7c31ab2e8ab7d34f3d78d"' \
--write-out "HTTP %{http_code}\n" \
--output checkout.md.download
```
An unchanged file returns `304 Not Modified` with no body and includes the current `ETag` and `Cache-Control` headers. Replace your local copy only when the response is `200 OK`.
## Knowledge-specific errors
In addition to the [generic status codes](/api/overview#status-codes), these responses are common:
| Status code | Error code | When it occurs |
| ----------- | ------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------- |
| `400` | `invalid_argument` | The path is invalid or outside the API-visible roots, or a `SKILL.md` file has invalid frontmatter or a duplicate skill name. |
| `404` | `not_found` | The path does not exist or is hidden. |
| `409` | `conflict` | A concurrent write or delete still conflicts after one retry, a skill name is already taken, or stored metadata is inconsistent. |
| `412` | `precondition_failed` | `If-Match` is stale, the file moved, `If-Match` was sent for a missing path, or an `If-None-Match` create-only condition failed. |
| `413` | `payload_too_large` | The request body exceeds 10 MiB (10,485,760 bytes). |
| `415` | `unsupported_media_type` | The extension or `Content-Type` is unsupported, they do not match, or an unsupported non-identity `Content-Encoding` such as `gzip` is set. |
# Overview
Source: https://docs.traversal.com/api/overview
Introduction to the Traversal API — base URL, authentication, and error handling.
The Traversal API lets you integrate investigations into your own tools, pipelines, and workflows. Anything a member can do in the Traversal web app, you can do over the API.
To get started, [create an API key](/api/authentication) and make your first request.
## Base URL
All endpoints are served from:
```
https://api.traversal.com
```
Single-tenant SaaS and BYOC customers use the API endpoint exposed by their dedicated deployment instead — see [single-tenant SaaS](/architecture/single-tenant-saas) or [Bring Your Own Cloud](/architecture/byoc) for details.
## Error format
All errors follow a consistent envelope:
```json theme={null}
{
"error": {
"code": "resource_exhausted",
"message": "A human-readable explanation of the error.",
"retry_after": 30
}
}
```
| Field | Type | Description |
| ------------------- | --------- | ----------------------------------------------------------------------------------------------------------------------------------------------- |
| `error.code` | `string` | Stable machine-readable category. Use this field instead of matching the message text. |
| `error.message` | `string` | Human-readable explanation of the error. |
| `error.retry_after` | `integer` | Optional suggested seconds to wait before retrying. Present only when the server can recommend a delay, such as some `429` and `503` responses. |
When `retry_after` is present in the body, the response also includes a standard `Retry-After` HTTP header with the same value.
### Status codes
| Status code | Meaning | When it occurs |
| ----------- | ---------------------- | ------------------------------------------------------------------------------------------------------------ |
| `400` | Bad Request | An invalid body, path, query parameter, or header value. The `message` identifies the failed validation. |
| `401` | Unauthorized | Missing, invalid, or revoked API key. |
| `403` | Forbidden | Insufficient role, ownership, permission, or endpoint-specific access. |
| `404` | Not Found | The requested resource does not exist or is not visible to the authenticated organization. |
| `405` | Method Not Allowed | The path exists, but it does not support the requested HTTP method. The response includes an `Allow` header. |
| `409` | Conflict | The request conflicts with the current resource state. |
| `412` | Precondition Failed | A conditional request validator such as `If-Match` is stale. |
| `413` | Content Too Large | The request body exceeds the endpoint limit. |
| `415` | Unsupported Media Type | The request uses an unsupported content format or encoding. |
| `429` | Too Many Requests | The organization has reached an endpoint-specific concurrency or rate limit. |
| `500` | Internal Server Error | An unexpected error occurred on the server. |
| `503` | Service Unavailable | API infrastructure is not available. |
Individual endpoints may define additional status codes specific to their behavior — see the relevant API reference page for details.
# Sessions API
Source: https://docs.traversal.com/api/sessions
Programmatically create investigations, send follow-up questions, and retrieve results from Traversal using the V1 Sessions API.
The Sessions API lets you launch investigations, send follow-up questions, and fetch results programmatically. Sessions are the same investigation primitive that powers the Traversal web application — anything a member can do in the UI, you can do over the API.
All endpoints require authentication — see [Authentication](/api/authentication) to create an API key. For base URL, error envelope, and generic status codes, see the [API overview](/api/overview).
## Endpoints
See the **Endpoints** section in the sidebar for the full API reference, generated from the OpenAPI spec.
### Required roles
API keys inherit the role of the user who created them. Most Sessions endpoints work for any `member`, but `GET /v1/sessions` is admin-only because it returns sessions across the whole organization.
| Endpoint | Minimum role |
| -------------------------------------------------------------- | ------------ |
| `POST /v1/sessions` — Create a session | `member` |
| `GET /v1/sessions` — List sessions | **`admin`** |
| `GET /v1/sessions/{session_id}` — Get a session | `member` |
| `GET /v1/sessions/{session_id}/events` — Stream session status | `member` |
| `POST /v1/sessions/{session_id}/messages` — Send a follow-up | `member` |
A request from an underprivileged key returns `403 Forbidden` with a message naming the required role. If `GET /v1/sessions` is failing for you, check the role of the user who issued the key in **[Settings > User Management](https://app.traversal.com/settings/user-management)**.
## Investigation depth
Both `POST /v1/sessions` and `POST /v1/sessions/{session_id}/messages` accept an optional `thinking_mode` field that controls how much depth Traversal applies:
| Mode | Behavior |
| ---------------- | ---------------------------------------------------------------- |
| `auto` (default) | Traversal classifies the request and picks an appropriate depth. |
| `deep` | Forces a thorough root-cause analysis. |
| `fast` | Runs a quick, exploratory pass. |
If omitted, `thinking_mode` defaults to `auto`, so existing integrations need no changes.
```bash theme={null}
curl -X POST https://api.traversal.com/v1/sessions \
-H "Authorization: Bearer $TRAVERSAL_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"input": "Checkout latency spiked at 14:30 UTC",
"idempotency_key": "pagerduty-incident-P12345",
"thinking_mode": "deep"
}'
```
## Session lifecycle
Investigations are **asynchronous**. Creating a session or sending a follow-up returns immediately while Traversal investigates in the background. Connect to `GET /v1/sessions/{session_id}/events` to wait for a terminal status, then retrieve the result with `GET /v1/sessions/{session_id}`.
### Status values
| Status | Meaning |
| ------------------- | ---------------------------------------------------- |
| `running` | A new investigation is in progress. |
| `idle` | Investigation is complete and ready for follow-ups. |
| `follow_up_running` | A follow-up message is being processed. |
| `failed` | The investigation or follow-up errored or timed out. |
| `cancelled` | The investigation was cancelled. |
`idle`, `failed`, and `cancelled` are terminal statuses. Once the session reaches one of these states, no further work is in progress.
### Status transitions
```mermaid theme={null}
stateDiagram-v2
[*] --> running: POST /v1/sessions
running --> idle: Investigation complete
running --> failed: Investigation error
running --> cancelled: Investigation cancelled
idle --> follow_up_running: POST /v1/sessions/{id}/messages
follow_up_running --> idle: Follow-up complete
follow_up_running --> failed: Follow-up error
follow_up_running --> cancelled: Follow-up cancelled
```
### Stream status updates
The session events endpoint uses [Server-Sent Events](https://html.spec.whatwg.org/multipage/server-sent-events.html) to report lifecycle changes without polling. Send the same bearer token you use for other API requests and request `text/event-stream`:
```bash theme={null}
curl --no-buffer \
-H "Authorization: Bearer $TRAVERSAL_API_KEY" \
-H "Accept: text/event-stream" \
https://api.traversal.com/v1/sessions/$SESSION_ID/events
```
The stream sends a named `status` event when you connect and whenever the status changes:
```text theme={null}
event: status
data: {"id":"a1b2c3d4-e5f6-7890-abcd-ef1234567890","status":"running","title":"Elevated error rate in checkout service","input":"Checkout latency spiked at 14:30 UTC","created_at":"2026-08-10T14:35:00Z","updated_at":"2026-08-10T14:35:00Z","messages":null}
: keepalive
event: status
data: {"id":"a1b2c3d4-e5f6-7890-abcd-ef1234567890","status":"idle","title":"Elevated error rate in checkout service","input":"Checkout latency spiked at 14:30 UTC","created_at":"2026-08-10T14:35:00Z","updated_at":"2026-08-10T14:38:12Z","messages":null}
```
Each `data` value uses the same session shape as other Sessions endpoints, but `messages` is always `null`. Fetch `GET /v1/sessions/{session_id}` after the terminal event to retrieve the conversation and final result.
While the status remains `running` or `follow_up_running`, the server sends a `: keepalive` comment every 15 seconds when there is no status change. SSE clients ignore comment lines automatically. The stream closes after it sends an `idle`, `failed`, or `cancelled` status.
If the connection closes before you receive a terminal status, reconnect to the same endpoint. The first event always contains the current status, so you do not need an event cursor or `Last-Event-ID` header.
### Python example
The following example creates an investigation, waits on the event stream, and retrieves the final messages:
```python theme={null}
import json
import os
import requests
base_url = "https://api.traversal.com"
headers = {"Authorization": f"Bearer {os.environ['TRAVERSAL_API_KEY']}"}
create_response = requests.post(
f"{base_url}/v1/sessions",
headers=headers,
json={
"input": "Checkout latency spiked at 14:30 UTC",
"idempotency_key": "pagerduty-incident-P12345",
},
)
create_response.raise_for_status()
created = create_response.json()
with requests.get(
f"{base_url}/v1/sessions/{created['id']}/events",
headers={**headers, "Accept": "text/event-stream"},
stream=True,
timeout=(10, 30),
) as response:
response.raise_for_status()
for line in response.iter_lines(chunk_size=1, decode_unicode=True):
if not line.startswith("data: "):
continue
session = json.loads(line.removeprefix("data: "))
if session["status"] in {"idle", "failed", "cancelled"}:
break
result_response = requests.get(
f"{base_url}/v1/sessions/{created['id']}",
headers=headers,
)
result_response.raise_for_status()
result = result_response.json()
```
Set a read timeout longer than the 15-second keepalive interval. If your client raises a timeout or the connection drops, reconnect and continue waiting.
### Polling fallback
If your HTTP client or network path does not support streaming responses, poll `GET /v1/sessions/{session_id}` at a steady interval until the session is no longer `running` or `follow_up_running`. A 5-second interval is a reasonable default.
Investigations have a **1-hour server-side timeout**. If an investigation has not completed within that window, the session status transitions to `failed`.
## Session-specific errors
In addition to the [generic status codes](/api/overview#status-codes), the Sessions API returns these session-specific responses:
| Status code | Meaning | When it occurs |
| ----------- | ----------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `404` | Not Found | Session does not exist or does not belong to your organization. This validation happens before the event stream opens. |
| `409` | Conflict | Session is not `idle` (e.g., still `running` or `follow_up_running`). Includes `retry_after`. |
| `429` | Too Many Requests | Organization has reached the limit of 15 concurrent running sessions. Both new investigations (`POST /v1/sessions`) and in-flight follow-ups (`POST /v1/sessions/{id}/messages`) count toward this limit. Includes `retry_after` (default 30s). |
# Bring Your Own Cloud
Source: https://docs.traversal.com/architecture/byoc
Learn about Traversal's BYOC (Bring Your Own Cloud) deployment option: requirements, expectations, deployment process, and responsibility matrix.
For customers with strict security and data protection requirements, Traversal offers a BYOC (Bring Your Own Cloud) deployment option on AWS. Deployed inside of your cloud account, this option provides a private, fully segregated, single-tenant, and shared-nothing deployment of Traversal. All connections to your private data sources occur over private, secure, and always-encrypted network paths.
## Architecture
Traversal BYOC is deployed, fully-managed, and operated by Traversal inside of a separate, dedicated cloud account that you own; for example, a separate AWS account part of your AWS organization.
This separation helps protect the security of your systems and applications while providing Traversal with the access necessary to deliver and maintain a highly-available service.
Access to your private data sources and environments can be made through the [Traversal Connector](/architecture/connector) architecture (pictured above) and through dedicated, secure, PrivateLink service endpoints to cross the VPC boundary.
With the exception of reporting system and usage telemetry to Traversal, Traversal BYOC deployments do not interact with Traversal's SaaS/cloud deployments. No customer data ever leaves the BYOC environment's boundary.
Traversal's web application and authentication are currently handled by external providers outside of the BYOC deployment. A private deployment option for those services will be available in the future.
### Networking
The entirety of the Traversal BYOC environment is deployed in a dedicated VPC, with all relevant infrastructure spread across multiple availability zones and hosted in private subnets.
Traversal requires a dedicated `/20` Class A CIDR block for its VPC, with at least three `/22` private subnets — one per distinct availability zone — for its deployment and operations. Public subnets, when used, are sized as `/24` and fit within the remaining `/22` of address space.
### Compute
All compute powering Traversal's services, data stores, data pipelines, agentic workflows, and applications, runs on cloud instances (in AWS, EKS-on-EC2) and managed cloud services inside of the provided, dedicated cloud account.
### AI Models and Inference
Traversal uses frontier AI models (LLMs) to power its AI SRE agent. At your discretion, those models can be cloud-hosted (from LLM providers directly), routed through a private gateway, or fully hosted in your cloud environment (in AWS, with Bedrock).
### Access control
Access to your BYOC deployment of Traversal by your users is always secured and controlled by Single Sign-on (SSO) with your Identity Provider.
Deployment and management of infrastructure, as well as operational access by Traversal, is performed through a cross-account IAM AssumeRole, scoped to the dedicated AWS account that hosts your Traversal BYOC deployment.
Access by Traversal operators for maintenance and support is made through a secure and audited [Tailscale](https://tailscale.com)-based VPN, restricted to authorized Traversal personnel only, and always secured by SSO with two-factor authentication.
### Telemetry
Two categories of telemetry data leave your BYOC deployment: **system telemetry**, used to monitor and support your deployment, and **usage telemetry**, used for product analytics, customer success, and billing. They're handled differently.
#### System telemetry
By default, system telemetry (metrics, logs, and traces about your Traversal deployment) is sent to Traversal's centralized observability solution outside of your environment. If needed, some telemetry attributes can be redacted or filtered before leaving your BYOC deployment.
If your requirements call for it, you can instead opt to keep system telemetry and monitoring entirely within your environment. In that configuration, alerts derived from that telemetry must still be allowed to egress to Traversal's on-call solution ([PagerDuty](https://pagerduty.com)), so Traversal operators can be notified of and respond to issues affecting your deployment.
#### Usage telemetry
Regardless of your system telemetry configuration, Traversal always collects the following usage telemetry, which leaves your environment:
* **Client-side user analytics** — events generated by user interactions in the web app
* **Server-side user analytics** — user events and feedback signals, such as thumbs up/down reactions
* **Usage data** — investigations initiated, follow-ups initiated, and activity of Traversal workers
Usage telemetry never contains customer data.
This list reflects what Traversal collects today and may change at any time as our needs evolve, particularly to support product analytics, customer success, and customer billing.
All telemetry data is always sent securely over HTTPS to authenticated endpoints.
### Data security
All data stored by Traversal in databases, network-attached storage volumes, and cloud storage buckets remains in your AWS account dedicated for Traversal's BYOC deployment.
Data is always encrypted at rest and in-transit by best-in-class encryption from your cloud provider (AES-256 and TLS1.3).
### Network security
Within the Traversal platform, all service-to-service traffic is authenticated and encrypted in-transit with mTLS/TLS1.3.
Connections to your private data sources and environments are always made through dedicated, private, secure, and encrypted network paths (PrivateLink) with the [Traversal Connector](/architecture/connector).
#### Ingress
By default, no inbound connections to your Traversal BYOC deployment can be made from the public internet. Your access and connection to Traversal is routed through your private network.
The following inbound connections (through secure PrivateLink service endpoints) must be established and allowed:
* From your users to Traversal's web application and API;
* From your Traversal Connector to the Edge Controller in your Traversal BYOC deployment;
* If deployed, from your Traversal Processor to your Traversal BYOC deployment's data ingest endpoint.
If you use the [Traversal Slack integration](/integrations/slack), inbound webhook connections from Slack to the Traversal API in your BYOC deployment must be allowed. This is typically `https://slack..traversal.com`.
#### Egress
To ensure the proper operation of your Traversal BYOC deployment, the following outbound connections must be allowed:
* To `login.traversal.com` and `*.auth0.com` for authentication;
* To `telemetry.traversal.com` for reporting usage telemetry, and — unless you've opted to keep system telemetry within your environment — system telemetry, to Traversal;
* To `*.rudderstack.com` for user telemetry;
* To `*.pagerduty.com` for monitoring and alerting;
* To `api.sendgrid.com` for transactional email delivery;
* To `*.tailscale.com` for Traversal's Tailscale-based operational VPN (required for upgrades, maintenance, and support);
* To `*.okta.com` for authentication of Traversal operators with our identity provider;
* To `*.letsencrypt.org` for issuance of SSL/TLS certificates;
* To `*.github.com` and `*.githubusercontent.com` for pulling Traversal platform repositories (required for deployment and upgrades);
* To `*.ghcr.io` and `*.docker.io` (and `*.cloudflare.docker.com` for Docker Hub image layers) for pulling third-party Docker images;
* To `*.pypi.org` and `files.pythonhosted.org` for pulling Python dependencies;
* If configuring integrations with any third-party vendor, to their endpoint domains:
* To `*.slack.com` for the Traversal Slack app
* To `api.anthropic.com` and/or `api.openai.com` for access to our LLM providers if you're not using Bedrock
Your Traversal BYOC deployment runs on AWS services (such as EKS, EC2, S3, ECR, STS, and — if you use Bedrock for inference — Amazon Bedrock) and must be able to reach their service APIs. You can satisfy this in one of two ways:
* **Allow egress to AWS service endpoints** by permitting outbound HTTPS to `*.amazonaws.com` and `*.api.aws`; or
* **Provision VPC endpoints** for the required AWS services within your VPC, keeping traffic on the AWS network so it never traverses your egress path.
Additionally, you must ensure that the following domains and endpoints are reachable by your end users:
* Your Traversal BYOC deployment at `.traversal.com` and `api..traversal.com`;
* `login.traversal.com` and `*.auth0.com` for authentication;
* Our user telemetry data processors:
* `*.rudderstack.com`
* `*.ingest.sentry.io`
## Deployment
To deploy Traversal BYOC, after establishing your agreement with Traversal and reviewing the responsibility matrix below, you must:
Provision a new, dedicated AWS account attached to your organization, specifically for use by and for Traversal.
Deploy the [Traversal BYOC AWS CloudFormation Template](https://templates.traversal.com/traversal-deploy-bootstrap.yaml) into this account to initialize a provisioning IAM role used by Traversal for deployment and operations.
Provide the AWS account ID and IAM role reference (ARN) from the outputs of the CloudFormation stack to Traversal.
Traversal then proceeds with the deployment of its platform and provides you with an access endpoint URL – typically `https://.traversal.com`. Following this initial deployment, and depending on the needs of your environment, you will:
Deploy the [Traversal Connector](/architecture/connector) to enable access to your private telemetry and data sources in your environment
Work with Traversal's Forward Deployed Engineers to establish PrivateLink endpoints between your environment and your Traversal BYOC deployment
## Operations
Traversal BYOC deployments are fully managed and operated by Traversal, and providing the same high-availability guarantees as our SaaS offering. Experienced Traversal operators continuously monitor, tune, and deploy updates to your Traversal BYOC deployment to ensure the highest quality of service for Traversal users.
## Responsibility matrix
| Category | Component | Traversal | Customer |
| :------------- | :------------------------------------------------------------------------------------ | :-------------------: | :-------------------: |
| Infrastructure | Provision and initialize a dedicated cloud account | | |
| | Management of compute and storage quotas | | |
| Networking | VPC and networking infrastructure in the BYOC account | | |
| | PrivateLink / VPC Service Endpoints | | |
| | AWS service connectivity (egress allowlisting or VPC endpoints) | | |
| | Firewall | | |
| Security | Provisioning IAM role | | |
| | Data encryption at rest and in transit | | |
| | Secrets management | | |
| | Application-level authentication and authorization | | |
| Data | Source data sovereignty and governance | | |
| | Governance on Traversal indexed data | | |
| | Backups and data retention | | |
| Software | Deploy the Traversal BYOC environment | | |
| | Maintain and operate the Traversal BYOC environment | | |
| | Deliver a highly-available Traversal experience | | |
| | Configure users, roles, and data/service integrations | | |
| | Use the deployed Traversal BYOC environment to investigate and resolve your incidents | | |
| Billing | Cloud account and infrastructure costs | | |
| | Licensing costs of Traversal | | |
| | Provide metering-based invoices for usage of Traversal | | |
# Traversal Connector
Source: https://docs.traversal.com/architecture/connector
The Traversal Connector is a lightweight service deployed in your environment that enables the Traversal platform to securely access your private data sources and internal systems — without requiring any inbound network access to your environment.
## Architecture
The Traversal Connector operates on a pull-based, outbound-only connection model. Rather than the Traversal platform connecting inbound to your environment, the Traversal Connector initiates a persistent, bidirectional, encrypted tunnel to the Edge Controller in Traversal's SaaS control plane or in your [Traversal BYOC deployment](/architecture/byoc). The Traversal platform sends requests over this tunnel, and the Traversal Connector executes them against your internal services and returns the responses over the same tunnel.
This design means:
* No inbound firewall rules are required — the Traversal Connector only makes outbound connections
* Your network perimeter is preserved — no services are exposed or listening for external traffic
* You retain full control — the Traversal Connector runs in your environment, under your operational purview
| Component | Runs in | Role |
| :--------------------- | :--------------------------------------------------------------------------- | :--------------------------------------------------------------------------------------------------------------- |
| Your internal services | Your environment |
Data sources and APIs queried by the Traversal Connector over your private network
|
| Traversal Connector | Your environment |
Initiates the outbound tunnel
Executes HTTP requests against your internal services
|
| Edge Controller | Traversal's SaaS control plane or your [BYOC](/architecture/byoc) deployment |
Receives the tunnel connection
Dispatches requests on behalf of the Traversal platform
|
The data flow is:
```
Traversal Platform → Edge Controller → (mTLS tunnel) → Traversal Connector → (HTTPS) → Your Internal Services
```
Responses return along the same path. The tunnel is always initiated outbound by the Traversal Connector.
## Security
### Encryption
The Traversal Connector establishes a dedicated, private tunnel to the Edge Controller, encrypted end-to-end with mTLS. This tunnel can optionally be established through a PrivateLink service endpoint, ensuring traffic never traverses the public internet. The Traversal Connector authenticates itself to the controller using a client certificate, and verifies the controller's identity using a trusted CA certificate.
### No inbound network access
The Traversal Connector initiates all connections outbound. It does not listen on any ports for external traffic, and does not require any inbound firewall rules or publicly reachable endpoints.
The only listening port is an internal health-check endpoint used by your container orchestrator to monitor the service's readiness.
### No data persistence
The Traversal Connector does not store, cache, or log any request or response data. It is a stateless, transient forwarding service — data flows through it and is not retained.
### Forward proxy support
For environments that require all outbound traffic to route through a corporate proxy, the Traversal Connector supports connecting to the Edge Controller through an HTTP CONNECT proxy.
## Telemetry
The only external connection the Traversal Connector makes — beyond the tunnel to the Edge Controller — is to `telemetry..traversal.com` to report its own operational telemetry (metrics, traces, and logs) using OpenTelemetry (OTLP).
No customer data is included in the telemetry. Only operational metrics such as tunnel connectivity status, request latency, and error rates are reported.
# Deployment options
Source: https://docs.traversal.com/architecture/intro
Traversal's architecture, security model, and deployment options — SaaS, single-tenant SaaS, and BYOC.
Traversal is agent-less and always read-only. In most cases, Traversal does not require the deployment of any software in your environment, and it cannot modify your systems or data. The [Traversal Connector](/architecture/connector) can be deployed to securely reach private data and observability sources, and the [Traversal Processor](/setup/processor) can be deployed to minimize egress volumes and apply data redaction before data leaves your environment.
Both the [Traversal Connector](/setup/connector) and [Traversal Processor](/setup/processor) run in any environment — AWS, GCP, Azure, OCI, on-prem Kubernetes, or any container runtime. Kubernetes is the easiest path: we publish Helm charts for both.
Traversal offers three deployment options: **SaaS** to get going in minutes with Traversal's cloud, **single-tenant SaaS** for a dedicated, isolated deployment fully managed by Traversal in Traversal's own cloud, or **BYOC** (Bring Your Own Cloud) for a dedicated, single-tenant, fully-managed deployment in your own cloud account.
Traversal has received and maintains its SOC 2 Type II attestation, applicable to all deployment options.
The easiest way to get started in minutes with Traversal is by signing up on Traversal's SaaS/cloud offering. Your data and connections to your observability providers remain fully secure, whether they are cloud vendors or you host them in your internal infrastructure.
Communication from the [Traversal Connector](/setup/connector) and [Traversal Processor](/setup/processor) to the Traversal SaaS travels over the public internet and is secured with mTLS — both sides authenticate with certificates, and all traffic is encrypted in transit.
At every layer — from frontend to database and agentic workflows.
On LLM calls, under zero-data-retention agreements with Traversal's LLM providers.
Connect to your private data sources and systems with the [Traversal Connector](/architecture/connector).
Reduce data leaving your environment with the [Traversal Processor](/setup/processor).
Customer-configured data redaction and obfuscation at the edge.
SSO and MFA for all users.
With Traversal single-tenant SaaS, you get a dedicated, isolated deployment of Traversal, fully owned, managed, and operated by Traversal — but running in its own dedicated cloud account, separate from Traversal's multi-tenant SaaS environment, exclusively for your organization.
An isolated Traversal environment in its own dedicated cloud account, exclusive to your organization.
All compute, storage, data stores, and encryption keys are exclusive and dedicated — never shared with other customers.
Reachable at your own dedicated domain, which can be a domain you delegate to Traversal.
Default connectivity is the same as multi-tenant SaaS (public internet with mTLS), with PrivateLink available as an add-on, on request.
Customer-configured data redaction and obfuscation at the edge.
Secure and fully auditable management by skilled Traversal operators — no cloud account for you to provision.
For the full architecture details, see the [single-tenant SaaS guide](/architecture/single-tenant-saas).
With Traversal BYOC, you get a dedicated, single-tenant, fully-managed deployment of Traversal in your own cloud account, with fully private access and network paths to your Traversal deployment, and full custody of your data in dedicated data stores and cloud storage buckets.
Full Traversal environment in a dedicated cloud account within your infrastructure boundary.
All compute, storage, encryption keys, and agentic runtime is exclusive and dedicated.
All traffic flows through private network paths and customer-controlled proxies (PrivateLink).
Options for BYOK, LLM gateways, and private inference endpoints.
Customer-configured data redaction and obfuscation at the edge.
Secure and fully auditable management by skilled Traversal operators.
For the full architecture details, deployment process, and responsibility matrix, see the [BYOC guide](/architecture/byoc).
# Single-tenant SaaS
Source: https://docs.traversal.com/architecture/single-tenant-saas
Learn about Traversal's single-tenant SaaS deployment option: a dedicated, isolated deployment of Traversal fully owned and operated by Traversal in its own cloud.
Traversal's single-tenant SaaS deployment option provides a dedicated, isolated deployment of Traversal exclusively for your organization, running in its own dedicated cloud account — separate from Traversal's multi-tenant SaaS environment and from every other customer's deployment. Unlike multi-tenant SaaS, no infrastructure is shared with other customers. Unlike BYOC, that dedicated cloud account is owned and operated by Traversal, not yours — so there's no cloud account for you to provision or maintain.
## Architecture
The Traversal Cloud Environment runs in a dedicated, Traversal-owned AWS account — separate from the shared account that powers multi-tenant SaaS — isolated to your organization alone, and reachable at your own dedicated domain, which can be a domain you delegate to Traversal.
### Networking
By default, connectivity to your single-tenant SaaS deployment is the same as multi-tenant SaaS: traffic travels over the public internet and is secured with mTLS — both sides authenticate with certificates, and all traffic is encrypted in transit.
[PrivateLink](/setup/privatelink) is available as an optional add-on, on request, for organizations that need private network paths to their dedicated deployment.
### Compute
All compute powering your single-tenant SaaS deployment — services, data stores, data pipelines, agentic workflows, and applications — runs on dedicated cloud instances and managed cloud services, exclusively for your organization, within Traversal's own AWS account(s).
### AI Models and Inference
Traversal uses frontier AI models (LLMs) to power its AI SRE agent. As with Traversal's other deployment options, LLM calls are routed through Traversal's managed default, under zero-data-retention agreements with its LLM providers.
### Access control
Access to your single-tenant SaaS deployment by your users is always secured and controlled by Single Sign-on (SSO) with your Identity Provider.
Because your single-tenant SaaS deployment runs inside Traversal's own cloud account rather than a customer network boundary, there's no VPN or cross-account bridge for Traversal operators to cross. Operational and administrative access by Traversal is made directly to Traversal's own cloud account, through Traversal's normal internal access controls, secured by SSO with second-factor authentication and fully audited.
### Telemetry
System telemetry used to monitor your single-tenant SaaS deployment, as well as usage data for billing purposes, flows to Traversal's internal observability and usage metering solutions — the same as Traversal's multi-tenant SaaS offering, since your deployment already runs within Traversal's own environment.
All telemetry data is always sent securely over HTTPS to authenticated endpoints.
### Data security
All data stored by Traversal in databases, network-attached storage volumes, and cloud storage buckets for your single-tenant SaaS deployment is dedicated and exclusive to your organization — never shared with other customers.
Data is always encrypted at rest and in-transit by best-in-class encryption (AES-256 and TLS1.3).
### Network security
Within the Traversal platform, all service-to-service traffic is authenticated and encrypted in-transit with mTLS/TLS1.3, consistent with the rest of Traversal's platform.
Your single-tenant SaaS deployment has dedicated ingress at your own domain, isolated from other customers' traffic.
## Operations
Single-tenant SaaS deployments are fully managed and operated by Traversal, with the same operational rigor and high-availability guarantees as Traversal's BYOC and multi-tenant SaaS offerings. Experienced Traversal operators continuously monitor, tune, and deploy updates to your deployment to ensure the highest quality of service.
# Changelog
Source: https://docs.traversal.com/changelog
New features, improvements, and updates to Traversal.
## Edit custom instructions from chat
You could already tailor a Worker by editing its
[custom instructions](/using-traversal/worker-features#custom-instructions) on
the Workers page. Now you can just tell it what you want in chat — from Slack,
Microsoft Teams, or a web Worker — and it updates those instructions itself.
Last-mile tailoring happens directly in the tools where you already work.
Say what you want in plain language. For example, you can ask your Worker to:
* **Prioritize or ignore certain alerts** — tell an Alert Worker which alerts
matter and which ones to leave alone.
* **Tag the responsible team** — have an Incident Worker notify the right
people as more information emerges.
* **Check the deployment dashboard first** — point a Worker at the evidence
you'd reach for yourself.
The direction still applies days later, long after the conversation has
scrolled away.
**How this differs from memories and schedules.** A
[memory](/using-traversal/knowledge-bank) is a fact about your systems —
learned during an investigation, held until a person approves it, then
available to every future investigation across your organization. A schedule
is *when* a Worker checks in. A standing instruction is *how one Worker
handles one channel*: what it picks up and what it ignores, scoped to that
channel and nowhere else.
See [Workers](/using-traversal/workers).
## Custom Workers
[Workers](/using-traversal/workers) are a powerful, flexible harness for
Traversal's Production World Model and Causal Search Engine.
[Incident Workers](/using-traversal/incident-workers) and
[Alert Workers](/using-traversal/alert-workers) package that intelligence into
opinionated experiences for specific SRE use cases. Custom Workers open the
same capabilities to the long tail of work your team wants Traversal to take
on.
Give a Custom Worker a mission in plain language, and it follows the channel
and acts on that direction. Deploy one into:
* **A deployment channel** — to watch a rollout as it goes out and flag the
change that caused a regression.
* **A support channel** — to surface the recurring problem behind a run of
customer reports.
* **A team or project channel** — to follow ongoing operational work and keep
a running picture of where it stands.
Anywhere you want the flexibility and power of a Worker and the channel is
neither an incident bridge nor an alert feed.
Pick **Custom** as the use case when you deploy a Worker, then write its
mission. It appears as "Traversal Custom Worker" and can be filtered on the
Workers page like any other kind.
See [Workers](/using-traversal/workers).
## The Traversal Workers release
Incident Workers in general availability, Alert Workers in public beta, and a
new web experience for both.
**[Incident Workers](/using-traversal/incident-workers) are now generally
available.** A Worker is the easiest way to get Traversal's full value during
an incident, because there is nothing to invoke: nobody has to tag it, and
nobody has to learn how to engage with it. It joins when the channel opens and
helps across the whole lifecycle — including the questions you wouldn't have
thought to ask — investigating before anyone asks, adapting as the evidence
changes, verifying recovery against live telemetry rather than a deployed fix,
and drafting the post-mortem when it's over. Feedback across thousands of
incidents in public beta has been overwhelmingly positive, and Workers are now
the default incident experience.
**[Alert Workers](/using-traversal/alert-workers) are now in public beta.**
This brings the same proactive Worker to alert channels, optimized for the
high-volume stream an on-call engineer actually faces. The intent is to take
the toil out of monitoring an alert channel: the Worker triages and prioritizes
every alert that fires, escalates the ones worth your attention, reuses its
earlier work on a familiar fire rather than investigating twice, and points out
the alert definitions worth fixing.
Workers were built to be native to Slack and Microsoft Teams, and there is now
a **[web experience](/using-traversal/workers-console)** for both. It covers the
console — where you deploy Workers, set auto-deploy rules, and see your entire
fleet at a glance — and a detailed view of each Worker, with its live
scratchpad, activity feed, private chat, schedules, and custom instructions.
Read the [Workers documentation](/using-traversal/workers) and the
[launch announcement](http://traversal.com/blog/traversal-workers-are-now-generally-available).
## Approve memories from Slack
When Traversal proposes a [memory](/using-traversal/knowledge-bank) during a
Slack investigation, you can now approve or reject it in the same thread.
The card shows the memory name and the text being saved. The memory stays
pending until a person decides — same as in the web app.
See [using Traversal in Slack](/using-traversal/slack).
## Amazon CloudWatch integration
Traversal now connects directly to [Amazon CloudWatch](/integrations/cloudwatch)
to pull metrics and logs during investigations, using a cross-account IAM
role. Deploy a CloudFormation template in the AWS account you want Traversal
to read from, and it correlates CloudWatch signals with your other connected
telemetry.
See the [setup guide](/integrations/cloudwatch) to connect your first account.
## Manage Knowledge Bank content through the API
The new [Knowledge Files API](/api/knowledge-files) lets you manage the
knowledge your team contributes to Traversal programmatically. List, upload,
download, replace, and delete runbooks, reference documents, and agent skills
to keep Traversal in sync with your existing documentation workflows.
The API covers team-authored documentation and skills. It does not expose
Traversal's Auto-created knowledge or agent memories.
## The new Knowledge Bank
Knowledge Bank is now one place to see everything Traversal knows about your
systems — what it worked out on its own, and what your team has taught it.
Four kinds of knowledge live there:
* **Auto-created** — Traversal's own documentation of your environment, written
from your connected integrations: code, metrics, logs, traces, alerts,
tickets, and existing docs. It takes no input from your team to produce, and
Traversal keeps regenerating it as your systems change — so anyone can read
up on an unfamiliar service instead of asking around. Where you know
something the data doesn't show, leave a comment and Traversal folds it in.
* **Skills** — reusable procedures that shape the steps Traversal takes, so an
investigation follows the way your team actually works.
* **Documentation** — runbooks, context files, and the constraints and
workarounds your systems don't record, expanding what Traversal has to reason
over.
* **Memories** — what Traversal has learned working alongside you. It proposes
what's worth keeping and waits for a person to approve, so it gets sharper
the more you use it.
[Read the docs](/using-traversal/knowledge-bank).
## Traversal comes to Microsoft Teams
Traversal is now available in Microsoft Teams. Mention **@Traversal** in a
channel, group chat, or direct chat with an incident description, and it runs
a root-cause investigation across your connected observability tools — posting
the report back into the thread with citations and a link to the full session
in the web app. Ask follow-up questions in the same thread to dig deeper.
A Teams administrator installs the app once; see the
[installation guide](/integrations/microsoft-teams) to get set up, and
[using Traversal in Teams](/using-traversal/microsoft-teams) for what you can
do with it.
## Stream session status over the API
The [Sessions API](/api/sessions#stream-status-updates) now provides a
Server-Sent Events endpoint at `GET /v1/sessions/{session_id}/events`. Connect
once to receive the current status and each status change, instead of polling
while Traversal investigates.
The stream sends keepalive comments during long-running investigations and
closes when the session becomes `idle`, `failed`, or `cancelled`. You can then
retrieve the final conversation with `GET /v1/sessions/{session_id}`.
## Throughput improvements for the Traversal Processor
[Traversal Processor](/setup/processor) 0.4.0 ships alongside the 0.4.0 Helm
chart with a rebuilt ingest and processing engine, delivering massive
improvements to throughput and reliability under heavy load.
* **Higher throughput** — up to \~80,000 records/s (\~300 MB/s) per pod on the
`large` size tier.
* **One-knob sizing** — `size: normal|large` sets CPU, memory, and internal
tuning to validated presets. See [sizing and throughput](/setup/processor#sizing-and-throughput).
* **Deeper observability** — a new and more detailed set of `processor.*`
metrics.
See the [Processor Helm chart changelog](/setup/processor-changelog) for
the chart-side changes in this release, including a breaking values-file
reorganization.
## Choose investigation depth over the Sessions API
The [Sessions API](/api/sessions) now accepts an optional `thinking_mode` on
both session creation and follow-ups. Pass `deep` to force a thorough
root-cause analysis, `fast` for a quick exploratory pass, or leave it as the
default `auto` to let Traversal pick — existing integrations need no changes.
Follow-ups now also count toward the same per-organization concurrency limit
as new sessions, so a burst of follow-ups is throttled with a `429` the same
way session creation already is.
## A clearer picture of Traversal adoption
We've fully revamped the **Analytics** page, now available to Org Admins at
[app.traversal.com/analytics](https://app.traversal.com/analytics). It gives
you a clear picture of how Traversal is being adopted and used across your
organization, all in one place.
* **Usage over time** — sessions and active users, with period-over-period
comparison so you can track adoption week over week.
* **Who's using Traversal** — top users and top channels, a Top Investigators
panel, and an activity heatmap showing engagement across your team.
* **Session detail** — breakdowns by origin (Web, Slack, API, MCP) so you can
see where investigations are being started.
## Workers read live incident transcripts
When your team runs an incident on a live video call, a
[Worker](/using-traversal/workers) can now read the call's transcript in real
time and fold it into its work. It picks up what's said out loud but never
typed into Slack — the fix someone just tried, the theory forming on the call,
even how people are reacting to Traversal's own suggestions — and uses those
cues to adjust course and stay current with where the room actually is.
Live calls generate a lot of chatter, so under the hood Traversal distills the
transcript down to what matters — cleverly filtering the noise rather than
handing the Worker the raw feed — so it stays sharp and focused without
overloading its context.
Live transcripts are in private beta. Talk to your Traversal contact to
connect your team's calls.
## Jump to anything with ⌘K
One for the power users: press **⌘K** anywhere in the web app to open a quick
search. Start a new investigation, jump to any past one by typing its name, or
open your integrations, all from the keyboard. Arrow keys move through the
list and **Enter** opens, so you never have to reach for the mouse.
## 24-hour time format
Prefer a 24-hour clock? A new **Time format** option in your
[personalization settings](https://app.traversal.com/settings/personalization)
switches Traversal from 12-hour AM/PM to 24-hour. It flows through the entire
app — evidence displays, investigation timelines, the times you type into an
investigation, everywhere a timestamp appears — so times read the way you
expect no matter where you are.
## Single-tenant SaaS deployment
A new deployment option that sits between multi-tenant SaaS and full BYOC: a
dedicated, single-tenant Traversal instance running in an isolated,
Traversal-operated AWS account — its own domain, no infrastructure shared with
any other customer, SSO required, and data encrypted at rest (AES-256) and in
transit (TLS 1.3). You get hard tenant isolation without having to run and
maintain Traversal in your own cloud. See
[deployment options](/architecture/intro).
## Navigate long sessions
When you're deep in an investigation, it's natural to keep asking follow-ups —
and a session can grow into a long back-and-forth. A new navigator makes those
long sessions easy to move through by their question-and-answer pairs — jump
straight back to any exchange instead of scrolling, and hover to preview what's
there.
## Traversal Workers
A new, more capable way to work with Traversal. Rather than waiting to be
summoned, a **Worker** joins a channel and works alongside your team like a
teammate — reading the situation, gathering context, and doing what's worth
doing on its own. During an incident, a Worker joins the moment the channel is
created, investigates autonomously, follows the incident in real time, and
drafts a post-mortem when it resolves.
Workers are in beta, enabled per team. [Read the docs](/using-traversal/workers)
or the [launch announcement](https://www.traversal.com/blog/ai-sre-proactive-incident-response-traversal-workers).
## Timezone preference
Set a personal timezone in your
[personalization settings](https://app.traversal.com/settings/personalization)
so every timestamp across Traversal shows in your local time instead of UTC —
no more mental math when reading an incident timeline.
## Connect your own MCP servers
Connect any external [MCP server](/integrations/mcp-servers) as a data source —
a great way to tailor Traversal to your stack with tools and systems we don't
support out of the box. Point it at an internal service or a third-party MCP
server and its tools become available to the agent during investigations.
## New navigation rail
A redesigned left navigation rail makes it faster to move between your
sessions, integrations, and settings, with a cleaner layout that stays out of
your way.
## Run MCP investigations in the background
Investigations over [MCP](/using-traversal/mcp) can now run in the background.
Your agent can kick one off and keep working instead of waiting on every call,
then collect the result once it's ready — and run several investigations at
once.
## Channel management
A new settings page to see every Slack channel Traversal is active in and
manage them in one place — turn on
[Alert Intelligence](/using-traversal/alert-intelligence) for a channel and set
its [custom instructions](/using-traversal/customization), so every
investigation in that channel follows the same guidance no matter who runs it.
## Smoother follow-up quoting
Quoting part of a previous response in your follow-up is smoother — highlight
the text you want to reference, hit **Add to chat**, and it's attached to your
next message with the composer focused and ready, so it's clear exactly what
you're asking about.
## Generate files to download
Traversal can now generate files you can download and share — reports, PDFs,
spreadsheets, and more. Ask for a deliverable and the agent produces it inline,
ready to download straight from the session.
## Slack session visualization
Sessions that originate in Slack now render with a custom Slack visualization
in the web chat — showing who sent each message, which channel it came from,
and whether any custom instructions were applied. Because Slack channels are
multiplayer, knowing *who* said what matters — a shared incident channel has
many voices, and the visualization keeps them straight. Alerts are rendered
more cleanly too, so a Slack-triggered investigation reads clearly from the
web app.
## Ask Traversal how it works
Traversal can now answer questions about its own documentation, so you can ask
how a feature works right where you're working. New users can even have
Traversal onboard them — walking through what it does and how to get the most
out of it.
## Custom instructions
Give Traversal standing instructions so you don't have to re-type your
preferences every time — point it at specific runbooks, prescribe steps, or
set output preferences like "be brief" or "suggest an owning team." Two layers:
* **Personalization** — your own instructions, applied to every investigation
you run, in Slack and the web app.
* **Channel custom instructions** — instructions applied to every
investigation in a Slack channel, no matter who runs it. Admins can also
toggle [Alert Intelligence](/using-traversal/alert-intelligence) mode per
channel from the same console.
See [customization](/using-traversal/customization).
## Connect over MCP with or without a trailing slash
Connecting over [MCP](/using-traversal/mcp) is more forgiving — the endpoint
now works whether or not your client includes a trailing slash, so a small
config difference no longer blocks the connection.
## Overhauled chain-of-thought
Traversal's "chain-of-thought" loading state has been redesigned to be more
legible and to better represent what the agent is actually doing as it
investigates — including a clearer view of which integrations it's using, and
when.
## Redesigned login page
Your first touchpoint with Traversal got a full refresh — rebuilt on the new
design system for a cleaner, more modern first impression, consistent with the
rest of the product.
## A new design system
We've rolled out a new design system across every page of the product — a more
open, modern feel with lighter borders and larger corner radii. We also
refined the color palette and sharpened legibility throughout, including a
polished dark mode. It's especially noticeable on the home page and the
session view, and in how we render code snippets.
## Lightning mode
A fast investigation path built for the rapid back-and-forth of agent-to-agent
work over [MCP](/using-traversal/mcp). A first pass typically returns in 30
seconds to two minutes — fast enough for a real conversation between your agent
and Traversal rather than a single slow round trip. You can always follow up to
go deeper.
## Tune the definitions of noisy alerts
When [Alert Intelligence](/using-traversal/alert-intelligence) flags an alert
as noise, its response now includes a **Tune Alert Definition** button. Click
it and Traversal drafts an updated alert definition — the specific change that
stops the alert from firing on non-issues — ready for you to review and apply.
## PrivateLink connectivity
Connect your environment to Traversal over AWS PrivateLink so traffic between
your network and your Traversal deployment stays on the AWS backbone and never
crosses the public internet. Paired with the outbound-only Connector, it gives
you a fully private data path — no public endpoints, nothing exposed. See
[PrivateLink](/setup/privatelink).
## Traversal Connector
A lightweight, outbound-only service you deploy in your environment that lets
Traversal securely reach your private data sources — databases, internal APIs,
and on-prem observability — without any inbound access. It opens a single
encrypted, outbound tunnel, so there are no inbound firewall rules to manage
and nothing in your network is exposed or listening for external traffic. It
runs entirely in your environment, under your operational control. See the
[Traversal Connector](/architecture/connector).
## Redaction with the Processor
The Traversal [Processor](/setup/processor) runs alongside the Connector in
your environment and [redacts](/setup/redaction) sensitive data — PII, secrets,
customer identifiers, or any fields you define — before it ever leaves your
network. You own the rules (regex patterns and structured JSON keys), so
Traversal only ever receives what you've explicitly allowed, and sensitive
values never reach our control plane.
## Request analysis
When an investigation traces a request across services, Traversal now renders
an inline **request-analysis** component right in its answer — a visual diagram
of the request's path across services, where it broke, and the errors behind
each hop, alongside the key logs. Because it can pull from many log indices at
once, you can see exactly where in a request an issue arose — the failing hop,
the service, and the error in context.
## Personal API keys
Create and manage your own API keys from settings to authenticate with the
[Traversal API](/api/overview) and the [MCP server](/using-traversal/mcp).
## Alert summary reports
Run `/traversal alert-summary` in any alert channel to generate an insights
report over a time period you choose — a digest of what fired, how often, and
what's worth attention — so you can spot noisy monitors and recurring issues at
a glance.
## Alert Intelligence
Traversal now replies to **every** alert in the Slack channels it watches with
an assessment — how urgent it is and why, and, for alerts that are just noise,
how to tune them. You never have to tag anything.
It's also efficient. Alert channels are noisy, and the same underlying issue
often trips many alerts at once. Rather than re-investigate each one from
scratch, Traversal recognizes when a new alert shares a root cause with one it
already looked into and reuses that finding — caching the response instead of
spending tokens re-deriving the same conclusion. See
[Alert Intelligence](/using-traversal/alert-intelligence).
## Programmatic investigations (Sessions API)
Kick off investigations, ask follow-ups, and pull results entirely over HTTP
with a self-service API key — the same capabilities you get in the app, now
available to your own scripts, runbooks, and internal tools. See the
[API reference](/api/overview).
## Easier integration setup
Connecting a data source is more self-serve: every integration shows a live
connection-health badge, credentials are verified the moment you connect (and
rolled back if they fail), and you can test a query against a connected
integration right from settings.
## Work with Traversal from your AI client (MCP)
The [Traversal MCP server](/using-traversal/mcp) lets any compatible AI client
— Claude Code, Cursor, Claude Desktop, and others — run investigations, ask
follow-ups, and retrieve results. When called from another agent, Traversal
acts as a **peer your local agent works alongside**, handing work back and
forth until the incident is understood.
## Post-mortems
Traversal produces a first-draft post-mortem so your team can skip the assembly
work — reconstructing the timeline, writing up what happened, drafting
follow-ups — and focus on the fixes. Generate one on demand by tagging
Traversal. See [post-mortems](/using-traversal/post-mortems).
## Knowledge Bank
Teach Traversal the operational context that lives only in your team's heads —
which services are business-critical, which alerts are usually noise, how your
team actually debugs an issue. See [Knowledge Bank](/using-traversal/knowledge-bank).
# Authentication
Source: https://docs.traversal.com/get-started/authentication
How users sign in to Traversal, how organizations manage access, and how to troubleshoot common authentication issues.
Traversal uses single sign-on (SSO) to authenticate users. Your organization administrator controls who can access Traversal and which sign-in methods are available.
## How sign-in works
Visit [app.traversal.com](https://app.traversal.com) (or your dedicated deployment's address, if you're on [single-tenant SaaS](/architecture/single-tenant-saas) or [BYOC](/architecture/byoc)) and enter your email address. Traversal uses your email domain to match you to your organization and present the available sign-in methods. For SSO users, Traversal identifies your organization's identity provider and redirects you there automatically.
If no organization matches your email, Traversal displays an **"Email not found"** error.
Traversal matches your email to an organization and displays the available sign-in methods.
Sign in using the method your organization has configured (e.g., enterprise SSO, Google, or email magic link).
After successful authentication, you are signed in and can start using Traversal immediately.
## Organization access models
Your administrator chooses one of two access models when setting up your organization.
### Invite-only
New users **must** receive an email invitation before they can sign in. This is the default for most organizations and provides the tightest access control.
* An administrator sends an invite to your email address.
* You receive an email with a sign-in link containing a one-time invite code.
* Click the link and authenticate with your provider to complete sign-in.
* The invite code is consumed on first use and cannot be reused.
### Open access (domain-based)
Any user with a matching email domain can sign in without an explicit invite. This is useful for organizations that want frictionless onboarding for all employees.
* No invite is required — just sign in with your corporate email.
* Your email domain must match one of the domains your administrator has registered.
Even in open-access organizations, users with email domains that are not registered to the organization still require an invite.
## Inviting users
Organization administrators can invite new users from the Traversal web app.
Navigate to **Settings** and select **User Management**.
Enter one or more email addresses (comma-separated) and choose a role:
* **Member** — can run investigations, view results, and use integrations.
* **Admin** — everything a member can do, plus manage users, integrations, and organization settings.
Each invitee receives an email with a personalized sign-in link. The link includes a one-time invite code tied to their email address.
### Inviting an existing user
If the email address you invite belongs to an existing Traversal user in your organization, the system checks their current role:
* **User already has the role** — no action is taken.
* **User needs a role upgrade** — the new role is granted immediately (e.g., promoting a member to admin).
No duplicate accounts are created.
## Enterprise SSO onboarding
Enterprise Single Sign-On (SSO) allows your organization to authenticate users through an existing identity provider (IdP) such as Okta, Microsoft Entra ID, Google Workspace, or any SAML 2.0 / OIDC-compliant provider. This eliminates separate credentials — your team signs in to Traversal with the same accounts they already use.
Across all identity providers, you **must** assign users or groups to the Traversal application in your IdP. Users who are not assigned to the client application cannot sign in, even if they have valid credentials. This is the most common cause of failed SSO logins.
### Attribute mapping
Traversal identifies users by their email address. Regardless of which identity provider you use, the **NameID** (SAML) or **sub/email** claim (OIDC) must map to the user's **primary email address**. This is how Traversal matches authenticated users to their organization and account.
| Protocol | Required mapping |
| -------- | ------------------------------------------------------- |
| **SAML** | NameID → primary email address (format: `EmailAddress`) |
| **OIDC** | `email` scope and claim → primary email address |
If the NameID or email claim does not return the user's primary email, Traversal cannot match the user to an organization, and sign-in will fail with an "Email not found" error.
### SCIM provisioning
Traversal does not currently support SCIM. User provisioning and deprovisioning must be managed manually through Traversal's invite system or by assigning and unassigning users in your identity provider.
### Setup instructions
Go to **Okta Admin Console → Applications → Applications → Create App Integration**. Select **OIDC - OpenID Connect** as the sign-in method and **Web Application** as the application type.
Set the **Sign-in redirect URI** to:
```
https://dev-ppocc0m78uclwopp.us.auth0.com/login/callback
```
Optionally, set the **Initiate login URI** to:
```
https://app.traversal.com
```
In the **Assignments** tab, add every user or group that needs access to Traversal. Unassigned users will receive a "User is not assigned to the client application" error and cannot sign in.
From the application settings, copy the **Client ID** and **Client Secret**. Verify that the granted scopes include `openid`, `email`, and `profile`.
Provide the following to the Traversal team:
* Okta domain (e.g., `yourcompany.okta.com`)
* Client ID
* Client Secret
* Scopes used (`openid email profile`)
Entra exposes Traversal through two objects: the **App registration** (where you configure authentication, redirect URIs, and client secrets) and the **Enterprise application** (where you manage user/group assignment and admin consent). Both are created automatically when you register the app, but each blade lives under a different page. The steps below call out which one to use.
Go to **Microsoft Entra admin center → Entra ID → App registrations → New registration**. Give the application a name (e.g., "Traversal").
On the **App registration** page, go to **Authentication → Platform configurations** and add a **Web** platform with the redirect URI:
```
https://dev-ppocc0m78uclwopp.us.auth0.com/login/callback
```
On the **App registration** page, go to **Certificates & secrets → Client secrets → New client secret**. Copy the secret **Value** immediately — it is only displayed once.
On the **App registration** page, go to **API permissions** and click **Grant admin consent for \**. The status column flips to a green checkmark for each permission.
Skipping this step is the most common cause of failed Entra sign-ins. Without admin consent, assigned users see a **"Request pending — your admin has been notified"** screen even when they belong to a group that is assigned to the application.
Go to **Enterprise applications → Traversal → Properties** and set **Assignment required** to **Yes**. Then under **Users and groups**, assign the users or groups that need access. Users who are not assigned cannot sign in.
Provide the following to the Traversal team:
* Domain (e.g., `yourcompany.onmicrosoft.com` or your custom domain)
* Application (client) ID (found on the **App registration** **Overview** page)
* Client Secret value
If your identity provider supports SAML 2.0, you can configure Traversal as a service provider.
Send Traversal **either** your IdP's metadata URL (recommended — it contains the sign-in URL, signing certificate, and supported bindings) **or** the individual sign-in URL and signing certificate. Either path works; the metadata URL is faster and less error-prone.
| Field | Required | Description |
| --------------------------------- | ------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Metadata URL** | Send this **or** the next two fields | The URL where your IdP publishes its SAML metadata XML. The metadata document declares the IdP's endpoints, signing certificate, and supported bindings. If you send this, you can skip the Sign In URL and X509 Signing Certificate. |
| **Sign In URL** | Required if no Metadata URL | The URL where SAML authentication requests are sent. This is also called the single sign-on (SSO) endpoint. |
| **X509 Signing Certificate** | Required if no Metadata URL | The public-key certificate required by the SP to validate the signature of the authentication assertions that have been digitally signed by the IdP. Traversal accepts the `.pem` and `.cer` formats. |
| **Sign Out URL** | Optional | The URL where SAML logout requests are sent. This is also called the single logout (SLO) endpoint. Provide this if you want users to be signed out of your IdP when they sign out of Traversal. |
| **User ID Attribute** | Optional | The attribute in the SAML assertion that identifies the user. Defaults to the NameID. Provide this only if your IdP exposes the user's primary email under a different attribute. |
| **Protocol Binding** | Optional | The HTTP binding the IdP expects for SAML authentication requests (`HTTP-POST` or `HTTP-Redirect`). Defaults to `HTTP-Redirect`. |
| **Signed authentication request** | Optional | Whether your IdP requires Traversal to sign outbound SAML authentication requests (`yes` or `no`, defaults to `no`). If `yes`, also specify the signing algorithm (`RSA-SHA256` or `RSA-SHA1`, defaults to `RSA-SHA256`) and the digest algorithm (`SHA256` or `SHA1`, defaults to `SHA256`). Traversal will share back a signing certificate to upload to your IdP. |
You will also need **attribute mappings** for `email`, `name`, and optionally `groups` or `roles`. The NameID (or the User ID Attribute, if specified) must map to the user's primary email address (see [Attribute mapping](#attribute-mapping)).
In your IdP, assign the users or groups that should have access to Traversal. Users who are not assigned to the SAML application cannot sign in.
Provide the required fields, plus any of the optional fields above that apply to your IdP, to the Traversal team. Traversal will provide you with the **Assertion Consumer Service (ACS) URL** and **SP Entity ID** to complete configuration on your side. If signed authentication requests are required, Traversal will also provide a signing certificate.
For any OIDC-compliant identity provider not listed in the other tabs:
Obtain the following:
* **Issuer URL** (e.g., `https://idp.example.com`) — Traversal uses this to auto-discover endpoints via `/.well-known/openid-configuration`
* **Client ID** and **Client Secret**
* **Scopes** (`openid email profile`)
In your IdP, register the following redirect URI for the Traversal application:
```
https://dev-ppocc0m78uclwopp.us.auth0.com/login/callback
```
Ensure all users who need access to Traversal are assigned to the application in your IdP. Unassigned users cannot sign in.
Provide the following to the Traversal team:
* Issuer URL
* Client ID
* Client Secret
* Scopes
Go to **Google Admin Console → Apps → Web and mobile apps → Add app → Add custom SAML app**. Follow the wizard and download the **IdP metadata XML** when prompted.
When asked for service provider details, enter the **ACS URL** and **Entity ID** provided by the Traversal team. Set the **Name ID format** to `EMAIL` and map the `email` and `name` attributes.
In the SAML app settings, go to **User access** and set the app to **On for everyone** — or scope it to specific organizational units. Users in disabled OUs cannot sign in.
Provide the following to the Traversal team:
* Google Workspace domain (e.g., `yourcompany.com`)
* Downloaded IdP metadata XML file
* Attribute mappings (`email`, `name`, and optionally `groups`)
After SSO is configured, coordinate a test sign-in with the Traversal team before rolling out to all users. This catches misconfigured redirect URIs, missing attribute mappings, or unassigned user groups early.
## Common issues
Traversal could not match your email to an organization. Either your email domain is not registered with any Traversal organization, or you have not been invited. Contact your administrator or reach out to [support@traversal.com](mailto:support@traversal.com).
Your organization uses invite-only access, and you do not have a valid invite. Ask your organization administrator to send you an invitation from the **User Management** settings page.
Invite codes expire after a set period. Ask your administrator to resend the invitation — this generates a fresh code and invalidates the old one.
The redirect URI configured in your identity provider does not match the expected value. Verify that the redirect URI is set to exactly:
`https://dev-ppocc0m78uclwopp.us.auth0.com/login/callback`
Extra trailing slashes, `http` instead of `https`, or incorrect paths will cause the redirect to fail.
You are authenticated, but your organization's integrations may not be configured yet. Ask your administrator to connect your observability, code, and communication tools in **Settings > Knowledge Base**.
## Roles and permissions
Traversal uses role-based access control (RBAC). Roles are assigned per organization.
| Role | Capabilities |
| ---------- | ------------------------------------------------------------------------------------------------------- |
| **Member** | Run investigations, view results, interact with Slack, access the knowledge bank |
| **Admin** | Everything a member can do, plus manage users, configure integrations, and update organization settings |
Roles are assigned when a user is invited, and can be updated by any administrator.
## Security
All authentication flows use industry-standard protocols:
* **OIDC / OAuth 2.0** with PKCE for browser-based sign-in.
* **Short-lived access tokens** that refresh automatically in the background.
* **HTTPS-only cookies** with strict same-site policies.
* **CSRF protection** on all state-changing requests.
For details on Traversal's broader security posture, certifications, and architecture, see the [Security](/responsible-use/security) page.
# Integrations
Source: https://docs.traversal.com/get-started/integrations
Connect your existing observability stack so Traversal can analyze live incidents with full context.
Once connected, Traversal begins analyzing metrics, events, logs, traces, deployments, system knowledge, Slack context, and more. The more data sources you connect, the more accurately Traversal can reason about what's happening in production.
## Access model — read-only by design
Traversal only requires read-only API access to your systems and repositories. There are no agents to deploy and no write privileges to your data stores or code.
For chat platforms — Slack and Microsoft Teams — Traversal uses messaging-surface permissions only: reading conversation context where the bot participates, and posting investigation results back. These permissions do not grant data-plane writes to your systems.
***
## Connect your stack
After logging in, go to **Company Knowledge > Integrations**. Use the Onboarding Guide on the Integrations page to navigate the setup process.
Traversal needs real-time access to your metrics, logs, and traces to understand what's happening as it happens — not after the fact.
Start with telemetry — it's the most important category. If you're on Datadog, connect that first. Once telemetry is in place, add GitHub and productivity tools.
Traversal supports the following integrations:
* [Alertmanager](/integrations/alertmanager)
* [AppDynamics](/integrations/appdynamics)
* [CloudWatch](/integrations/cloudwatch)
* [Coralogix](/integrations/coralogix)
* [Datadog](/integrations/datadog)
* [Dynatrace](/integrations/dynatrace)
* [Elasticsearch](/integrations/elasticsearch)
* [Grafana](/integrations/grafana)
* [Loki](/integrations/loki)
* [Mimir](/integrations/mimir)
* [OpenSearch](/integrations/opensearch)
* [Prometheus](/integrations/prometheus)
* [Sentry](/integrations/sentry)
* [Splunk](/integrations/splunk)
* [Tempo](/integrations/tempo)
* [ThousandEyes](/integrations/thousandeyes)
* [VictoriaMetrics](/integrations/victoria-metrics)
* [AWS Account](/integrations/aws-cli)
* [GitHub](/integrations/github)
* [GitLab](/integrations/gitlab)
* [MotherDuck](/integrations/motherduck)
* [Confluence](/integrations/confluence)
* [FireHydrant](/integrations/firehydrant)
* [incident.io](/integrations/incident-io)
* [Jira](/integrations/jira)
* [Linear](/integrations/linear)
* [Microsoft Teams](/integrations/microsoft-teams)
* [Notion](/integrations/notion)
* [ServiceNow](/integrations/servicenow)
* [Slack](/integrations/slack)
* [MCP Servers](/integrations/mcp-servers) — connect tools from any MCP server
Click any integration above for setup instructions, or select one in Traversal for a step-by-step walkthrough.
If a source you need is missing, reach out and the Traversal team will guide you through the custom integration workflow.
Completed integrations appear highlighted in green.
You can immediately start exploring your system by asking natural-language questions on Traversal's Home page. For example:
* What are the most recent PRs?
* What are the impacts of my deployment on the `[endpoint]` endpoint?
# Quickstart
Source: https://docs.traversal.com/get-started/quickstart
Spin up your first investigation, ask about an incident, or explore your system using natural-language questions.
## Start an investigation
The Traversal Investigation page lets you describe an incident or system question in plain language. Traversal analyzes your connected stack and returns a triage with root cause candidates, timelines, and cited evidence.
For the best experience, deploy the [Traversal Slack app](/integrations/slack) so your team can start investigations, receive reports, and ask follow-up questions directly from Slack.
### Add context quickly
**@-mention entities** — Type `@` to reference teams, services, applications, or environments. Use the dropdown or autocomplete to select the right item.
**Paste an incident description** — Paste in a snippet from a ticket, Slack thread, or alert. Traversal parses entity names (teams and services) and extracts the date and time if present.
Include when the issue started, error messages, affected teams and services, and any alert or runbook links. More specific context produces better diagnostics.
### Set the time window
Deep mode requires a start date, time, and timezone. If you paste an incident description that includes a timestamp, Traversal parses it automatically. Otherwise, use the date and time controls on the page.
Examples that parse correctly:
* `Oct 23 20:00 UTC`
* `10/23/2024 at 20:00 UTC`
* `Oct 23 4pm ET 2024`
* `5pm EST yesterday`
### Ask follow-up questions
If the first pass doesn't capture the full picture, ask follow-up questions in the web app or the Slack thread.
* Drill into any symptom
* Slice the timeline
* Request more detail on a specific signal
To change the analysis window, start a new investigation with the revised time range.
Reply with `@Traversal` and additional context to sharpen the analysis — for example: "focus on payments-api in prod-us-east after 12:05 UTC".
# Alertmanager
Source: https://docs.traversal.com/integrations/alertmanager
Connect Alertmanager to query active and historical alerts during investigations.
Connecting Alertmanager allows Traversal to query active and historical alerts from your Alertmanager endpoint, including Grafana-managed Alertmanager deployments.
## What Traversal reads
* **Alerts** — active and grouped alerts via the Alertmanager API
## Setup
Enter the base URL Traversal should send Alertmanager API requests to.
If you already use Alertmanager in Grafana, go to **Connections > Data sources > Alertmanager** and copy the value in **HTTP URL**.
Common examples:
* `https://alertmanager.example.com:9093`
* `https://alertmanager-us-central1.grafana.net`
Do not enter:
* a Grafana dashboard URL
* a full API path such as `/api/v2/alerts` or `/api/v2/alerts/groups`
If the Grafana data source is provisioned or Grafana is handling authentication for you, ask your administrator for the direct URL and credentials Traversal should use.
If your endpoint requires authentication, enter a bearer token.
Only set **Organization ID** if your deployment expects the `X-Scope-OrgID` header for multi-tenant access.
Go to **Company Knowledge > Integrations**, select Alertmanager, and enter the URL, optional token, and optional organization ID.
## More information
* [Find the Alertmanager HTTP URL in Grafana](https://grafana.com/docs/grafana/latest/datasources/alertmanager/)
# AppDynamics
Source: https://docs.traversal.com/integrations/appdynamics
Connect AppDynamics to pull application metrics and health signals during investigations.
Connecting AppDynamics lets Traversal pull application metrics, health signals, and alert context during investigations so it can correlate symptoms to likely causes with supporting evidence.
## What Traversal reads
* **Metrics** — application performance metrics across tiers, nodes, and backends
* **Events** — application events and health rule violations
* **Request snapshots** — transaction snapshots for slow or erroring requests
## Setup
Enter the API client's **Client ID** and **Client Secret**. Traversal uses them to request and refresh OAuth access tokens automatically. Do not generate a temporary access token in the AppDynamics UI for this integration.
1. Sign in to the Controller UI as an Account Owner, or as an admin who can manage API clients. If you are
in the Account Overview section click on Launch Controller.
2. Click your user name, then go to **Administration > API Clients**.
3. Click **+ Create**.
4. Enter a **Client Name** and description for Traversal.
5. Click **Generate Secret**, then copy the secret immediately. AppDynamics shows it only once.
6. Under **Roles**, click **Add** and select **Applications & Dashboards Viewer (Default)** and **Dashboards Viewer (Default)**.
7. Click **Save**.
Click your user name and go to **My Splunk AppDynamics Account**, or open **Administration > Account Overview**. You need two values from this page:
* **Controller URL** — listed under your license details (e.g. `https://example.saas.appdynamics.com`)
* **Controller Account** — the account name shown on the same page
* **Client ID:** `@` — for example, if your Client Name is `traversal` and the account name from the previous step is `mycompany`, enter `traversal@mycompany`.
* **Client Secret:** The secret you copied when you created the API client.
* **Base URL:** The controller URL from the previous step (e.g. `https://example.saas.appdynamics.com`).
Do not enter the full token URL. Traversal appends the AppDynamics `/controller/...` API paths itself.
Go to **Company Knowledge > Integrations**, select AppDynamics, and enter the **Client ID**, **Client Secret**, and **Base URL** you gathered above.
Click **Save**.
If the integration returns empty or unauthorized responses, confirm that the API client has the required roles and that you entered the `client_id` in the `@` format.
## More information
* [Configure and troubleshoot AppDynamics API client](https://www.cisco.com/c/en/us/support/docs/security/endpoint-security-analytics-built-on-splunk/223150-configure-and-troubleshoot-appdynamics.html#toc-hId-1235019416)
* [Splunk AppDynamics API clients](https://help.splunk.com/en/appdynamics-saas/extend-splunk-appdynamics/25.7.0/extend-splunk-appdynamics/splunk-appdynamics-apis/api-clients)
# AWS Account
Source: https://docs.traversal.com/integrations/aws-cli
Let Traversal's agent run read-only queries against a single AWS account via a cross-account IAM role.
Connecting an AWS Account lets Traversal query AWS to discover and inspect your AWS resources during investigations — describing resources, reading configuration, pulling logs, etc. — using a cross-account, read-only IAM role. Traversal assumes this role via STS, minting short-lived credentials rather than using long-lived AWS access keys. No secrets are ever accessible.
## Setup
In the Traversal Web UI, go to **Company Knowledge > Integrations**, select "AWS Account", and download the CloudFormation template. It's pre-filled with your external ID and Traversal's AWS account ID so it's ready to be deployed in your AWS account.
Deploy the CFN template in your AWS account, either through the [AWS console](https://console.aws.amazon.com/console/home/?nc2=h_si\&src=header-signin) or using `aws cloudformation deploy` from the command line. This creates a single read-only IAM role that trusts Traversal's AWS principal, with the `sts:ExternalId` condition pinned to `traversal:`.
Once the stack finishes, copy the role ARN from its outputs. Also copy the external ID embedded in the template — you'll enter both in the Traversal Web UI.
Back in the "AWS Account" integration form, enter:
* **AWS Account ID** — the 12-digit account this integration represents
* **Role ARN**
* **External ID** — must match the value in the deployed role's trust policy
* **Default Region** — used by default; Traversal can inspect resources in other regions during an investigation
Traversal uses the account ID to select the right connection when multiple AWS accounts are configured, so it must be unique per AWS Account integration.
To connect additional AWS accounts, repeat these steps for each account — AWS Account supports multiple integration instances, one per account.
## Required permissions
The CloudFormation template attaches AWS's managed **`AIDevOpsAgentAccessPolicy`** policy — broad read-only `Describe`/`Get`/`List` access across AWS services, intended for AI agent tooling — plus `sts:GetCallerIdentity`. No write, delete, or modify permissions are granted.
## More information
* [IAM roles with external ID conditions](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_create_for-user_externalid.html)
* [AIDevOpsAgentAccessPolicy's permissions](https://docs.aws.amazon.com/aws-managed-policy/latest/reference/AIDevOpsAgentAccessPolicy.html)
# CloudWatch
Source: https://docs.traversal.com/integrations/cloudwatch
Connect Amazon CloudWatch to query metrics and logs via a cross-account IAM role during investigations.
Connecting CloudWatch lets Traversal query your AWS CloudWatch metrics and logs during investigations, using a cross-account IAM role rather than long-lived AWS credentials, so it can correlate infrastructure signals and surface anomalies alongside your other telemetry.
## What Traversal reads
* **Metrics** — time-series data via `GetMetricData`
* **Logs** — log group events via CloudWatch Logs
## Setup
In Traversal, go to **Company Knowledge > Integrations**, select CloudWatch, and download the CloudFormation template. It's pre-filled with your external ID and Traversal's AWS account ID — no parameters required.
Deploy the template in the AWS account whose CloudWatch data you want Traversal to read. This creates a read-only IAM role that trusts Traversal's AWS principal, with the `sts:ExternalId` condition pinned to `traversal:`.
Traversal assumes this role via STS for each request — it never receives or stores long-lived AWS access keys.
Once the stack finishes, copy the role ARN from its outputs.
Back in the CloudWatch integration form, enter:
* **Role ARN**
* **Regions** — every AWS region that hosts CloudWatch metrics or log groups you want investigated
Metrics and log groups are region-scoped, so a region not listed here won't be queried.
To connect additional AWS accounts, repeat these steps for each account — CloudWatch supports multiple integration instances, one per account.
## More information
* [IAM roles with external ID conditions](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_create_for-user_externalid.html)
# Confluence
Source: https://docs.traversal.com/integrations/confluence
Connect Confluence to surface runbooks and documentation during investigations.
Connecting Confluence allows Traversal to surface runbooks, procedures, and postmortems during investigations and cite relevant internal documentation as evidence.
## What Traversal reads
* **Pages** — page content searched via CQL and fetched by ID
* **Spaces** — space metadata used to scope searches
## Required scopes
When issuing a scoped Atlassian API token, grant the following minimum Confluence scopes for optimal agent performance:
| Scope | Used for |
| ---------------------------- | --------------------------------------------------------------------------------------- |
| `search:confluence` | CQL search across pages — required for the agent to discover relevant runbooks and docs |
| `read:page:confluence` | Fetching full page content by ID after a search hit |
| `read:space:confluence` | Resolving space metadata used to scope searches and label results |
| `read:attachment:confluence` | Reading attachments referenced from pages |
Only `search:confluence` is needed for the connection health check on the Traversal UI to pass.
## Setup
1. Open [Atlassian API tokens](https://id.atlassian.com/manage-profile/security/api-tokens) and sign in
2. Create a standard Atlassian API token, give it a descriptive name, and choose an expiration date
3. Copy the generated token right away. Atlassian only shows it once
Go to **Company Knowledge > Integrations**, select Confluence, and enter:
* Confluence site URL (usually `https://company.atlassian.net/wiki`)
* Your Atlassian account email
* API token
1. In Confluence, select your avatar, then go to **Settings > Personal access tokens**
2. Create a token, optionally set an expiration date, and copy it right away
Go to **Company Knowledge > Integrations**, select Confluence, choose **Bearer Token**, and enter:
* Confluence site URL
* Personal access token
Traversal exposes two Confluence authentication modes:
* **Basic Authentication** for Atlassian Cloud
* **Bearer Token** for self-hosted Confluence Server or Data Center
## More information
* [Atlassian API tokens](https://support.atlassian.com/atlassian-account/docs/manage-api-tokens-for-your-atlassian-account/)
* [Confluence Data Center personal access tokens](https://confluence.atlassian.com/display/ENTERPRISE/Using+personal+access+tokens)
# Coralogix
Source: https://docs.traversal.com/integrations/coralogix
Connect Coralogix to query logs and metrics during investigations.
Connecting Coralogix lets Traversal query your logs and metrics during investigations to provide comprehensive observability insights.
## What Traversal reads
* **Logs** — queried during investigations to surface relevant signals
## Setup
1. In Coralogix, go to **Settings > API Keys > Personal Keys**
2. Click **+ New Key**, give it a name (e.g., "Traversal")
3. Select the **DataQuerying** role preset
4. Click **Create** and copy the API key (it's only shown once)
Select the API endpoint for your Coralogix region:
* EU1: `https://api.coralogix.com`
* EU2: `https://api.eu2.coralogix.com`
* US1: `https://api.coralogix.us`
* US2: `https://api.cx498.coralogix.com`
* AP1 (India): `https://api.app.coralogix.in`
* AP2 (Singapore): `https://api.coralogixsg.com`
You can also identify your region from your Coralogix account URL — for example, `https://your-team.cx498.coralogix.com` indicates US2.
Go to **Company Knowledge > Integrations**, select Coralogix, and enter your API endpoint and API key.
## More information
* [Coralogix API keys](https://coralogix.com/docs/api-keys/)
# Datadog
Source: https://docs.traversal.com/integrations/datadog
Connect Datadog to pull metrics, logs, traces, alerts, and events during investigations.
Connecting Datadog lets Traversal use your logs, metrics, traces, and monitors during investigations so it can surface anomalies, correlate signals, and generate grounded root-cause explanations.
## What Traversal reads
* **Logs** — queried from your log indexes
* **Metrics** — time-series data and tag metadata
* **Spans / Traces** — APM span data
* **Incidents** — active and historical incident records
* **Events** — event stream data
* **Monitors** — monitor state and alert history
* **Dashboards** — dashboard panel definitions
## Setup
The API key identifies your Datadog organization. In Datadog, go to:
**Organization Settings → API Keys → New Key**
Name it (e.g. `traversal`) and copy the value — it won't be shown again.
The application key authorizes API queries on behalf of a user. Go to:
**Organization Settings → Application Keys → New Key**
Name it (e.g. `traversal`) and copy the value — it won't be shown again.
Create the application key under a **dedicated read-only Datadog user**.
The key inherits that user's permissions — no explicit scopes needed. Traversal will have read-only access to everything that user can see.
Recommended for most setups. Create a service user with the **Datadog Read Only Role** and generate the application key under that account.
Create the application key under any user and add the following scopes explicitly:
| Scope | What it covers |
| ------------------ | ------------------------------------- |
| `logs_read_data` | Log queries |
| `metrics_read` | Metric tag configuration and metadata |
| `timeseries_query` | Metric time-series data |
| `apm_read` | Spans and traces |
| `incident_read` | Incident records |
| `events_read` | Event stream |
| `monitors_read` | Monitor state and alerts |
| `dashboards_read` | Dashboard definitions |
`metrics_read` alone is **not** sufficient for metric queries. It only grants access to metric names, metadata, and tag configurations. You must also add `timeseries_query` to allow Traversal to read actual metric values.
Use this if your security policy requires explicit permission grants, or if the creating user has broader access you don't want to expose.
Your API URL is based on your Datadog site. To find it, look at your browser URL when logged in to Datadog and match it below:
| Browser URL | API URL to use |
| ------------------- | ------------------------------- |
| `app.datadoghq.com` | `https://api.datadoghq.com` |
| `us3.datadoghq.com` | `https://api.us3.datadoghq.com` |
| `us5.datadoghq.com` | `https://api.us5.datadoghq.com` |
| `app.datadoghq.eu` | `https://api.datadoghq.eu` |
| `ap1.datadoghq.com` | `https://api.ap1.datadoghq.com` |
| `ap2.datadoghq.com` | `https://api.ap2.datadoghq.com` |
| `app.ddog-gov.com` | `https://api.ddog-gov.com` |
Go to **Company Knowledge → Integrations**, select **Datadog**, and enter:
* Datadog API URL
* API key
* Application key
## More information
* [Datadog API and Application keys](https://docs.datadoghq.com/account_management/api-app-keys/)
* [Datadog OAuth scopes](https://docs.datadoghq.com/api/latest/scopes/)
* [Datadog site URLs](https://docs.datadoghq.com/getting_started/site/)
# Dynatrace
Source: https://docs.traversal.com/integrations/dynatrace
Connect Dynatrace to pull entities, problems, metrics, logs, traces, and Grail data during investigations.
Connecting Dynatrace lets Traversal pull entities, Davis-detected problems, metrics, events, logs, and traces during investigations so it can correlate symptoms across your services and produce grounded root-cause explanations.
## What Traversal reads
* **Entities and topology** — host, service, process, and application entities, plus their dependency relationships
* **Problems** — Davis-detected problems and their evidence
* **Metrics** — time-series data and dimensional metadata
* **Events** — deployment events and other point-in-time signals
* **Logs** — classic log queries and Grail (DQL) log queries
* **Spans / Traces** — APM spans and trace context
* **Settings** — read-only access to configuration objects (e.g. anomaly-detection rules)
## Authentication
Traversal talks to Dynatrace through two complementary API surfaces, and you can connect **either, or both at the same time**:
| Surface | Host | Authentication | Covers |
| --------------------------- | ---------------------- | ---------------------------- | --------------------------------------------------------------- |
| **Classic Environment API** | `*.live.dynatrace.com` | API token (`dt0c01...`) | Entities, problems, events, classic metrics, classic logs, SLOs |
| **Platform / Grail** | `*.apps.dynatrace.com` | OAuth 2.0 client credentials | DQL queries against logs, events, spans, metrics, and bizevents |
**Pick what fits your tenant:**
* **Dynatrace Managed / self-hosted** — Classic API only (Grail is SaaS-only).
* **SaaS, Grail-only** — OAuth client only. Problems and entities are still reachable via DQL.
* **SaaS, mid-migration or both surfaces in use** — provide both. Traversal automatically routes each query to the right surface.
You only need to fill in the section(s) for the surface(s) you want to connect; leave the other section empty. Traversal's connection check exercises every credential you provide and reports per-surface pass/fail, so you'll see exactly which half is working if one side is misconfigured.
## Setup
Best for **Dynatrace Managed**, **self-hosted**, or any tenant where you only need entities, problems, events, classic metrics, classic logs, and SLOs. Works on every Dynatrace deployment.
Best for **SaaS-only** tenants that have moved to Grail for logs, events, spans, metrics, and bizevents. Required for DQL queries.
Best for **SaaS tenants mid-migration** or any tenant using both surfaces in production. Traversal routes each query to the surface that owns the data, so you get the broadest coverage.
1. In Dynatrace, open **Settings → Integration → Dynatrace API → Generate token**.
2. Name it (e.g. `traversal`).
3. Choose the read-only scopes you want to grant. Traversal will exercise whichever scopes the token carries; the more scopes you grant, the more data Traversal can use during investigations.
**Minimum**:
| Scope | What it covers |
| --------------- | -------------------------------------------------------------------------- |
| `entities.read` | Required to connect. Powers entity-listing and service-dependency lookups. |
**Recommended additions** — grant whichever your security policy allows:
| Scope | What it unlocks |
| --------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `problems.read` | Davis-detected problems (`/api/v2/problems`) |
| `metrics.read` | Metric time-series queries (`/api/v2/metrics/query`) |
| `events.read` | Events including deployments (`/api/v2/events`) |
| `releases.read` | Curated release lists (`/api/v2/releases`); only available on tenants where the Releases feature is enabled. Traversal falls back to events automatically when this scope isn't granted. |
| `settings.read` | Settings 2.0 objects, including anomaly-detection rules (`/api/v2/settings/objects`) |
| `logs.read` | Classic log queries |
| `slo.read` | SLO definitions and status |
| `traces.lookup` | Classic trace lookups |
Granting only `entities.read` is enough to connect, but Traversal won't have anything else to read. Grant the additional scopes for the data types you want surfaced during investigations.
4. Click **Generate token** and copy the token immediately. Dynatrace shows it only once.
5. Note your **Classic Environment URL** in the format `https://.live.dynatrace.com`. The environment ID is the subdomain of your Dynatrace UI URL.
For Dynatrace Managed deployments the URL format is `https:///e/` instead. Traversal supports both.
1. In Dynatrace, open **Account Management → Identity & access management → OAuth clients → Create client**.
2. Name it (e.g. `traversal`) and choose which storage scopes to grant. Traversal queries whichever data types the token can read, so grant the storage scopes for the signals you want available during investigations.
**Minimum**:
| Scope | What it covers |
| ------ | ----------------------------------------------------------------------------------------------------------- |
| *none* | No storage scopes are required to connect — you can save the integration first and grant data scopes later. |
**Recommended additions** — grant whichever your security policy allows:
| Scope | What it unlocks |
| ------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `document:documents:read` | Grail dashboards — lets Traversal surface relevant panels during investigations |
| `storage:buckets:read` | **Required alongside any `storage:
:read` scope.** DQL queries (`timeseries`, `fetch metric.*`, `fetch logs`, etc.) succeed but return zero records with a `No bucket permissions for table ` notification until this scope is granted. The only DQL table that does *not* need it is `dt.entity.*`. |
| `storage:metrics:read` | Metric records in Grail (pair with `storage:buckets:read`) |
| `storage:logs:read` | Log records in Grail (pair with `storage:buckets:read`) |
| `storage:events:read` | Event records in Grail, including deployment events (pair with `storage:buckets:read`) |
| `storage:spans:read` | Span records in Grail (pair with `storage:buckets:read`) |
| `storage:bizevents:read` | Business event records (pair with `storage:buckets:read`) |
| `storage:system:read` | System tables (e.g. `dt.system.tables`); useful for schema discovery |
You can save the integration with zero storage scopes if you want to set up authentication first and add data scopes later. Until you grant the relevant `storage:*:read` scopes, Traversal won't be able to read any data from Grail.
3. Click **Create client** and copy the **Client ID**, **Client secret**, and **Account URN**. The secret is shown only once.
4. Note your **Platform URL** in the format `https://.apps.dynatrace.com`. The environment ID is the same as your Classic URL — only the host suffix differs (`.apps.` vs `.live.`).
Go to **Company Knowledge → Integrations**, select **Dynatrace**, and fill in the section(s) for the surface(s) you're connecting:
**Classic API**
* **Classic Environment URL** — e.g. `https://abc12345.live.dynatrace.com`
* **API Token** — the `dt0c01...` token from Step 2
**Platform / Grail**
* **Platform URL** — e.g. `https://abc12345.apps.dynatrace.com`
* **OAuth Client ID** — starts with `dt0s02.`
* **OAuth Client Secret** — the secret you copied
* **Account URN** — format `urn:dtaccount:`, found in **Account Management → Overview**
Click **Save**. Traversal exercises every credential you provided and reports the result per surface — if you connected both, you'll see a separate pass/fail for Classic and Grail.
## Troubleshooting
* **Classic connection fails with an `entities.read` permission error** — `entities.read` is the only scope required to connect; other scopes are only needed for the corresponding queries. Re-generate the token with `entities.read` granted.
* **Integration saves but a specific Dynatrace query returns "no data" or a permission error** — the corresponding scope is likely not granted on the token (Classic) or OAuth client (Grail). Add the scope from the tables above and re-save the integration.
* **Classic URL pasted on the Platform host (or vice-versa)** — Traversal rejects URLs that mix the two surfaces. Use `*.live.dynatrace.com` for Classic and `*.apps.dynatrace.com` for Grail.
* **Grail OAuth connection fails with an SSO error** — check that all three of Client ID, Client Secret, and Account URN are correct. The Account URN must be the format `urn:dtaccount:` from your Dynatrace Account Management page, not the environment ID.
## More information
* [Dynatrace API documentation](https://docs.dynatrace.com/docs/dynatrace-api)
* [Generate access tokens (Classic API)](https://docs.dynatrace.com/docs/manage/access-control/access-tokens)
* [Create OAuth clients (Platform)](https://docs.dynatrace.com/docs/manage/identity-access-management/access-tokens-and-oauth-clients)
* [Dynatrace Query Language (DQL)](https://docs.dynatrace.com/docs/discover-dynatrace/references/dynatrace-query-language)
# Elasticsearch
Source: https://docs.traversal.com/integrations/elasticsearch
Connect Elasticsearch to query log indices during investigations.
Connecting Elasticsearch allows Traversal to query your log indices, detect relevant errors or patterns, and use those logs as evidence during investigations.
## What Traversal reads
* **Logs** — documents queried from your log indices
## Setup
In your Elastic Cloud project, click the **Help** icon (top right) and select **Connection details**. Copy the **Elasticsearch endpoint** URL (e.g., `https://my-project-XXXX.es.us-central1.gcp.elastic.cloud`).
Go to **Settings > Access > API keys** and click **Create API key**. Give it read access to the indices Traversal should query.
Create a user API key, not a cross-cluster API key.
After creation, Elastic shows you a single encoded API key value — copy it.
Go to **Company Knowledge > Integrations**, select **Elasticsearch**, and enter:
* **Host**: the Elasticsearch endpoint from step 1
* **Authentication**: select **API Key** and paste the encoded key
Identify your Elasticsearch host URL (e.g., `https://elasticsearch.mycompany.com:9200`).
Use the username and password of a user with read access to the indices Traversal should query.
In Kibana, go to **Stack Management > Security > API keys** and click **Create API key**. Give it read access to the indices Traversal should query.
If Elastic shows you a single encoded API key value, paste it into **API Key** in Traversal.
If Elastic shows you separate `id` and `api_key` values after creation, select **API Key with ID** in Traversal and enter both values.
Go to **Company Knowledge > Integrations**, select **Elasticsearch**, and enter your host URL and chosen credentials.
If the integration does not work as expected, check the following:
* You entered the Elasticsearch endpoint URL, not a Kibana URL
* You selected the authentication method that matches the credential you created
* The user or API key has read access to the log indices Traversal should query
## More information
* [Elasticsearch API keys](https://www.elastic.co/docs/deploy-manage/api-keys/elasticsearch-api-keys)
# FireHydrant
Source: https://docs.traversal.com/integrations/firehydrant
Connect FireHydrant for incident management context during investigations.
Connecting FireHydrant lets Traversal pull incident context and postmortems so it can reference ongoing incidents and past learnings during investigations.
## What Traversal reads
* **Incidents** — names, severity, current status, and postmortem data
## Setup
1. In FireHydrant, go to **Settings > API Keys**
2. Click **+ Create API key**, name it (e.g., "Traversal"), and click **Create**
3. Copy the API key (it's only shown once)
Go to **Company Knowledge > Integrations**, select FireHydrant, and paste the API key.
## More information
* [FireHydrant API keys](https://docs.firehydrant.com/docs/api-keys)
# GitHub
Source: https://docs.traversal.com/integrations/github
Connect GitHub to correlate incidents with recent code changes.
Connecting GitHub lets Traversal correlate incidents with recent code changes by pulling commits and pull requests, helping it explain "what changed" alongside telemetry evidence.
## What Traversal reads
* **Pull requests** — recent PRs, diffs, and review comments
* **Commits** — commit history and changed files
* **Code** — file contents and code search results
## Setup
In Traversal, go to **Company Knowledge > Integrations**, select GitHub, and click **Install GitHub App**. You'll be redirected to GitHub to authorize the Traversal app.
Choose the GitHub organization and repositories you want Traversal to access. After authorization, you'll be redirected back to Traversal with the Installation ID populated automatically.
If the Traversal GitHub App is already installed, you can paste the existing **Installation ID** instead of running the install flow again.
Click **Save**.
Go to [github.com/settings/tokens](https://github.com/settings/tokens) and create a new token with scopes: `repo`, `read:org`.
In **Company Knowledge > Integrations**, select GitHub, switch to **Personal Access Token**, enter the token, and click **Save**.
On your GitHub Enterprise instance, create a personal access token with scopes: `repo`, `read:org`.
In **Company Knowledge > Integrations**, select GitHub, switch to **GitHub Enterprise**, enter your API URL (e.g., `https://github.yourcompany.com/api/v3`) and the token, and click **Save**.
### Optional scoping
You can optionally narrow GitHub access in Traversal:
* Use **Repositories** for a list of repositories in `owner/repo` format
* Use **Organization** for organization-wide access
### Required permissions (GitHub App, read-only)
**Repositories:** Commit statuses, Contents, Custom properties, Dependabot alerts, Deployments, Issues, Metadata, Pull Requests
**Organization:** Events, Knowledge bases, Issue Types, Members
## More information
* [GitHub App authentication](https://docs.github.com/en/apps/creating-github-apps/authenticating-with-a-github-app)
* [Installing GitHub Apps](https://docs.github.com/en/apps/using-github-apps/installing-your-own-github-app)
# GitLab
Source: https://docs.traversal.com/integrations/gitlab
Connect GitLab to correlate incidents with recent code changes.
Connecting GitLab lets Traversal correlate incidents with recent code changes by pulling commits and merge requests, helping it explain "what changed" alongside telemetry evidence. Supports both **gitlab.com** and **self-hosted GitLab instances**.
## What Traversal reads
* **Merge requests** — recent MRs, diffs, and discussions
* **Commits** — commit history and changed files
* **Code** — file contents and code search results
## Setup
1. Go to **User Settings > Access Tokens** ([gitlab.com/-/user\_settings/personal\_access\_tokens](https://gitlab.com/-/user_settings/personal_access_tokens))
2. Click **Add new token**
3. Set scopes: `read_api`, `read_repository`, or `api`
4. Click **Create personal access token** and copy the token immediately
Go to **Company Knowledge > Integrations**, select GitLab, and paste the token.
You can optionally scope access further:
* Use **Project IDs** for a comma-separated list of project IDs or project paths such as `mygroup/service-api`
* Use **Group ID** for group-wide access
For self-hosted GitLab, expand **Self-Hosted Configuration** and enter the host base URL, such as `https://gitlab.yourcompany.com`. Do not include `/api/v4`. Traversal appends that path automatically.
Go to your GitLab project's **Settings > Access Tokens** and create a token with `read_api` or `api` scope.
Select **Project Access Token**, enter the numeric **Project ID** from project settings, and paste the token.
For self-hosted GitLab, enter the host base URL only. Do not include `/api/v4`.
Go to your GitLab group's **Settings > Access Tokens** and create a token with `read_api` or `api` scope.
Select **Group Access Token**, enter the numeric **Group ID** from group settings, and paste the token.
For self-hosted GitLab, enter the host base URL only. Do not include `/api/v4`.
### Self-hosted GitLab
For self-hosted instances, expand **Self-Hosted Configuration** and enter your GitLab host base URL (for example, `https://gitlab.yourcompany.com`). Do not include `/api/v4`. Ensure the instance is accessible from Traversal.
### Required token scopes
* **Read-only:** `read_api` + `read_repository`
* **Broader access:** `api`
## More information
* [GitLab personal access tokens](https://docs.gitlab.com/ee/user/profile/personal_access_tokens.html)
* [GitLab API documentation](https://docs.gitlab.com/ee/api/rest/)
# Grafana
Source: https://docs.traversal.com/integrations/grafana
Connect Grafana to query dashboards, metrics, logs, traces, and alerts during investigations.
Connecting Grafana lets Traversal query your dashboards, metrics, logs, and alerts during investigations to provide comprehensive observability insights.
## What Traversal reads
* **Dashboards** — panel definitions and queries
* **Alerts** — Grafana-managed alert rules and their states
* **Data source proxy** — metrics, logs, and traces queried through Grafana's data source proxy
## Setup
Find your Grafana server URL (e.g., `https://your-instance.grafana.net` or your self-hosted URL).
1. Log in to your Grafana instance
2. Go to **Administration > Service Accounts**
3. Create a new service account with the **Viewer** role
4. Click **Add service account token** and copy the token value — it won't be shown again
If your Grafana instance uses SSO (Okta, Azure AD, Google, etc.), use a service account token — username/password will not work for API access.
Use the username and password of a user with read access to the dashboards and data sources Traversal should query.
Username/password does not work if your Grafana instance uses SSO (Okta, Azure AD, Google, etc.). Use a service account token instead.
Go to **Company Knowledge → Integrations**, select **Grafana**, and enter your URL and credentials.
## More information
* [Grafana service accounts](https://grafana.com/docs/grafana/latest/administration/service-accounts/)
# incident.io
Source: https://docs.traversal.com/integrations/incident-io
Connect incident.io for incident management context during investigations.
Connecting incident.io lets Traversal reference incident details, linked alert details, severities, and incident statuses during investigations.
## What Traversal reads
* **Incidents** — incident details, severities, statuses, and timelines
* **Alerts** — alerts linked to incidents
## Setup
1. In incident.io, go to **Settings > API Keys**
2. Click **Create API key** and name it (e.g., "Traversal")
3. Grant read access to incidents and related metadata. If your workspace uses team-scoped API keys, make sure the key covers every team whose incidents Traversal should read.
4. Copy the API key (starts with `inc_`, only shown once)
Go to **Company Knowledge > Integrations**, select incident.io, and paste the API key.
This integration only requires an incident.io API key. You do not need to enter a base URL or any additional authentication fields.
## Auto-add the Traversal Slack bot to incident channels
incident.io can automatically create a dedicated Slack channel for each incident (for example, `#inc-2026-07-01-checkout-latency`). To have Traversal join those channels the moment they're created — even when they're private — use an incident.io [Workflow](https://docs.incident.io/workflows/getting-started) that invites the Traversal Slack bot.
This requires the [Traversal Slack app](/integrations/slack) to be installed in the same workspace where incident.io creates incident channels.
In incident.io, go to **Settings > Workflows**, click **New Workflow**, and select the trigger **An incident is created or changed**.
Feel free to configure the workflow's filters to select for the incidents you want Traversal's bot in — for example, restrict it to incidents at or above a given severity. Leave the filters empty to have Traversal join every incident channel.
Add a step and choose **Invite a Slack bot user to the incident channel**. Select the **Traversal** bot as the user to invite.
Once your steps are added, click **Save as draft**, then **Save and set live**. If asked whether to run the workflow on incident changes, select **No** so it only applies to incidents going forward.
From then on, Traversal is added to each new incident channel automatically and can begin assisting as soon as the incident opens.
After enabling the workflow, trigger a test incident and confirm the Traversal bot appears in the new incident channel before relying on it in production.
## More information
* [incident.io API keys](https://api-docs.incident.io/admin/api-keys)
* [incident.io API reference](https://api-docs.incident.io/)
# Jira
Source: https://docs.traversal.com/integrations/jira
Connect Jira for issue tracking and project context during investigations.
Connecting Jira allows Traversal to access issues and projects, enabling correlation of incidents with engineering work.
## What Traversal reads
* **Issues** — issue details, status, priority, and assignee via JQL search
* **Comments** — full comment threads on issues
## Setup
1. Open [Atlassian API tokens](https://id.atlassian.com/manage-profile/security/api-tokens) and sign in
2. Create an API token for Jira, give it a descriptive name, and choose an expiration date
3. If Atlassian shows a scoped-token flow in your organization, select the Jira app and grant read-only access. `read:jira-work` covers the Jira endpoints Traversal uses to read issues, comments, JQL search results, and attachments
4. Copy the generated token right away. Atlassian only shows it once
Go to **Company Knowledge > Integrations**, select Jira, and enter:
* Jira instance URL (usually your site URL, such as `https://company.atlassian.net`)
* Email address associated with your Atlassian account
* API token
* Cloud ID, only if your Jira URL uses Atlassian's API gateway instead of your site URL
### Jira URL and Cloud ID
Use your Jira site URL, such as `https://company.atlassian.net`, whenever possible.
If your organization routes Jira API traffic through Atlassian's API gateway instead of your site URL, also provide the site's **Cloud ID**. Atlassian's current docs refer to this as `cloudId` in gateway URLs such as `.../ex/jira/{cloudId}/...`, and Traversal exposes a separate `Cloud ID` field for those setups.
## More information
* [Atlassian API tokens](https://support.atlassian.com/atlassian-account/docs/manage-api-tokens-for-your-atlassian-account/)
* [Jira scopes for Atlassian scoped tokens](https://developer.atlassian.com/cloud/jira/platform/scopes-for-oauth-2-3LO-and-forge-apps/)
* [Atlassian API gateway and Cloud ID](https://developer.atlassian.com/cloud/oauth/getting-started/making-calls-to-api/)
* [Find your Atlassian Cloud ID](https://support.atlassian.com/jira/kb/retrieve-my-atlassian-sites-cloud-id/)
# Linear
Source: https://docs.traversal.com/integrations/linear
Connect Linear for issue tracking and project context during investigations.
Connecting Linear allows Traversal to access issues and projects, enabling correlation of incidents with engineering work.
## What Traversal reads
* **Issues** — title, description, status, priority, labels, and assignee
* **Comments** — full comment thread on each issue
* **Attachments** — images attached to issues and comments
## Setup
1. In Linear, go to **Settings → Security and Access → Personal API Keys**
2. Click **New API Key**, give it a name (e.g. `traversal-prod`), and click **Create**
3. Copy the generated API key (starts with `lin_api_`)
Go to **Company Knowledge → Integrations**, select **Linear**, and paste the API key.
Click **Save**.
## More information
* [Linear API keys](https://developers.linear.app/docs/graphql/working-with-the-graphql-api#personal-api-keys)
# Loki
Source: https://docs.traversal.com/integrations/loki
Connect Grafana Loki to query and analyze logs during investigations.
Connecting Loki allows Traversal to query and analyze logs from your Grafana Loki instance during incident investigations.
## What Traversal reads
* **Logs** — queried via LogQL against your log streams
* **Labels** — stream label names and values used to scope queries to the right services
## Setup
Use this if your Loki instance is publicly accessible and you want to connect to it independently of Grafana.
Use your Loki server's base URL, e.g. `https://loki.yourcompany.com`.
If your Loki URL is a cluster-internal address (e.g. a Kubernetes service URL), it will not be reachable from Traversal. Use the Grafana proxy path instead.
If your Loki instance requires authentication, provide an API token. Leave blank if your instance is unauthenticated.
If your Loki deployment uses multi-tenancy, find your tenant/org ID. Traversal sends it as the `X-Scope-OrgID` header on every request.
Go to **Company Knowledge → Integrations**, select **Loki**, and enter:
* Loki URL
* API token (optional)
* Organization ID (optional, for multi-tenant deployments only)
If Loki is configured as a datasource in your Grafana instance, Traversal can reach it through the Grafana datasource proxy. This is useful when Loki is only accessible internally (e.g. within a cluster).
No additional setup is needed beyond the [Grafana integration](/integrations/grafana). Traversal auto-discovers Loki and queries it using the same Grafana service account token.
Multi-tenancy is handled automatically by Grafana when going through this path. You do not need to provide an org ID in Traversal.
## More information
* [Loki HTTP API](https://grafana.com/docs/loki/latest/reference/loki-http-api/)
* [Loki multi-tenancy](https://grafana.com/docs/loki/latest/operations/multi-tenancy/)
# MCP Servers
Source: https://docs.traversal.com/integrations/mcp-servers
Connect any Model Context Protocol server so Traversal can call its tools during investigations.
Connecting an [MCP server](https://modelcontextprotocol.io/) lets Traversal call that server's tools during investigations — pulling in context and capabilities from systems that don't have a dedicated Traversal integration.
## How it works
* Each MCP server is its own integration instance — connect as many as you need, each with its own URL and authentication.
* Traversal connects to remote servers over HTTP-based transports — both **Streamable HTTP** and **SSE** are supported.
* Traversal discovers the tools the server exposes, and you choose which ones to enable from the integration's tool list in the Traversal UI.
Traversal is [read-only by design](/architecture/intro). If an MCP server exposes tools that write or mutate state, add them to the server's **blocked tools** list so Traversal never calls them.
## Authentication
Traversal supports three authentication methods for MCP servers. Choose the one your server uses when you configure the integration.
A static token that Traversal sends as the `Authorization: Bearer ` header on every request. Use this when your server authenticates with a long-lived API key or personal access token.
Generate an API key or access token in the system that hosts the MCP server. Copy it — you'll paste it into Traversal.
Go to **Company Knowledge > Integrations**, add an **MCP Server**, and enter:
* **Server URL** — the server's streamable-http endpoint (e.g. `https://mcp.example.com/mcp`)
* **Auth method** — **Bearer token**
* **Token** — the token you copied
A service-account **client ID** and **client secret** that Traversal exchanges for a short-lived bearer token at a token endpoint before each session. Use this for the OAuth 2.0 `client_credentials` grant and similar service-to-service token flows.
In your identity provider, create a service account (or app registration) and note its client ID and client secret, plus the token endpoint URL that issues tokens for it.
Go to **Company Knowledge > Integrations**, add an **MCP Server**, and enter:
* **Server URL** — the server's streamable-http endpoint
* **Auth method** — **Client credentials**
* **Token URL** — the endpoint that issues tokens (e.g. `https://auth.example.com/token`)
* **Client ID** and **Client secret** — sent as HTTP basic auth to the token endpoint
If your provider needs a custom request body or returns the token under a non-standard JSON key, set the optional **request body** and **token response key** fields. The token key defaults to `access_token`.
Traversal fetches a fresh token as needed, so you never store a long-lived bearer token — only the service-account credentials.
Standard OAuth 2.0 authorization-code flow with PKCE (RFC 7636) and Dynamic Client Registration (RFC 7591). You authorize Traversal once, in a popup, at setup time. Use this when your server delegates authorization to an OAuth provider.
Go to **Company Knowledge > Integrations**, add an **MCP Server**, enter the **Server URL**, and choose **OAuth** as the auth method.
Traversal discovers the server's OAuth metadata (RFC 9728 and RFC 8414), registers itself as a client, and opens the provider's authorization page in a popup. Sign in and approve the requested scopes.
Access and refresh tokens are stored encrypted. Traversal refreshes the access token automatically when the server issues a refresh token, so you don't need to re-authorize on every session.
The OAuth authorization and token endpoints must use HTTPS.
## Enabling tools
After a server connects, Traversal lists the tools it exposes. In the Traversal UI you choose which tools to enable and give each one an **agent-facing description** — the text the agent uses to decide when to call the tool. A tool with no configured description won't be surfaced to the agent, so write a clear description for every tool you enable.
Two limits apply, both enforced when you save:
| Limit | Value |
| ----------------------- | -------------------------------------------------------------------- |
| Enabled tools | Up to **20** enabled tools across all connected MCP servers combined |
| Tool description length | **500** characters or fewer per tool |
Enabled tools and their descriptions are configured per server in the Traversal UI.
## Servers on a private network
If your MCP server — or its OAuth token endpoint — isn't reachable from the public internet, Traversal routes requests through your [Traversal Connector](/architecture/connector), the same relay used by other integrations. No changes to the server are required beyond making it reachable from the connector's network.
## Settings
| Setting | Description |
| --------------------- | ----------------------------------------------------------------------------------------- |
| **Server URL** | The MCP server's streamable-http endpoint. |
| **Auth method** | `Bearer token`, `Client credentials`, or `OAuth`. |
| **Blocked tools** | Tools Traversal will never call, even if the server exposes them. |
| **Tool call timeout** | How long Traversal waits for a single tool call before giving up. Defaults to 30 seconds. |
## FAQ
Generally, no — a native integration is the better choice. Traversal puts a lot of thought into each native integration and how it fits into the agent's harness, so native integrations tend to perform better, are far less likely to time out, and work more smoothly with the agent overall. Reach for MCP to bring in systems that *don't* have a native Traversal integration. Running both against the same system at once isn't ideal either — it adds duplicate, less-tuned tools that compete for the agent's attention.
Many companies build their own custom MCP servers. When you do, expose granular, single-purpose tools — not one tool that wraps a general-purpose agent. Traversal *is* the reasoning agent: it picks each tool, reads the result, decides the next step, and cites the underlying evidence in its report. Fine-grained tools ("query logs", "get a trace", "look up a ticket") let it do that well. A single black-box "agent" tool hides that reasoning and returns opaque output Traversal can't cite or build on.
If you already have a general-purpose agent exposed over MCP, you don't have to expose it wholesale — configure the specific underlying tools you want Traversal to access. That gives the agent the granularity it works best with.
Traversal is deliberate about what the agent can reach. The agent first sees your enabled tools and their descriptions — not their full schemas. When a description looks relevant to the investigation, Traversal loads that specific tool's full schema before calling it. Keeping the working set focused, and the descriptions clear, is what keeps tool selection accurate — which is why we cap enabled tools at **20** and ask for a clear description on each. Treat the description as the agent's only cue for *when* to use a tool, and write it accordingly.
## More information
* [Model Context Protocol](https://modelcontextprotocol.io/)
* [Security, data & permissions](/integrations/mcp-servers-security) — data access, controls, and authentication for security and compliance reviews
* [Traversal MCP server](/using-traversal/mcp) — for driving Traversal from your AI clients
# Microsoft Teams
Source: https://docs.traversal.com/integrations/microsoft-teams
Install the Traversal app in your Microsoft Teams workspace.
Bring Traversal into your team's Microsoft Teams workspace so engineers can start investigations and receive reports without leaving Teams.
## Install the Traversal Teams app
Traversal provides a Teams app package that a Teams administrator installs into your tenant.
Contact the Traversal team to receive the Traversal Teams app package for your organization.
A Teams administrator uploads the package in the **Teams Admin Center → Teams apps → Manage apps → Upload new app**, making it available to your organization.
A tenant administrator grants consent for the Microsoft Graph permissions the app uses. See [Microsoft Teams — data privacy and security](/integrations/microsoft-teams-data-privacy-and-security) for the full permission list and what each one is used for.
Provisioning is self-serve. The first time someone interacts with Traversal, it replies with a prompt to connect your Microsoft Teams workspace to your Traversal organization. An admin clicks **Connect to Traversal** on that message to pair the workspace with your organization.
Once connected, you can begin using the Traversal bot in your Teams workspace.
## Verify it works
Once your workspace is connected, mention `@Traversal` in any channel and say hello.
## Uninstall and data deletion
To remove the app from your workspace, go to **Teams Admin Center → Teams apps → Manage apps → Traversal → Block**, or remove it from individual teams via the team's **Manage apps** page.
For data export or deletion, email [support@traversal.com](mailto:support@traversal.com) with your tenant ID.
## FAQ
Your Microsoft Teams workspace hasn't been paired with a Traversal organization yet. An admin can click **Connect to Traversal** on that message to pair it.
This likely means the app isn't correctly installed in your workspace. Verify the upload and admin-consent steps above, or reach out to a Traversal team member.
Learn how to start investigations and ask follow-up questions in Teams.
How the Teams integration handles data — permissions, PII, and Traversal's security posture.
# Mimir
Source: https://docs.traversal.com/integrations/mimir
Connect Grafana Mimir for long-term Prometheus metrics storage during investigations.
Connecting Mimir lets Traversal query your long-term metrics storage during investigations so it can detect anomalies, compare baselines, and correlate changes across services.
## What Traversal reads
* **Metrics** — time-series data via PromQL instant and range queries
* **Labels** — label names and values used to scope queries to the right services and environments
## Setup
Use this if your Mimir instance is publicly accessible and you want to connect to it independently of Grafana.
Use your Mimir HTTP API endpoint. The URL should include the `/prometheus` path, e.g. `https://mimir.yourcompany.com/prometheus`.
If your Mimir URL is a cluster-internal address (e.g. a Kubernetes service URL), it will not be reachable from Traversal. Use the Grafana proxy path instead.
Mimir has no built-in authentication — credentials depend on what proxy or gateway sits in front of your Mimir instance. Traversal supports bearer token or basic auth. If your Mimir instance is unauthenticated, leave the credentials blank.
If your Mimir deployment uses multi-tenant, find your tenant/org ID. Traversal sends it as the `X-Scope-OrgID` header on every request.
Go to **Company Knowledge → Integrations**, select **Mimir**, and enter:
* Mimir URL (including `/prometheus` path)
* Credentials (optional — bearer token or basic auth if required)
* Organization ID (optional — for multi-tenant deployments only)
If Mimir is configured as a datasource in your Grafana instance, Traversal can reach it through the Grafana datasource proxy. This is useful when Mimir is only accessible internally (e.g. within a cluster).
No additional setup is needed beyond the [Grafana integration](/integrations/grafana). Traversal auto-discovers Mimir and queries it using the same Grafana service account token.
Multi-tenancy is handled automatically by Grafana when going through this path. You do not need to provide an org ID in Traversal.
## More information
* [Mimir HTTP API](https://grafana.com/docs/mimir/latest/references/http-api/)
# MotherDuck
Source: https://docs.traversal.com/integrations/motherduck
Connect MotherDuck for serverless analytics queries during investigations.
Connecting MotherDuck allows Traversal to run analytical queries against your data using the serverless DuckDB platform.
## Setup
1. Log in to [app.motherduck.com](https://app.motherduck.com)
2. Click your organization name in the top left → **Settings**
3. Click **+ Create token**, give it a name (e.g. `traversal-prod`)
4. Set the token type to **Read Scaling**
5. Copy the generated token — it won't be shown again
Go to **Company Knowledge → Integrations**, select **MotherDuck**, and paste the access token.
Click **Save**.
## More information
* [MotherDuck authentication](https://motherduck.com/docs/key-tasks/authenticating-and-connecting-to-motherduck/authenticating-to-motherduck/)
# Notion
Source: https://docs.traversal.com/integrations/notion
Connect Notion to surface internal documentation during investigations.
Connecting Notion allows Traversal to surface your runbooks, incident procedures, and postmortems during investigations and enrich responses with relevant internal documentation.
## What Traversal reads
* **Pages** — page content from pages shared with the integration
* **Databases** — database entries from databases shared with the integration
## Setup
1. Go to [Notion's internal integrations page](https://www.notion.so/profile/integrations/internal)
2. Click **New integration**
3. Create an internal integration and copy the token from the **Secrets** section (starts with `secret_`)
On the integration page, go to the **Content access** tab and click **Edit access** to select which pages and databases Traversal should be able to read.
Alternatively, you can share individual pages by opening them, clicking **Share**, and inviting the integration with at least **Can read** access.
Go to **Company Knowledge > Integrations**, select Notion, and paste the integration token.
### Supported Notion links
After the integration is connected, Traversal can fetch shared content from `notion.so` and `notion.site` links. If a Notion page is missing from results, verify that the page or database is shared with the integration.
## More information
* [Notion integrations](https://www.notion.com/help/create-integrations-with-the-notion-api)
# OpenSearch
Source: https://docs.traversal.com/integrations/opensearch
Connect OpenSearch to query log and event indices during investigations.
Connecting OpenSearch allows Traversal to query your log/event indices and use matching documents as evidence during investigations.
## What Traversal reads
* **Logs** — documents queried from your log and event indices
## Setup
Enter one or more OpenSearch host URLs, such as `https://opensearch.mycompany.com:9200`.
OpenSearch deployments expose credentials in different formats. In Traversal, choose the option that matches the credential your OpenSearch administrator gives you.
In OpenSearch Dashboards, create or choose a read-only role that can query the indices Traversal should access, then create a user and assign that role.
Use this option if your OpenSearch deployment gives you a single encoded API key value.
Ask your OpenSearch administrator to create an API key with read-only access to the indices Traversal should query, then copy the full encoded key value into Traversal.
Use this option if your OpenSearch deployment gives you two values: an API key ID and an API key secret.
Ask your OpenSearch administrator to create a read-only API key, then copy the **API Key ID** and **API Key Secret** into the matching fields in Traversal.
Use this option only if your OpenSearch cluster already accepts `Authorization: Bearer ` headers, typically through JWT authentication.
Ask your OpenSearch administrator or identity team to issue a token for a user or service principal that has read access to the target indices. The token must be signed by an issuer your cluster trusts, and its subject or role claims must map to permissions OpenSearch recognizes.
Before saving the integration, test the token with a simple read request:
```bash theme={null}
curl -H "Authorization: Bearer $TOKEN" https://opensearch.example.com:9200/_cluster/health
```
OpenSearch On-Behalf-Of tokens are short-lived, so they are usually not a good fit for a saved integration unless you rotate them automatically.
Go to **Company Knowledge > Integrations**, select OpenSearch, and enter your host URLs and credentials.
If you want clickable links from Traversal back to your OpenSearch UI, optionally set **Base URL for UI** to your OpenSearch Dashboards URL.
## More information
* [OpenSearch access control](https://docs.opensearch.org/docs/latest/security/access-control/)
* [OpenSearch JWT authentication](https://opensearch.org/docs/latest/security/authentication-backends/jwt/)
* [OpenSearch authorization tokens](https://docs.opensearch.org/latest/security/access-control/authentication-tokens/)
# Prometheus
Source: https://docs.traversal.com/integrations/prometheus
Connect Prometheus to query time-series metrics during investigations.
Connecting Prometheus lets Traversal query your metrics during investigations so it can detect anomalies, compare baselines, and correlate changes across services.
## What Traversal reads
* **Metrics** — time-series data via PromQL instant and range queries
* **Labels** — label names and values used to scope queries to the right services
## Setup
Log in to your Grafana Cloud account at [grafana.com/auth/sign-in](https://grafana.com/auth/sign-in). On the portal page, find your stack and click **Launch** to open the Grafana instance (URL looks like `https://.grafana.net`).
In your Grafana instance, go to **Connections > Data sources** and click the provisioned Prometheus data source (named `grafanacloud--prom`).
* Under **Connection**, copy the **Prometheus server URL** (e.g., `https://prometheus-prod-XX-prod-us-west-0.grafana.net/api/prom`)
* Under **Authentication**, note the **User** field — this is your numeric instance ID (e.g., `3116090`)
1. In the Grafana Cloud portal, go to **Security > Access Policies**
2. Click **Create access policy**, give it a name, and add the `metrics:read` scope for your stack
3. Click **Add token** and copy the generated `glc_...` token
Go to **Company Knowledge > Integrations**, select **Prometheus**, and enter:
* **Server URL**: the Prometheus query endpoint from step 2
* **Authentication**: select **Basic Authentication**
* **Username**: your numeric instance ID
* **Password**: the `glc_...` access policy token
Grafana Cloud requires basic auth — the username identifies your tenant in Grafana's multi-tenant Prometheus backend. Bearer token authentication will not work.
Identify the Prometheus HTTP API endpoint (e.g., `http://prometheus:9090`).
If your Prometheus is behind a reverse proxy or accessed through Grafana, find the direct URL in **Grafana > Connections > Data sources > Prometheus** under the **Connection** section. Do not use the Grafana dashboard URL.
If the data source is provisioned or uses a proxy, ask your administrator for the direct Prometheus URL and credentials.
Go to **Company Knowledge > Integrations**, select **Prometheus**, and enter:
* **Server URL**: your Prometheus endpoint
* **Authentication** (optional): basic auth credentials if your instance requires them
For managed Prometheus services (e.g., Amazon Managed Prometheus, Coralogix), find the remote read/query endpoint and generate an API token or bearer token with read access to metrics. Refer to your provider's documentation.
Go to **Company Knowledge > Integrations**, select **Prometheus**, and enter:
* **Server URL**: the query endpoint
* **Authentication**: select **Bearer Token** and paste your token
## More information
* [Prometheus HTTP API](https://prometheus.io/docs/prometheus/latest/querying/api/)
* [Grafana Cloud Prometheus documentation](https://grafana.com/docs/grafana-cloud/send-data/metrics/metrics-prometheus/)
# Sentry
Source: https://docs.traversal.com/integrations/sentry
Connect Sentry for error tracking and issue context during investigations.
Connecting Sentry lets Traversal pull error context and issue information so it can reference ongoing and past incidents during investigations.
## What Traversal reads
* **Events** — error and performance events
* **Issues** — grouped errors with status and metadata
* **Releases** — release metadata used to correlate errors with deploys
Traversal only ever reads from Sentry — every scope below is read-only.
## Required scopes
Grant the following read-only scopes. If your Sentry instance offers Read / Write / Admin access per resource, set everything to **Read**.
| Scope | Used for |
| ------------------ | ------------------------------------------------------------- |
| `event:read` | Reading error and performance events |
| `project:read` | Resolving projects to scope queries |
| `project:releases` | Reading release metadata to correlate errors with deploys |
| `org:read` | Reading organization metadata |
| `member:read` | Resolving members referenced on issues |
| `team:read` | Resolving teams that own projects |
| `alerts:read` | Reading alert rules to correlate issues with triggered alerts |
Use the API **auth token**, not a DSN — the DSN is the write-only SDK ingestion key and will not authorize reads.
## Setup
An internal integration is org-scoped, least-privilege, and survives people leaving the team.
1. In Sentry, go to **Settings → Custom Integrations → Create New Integration → Internal**
2. Give it a name (e.g. `traversal-prod`)
3. Grant read access on the [scopes listed above](#required-scopes)
4. Copy the generated auth token
Go to **Company Knowledge → Integrations**, select **Sentry**, and enter:
* **Auth Token** — the internal-integration token you just created
* **Organization Slug** — the org segment in your Sentry URLs (e.g. `my-org` in `sentry.io/organizations/my-org`)
* **Base URL** *(optional)* — leave blank for `sentry.io`. Set `https://de.sentry.io` for the EU region, or your instance URL for single-tenant / self-hosted Sentry
Click **Save**.
A personal auth token is tied to an individual account — use it as a fallback if you cannot create an internal integration.
1. In Sentry, go to **Settings → Auth Tokens → Create New Token**
2. Give it a name (e.g. `traversal-prod`)
3. Select the read [scopes listed above](#required-scopes)
4. Make it non-expiring and copy the token — it is only shown once
Go to **Company Knowledge → Integrations**, select **Sentry**, and enter:
* **Auth Token** — the personal auth token you just created
* **Organization Slug** — the org segment in your Sentry URLs (e.g. `my-org` in `sentry.io/organizations/my-org`)
* **Base URL** *(optional)* — leave blank for `sentry.io`. Set `https://de.sentry.io` for the EU region, or your instance URL for single-tenant / self-hosted Sentry
Click **Save**.
## More information
* [Sentry API authentication](https://docs.sentry.io/api/auth/)
* [Sentry internal integrations](https://docs.sentry.io/organization/integrations/integration-platform/internal-integration/)
# ServiceNow
Source: https://docs.traversal.com/integrations/servicenow
Connect ServiceNow for ITSM, ITOM / Event Management, CMDB, and ACC context during investigations.
Connecting ServiceNow lets Traversal pull read-only operational context during investigations.
## What Traversal reads
Traversal reads each of the following tables with `GET /api/now/table/{table}`.
| Table | What Traversal reads | Module |
| ---------------------- | -------------------------------------------------- | ----------------------- |
| `incident` | Incidents, including the initial connection test | ITSM |
| `change_request` | Change requests | ITSM |
| `sys_user_group` | Assignment groups | ITSM |
| `sys_attachment` | Attachment metadata on the records Traversal reads | ITSM |
| `em_alert` | Alert records | ITOM / Event Management |
| `cmdb_ci` | Configuration items | CMDB |
| `cmdb_rel_ci` | Relationships between configuration items | CMDB |
| `cmdb_ci_service` | Business services | CMDB |
| `cmdb_ci_business_app` | Business applications | CMDB |
| `service_offering` | Service offerings | CMDB |
| `kb_knowledge` | Knowledge base articles | Knowledge |
Use this table as the source of truth for the read access the ServiceNow integration needs on a standard instance. Traversal needs read access to these tables and no others, and no write access to any of them.
### Additional endpoints
Traversal also calls two endpoints outside the Table API:
| Endpoint | Purpose |
| --------------------------------------- | -------------------------------------------------------------------------------------------- |
| `GET /api/now/attachment/{sys_id}/file` | Downloads the contents of an attachment found on a record Traversal has already read |
| `POST /api/now/v1/batch` | Reads ACC / Clotho metrics, such as CPU, memory, and disk, for supported configuration items |
The Batch API call carries its sub-requests in the request body. Traversal puts one `GET` per metric there, in the form:
```text theme={null}
GET /api/now/v1/clotho/table/cmdb_ci_computer/{ci_sys_id}/{metric}
```
The ACC / Clotho metrics are optional. Traversal reads them only for supported hardware configuration items, so you can leave that access out if you do not want Traversal to read metrics.
The Batch API call is an HTTP `POST`, but it is still read-only. The method is a `POST` because ServiceNow's Batch API takes its sub-requests in the request body, and Traversal only ever puts `GET` requests there. The call creates and modifies no records.
### Custom tables
Some ServiceNow instances are customized enough that Traversal needs to read an additional or custom table, such as a database view that replaces a standard table. Traversal identifies any such tables with you during onboarding and adds them to the list above for your deployment. The tables in this page are what a standard ServiceNow instance requires.
## Before you start
Ask your ServiceNow administrator for:
* your ServiceNow instance URL, such as `https://your-instance.service-now.com`
* your token URL, which is usually `https://your-instance.service-now.com/oauth_token.do`
* an integration user with read access to the tables listed in [What Traversal reads](#what-traversal-reads)
* if you want ACC metrics, whatever Clotho or ACC permissions your instance requires for metrics access
Some teams use roles such as `snc_read_only` for general table reads and `sn_clotho_user` for Clotho metrics access, but ServiceNow roles vary by module, release, and instance customization. Confirm the exact role set with your ServiceNow administrator instead of assuming one universal combination.
## Setup
Use this option when your administrator gives you a dedicated OAuth application's **Client ID**, **Client Secret**, and token URL.
Use this option when your administrator gives you a ServiceNow **Username**, **Password**, and token URL for the integration user.
Traversal uses those credentials to obtain access tokens automatically. You do not need to generate and paste a bearer token yourself.
In Traversal, you will enter:
* **Instance URL**
* **Token URL**
* either **Client ID** and **Client Secret**, or **Username** and **Password**
The token URL is usually:
```text theme={null}
https://.service-now.com/oauth_token.do
```
Go to **Company Knowledge > Integrations**, select ServiceNow, and enter:
* **Instance URL** (e.g., `https://your-instance.service-now.com`)
* **Auth type** — select the option that matches the credentials your administrator provided
* **Token URL**
* For client credentials: **Client ID** and **Client Secret**
* For username and password: **Username** and **Password**
If the integration does not work after you save it, confirm that:
* the credentials have read access to every table in [What Traversal reads](#what-traversal-reads)
* the token URL is the correct `oauth_token.do` endpoint for your instance or custom domain
* the integration user is allowed to use ServiceNow web services
## FAQ
Use **OAuth client credentials** if your administrator can issue a dedicated OAuth app for Traversal. Use **Username and password** if your administrator prefers an integration user account. Both methods let Traversal obtain OAuth access tokens automatically, but the operational difference is whether your administrator manages an OAuth app or a user credential.
No. Traversal only makes read-only API calls. No records are created, updated, or deleted.
One call uses the `POST` method: `POST /api/now/v1/batch`, which Traversal uses to fetch ACC / Clotho metrics. ServiceNow's Batch API takes its sub-requests in the request body, which is why the call is a `POST`, and Traversal only puts `GET` requests there. If your security review flags the method, this is the call it found.
Read-only access to the tables in [What Traversal reads](#what-traversal-reads), plus the [additional endpoints](#additional-endpoints) if you want attachments and ACC metrics. Traversal needs nothing beyond read access to those. See [Before you start](#before-you-start) for guidance on the roles that grant it.
## More information
* [ServiceNow REST API](https://developer.servicenow.com/dev.do#!/reference/api)
* [ServiceNow OAuth setup](https://support.servicenow.com/kb?id=kb_article_view\&sysparm_article=KB2058319)
# Slack
Source: https://docs.traversal.com/integrations/slack
Install the Traversal Slack app in your workspace.
Bring Traversal into your team's Slack workspace so engineers can start investigations and receive reports without leaving Slack.
## Install the Traversal Slack app
### SaaS installation
Install Traversal for Slack directly from the Integrations page in Traversal. A Slack workspace administrator for your company must be the one to complete this install.
In Traversal, go to **Company Knowledge → Integrations**, select **Slack**, and click **Add to Slack**. You'll be redirected to Slack to authorize the app.
Review the requested permissions and approve the install.
Slack returns you to the Integrations page. Confirm the Slack tile passes its health check.
You must be signed into Traversal and start the install from the Slack tile so the app is provisioned for the right organization.
### Dedicated deployment installation (Single-tenant SaaS & BYOC)
For [single-tenant SaaS](/architecture/single-tenant-saas) or [BYOC](/architecture/byoc) customers, Traversal creates a dedicated Slack app for your organization.
Contact the Traversal team to receive a custom Slack app installation URL for your organization.
A Slack workspace administrator opens the provided link and installs the Traversal app to your workspace.
Reach out to the Traversal team to provision your app and start using it.
## Verify it works
Once your app is provisioned, tag `@Traversal` in any channel it's been added to and say hello.
## Uninstall and data deletion
To remove the app from your workspace, go to **Slack → Settings & administration → Manage apps → Traversal → Remove**.
For data export or deletion, email [support@traversal.com](mailto:support@traversal.com) with your workspace ID.
## FAQ
This likely means your Slack app isn't provisioned correctly. Reach out to a Traversal team member to verify your app is properly set up.
Learn how to start investigations, configure triggers, and ask follow-up questions in Slack.
# Splunk
Source: https://docs.traversal.com/integrations/splunk
Connect Splunk to run read-only log searches during investigations.
Connecting Splunk lets Traversal run read-only searches on your indexed logs and use the results as evidence during investigations.
## What Traversal reads
* **Logs** — search results from your indexed data via SPL queries
## Setup
Use the Splunk `splunkd` management URL over HTTPS, for example `https://splunk.mycompany.com:8089`.
This should be the base URL before any `/services/...` path.
In standard deployments, this is usually port `8089`. Do not use the Splunk Web UI URL if it points to a different port such as `8000`.
**Splunk Cloud:** Port `8089` is not open by default. Ask your Splunk Cloud administrator to add the IP addresses that Traversal connects from to the IP allow list for the management port before the integration will work.
Before you create a token:
* Make sure token authentication is enabled in Splunk.
* Your account must have a role with the `edit_tokens_own` capability to create tokens for yourself, or the `edit_tokens_all` capability to create tokens for any user on the instance.
* The user the token is created for must already exist on that Splunk instance.
* If your Splunk instance uses LDAP or SAML authentication, token creation can depend on your identity provider or directory configuration. Check with your Splunk administrator if token creation fails for an existing user.
* Be ready to copy the full token immediately after you create it. Splunk only shows it once, and you cannot retrieve the full token later.
1. In Splunk Web, sign in as a user who can create tokens.
2. Go to **Settings > Tokens**.
3. Click **New Token**.
4. Create the token for a user with read access to the indexes Traversal should search.
5. Click **Create** and copy the full token immediately. Splunk only shows it once.
Use a Splunk user with read access to the indexes Traversal should search.
Traversal signs in to Splunk and manages the session key automatically. You do not need to create or paste a session key yourself.
Go to **Company Knowledge > Integrations**, select Splunk, and enter the REST API base URL plus your chosen credentials.
Click **Save**.
If the integration does not work as expected, confirm that:
* the URL points to the Splunk management API over HTTPS
* the token or user has read access to the indexes Traversal should search
* the URL does not already include a `/services/...` path
## More information
* [Splunk authentication tokens](https://help.splunk.com/en/splunk-enterprise/administer/manage-users-and-security/9.1/authenticate-into-the-splunk-platform-with-tokens/create-authentication-tokens#ariaid-title6)
* [Basic concepts about the Splunk platform REST API](https://help.splunk.com/en/splunk-enterprise/leverage-rest-apis/rest-api-user-manual/9.3/rest-api-user-manual/basic-concepts-about-the-splunk-platform-rest-api)
# Tempo
Source: https://docs.traversal.com/integrations/tempo
Connect Grafana Tempo to query distributed traces during investigations.
Connecting Tempo allows Traversal to query and analyze distributed traces from your Grafana Tempo instance during incident investigations.
## What Traversal reads
* **Traces** — distributed traces searched by tags and fetched by trace ID
## Setup
Use this option if Traversal can reach Tempo, a Tempo gateway, or Tempo query-frontend directly.
If you already view traces in Grafana, start there:
1. In Grafana, open **Connections > Data sources**.
2. Select the Tempo data source you use for traces.
3. Under **Connection**, copy the Tempo URL.
Enter that URL in Traversal as the base URL of the Tempo HTTP API.
* **Microservices deployments:** Use the query-frontend URL or a gateway in front of Tempo.
* **Monolithic deployments:** Use the Tempo base URL.
* **Grafana Cloud:** Use your stack Tempo URL, which is typically `https://.grafana.net/tempo`.
Do not enter a trace-specific path such as `/api/traces/`.
Use this option if Traversal should query Tempo through Grafana instead of connecting to Tempo directly.
1. In Grafana, open **Connections > Data sources**.
2. Select the Tempo data source you use for traces.
3. Note the data source UID from the page URL. In Grafana, the edit page URL ends with `/connections/datasources/edit/` or `/datasources/edit/`.
4. Build the Traversal URL as `https:///api/datasources/proxy/uid/`.
For example, if your Grafana URL is `https://grafana.example.com` and the data source UID is `tempo`, enter `https://grafana.example.com/api/datasources/proxy/uid/tempo`.
Do not copy the **Connection > URL** field for this option. That field is the underlying Tempo endpoint, not the Grafana datasource proxy URL.
Traversal can send an optional bearer token and `X-Scope-OrgID` header with either option.
* If you use a direct Tempo endpoint, the token usually authenticates to a Tempo gateway or reverse proxy.
* If you use a Grafana datasource proxy URL, the token must authenticate to Grafana or whatever sits in front of Grafana.
If you are not sure which option to use, ask your Grafana or Tempo administrator which URL Traversal should call directly.
Only set **Organization ID** if your administrator told you to send the `X-Scope-OrgID` header. In many multi-tenant deployments, the gateway or Grafana proxy adds this header for you.
Go to **Company Knowledge > Integrations**, select Tempo, and enter the URL from the option you chose and any optional authentication details.
Click **Save**.
If the integration does not work as expected, confirm that you entered either a direct Tempo URL or a Grafana datasource proxy URL, not a trace-specific path, and that any required authentication or tenant headers are configured correctly.
## More information
* [Tempo HTTP API](https://grafana.com/docs/tempo/latest/api_docs/)
* [Grafana integration](/integrations/grafana)
* [Grafana data source HTTP API](https://grafana.com/docs/grafana/latest/developers/http_api/data_source/)
* [Locate your stack's URL, user, and password](https://grafana.com/docs/grafana-cloud/send-data/traces/set-up/locate-url-user-password/)
* [Enable multi-tenancy](https://grafana.com/docs/tempo/latest/setup/operator/multitenancy)
# ThousandEyes
Source: https://docs.traversal.com/integrations/thousandeyes
Connect ThousandEyes for network intelligence and monitoring data during investigations.
Connecting ThousandEyes lets Traversal pull network test data and alert context during investigations.
## What Traversal reads
* **Tests** — network, HTTP, and DNS test results
* **Alerts** — active and historical alert records
## Setup
1. Log in to ThousandEyes
2. Go to **Manage → Account Settings → Users and Roles**
3. Select your user and go to the **Profile** tab
4. Under **User API Tokens**, select **OAuth Bearer Token**
5. Click **Create** — you may be prompted for a verification code as part of MFA
6. Copy the token — it is only shown once
Go to **Company Knowledge → Integrations**, select **ThousandEyes**, and enter the API token.
Click **Save**.
## More information
* [ThousandEyes API documentation](https://developer.thousandeyes.com/)
# VictoriaMetrics
Source: https://docs.traversal.com/integrations/victoria-metrics
Connect VictoriaMetrics for high-performance metrics querying during investigations.
Connecting VictoriaMetrics lets Traversal query your metrics during investigations so it can detect anomalies, compare baselines, and correlate changes across services.
## What Traversal reads
* **Metrics** — time-series data queried via PromQL instant and range queries
* **Labels** — label names and values used to scope queries to the right services and environments
## Setup
Contact your Traversal administrator to configure this integration.
Have the following ready:
| Information | Details |
| ------------------ | -------------------------------------------------------------------------- |
| **Cluster URLs** | One or more endpoint URLs (e.g. `https://vm.example.com:8428`) |
| **Authentication** | Basic auth (username + password), bearer token, or none if unauthenticated |
## More information
* [VictoriaMetrics documentation](https://docs.victoriametrics.com/)
# About Traversal
Source: https://docs.traversal.com/introduction
AI SRE that automatically detects incidents, triages alerts, and performs root cause analysis across your entire observability stack.
Traversal is an AI site reliability engineering (SRE) platform built for teams responsible for keeping production running. It continuously gathers context across your observability stack, denoises and triages alerts, and performs root cause analysis automatically — so you understand what's happening in production without war rooms, manual correlation, or endless dashboard-switching.
Start your first investigation in minutes
Connect your observability stack to Traversal
Investigate incidents directly from Slack
A proactive agent that joins incident channels and owns the response
Use Traversal from Claude, Cursor, and other AI clients
Teach Traversal your team's runbooks and tribal knowledge
## What You Can Do With Traversal
* **Automatically detect incidents and perform RCA** across metrics, logs, traces, and alerts
* **Own live incident response** with a proactive [Incident Worker](/using-traversal/incident-workers) that joins the channel, investigates on its own, and produces a draft post-mortem
* **Triage high-volume alert streams** into a prioritized list of impactful issues
* **Investigate complex failures** across millions of events using causal reasoning, not manual filtering
* **Capture and apply runbooks and tribal knowledge** to improve accuracy over time
* **Prevent SLA/SLO breaches** through early detection and context-aware alerting
* **Generate post-mortems** from any investigation in one click
## Who Traversal Is For
Traversal is built for teams closest to incidents and customer impact:
* **SREs and platform engineers** operating complex, distributed systems at scale
* **On-call engineers** overwhelmed by alert volume and manual correlation
* **Mission control / first line of defense teams** who monitor alerts and triage incidents under time pressure
* **New or junior engineers** ramping up in unfamiliar systems and learning institutional knowledge quickly
* **Product managers and customer success teams** who need accurate, real-time system understanding without deep observability expertise
## How Traversal Works
Traversal is agent-less and schema-less, operating with accuracy and speed across fast-changing environments without manual setup.
* **Builds a system world model** by mining telemetry and code to represent millions of entities, statistical baselines, and relationships across your production environment
* **Applies causal search to observability data**, compressing, re-indexing, and ranking telemetry to enable deep investigations at machine scale
* **Learns autonomously through self-play and passive observation**, improving accuracy over time without hard-coded prompts or brittle heuristics
* **Scales from team to enterprise** with no agents, schemas, or per-service configuration required
## Where Traversal Works
Traversal meets you where you are:
* **Web app** — rich timelines, signal previews, and post-mortem creation at [app.traversal.com](https://app.traversal.com)
* **Slack** — start or auto-trigger investigations, receive RCA summaries, and ask follow-ups in-thread
* **MCP** — run investigations from Claude, Cursor, ChatGPT, and other AI clients via the [Model Context Protocol](/using-traversal/mcp)
## What's new in Traversal
Traversal ships continuously. See the [changelog](/changelog) for the latest
features, improvements, and updates.
***
Have feedback or questions? Reach the team at [support@traversal.com](mailto:support@traversal.com).
# Data privacy
Source: https://docs.traversal.com/responsible-use/data-privacy
How Traversal handles your data — including isolation guarantees, read-only access, integration scopes, and compliance alignment.
Traversal is built on a foundation of strict data privacy. Your data is never shared with other customers, never used to train models, and never processed beyond the scope of your own investigations.
## Core privacy principles
Your data is never used for cross-customer training, model improvement, or service optimization.
Any data processing is limited to in-context use for your organization only. Data from your investigations does not leave your context.
All customer data is protected through strict tenant isolation, access controls, and privacy safeguards.
Traversal cannot modify your systems or data. All integrations use read-only access only.
## What data Traversal accesses
Traversal only accesses what is necessary to perform investigations. All access is read-only.
Traversal reads metrics, logs, traces, and alerts from your connected observability tools. This data is used in-context to perform root cause analysis and is not retained beyond the investigation.
Traversal requests read-only API access to your repositories to correlate deployment events and code diffs with incidents. Traversal has no write privileges to your codebase.
Traversal runs as a Slack app installed by a workspace administrator, and can only see channels the app has been added to. Within those channels it:
* **Reads conversation context** — messages, threads, reactions, pinned items, shared files, and link previews — to understand the incident and any follow-up questions (`app_mentions:read`, `channels:history`, `groups:history`, `im:history`, `mpim:history`, `channels:read`, `groups:read`, `im:read`, `mpim:read`, `reactions:read`, `pins:read`, `links:read`, `files:read`).
* **Resolves requester identity** — display name and email — to attribute requests and reports (`users:read`, `users:read.email`).
* **Posts back into Slack** — investigation results, replies, reactions, uploaded report files, direct messages, and slash-command responses (`chat:write`, `im:write`, `reactions:write`, `files:write`, `links:write`, `commands`, `incoming-webhook`).
* **Joins channels** it is invited to (`channels:join`).
These are messaging-surface permissions only. They do not grant Traversal any write or data-plane access to your connected systems.
Traversal runs as a Microsoft-approved Teams bot (an Azure AD app plus Azure Bot) installed by a Teams administrator. Your organization decides where it participates — it acts only on mentions, configured triggers, or Workers your team has enabled. In those conversations it:
* **Reads conversation context** — the content of messages that mention or trigger the bot, thread context, and channel/team names, plus any attached files or images (`ChannelMessage.Read.All`, `Team.ReadBasic.All`, `Channel.ReadBasic.All`; `Chat.Read.All` only where 1:1 and group chats are in scope).
* **Resolves requester identity** — the sender's Azure AD user ID, display name, and email — to attribute requests (`User.Read.All`).
* **Reads membership**, where enabled — who belongs to a channel or team (`ChannelMember.Read.All`, `TeamMember.Read.All`).
* **Posts back into Teams** — authenticates as its registered app and posts investigation results into the originating channel or thread.
These are messaging-surface permissions only. They do not grant Traversal any write or data-plane access to your connected systems.
No agents are deployed in your environment. Traversal operates without sidecars or background processes running in your infrastructure.
## Compliance alignment
Traversal aligns with the following compliance frameworks:
| Framework | Status |
| ------------- | -------- |
| SOC 2 Type II | Attested |
| GDPR | Aligned |
| HIPAA | Aligned |
Traversal also undergoes regular third-party penetration testing.
Visit the [Traversal Trust Center](https://trust.traversal.com/) to request compliance reports or review security documentation.
## Contact
For questions about data privacy, compliance requirements, or to request detailed documentation, contact [security@traversal.com](mailto:security@traversal.com).
# Security
Source: https://docs.traversal.com/responsible-use/security
Traversal's security architecture, certifications, and controls — including SOC 2 Type II, GDPR, HIPAA, and read-only access design.
Traversal maintains robust controls across access, data handling, and system operations. Security is a core design principle, not an add-on.
## Security certifications
Traversal maintains a **SOC 2 Type II** attestation, aligns with **GDPR** and **HIPAA** requirements, and undergoes regular third-party penetration testing.
Request compliance reports, review security documentation, and find more information about Traversal's certifications.
## Security-first architecture
Traversal is built from the ground up with security as a core design principle.
Traversal deploys no sidecars or background processes in your environment. There is nothing running in your infrastructure on Traversal's behalf.
Traversal cannot modify your systems or data. All integrations require read-only access only — no write privileges to your data stores or code.
Traversal does not rely on brittle integrations or sensitive structural dependencies. There is no schema configuration required.
Traversal avoids hard-coded prompts or hidden logic that can introduce risk or unpredictable behavior.
## Network allowlisting
If your network policies or firewalls require allowlisting of external IP addresses, the relevant IPs depend on how Traversal connects to your environment.
These IP addresses are expected to remain stable, but may change due to infrastructure maintenance operations. Traversal does not guarantee IP address permanence.
### Direct integrations
When Traversal's [SaaS](/architecture/intro#saas) environment connects directly to your integrated services (not through the [Traversal Connector](/architecture/connector)), it uses the following IP addresses:
| IP address |
| -------------- |
| `50.112.45.78` |
| `35.162.155.1` |
| `52.43.83.98` |
Add these to your service-side allowlists.
### Traversal Connector
When the [Traversal Connector](/architecture/connector) is deployed in your environment to reach Traversal's [SaaS](/architecture/intro#saas), it makes outbound connections to two destinations: it connects to `edge.traversal.com`, and it exports its own telemetry to `telemetry.traversal.com`. If your environment restricts egress traffic, allowlist the IP addresses for both.
`edge.traversal.com`:
| IP address |
| --------------- |
| `44.254.181.73` |
| `16.148.183.36` |
| `54.71.194.238` |
`telemetry.traversal.com`:
| IP address |
| ---------------- |
| `184.33.54.68` |
| `44.230.130.223` |
| `100.22.80.68` |
Telemetry egress uses port `4317` by default. If you restrict egress by port as well as by address, allow the port named by the telemetry endpoint your deployment is configured with.
### Traversal Processor
When the [Traversal Processor](/setup/processor) is deployed in your environment to reach Traversal's [SaaS](/architecture/intro#saas), it makes outbound connections to two destinations: it sends data to `ingest.traversal.com`, and it exports its own telemetry to `telemetry.traversal.com`. If your environment restricts egress traffic, allowlist the IP addresses for both.
`ingest.traversal.com`:
| IP address |
| ---------------- |
| `52.13.31.181` |
| `16.146.208.114` |
| `44.241.232.92` |
`telemetry.traversal.com`:
| IP address |
| ---------------- |
| `184.33.54.68` |
| `44.230.130.223` |
| `100.22.80.68` |
Telemetry egress uses port `4317` by default. If you restrict egress by port as well as by address, allow the port named by the telemetry endpoint your deployment is configured with.
For [single-tenant SaaS](/architecture/single-tenant-saas) and [BYOC](/architecture/intro#byoc-bring-your-own-cloud) deployments, these IPs do not apply — the Traversal Connector and Processor reach the Traversal control plane within your dedicated deployment instead.
## Data privacy
Customer data is not used for cross-customer training, model improvement, or service optimization. Any data processing is limited to in-context use for the originating customer only. All customer data is protected through strict isolation, access controls, and privacy safeguards.
For more detail, see the [Data privacy](/responsible-use/data-privacy) page.
## Contact security
For detailed security documentation, compliance reports, or to discuss your specific requirements, contact the Traversal security team at [security@traversal.com](mailto:security@traversal.com).
# Traversal Connector
Source: https://docs.traversal.com/setup/connector
Deploy the Traversal Connector to Kubernetes via Helm or to any container runtime via environment variables.
The Traversal Connector is a lightweight service that establishes an outbound, mTLS-encrypted connection to the Traversal control plane, allowing Traversal to access your private data sources without any inbound network access. For an architectural overview, see [Traversal Connector](/architecture/connector).
The connector ships as a single container image and supports two deployment paths: a Helm chart for Kubernetes (the recommended path), or running the image directly on any container runtime — Docker on EC2, systemd-managed containers, Nomad, etc. — configured through environment variables.
If your environment restricts outbound traffic, see [Network allowlisting](/responsible-use/security#traversal-connector) for the IPs to allow before deploying.
## Client certificate
The connector authenticates to the Traversal control plane over mTLS. The recommended way to provision the client certificate is to generate the private key in your own environment and send Traversal a Certificate Signing Request (CSR) — the private key never leaves your environment, and Traversal countersigns the CSR with its certificate authority.
Before you start, you'll need:
* `openssl` (1.1.1 or newer, for `-addext` support), or any tool that can produce a PKCS#10 CSR with a SAN URI extension.
* Your **tenant UUID** and **tenant name**, both provided by Traversal.
The CSR's Subject Alternative Name encodes your tenant identity using a [SPIFFE URI](https://spiffe.io/docs/latest/spiffe-about/spiffe-concepts/#spiffe-id):
```
spiffe://traversal.com/tenant//
```
Traversal's control plane uses this SPIFFE ID to identify the sender of every request, so both fields must exactly match the values Traversal shared with you.
Generate a private key locally and store it in a secure location. The key never leaves your environment.
```bash theme={null}
openssl ecparam -name prime256v1 -genkey -noout -out client.key
```
P-256 is the SPIFFE-recommended algorithm. RSA 2048+ also works if your platform requires it (`openssl genpkey -algorithm RSA -pkeyopt rsa_keygen_bits:2048 -out client.key`).
Generate a CSR that embeds the SPIFFE URI as a SAN. Substitute `` and `` with the values Traversal shared with you.
```bash theme={null}
openssl req -new -key client.key -out client.csr \
-subj "/CN=" \
-addext "subjectAltName=URI:spiffe://traversal.com/tenant//"
```
Verify the SAN URI is encoded correctly before sending:
```bash theme={null}
openssl req -in client.csr -noout -text | grep -A1 "Subject Alternative Name"
```
You should see your `spiffe://...` URI on the next line.
Share `client.csr` with Traversal through the secure channel agreed upon during onboarding.
Do **not** share `client.key` — keeping the private key in your environment is the whole point of this flow.
Traversal signs the CSR with its certificate authority and returns your signed client certificate:
* `client.crt` — valid for the lifetime issued by Traversal.
Together with the `client.key` you generated in Step 1, these are the two files you'll plug into the deployment configuration below.
Where these files go depends on your deployment path:
* **Helm:** `client.crt` and `client.key` populate `controllerTLS.certPEM` and `controllerTLS.keyPEM`. See [Required values](#required-values).
* **Other:** the same two files populate `TLS_CERT_BASE64` and `TLS_KEY_BASE64` (each base64-encoded). See [Required](#required).
## Deployment
A Helm-based Kubernetes deployment is composed of three artifacts. The image and chart are published openly; only the values file requires authentication, since it may carry credentials issued for your deployment.
Public, built from the open-source connector repo.
Public OCI artifact. Generic across all deployments.
Per-deployment. Pulled with a Traversal-issued token.
## Container image
The connector is open source. The source lives in the [traversal-connector repository on GitHub](https://github.com/InteractionLabs/traversal-connector), and Traversal builds and publishes the image to the [traversalext/traversal-connector repository on Docker Hub](https://hub.docker.com/r/traversalext/traversal-connector). The image is public, so by default your cluster pulls it directly from Docker Hub.
The connector image is a multi-platform image with support for both x86 and ARM (`linux/amd64` and `linux/arm64`).
If your security policy requires it, you can also build the image yourself from the public source or mirror the published image into an internal registry — override `image.repository` (and `image.pullSecrets`, if your registry requires authentication) in your values when installing the chart.
## Helm chart
The Helm chart is published as a public OCI artifact at `oci://registry-1.docker.io/traversalext/traversal-connector-charts`. The chart is generic across all deployments — it contains no customer-specific configuration — so you can inspect or vendor it freely with standard Helm tooling:
```bash theme={null}
helm pull oci://registry-1.docker.io/traversalext/traversal-connector-charts \
--version --untar
```
The pulled chart includes a `values.yaml` template that documents every supported field and indicates which ones are required.
### Required values
The chart and connector together require the following fields. Traversal pre-populates all of them in the values file it builds for you, so you don't typically need to set them manually:
| Field | What it is |
| :-------------- | :--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `envName` | A short label identifying your deployment environment (for example, `acme-prod`). Tags the connector's metrics, logs, and traces, and identifies your deployment on the control plane side. |
| `connectorID` | A unique identifier (UUID) Traversal assigns to your connector deployment. The connector stamps it on every request to the control plane (via the `X-Traversal-Connector-ID` header) so Traversal can attribute connections to your deployment. Install fails if it is unset. |
| `controllerURL` | The URL of the Traversal control plane this connector reaches. |
| `controllerTLS` | The mTLS material the connector presents when authenticating to the control plane: a client certificate and its private key. Supplied inline as PEM strings (the chart renders a Kubernetes Secret) or by referencing the name of an existing Secret you manage. A custom CA certificate (`controllerTLS.caPEM`) is supported but optional — if not set, the system trust store is used. |
| `otel` | OTLP telemetry endpoints: `otel.metricsEndpoint`, `otel.tracesEndpoint`, and `otel.logsEndpoint`. All three default to Traversal's telemetry ingest — override them only to route through a collector you operate. See [Telemetry forwarding](#telemetry-forwarding). |
All other fields — replica count, resource requests, redaction rules, scheduling — are optional and have sensible defaults.
### Telemetry forwarding
The connector reports its own operational telemetry — metrics, traces, and logs — to Traversal over OTLP. It is how Traversal supports your deployment.
By default the chart adds a second container to the connector pod: a forwarder that buffers this telemetry and retries delivery, so a brief interruption reaching Traversal does not lose it. It authenticates with the connector's `controllerTLS` identity and needs no credentials of its own.
| Field | Default | What it is |
| :----------------------- | :------------------------ | :---------------------------------------------------------------------------------------------------------- |
| `otel.sidecar.enabled` | `true` | Runs the forwarder. Set to `false` to export directly from the connector, leaving one container in the pod. |
| `otel.sidecar.image` | `traversalext/alloy` | Forwarder image. Override `repository` to pull from an internal mirror. |
| `otel.sidecar.resources` | 50m CPU, 128–256Mi memory | Resource requests and limits for the forwarder container. |
The `otel` endpoints default to Traversal's telemetry ingest. Override all three to send to a collector you operate instead.
### Disabling telemetry
Setting `disableTelemetry: true` stops all telemetry export and removes the forwarder.
Disabling telemetry is highly discouraged. Telemetry is Traversal's only view into a connector running in your network, so without it Traversal can provide only limited support.
## Values file
The values file holds your deployment-specific configuration: your `envName`, `controllerURL`, and `connectorID`, your connector's mTLS material, and any deployment-specific tuning or telemetry destinations.
Because it may embed private key material, the values file is not published publicly. Instead, Traversal builds it for you and packages it as an OCI artifact in a private Docker Hub namespace, alongside the chart.
Through a secure channel, Traversal shares with you a Docker Hub access token, scoped read-only to your deployment's namespace, and the artifact reference for your values file (for example, `registry-1.docker.io/traversalext/traversal-connector-charts-:`).
The Docker Hub username is always `traversalext`. Only the token is per-customer.
The values file is published as an OCI artifact, which Helm doesn't natively pull. Install the [ORAS CLI](https://oras.land):
```bash theme={null}
brew install oras
```
See the [ORAS installation guide](https://oras.land/docs/installation) for non-macOS platforms.
Set the values shared by Traversal, then pull the artifact:
```bash theme={null}
export DOCKERHUB_REPO_NAME=... # provided by Traversal
export DOCKERHUB_REPO_TOKEN=... # provided by Traversal
export VALUES_FILE_VERSION=v0.7.0
oras pull \
--username traversalext --password "$DOCKERHUB_REPO_TOKEN" \
"registry-1.docker.io/traversalext/$DOCKERHUB_REPO_NAME:$VALUES_FILE_VERSION"
```
This writes the values file into your current directory. Pass `-o ` to ORAS to place it elsewhere.
## Installing
With the chart and values file in hand, install in whatever way fits your environment — `helm install` directly, a GitOps pipeline (ArgoCD, Flux) referencing the OCI chart, or an internal Helm registry mirror. As a minimal end-to-end example using Helm directly:
```bash theme={null}
helm install traversal-connector \
oci://registry-1.docker.io/traversalext/traversal-connector-charts \
--version 0.7.0 \
-f YOUR_VALUES_FILE
```
The pod establishes its connection to the Traversal control plane within a few seconds of starting; its readiness probe gates traffic until the connection is live.
## Redaction
Set `redaction.enabled` in your Helm values and provide a rules file using one of two options:
```yaml theme={null}
# Option A: inline rules — the chart creates and manages the ConfigMap
redaction:
enabled: true
rulesContent: |
version = "v1"
[[rules]]
name = "email"
type = "regex"
pattern = '[a-zA-Z0-9._%+\-]+@[a-zA-Z0-9.\-]+\.[a-zA-Z]{2,6}'
# Option B: reference a ConfigMap you manage externally
redaction:
enabled: true
existingConfigMap: my-redaction-rules
```
When using `existingConfigMap`, the key inside that resource must be named `redaction-rules.toml`.
The connector watches the rules file and hot-reloads it whenever the content changes. No restart is required. If the file is missing, unreadable, or contains an invalid pattern when hot-reloaded, the connector exits rather than using the new, invalid, redaction configuration.
See [Redaction](/setup/redaction) for the rules file format and field filtering options.
For environments without Kubernetes, run the same connector image directly on any container runtime — Docker, podman, systemd, Nomad, ECS, and so on. The image is identical to the one used in the Kubernetes path; the only difference is that you supply configuration through environment variables instead of a Helm values file.
## Container image
The connector is open source. The source lives in the [traversal-connector repository on GitHub](https://github.com/InteractionLabs/traversal-connector), and Traversal builds and publishes the image to the [traversalext/traversal-connector repository on Docker Hub](https://hub.docker.com/r/traversalext/traversal-connector). The image is public, so your runtime can pull it directly from Docker Hub. You can also build it yourself from the public source or mirror it into an internal registry.
The connector image is a multi-platform image with support for both x86 and ARM (`linux/amd64` and `linux/arm64`).
## Configuration via environment variables
Every connector setting is exposed as an environment variable. The required variables are listed first; the remaining sections are optional with sensible defaults.
### Required
| Variable | What it is |
| :--------------------------------------------------- | :----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| ENV\_NAME | A short label identifying your deployment environment (for example, `acme-prod`). Tags the connector's metrics, logs, and traces. |
| TRAVERSAL\_CONTROLLER\_URL | The URL of the Traversal control plane this connector reaches. |
| TRAVERSAL\_CONNECTOR\_ID | A unique identifier (UUID) Traversal assigns to your connector deployment. Stamped on every request to the control plane (via the `X-Traversal-Connector-ID` header) so Traversal can attribute connections to your deployment. The connector refuses to start if it is unset. |
| TLS\_CERT\_BASE64 | Base64-encoded PEM client certificate the connector presents when authenticating to the control plane. Include the leaf certificate followed by any intermediate, in that order. |
| TLS\_KEY\_BASE64 | Base64-encoded PEM private key for the client certificate above. |
| OTEL\_EXPORTER\_OTLP\_METRICS\_ENDPOINT | OTLP endpoint to which the connector exports its operational metrics. Must be an `https://` URL naming a host. |
| OTEL\_EXPORTER\_OTLP\_TRACES\_ENDPOINT | OTLP endpoint to which the connector exports its operational traces. Must be an `https://` URL naming a host. |
| OTEL\_EXPORTER\_OTLP\_LOGS\_ENDPOINT | OTLP endpoint to which the connector exports its operational logs. Must be an `https://` URL naming a host. Logs also always go to stdout. |
All three endpoints are required — the connector exits at startup if any is unset or is not an `https://` URL naming a host. Upgrade the image and the configuration that supplies these endpoints together.
### Connection tuning
| Variable | Default | What it is |
| :---------------------------------------- | :----------- | :----------------------------------------------------------------------- |
| ENV\_LEVEL | `production` | Environment level: `production` or `development`. Affects log verbosity. |
| PROXY\_URL | *none* | HTTP forward proxy for outbound traffic to the control plane. |
| HTTP\_PORT | `8080` | Port the connector listens on for health checks. |
| MAX\_TUNNELS\_ALLOWED | `2` | Maximum number of concurrent connections to the control plane. |
| MAX\_CONCURRENT\_REQUESTS | `10` | Maximum number of in-flight requests across all connections. |
| RECONNECT\_INTERVAL | `5s` | Initial reconnect interval after a connection drop. |
| MAX\_BACKOFF\_DELAY | `60s` | Maximum exponential backoff delay between reconnect attempts. |
| REQUEST\_TIMEOUT | `60s` | Per-request timeout. |
| MAX\_REQUEST\_BODY\_SIZE\_MB | `32` | Maximum HTTP request body size, in MB. |
### Control plane TLS
Optional overrides for the TLS handshake to the control plane. Set these only when the control plane uses a private CA or requires a non-default SNI.
| Variable | Default | What it is |
| :----------------------------- | :------------- | :----------------------------------------------------------------------------------------------------------------------------------------------- |
| TLS\_CA\_BASE64 | *system trust* | Base64-encoded PEM CA certificate used to verify the control plane's identity. Required only if the control plane uses a private/self-signed CA. |
| TLS\_SERVER\_NAME | *from URL* | Override SNI server name for the control plane TLS handshake. |
### Upstream TLS
Controls how the connector verifies TLS for the upstream services it queries on Traversal's behalf.
| Variable | Default | What it is |
| :------------------------------------- | :------------- | :----------------------------------------------------------------------------------- |
| UPSTREAM\_TLS\_VERIFY | `true` | Whether to verify TLS certificates of upstream services. |
| UPSTREAM\_TLS\_CA\_BASE64 | *system trust* | Base64-encoded PEM CA certificate used to verify upstream services with private CAs. |
### Telemetry
| Variable | Default | What it is |
| :------------------------------------------ | :-------------------- | :-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| OTEL\_SERVICE\_NAME | `traversal-connector` | Service name reported on all signals. |
| OTEL\_EXPORTER\_OTLP\_PROTOCOL | *empty* | `grpc` and `http/protobuf` select gRPC; `http/json` or empty selects HTTP. Endpoints take `https://host:port` for gRPC, and name the signal in the path for HTTP (`https://host/v1/metrics`). |
| TRAVERSAL\_DISABLE\_TELEMETRY | `false` | Stops all telemetry export, overriding any endpoints you configured. |
There is no forwarder container outside Kubernetes, so the connector exports directly.
Disabling telemetry is highly discouraged. Telemetry is Traversal's only view into a connector running in your network, so without it Traversal can provide only limited support.
### Redaction
The connector supports a regex-based redaction pipeline that rewrites sensitive text in upstream responses before they leave your environment. Set `REDACTION_RULES_FILE` to the in-container path of your rules file and mount it into the container:
| Variable | Default | What it is |
| :---------------------------------- | :------ | :--------------------------------------------------------------------------------- |
| REDACTION\_RULES\_FILE | *none* | Path inside the container to a TOML rules file. See [Redaction](/setup/redaction). |
```bash theme={null}
docker run --rm \
-e REDACTION_RULES_FILE=/etc/traversal/rules.toml \
-v /path/to/your/rules.toml:/etc/traversal/rules.toml:ro \
traversalext/traversal-connector:
```
The connector watches the rules file and hot-reloads it whenever the content changes. No restart is required. If the file is missing, unreadable, or contains an invalid pattern when hot-reloaded, the connector exits rather than using the new, invalid, redaction configuration. See [Redaction](/setup/redaction) for the full rules file format.
## Running the connector
Once you have the required values from Traversal, start the connector with any container runtime. As a minimal example with Docker:
```bash theme={null}
docker run --rm \
-e ENV_NAME=acme-prod \
-e TRAVERSAL_CONTROLLER_URL=https://controller.example.com \
-e TRAVERSAL_CONNECTOR_ID= \
-e TLS_CERT_BASE64="$(base64 < client.crt)" \
-e TLS_KEY_BASE64="$(base64 < client.key)" \
-p 8080:8080 \
traversalext/traversal-connector:
```
The connector exposes `/healthz` and `/readyz` on `HTTP_PORT` (default `8080`) for health and readiness checks. Wire these into your runtime's health checks so traffic is gated on the connection being live.
## Want to learn more?
Rules file format, field filtering, mount examples, and hot-reload behaviour.
How the connection works, what data flows where, and why no inbound network access is required.
Traversal Connector source code — inspect it, or build your own image.
The published multi-platform connector image (`linux/amd64` and `linux/arm64`).
Pull OCI artifacts from any registry.
Helm's native OCI support for chart distribution.
# PrivateLink
Source: https://docs.traversal.com/setup/privatelink
Establish AWS PrivateLink connections between your VPC and your dedicated Traversal deployment (single-tenant SaaS or BYOC) so traffic to Traversal services stays on the AWS private network.
AWS PrivateLink provides private connectivity between VPCs and services by keeping traffic on the AWS network backbone. It works through Interface Endpoints, eliminating the need for public IPs, internet gateways, or VPC peering. As a service provider (in the PrivateLink terminology), Traversal's dedicated deployments (single-tenant SaaS and BYOC) expose four endpoint services: its web application, API, Edge (for the Connector), and Ingest (for the Processor).
In a dedicated Traversal deployment, PrivateLink is how your VPCs reach Traversal's endpoints without exposing them to the public internet. After deploying the VPC endpoints in your VPC(s), Traversal-supplied hostnames resolve only inside VPCs you explicitly authorize, and the wildcard server certificate is issued by your own CA against a domain you own. For an architectural overview, see [single-tenant SaaS](/architecture/single-tenant-saas) or [Bring Your Own Cloud](/architecture/byoc).
You establish the connection by deploying a CloudFormation template that Traversal publishes. The template creates one VPC Interface Endpoint per service; you supply your own security groups and they are attached per endpoint, giving you independent access control per service. Each security group must allow inbound HTTPS (port 443) from the CIDR ranges or sources that need to reach that service. Each endpoint resolves a Traversal-supplied hostname to a private IP inside your VPC, keeping all traffic on the AWS network.
Multi-VPC access: Repeat the deployment steps in each VPC where Traversal services need to be reachable. Each VPC needs its own set of 4 Interface Endpoints and its own association with the private hosted zone.
## Before you begin
Traversal shares the following with you through a secure channel during onboarding of your dedicated deployment:
* **Endpoint Service names** — one per service (`api`, `app`, `edge`, `ingest`), in the form `com.amazonaws.vpce..vpce-svc-xxxxxxxxxxxxxxxxx`.
* **A Route53 private hosted zone ID** and its **four nameservers**.
* **The subdomain** that resolves the Traversal endpoints (for example, `traversal..com`).
On your side, you'll need:
* An AWS account, and a VPC where the Interface Endpoints will live.
* Subnet IDs in that VPC (one per AZ recommended).
* The CIDR ranges of every client network that should reach the endpoints.
* A parent zone you control where you can add a delegating `NS` record.
## DNS Resolution
Traversal creates a private Route53 hosted zone for a subdomain and shares four nameservers with you. You add an NS record on your parent zone pointing at those nameservers, delegating the subdomain to Traversal.
This lets Traversal use a subdomain you own, for example, `traversal..com`, so that Traversal service hostnames appear under your domain rather than a Traversal-owned one.
Traversal then populates the hosted zone with CNAME records pointing to your Interface Endpoint DNS names, which in turn resolve to the private IP of the endpoint inside your VPC. These hostnames only resolve inside VPCs that are associated with the hosted zone and are not resolvable from the public internet.
## Deployment
On your parent zone, add an `NS` record for your delegated subdomain pointing at the four nameservers Traversal shared with you.
Deploy the PrivateLink consumer CloudFormation template ([https://templates.traversal.com/traversal-privatelink-consumer.yaml](https://templates.traversal.com/traversal-privatelink-consumer.yaml)) in the AWS account and region where the Interface Endpoints will live. Pass the four `*EndpointServiceName` values Traversal shared with you, along with your VPC ID, subnet IDs, and the CIDR ranges of clients that should reach the endpoints.
The stack outputs include one DNS name per service (`ApiEndpointDnsName`, `AppEndpointDnsName`, `EdgeEndpointDnsName`, `IngestEndpointDnsName`).
Share your VPC IDs and regions with Traversal. Once Traversal authorizes them, run the following from your account for each VPC:
```bash theme={null}
aws route53 associate-vpc-with-hosted-zone \
--hosted-zone-id \
--vpc VPCRegion=,VPCId=
```
Share the four `*EndpointDnsName` outputs from the CloudFormation stack with Traversal. Traversal creates the CNAMEs in the private hosted zone so `.` resolves to your Interface Endpoint.
## DNS-only VPC access
If a VPC only needs to resolve Traversal hostnames but not connect directly to the endpoints — for example, an admin VPC — you can authorize it without deploying Interface Endpoints. Share the VPC IDs and regions with Traversal; we'll authorize them on our side. Then run the following from your account for each VPC:
```bash theme={null}
aws route53 associate-vpc-with-hosted-zone \
--hosted-zone-id \
--vpc VPCRegion=,VPCId=
```
Once associated, DNS queries for the Traversal subdomain resolve inside that VPC without Interface Endpoints.
## Want to learn more?
How Traversal single-tenant SaaS is deployed in Traversal's own cloud account, what's in scope, and the security boundary.
How Traversal BYOC is deployed in your AWS account, what's in scope, and the security boundary.
AWS documentation on PrivateLink, Interface Endpoints, and Endpoint Services.
# Traversal Processor
Source: https://docs.traversal.com/setup/processor
Deploy the Traversal Processor to Kubernetes via Helm or to any container runtime via environment variables.
The Traversal Processor is a high-performance telemetry ingestion and processing service that receives telemetry data from your infrastructure, and extracts and indexes structured data into the Traversal platform. Today, the Traversal Processor accepts logs data only, via HTTP endpoints and OpenTelemetry Protocol (OTLP) over both gRPC and HTTP.
The Traversal Processor ships as a single container image can be deployed via a Helm chart and a Values file provided by Traversal. The Traversal Processor is designed to operate as a horizontally scalable service, and does not require coordination across instances.
If your environment restricts outbound traffic, see [Network allowlisting](/responsible-use/security#traversal-processor) for the IPs to allow before deploying.
## Deployment
A Helm-based Kubernetes deployment is composed of three artifacts. While the chart is published openly, the image requires authentication. The values file also requires authentication, since it carries credentials issued for your deployment.
Private Docker image. Requires Traversal-issued pull credentials.
Public OCI artifact. Generic across all deployments.
Per-deployment. Pulled with a Traversal-issued token.
## Container image
Traversal builds and publishes the Traversal Processor image to the [traversalext/traversal-processor repository on Docker Hub](https://hub.docker.com/r/traversalext/traversal-processor). The image is private, so you will have to authenticate with Docker Hub.
The processor image is a multi-platform image with support for both x86 and ARM (`linux/amd64` and `linux/arm64`).
If your security policy requires it, you can mirror the published image into an internal registry and point the chart at it by overriding `image.repository` in your values. If the mirror requires authentication, attach the pull credentials to the deployment's service account (`imagePullSecrets`) in your cluster.
## Helm chart
The Helm chart is published as a public OCI artifact at `oci://registry-1.docker.io/traversalext/traversal-processor-charts`. The chart is generic across all deployments — it contains no customer-specific configuration — so you can inspect or vendor it freely with standard Helm tooling:
```bash theme={null}
helm pull oci://registry-1.docker.io/traversalext/traversal-processor-charts \
--version --untar
```
The pulled chart includes a `values.yaml` template that documents every supported field and indicates which ones are required.
See the [chart changelog](/setup/processor-changelog) for what's new in each version and any upgrade notes.
### Required values
The chart and Traversal Processor together require the following fields, all under the `traversalProcessor` key. Traversal pre-populates all of them in the values file it builds for you, so you don't typically need to set them manually:
| Field | What it is |
| :---------------------------- | :--------------------------------------------------------------------------------------------- |
| `account.id` | Your account UUID, used to associate ingested logs with your account. |
| `traversalIngestion.endpoint` | The Traversal ingest endpoint the processor streams processed log clusters to. |
| `mtls.certBase64` | Base64-encoded PEM client certificate the processor presents when authenticating to Traversal. |
| `mtls.keyBase64` | Base64-encoded PEM private key for the client certificate above. |
The mTLS material can be supplied three ways — provide exactly one:
* **Inline base64:** `mtls.certBase64` and `mtls.keyBase64` (the chart renders a Kubernetes Secret).
* **Existing Secret:** `mtls.existingSecret`, the name of a Secret you manage with keys `cert.pem` and `key.pem` (for example, one produced by cert-manager).
* **Files:** `mtls.certFile` and `mtls.keyFile`, PEM paths already present in the container (mounted via `extraVolumes`/`extraVolumeMounts`, a CSI driver, an init container, etc.).
The [telemetry forwarder](#telemetry-forwarding) reuses this identity, whichever form you choose.
All other fields — sizing, field extraction, redaction, replica count, TLS, scheduling — are optional and have sensible defaults.
### Sizing and throughput
A single value, `traversalProcessor.size`, selects a validated resource-and-tuning preset — pick the tier that matches your log volume and the chart configures CPU, memory, and internal ingest tuning for you.
| Size | Resources (per pod) | Recommended node | Guideline throughput (per pod) |
| :------- | :------------------ | :---------------- | :------------------------------ |
| `normal` | \~3 vCPU / 4 GiB | AWS `c8g.xlarge` | \~40,000 records/s · \~150 MB/s |
| `large` | \~7 vCPU / 8 GiB | AWS `c8g.2xlarge` | \~80,000 records/s · \~300 MB/s |
**Scale up** by moving `normal` → `large`; **scale out** by raising `replicaCount` (the processor is horizontally scalable with no cross-instance coordination). If your cluster mixes node families, pin pods to the recommended instance type with `nodeSelector`.
Throughput figures are **guidance, not guarantees**. They were measured with roughly 4 KB average log records, \~40% cluster compression, and redaction disabled. Actual throughput depends on record size, field cardinality, redaction, and downstream latency — size for headroom and validate against your own traffic.
### Telemetry forwarding
The processor reports its own operational telemetry — metrics, traces, and logs — to Traversal over OTLP. It is how Traversal supports your deployment, and is separate from the log data you send the processor to ingest.
By default the chart adds a second container to the processor pod: a forwarder that buffers this telemetry and retries delivery, so a brief interruption reaching Traversal does not lose it. It authenticates with the processor's `mtls` identity and needs no credentials of its own.
| Field | Default | What it is |
| :------------------------------------- | :------------------------ | :---------------------------------------------------------------------------------------------------------- |
| `observability.otel.sidecar.enabled` | `true` | Runs the forwarder. Set to `false` to export directly from the processor, leaving one container in the pod. |
| `observability.otel.sidecar.image` | `traversalext/alloy` | Forwarder image. Override `repository` to pull from an internal mirror. |
| `observability.otel.sidecar.resources` | 50m CPU, 128–256Mi memory | Resource requests and limits for the forwarder container. |
| `observability.otel.forwardProxy` | *none* | Forward proxy for telemetry egress. Set this where your network reaches Traversal only through a proxy. |
The `observability.otel` endpoints default to Traversal's telemetry ingest. Override all three to send to a collector you operate instead.
### Disabling telemetry
Setting `observability.otel.enabled: false` stops all telemetry export and removes the forwarder.
Disabling telemetry is highly discouraged. Telemetry is Traversal's only view into a processor running in your network, so without it Traversal can provide only limited support.
## Values file
The values file holds your deployment-specific configuration: your account details, Traversal API endpoint, mTLS material, and any deployment-specific tuning or telemetry destinations.
Because it may embed private key material, the values file is not published publicly. Instead, Traversal builds it for you and packages it as an OCI artifact in a private Docker Hub namespace, alongside the chart.
Through a secure channel, Traversal shares with you a Docker Hub access token, scoped read-only to your deployment's namespace, and the artifact reference for your values file (for example, `registry-1.docker.io/traversalext/traversal-processor-charts-:`).
The Docker Hub username is always `traversalext`. Only the token is per-customer.
The values file is published as an OCI artifact, which Helm doesn't natively pull. Install the [ORAS CLI](https://oras.land):
```bash theme={null}
brew install oras
```
See the [ORAS installation guide](https://oras.land/docs/installation) for non-macOS platforms.
Set the values shared by Traversal, then pull the artifact:
```bash theme={null}
export DOCKERHUB_REPO_NAME="" # provided by Traversal
export DOCKERHUB_REPO_TOKEN="" # provided by Traversal
export VALUES_FILE_VERSION="v0.1.0"
oras pull \
--username traversalext --password "$DOCKERHUB_REPO_TOKEN" \
"registry-1.docker.io/traversalext/$DOCKERHUB_REPO_NAME:$VALUES_FILE_VERSION"
```
This writes the values file into your current directory. Pass `-o ` to ORAS to place it elsewhere.
## Installing
With the chart and values file in hand, install in whatever way fits your environment — `helm install` directly, a GitOps pipeline (ArgoCD, Flux) referencing the OCI chart, or an internal Helm registry mirror. As a minimal end-to-end example using Helm directly:
```bash theme={null}
helm install traversal-processor \
oci://registry-1.docker.io/traversalext/traversal-processor-charts \
--version \
-f -values.yaml
```
The pod exposes a health endpoint at `/health` on its dedicated probe port (`healthPort`, default `3001`); its readiness probe gates traffic until the service is ready.
## Log ingestion from Amazon SQS
Instead of sending logs to the processor's HTTP or OTLP endpoints, you can have the processor pull them from an Amazon SQS queue. An optional sidecar container runs in the same pod, polls the queue, and hands each batch to the processor over the pod's loopback interface — so no inbound network path to the processor is needed, and there is no separate forwarder service to deploy.
This is opt-in and off by default:
```yaml theme={null}
traversalProcessor:
sqsAdapter:
enabled: true
queueUrl: "https://sqs.us-west-2.amazonaws.com/123456789012/my-log-queue"
region: "us-west-2"
```
Every replica polls the same queue and SQS divides messages between them, so raising `replicaCount` scales ingest with no further configuration.
The sidecar ships as its own image, `traversalext/traversal-processor-sqs-adapter`, kept separate from the processor image so deployments that don't use SQS carry no AWS dependencies. Like the processor image it is private, so the pull credentials for your deployment need to cover both.
### Message format
Message bodies must be NDJSON — one JSON object per line. A single message can carry many lines and all of them are read.
If your pipeline wraps the payload instead of sending NDJSON directly, set `ndjsonPayloadPath` to a `|`-delimited path to the string holding it. For example, `content|lines_str` reads the NDJSON from `{"content": {"lines_str": "..."}}`.
To control how each record's index and timestamp are derived, set `indexStrategy` and `timestampStrategy`. They accept the same strategies as the JSON ingest API; leave them empty to use the processor's defaults.
### AWS credentials
The sidecar needs permission to call `sqs:ReceiveMessage`, `sqs:DeleteMessage`, `sqs:GetQueueAttributes` and `sqs:ChangeMessageVisibility` on the queue. Grant it by exactly one of three routes:
| Route | When to use it | What to set |
| :------------------------------------ | :--------------------------------------------------------------- | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| EKS Pod Identity | Recommended on EKS, and what newer clusters use. | `serviceAccount.create: true` and `serviceAccount.name`, then create a pod identity association binding that ServiceAccount to your role. Leave `serviceAccount.roleArn` unset. |
| IAM roles for service accounts (IRSA) | Clusters with an OIDC provider and no Pod Identity agent. | `serviceAccount.create: true` and `serviceAccount.roleArn`. |
| Kubernetes Secret | Anywhere outside EKS, where neither role mechanism is available. | `sqsAdapter.existingCredentialsSecret`, naming a Secret that holds `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` (and optionally `AWS_SESSION_TOKEN`). |
To tell which mechanism your cluster uses, look for a running `eks-pod-identity-agent` DaemonSet in `kube-system` — if it's there, you have Pod Identity.
### Networking
**Check how SQS traffic leaves your VPC before you enable this.** Without an SQS interface endpoint, calls from a private subnet leave through your NAT gateway, which bills data processing per GB in both directions.
If you reach SQS through a VPC or PrivateLink endpoint whose private DNS is **not** enabled, set `endpointUrl` to that endpoint. Leave it empty for standard SQS, which is resolved from `region`.
## Redaction
The processor supports a regex-based redaction pipeline that rewrites sensitive text in log fields before data is sent to Traversal. Redaction is opt-in, disabled by default.
Set `redaction.enabled` in your Helm values and provide a rules file using one of two options:
```yaml theme={null}
# Option A: inline rules — the chart creates and manages the ConfigMap
redaction:
enabled: true
rulesContent: |
[[rules]]
name = "email"
type = "regex-structured-data"
pattern = '[a-zA-Z0-9._%+\-]+@[a-zA-Z0-9.\-]+\.[a-zA-Z]{2,6}'
# Option B: reference a ConfigMap you manage externally
redaction:
enabled: true
existingConfigMap: my-redaction-rules
```
When using `existingConfigMap`, the key inside that resource must be named `redaction-rules.toml`.
See [Redaction](/setup/redaction) for the rules file format and field filtering options.
## TLS for the ingestion endpoint
By default the processor serves plain HTTP on port 3000. If your environment requires end-to-end encryption — for example when a load balancer or service mesh terminates TLS and re-encrypts to the backend — you can enable TLS so the processor serves HTTPS directly.
TLS is configured through the `tls` block in your Helm values. Provide exactly one of two credential sources:
**PEM files** — provide paths to certificate and key files already mounted in the container (for example via a Kubernetes Secret volume):
```yaml theme={null}
traversalProcessor:
tls:
enabled: true
certFile: "/etc/tls/tls.crt"
keyFile: "/etc/tls/tls.key"
```
**Inline base64** — provide base64-encoded PEM content directly; the chart creates a Kubernetes Secret from it and mounts it read-only into the container, so you don't manage any files yourself:
```yaml theme={null}
traversalProcessor:
tls:
enabled: true
certB64: ""
keyB64: ""
```
The chart validates that exactly one credential source is provided — setting both file-based and inline values, or only one half of a pair, will produce a clear error at install time.
Enabling TLS affects only the main ingest port. Health and metrics stay on the separate probe port (`healthPort`), which always serves plain HTTP on its own runtime — so the Kubernetes liveness and readiness probes keep working unchanged whether or not ingest TLS is enabled.
## Want to learn more?
Rules file format, field filtering, caching behaviour, and mount examples.
Pull OCI artifacts from any registry.
Helm's native OCI support for chart distribution.
# Processor Helm chart changelog
Source: https://docs.traversal.com/setup/processor-changelog
What's new in each traversal-processor-charts release, and any migration notes for upgrading.
The `traversal-processor-charts` Helm chart is versioned independently from the Traversal Processor container image (see [Container image](/setup/processor#container-image)).
**The processor can now ingest logs from an Amazon SQS queue.** This functionality does not require a separate forwarder service. It is off by default — existing deployments are unaffected until you enable it. See [Log ingestion from Amazon SQS](/setup/processor#log-ingestion-from-amazon-sqs) for configuration details.
**Memory usage is more predictable if output latency spikes.** A slow or briefly unavailable output endpoint now applies backpressure instead of growing the processor's memory. No values file changes are required.
This chart defaults to processor image `0.5.0`.
**More ways to supply mTLS material.** In addition to inline base64, you can now point the processor at an existing Kubernetes Secret with `mtls.existingSecret`, or at PEM file paths already mounted in the container. Use whichever fits how your cluster manages certificates.
**Self-reported metrics can egress through a forward proxy.** If outbound traffic from your cluster must pass through a proxy, the processor's own operational metrics can now be routed via that path.
**Memory stability increased.** Internal ingest memory budgets are now effectively adjusted based on the `size` preset to make memory usage more predictable.
**The chart's values layout was reorganized.** Kubernetes deployment settings (replica count, image, resources, service, probes, scheduling, pod disruption budget) now sit at the top level of the values file, while processor-specific settings stay under the `traversalProcessor` key. A few settings were also renamed, consolidated, or removed in the cleanup — for example, log ingestion is now configured through a single `traversalProcessor.traversalIngestion.endpoint`.
Traversal builds and pre-populates your values file, so most deployments need no action. If you maintain your own overrides, re-check them against the updated `values.yaml` (`helm pull … --untar`) or the [setup guide](/setup/processor) — some field names and locations have moved.
Client certificate material is now mounted from a Kubernetes Secret as a read-only file instead of being injected as a literal pod environment variable. No values file changes are required to pick up this fix. The same `mtls.certBase64`/`keyBase64` fields you already set are now delivered more securely under the hood.
All customers should migrate to `processorIngestion.endpoint` going forward. It authenticates with the same `mtls` certificate as everything else.
**Metrics reporting is now on by default.** The processor reports its own operational metrics to Traversal automatically; no configuration is required.
# Redaction
Source: https://docs.traversal.com/setup/redaction
Scrub sensitive information from your data before it leaves your environment. Configurable and customizable.
Both the Traversal Connector and the Traversal Processor support redaction: a regex-based pipeline that rewrites sensitive text in your data before it is forwarded to Traversal. Rules are defined in a TOML file you author.
## How it works
When a rules file is configured, the payload of each request is scanned against your rules in order. Each rule is a named regex pattern with a replacement string. Rules are applied sequentially. The output of one rule becomes the input to the next. Thus, ordering matters when patterns could overlap.
Redaction which operates on JSON payloads replaces patterns in JSON keys *and* values while preserving the structure of the payload.
## Rules file format
```toml theme={null}
version = "v1"
# Optional. Fallback replacement for rules that omit their own.
# Defaults to [REDACTED] if not set.
default_replacement = "[REDACTED]"
[[rules]]
name = "email"
type = "regex-structured-data"
pattern = '[a-zA-Z0-9._%+\-]+@[a-zA-Z0-9.\-]+\.[a-zA-Z]{2,6}'
# Uses default_replacement
[[rules]]
name = "ssn"
type = "regex-structured-data"
pattern = '\b\d{3}-\d{2}-\d{4}\b'
replacement = "[SSN]"
```
### Fields
| Field | Required | Description |
| :-------------------- | :------- | :--------------------------------------------------------------------------------------------------------------------------- |
| `version` | Yes | Schema version. Use `"v1"`. |
| `default_replacement` | No | Fallback replacement for rules that omit `replacement`. Defaults to `[REDACTED]`. |
| `rules[].name` | Yes | Human-readable label for the rule. Appears in logs and metrics. |
| `rules[].type` | Yes | `"regex-structured-data"` for the Processor (walks JSON fields). Rules with an unrecognised type are skipped with a warning. |
| `rules[].pattern` | Yes | Regex pattern. All matches in the string value are replaced. |
| `rules[].replacement` | No | Text substituted for each match. Falls back to `default_replacement`. |
The regex engine does not support lookaheads, lookbehinds, or backtracking. Patterns using those features will cause startup to fail with a parse error.
### Field filtering
The redaction engine supports two optional per-rule fields that restrict which fields a rule applies to:
| Field | Description |
| :---------------------- | :--------------------------------------------------------------------------------------------------- |
| `rules[].redact_fields` | Allowlist of field names this rule applies to. When set, the rule only fires on fields in this list. |
| `rules[].skip_fields` | Blocklist of field names this rule skips. When set, the rule never fires on fields in this list. |
These rules can be combined on the same rule in tandem. When `skip_fields` and `redact_fields` are set, both must pass for the rule to fire on a given field. Rules without either filter apply to all fields.
```toml theme={null}
[[rules]]
name = "card-number"
type = "regex-structured-data"
pattern = '\b\d{16}\b'
replacement = "[CARD]"
redact_fields = ["message", "body"] # only apply to these fields
```
# Alert Workers
Source: https://docs.traversal.com/using-traversal/alert-workers
A proactive AI SRE for your alert channels — a verdict on every alert, the reasoning behind it, and the noise cut down.
Alert channels rarely get the attention they need. Engineers either tune them out and miss the warning that mattered, or spend hours keeping up and still miss things. Almost nobody finds time to prune the noisy ones, so the channel only gets louder.
An **Alert Worker** triages and prioritizes every single alert so that you don't have to, and escalates the ones you should actually focus on.
Alert Workers are in **public beta**. Contact your Traversal representative to gain access.
## What happens when an alert fires
Every time an alert fires, the Worker looks at it, investigates, and replies in that alert's thread with what it found and what it recommends. It also reacts to the alert with its assessment, so you can scan the channel by urgency and see which alerts are a critical issue, which are technical debt worth addressing at some point, and which are just a noisy alert that needs its definition fixed.
| Verdict | Meaning |
| :--------------------- | :--------------------------------------------------------------------------------------------------------------------- |
| **Address Now** | Active impact on users or the product right now. On-call should act, route it to the owning team, or escalate. |
| **Address Soon** | A real problem with no user-facing impact yet — tech debt, resource pressure, an internal issue. |
| **Alert Needs Update** | The system is healthy and the alert is at fault. Where the data supports it, the Worker names the specific tuning fix. |
Reply in the thread to push back or ask for more detail. Nothing rests on how often an alert fires — a verdict comes from your current telemetry, prior investigations, linked tickets, and whatever your team already said in earlier threads.
Alert channels run at high volume, so a Worker is built to be token-efficient: when a fire clearly repeats something it has already worked out, it says so and links to that earlier work rather than investigating twice.
## The scratchpad
The [scratchpad](/using-traversal/worker-features#scratchpad) is the Worker's up-to-date synthesis of the whole channel, written for the engineer who just came on call: what needs action now, what's recurring and why, the known causes and prior fixes, and where the alerts themselves are worth improving. Open the Worker from the [Workers console](/using-traversal/workers-console) to read it.
Use [private chat](/using-traversal/worker-features#private-chat) for the follow-ups — *why did this fire, is it getting more frequent, did other alerts share the same cause, how should we update this definition* — without adding channel noise.
## Steer it in plain English
Set policy with [custom instructions](/using-traversal/worker-features#custom-instructions) rather than regex or a rules file:
```text theme={null}
Only escalate Sev1 and Sev2 alerts. Ignore staging unless it affects a customer test.
Always run a fresh investigation for payments-api, and tag the payments on-call
when an alert affects checkout.
```
[Schedules](/using-traversal/worker-features#schedules) cover shift handoffs and alert-health reviews. Because the Worker has been in the channel the whole time, a handoff is what actually happened and what's still open — not a digest of subject lines.
## FAQ
Noise filters mostly count how often an alert fires. An Alert Worker investigates *why* it fired and grounds its verdict in real evidence — your telemetry, past investigations, linked tickets, and what your team already said about it in the thread. Most alerts aren't meaningless noise; they're real problems, unknowns, or misconfigured rules, and the Worker tells them apart.
No. An Alert Worker triages and explains — it posts its reasoning and suggests fixes. It never changes your alerting, paging, or routing. You stay in control of what happens next.
Every response goes in its alert's own thread, never the main channel, so the channel reads exactly as it did before — just with a verdict on each alert. When one monitor double-posts the same fire, the Worker responds once in full and links the duplicates to it.
Yes. Reply with `@Traversal` in the thread to add context and steer the analysis. The Worker will re-investigate when what you've told it changes the picture.
# Tips
Source: https://docs.traversal.com/using-traversal/best-practices
Get the most out of Traversal by anchoring investigations in time, asking specific questions, and sharing your team's institutional knowledge.
Traversal is designed to do the exhaustive, machine-scale work of investigation so you can focus on judgment. The more you ask good questions, provide clear time bounds, give feedback, and encode what your team already knows — the more Traversal becomes a true extension of your best engineers.
## Anchor investigations in time
Time is the single most important input for accurate investigations.
Use the moment symptoms were **first observed** as your investigation anchor — not when the alert fired or when the ticket was opened.
Traversal looks primarily backwards from the time you provide. If you set it too early, too late, or omit it entirely, key context may be missed and results may be vague or low confidence.
**Good:** "Investigate checkout failures starting 09:12 UTC."
**Risky:** "Why is checkout broken?" — no time context provided.
When in doubt, the incident ticket start time is a reasonable proxy.
If your results seem vague or the confidence is unexpectedly low, incorrect or missing time context is the most common cause.
## Ask questions naturally — but be specific
If you don't know where to start, enter your question in the **Investigations** search bar. You don't need to choose the right reasoning mode — Traversal determines that automatically.
Ask naturally and use follow-up questions to refine, drill down, or challenge hypotheses.
**Common use cases:**
* Incident investigation
* Alert context
* Service health checks
* Dependency exploration
* Onboarding into unfamiliar systems
## Include specific identifiers when you have them
The more specific your query, the more Traversal can focus its search. Generic descriptions force the agent to cast a wide net; specific identifiers let it go deep immediately.
Useful identifiers to include:
* **Trace or request IDs** — correlate across services instantly
* **Service or application names** — scope the investigation to the right part of your stack
* **Error messages or codes** — match logs and alerts precisely
* **Deployment SHAs or version numbers** — tie symptoms to a specific change
* **UUIDs from tickets or alerts** — link directly to the triggering event
**Good:** "Investigate latency spikes in `payments-api` starting 14:32 UTC. Trace ID: abc123, error: `connection pool exhausted`."
**Less specific:** "Something is slow in payments." — Traversal will still help, but it has to work harder to find the right signals.
## Know when to start a new investigation
* You want more detail on an existing root cause analysis
* You're exploring evidence or timelines
* You're validating or disproving a hypothesis
* The incident time changes
* You switch to a different alert or service
* New information fundamentally reframes the problem
## Share tribal knowledge with Traversal
Traversal learns autonomously, but it becomes more accurate when guided by your team's institutional knowledge.
Use **Knowledge Bank** to encode:
* Runbooks
* Preferred debugging paths
* Business-critical services
* Known failure modes
Give feedback when an investigation misses practical nuance. Traversal is strongest when paired with what your team already knows.
## Use Traversal for learning and onboarding
Traversal isn't only for active incidents. Use it to build team-wide system understanding.
Review historical incidents to understand patterns and common failure modes across your stack.
Query Traversal about a service's history to surface recurring issues before they become incidents.
Let new team members ask questions freely to ramp up on unfamiliar systems without requiring senior engineer time.
Over time, Traversal becomes a shared source of system understanding — not just an incident response tool.
# Customization
Source: https://docs.traversal.com/using-traversal/customization
Give Traversal standing instructions for your investigations, a channel, or a Worker.
Traversal works out of the box — you don't need to configure anything to get accurate investigations. Customization is the last-mile layer on top: when you have opinions about how Traversal should investigate or write things up, you set them once and Traversal applies them to every investigation, instead of re-typing them each time.
There are three ways to customize:
* [Personalization](https://app.traversal.com/settings/personalization) — your own instructions, applied to every investigation you run: Slack (channels and DMs) and the web app.
* [Channel custom instructions](https://app.traversal.com/settings/channel-management) — instructions applied to every investigation in a Slack channel, no matter who runs it.
* [Worker custom instructions](/using-traversal/worker-features#custom-instructions) — instructions applied to one Worker, or to future Workers created by an auto-deploy rule.
None can override Traversal's security guardrails or its evidence and citation rules.
| Layer | Applies to | Set it in |
| :-------------------------- | :----------------------------------------------------------- | :------------------------------------------------------------------------------------- |
| Personalization | Every investigation you run, in Slack and the web app | [Settings → Personalization](https://app.traversal.com/settings/personalization) |
| Channel custom instructions | Every investigation in the channel, from anyone | [Settings → Channel management](https://app.traversal.com/settings/channel-management) |
| Worker custom instructions | One Worker, or future Workers that match an auto-deploy rule | **Workers → Edit Worker** or **Auto-deploy settings** |
## What to include
Anything you'd otherwise find yourself repeating — the preferences and hard-won context that apply to you or your team:
* **Investigation steps** — where to look first, or what to rule out. *"Always check the deploys dashboard and recent releases before anything else."*
* **Known failure modes** — the usual suspects for your systems. *"Latency spikes on checkout are usually the payments provider rate-limiting at peak — check that first."*
* **Runbooks and references** — point Traversal at the docs and dashboards you trust. *"For database incidents, follow our failover runbook: ``."*
* **Response format** — how you want findings written up. *"Lead with customer impact, then the root cause with evidence. Keep it to a few bullets."*
* **Priorities** — what matters most. *"Prioritize customer-facing services; always call out anything touching checkout."*
Keep them specific and durable — standing preferences, not one-off asks.
## Personalization
Set your instructions in [Settings → Personalization](https://app.traversal.com/settings/personalization), under **Instructions for Traversal**. They follow you across every investigation you start, in Slack or the web app.
## Channel custom instructions
Set a channel's instructions in [Settings → Channel management](https://app.traversal.com/settings/channel-management), under **Custom instructions**. They apply to every investigation in that channel — from anyone, including auto-triggered ones — so they're ideal for alert channels, auto-trigger channels, or a shared debugging channel where the whole team wants consistent output.
## Worker custom instructions
Set instructions when you deploy a Worker, from **Edit Worker** on its detailed view, or on an auto-deploy rule so every future matching channel inherits them.
Worker instructions are the only layer that can shape *when* a Worker acts, not just how it investigates — which alerts to ignore, which to always investigate fresh, when to escalate, who to notify. They also cover where to look first, what to prioritize, and how to format an output such as a post-mortem. See [Deploy and manage Workers](/using-traversal/worker-features#custom-instructions) for examples.
## How they work together
Personalization and channel instructions shape **investigations**: in a Slack channel both apply, and the channel's take priority if they conflict; in a DM or the web app, only your personal instructions apply.
Worker instructions shape the **Worker** — its judgment about when to engage and what to say. If you have a Worker in a channel, that's the layer to reach for when you want to change its behavior.
None of these layers can override Traversal's security guardrails or its evidence and citation rules.
# Incident Workers
Source: https://docs.traversal.com/using-traversal/incident-workers
A proactive AI SRE that works your incident channel from the first signal through verified recovery.
An **Incident Worker** joins your incident channel the moment it opens, starts investigating, and stays with the team until recovery is verified. It follows the whole conversation, adapts as evidence changes, and decides for itself when to speak and when to stay out of the way.
Because it starts before anyone has to ask, responders walk into a channel where the first round of investigation is often already done. This is the default incident experience.
## What it does
An Incident Worker isn't running a checklist. It reads what the incident needs and works on that, the way an engineer on the call would — and you can always just ask it for something instead, in the channel or in private chat.
Most often that means:
* **Investigating** — the channel context plus the relevant telemetry, code, deployments, tickets, and knowledge.
* **Assessing impact** — which services, users, and regions are affected, kept current as things move.
* **Testing theories** — chasing new clues, ruling out weak paths, and correcting its own earlier conclusions when the evidence turns.
* **Weighing in on mitigation** — showing the evidence behind a proposed action, including when the action on the table would make things worse.
* **Verifying recovery** — confirming against live signals that the incident is really over, not just that a fix shipped.
* **Drafting the post-mortem** — turning the live record into a cited summary, timeline, root-cause analysis, and prevention recommendations.
Which of these it does, and in what order, depends on the incident. A Worker that arrives to find the root cause already known spends its time on impact and verification instead.
## Working with the team
A Worker behaves the way a good teammate would. Messages, threads, reactions, and — where configured — live incident-call transcripts all become context as they arrive. It grounds every substantive conclusion in evidence with citations you can check, threads its findings so the main channel stays for updates the whole room needs, and errs toward silence rather than reacting to chatter. Mention Traversal or reply in a thread whenever you want a deeper investigation, a clarification, or an update.
When you're pulled in late, its [scratchpad](/using-traversal/worker-features#scratchpad) is the fastest way back up to speed — an up-to-date synthesis of impact, conclusions, mitigation progress, remaining risk, and the evidence behind each, so you can skip the scrollback. Follow up in [private chat](/using-traversal/worker-features#private-chat) without any of it reaching the response channel.
[Schedules](/using-traversal/worker-features#schedules) are worth reaching for during an incident — *"recheck the error rate in 15 minutes and post only if recovery stalls."* The Worker comes back on its own, so nobody has to remember to look again once attention moves on.
## When the incident resolves
A Worker stays in sync the whole way through, so it knows when it's over — and it verifies that against live telemetry rather than taking a deployed fix as proof. Once the room reaches consensus it posts a draft [post-mortem](/using-traversal/post-mortems), reconciling its own investigation with everything the team said and did, and says plainly why it's posting.
It also assesses its own work — what it got right, where it was slow or wrong — and recommends ways to improve your alerting so the next incident like this is caught earlier, or avoided altogether.
See [Post-mortems](/using-traversal/post-mortems) for more detail.
## Staying in control
A Worker investigates, explains, and recommends. It never changes your alerting, paging, or routing, and it does not take destructive action. You can [stop proactive mode](/using-traversal/worker-features#stop-a-worker) at any time — you can still mention `@Traversal` with a question afterwards — or give it [custom instructions](/using-traversal/worker-features#custom-instructions) telling it where to look first, what to prioritize, or who to tag.
## Deploy an Incident Worker
Most teams use an [auto-deploy rule](/using-traversal/workers-console#auto-deploy-to-future-channels): point it at your incident channel convention — names starting with `inci-`, `sev-`, or `incident-` — and every future matching channel gets a Worker automatically. You can also [scan your workspace](/using-traversal/workers-console#scan-your-workspace) for recommendations, or [deploy to a specific channel](/using-traversal/workers-console#deploy-to-a-channel).
# Investigations
Source: https://docs.traversal.com/using-traversal/investigations
Start an investigation, ask about an incident, or explore your system using natural language.
The Investigations page is where you interact with Traversal's AI SRE. You can spin up an investigation, ask about an active incident, or explore system behavior — all in natural language.
## Starting an investigation
Type your question or paste an incident description directly into the input. Traversal parses the text for entities (teams, services) and extracts the date and time if present.
Type `@` to reference teams, services, applications, environments, or other scoped entities and add context quickly.
### Pasting incident descriptions
You can paste a snippet from a ticket, Slack thread, or alert directly. Traversal will extract the relevant entities and time window automatically.
**Time formats that parse well:**
* `Oct 23 20:00 UTC`
* `10/23/2024 at 20:00 UTC`
* `Oct 23 4pm ET 2024`
* `5pm EST yesterday`
When running in deep mode, Traversal requires a start date and time with a timezone. If you omit the time window, deep mode will not run.
### Writing effective descriptions
Better descriptions produce better results. Include:
* When the issue started
* Error messages you're seeing
* Affected teams or services
* Links to relevant alerts or runbooks
## What Traversal returns
Each investigation returns a root cause analysis (RCA) with the following:
A summary statement with a confidence level and supporting evidence.
Clickable citations including metrics anomalies, log lines, deployments, code diffs, and topology relationships.
A sequence showing when symptoms began, anomalies were detected, deployments occurred, and dependent services started failing.
Diagnostic suggestions, remediation steps, runbook-style guidance, and suggested team escalations.
### Confidence levels
| Level | Meaning |
| ---------- | ----------------------------------------------------------------------------- |
| **High** | Strong correlation patterns with consistent evidence across multiple signals. |
| **Medium** | Theory is plausible and likely, but missing full visibility or traces. |
| **Low** | Weak or partial signals. Requires human confirmation. |
Traversal's recommended next steps are suggestions, not actions. Traversal will not make changes to your system without your permission.
## Asking follow-up questions
You can continue an investigation by asking follow-up questions in the web app or in the Slack thread for that investigation. Examples:
* Drill into a specific symptom
* Slice the timeline to a narrower window
* Request more detail on a particular signal
In Slack, reply with `@Traversal` and additional context, for example: `@Traversal focus on payments-api in prod-us-east after 12:05 UTC`.
To change the analysis time window, start a new investigation with the revised time range rather than asking as a follow-up.
# Knowledge Bank
Source: https://docs.traversal.com/using-traversal/knowledge-bank
See everything Traversal has learned about your systems — and add the context that only lives in your team's heads.
The best engineers on your team carry an enormous amount of context that isn't written down anywhere: which services are business-critical, which alerts are usually noise, what broke last time and why. Most AI tools ask you to hand that over before they can help — write the runbooks, document the exceptions, keep it all current. That gets it backwards. The agent should do the learning.
Traversal builds its own understanding of your environment from the systems you've already connected, and **Knowledge Bank is where you can see it**. Browse what Traversal figured out on its own, correct anything that looks wrong, and add the last mile that only your team can know.
## Traversal learns your systems on its own
You do not have to write anything for Traversal to work. From the moment your integrations are connected, Traversal reads them and writes itself a **wiki of your environment**, assembled from the same sources it uses during an investigation:
* **Code** — your repositories in GitHub or GitLab, including how services are wired together and what they actually log
* **Metrics** — Prometheus, Mimir, Datadog, Dynatrace, AppDynamics, VictoriaMetrics
* **Logs and traces** — OpenSearch, Elasticsearch, Splunk, Loki, Honeycomb
* **Alerts and incidents** — Alertmanager, Datadog monitors, incident.io
* **Documentation and tickets** — Confluence, Notion, Jira, ServiceNow
The wiki falls out of building Traversal's **Production World Model**. As Traversal takes in those sources it works out how everything relates — this service to the metrics that describe it, the code that implements it, the alerts that fire on it, the traces that run through it, the tickets filed against it. Those relationships are what make the wiki possible: each entry is written from what the model already understands about how your environment fits together, not from any one system read in isolation. And because your systems don't hold still, Traversal keeps regenerating it as they change, so its picture of production doesn't quietly go stale.
**Connecting your integrations is the whole setup.** Everything in the Auto-created section was written without a single line of input from your team.
## What's in Knowledge Bank
| Section | What's in it | What you can do |
| :------------------------- | :------------------------------------------------------------------------------- | :-------------------------------------------------- |
| **Skills** | Reusable procedures that shape the steps Traversal takes | Create, upload, organize, download, and delete |
| **Uploaded documentation** | Docs, runbooks, and context files your team adds, expanding what Traversal knows | Create, upload, organize, download, and delete |
| **Memories** | What Traversal has learned working alongside your team | Review, approve, and reject what Traversal proposes |
| **Auto-created** | Traversal's own documentation of your environment, refreshed as it changes | Browse and comment on |
Everything your team authors is live immediately. Nothing needs approval, and there's no build step between adding a runbook and Traversal being able to use it.
## Auto-created: what Traversal figured out on its own
Open the **Auto-created** tab and you're reading Traversal's own working documentation of your environment, organized by service and written entirely from the evidence in your connected systems. A page might cover what a service does and who owns it, how it's put together, what it talks to, what it logs, or how it has broken before — the shape keeps evolving as Traversal gets better at describing your stack.
That makes it useful well beyond the agent. It's clear, current documentation your whole team can read: someone joining the team can come up to speed on an unfamiliar service without hunting down the one person who remembers, and anyone can answer "how does this actually behave in production?" by reading rather than guessing.
### Sharpen it with comments
Traversal wrote all of this without a word of input from your team — which also means it's working from evidence alone. Where you know something the data doesn't show, or where it drew a conclusion that isn't quite right, leave a **comment** and Traversal folds that nuance in as it maintains the page.
* Open a file and switch to the **Comments** tab to weigh in on the page as a whole.
* Or select any passage and choose **Quote in comment** to attach your note to that exact line.
Your note is visible to your team in context immediately, and Traversal carries it forward as it refreshes — so the improvement compounds instead of being re-derived away on the next pass. These pages stay Traversal's to maintain, which is why you tune them with notes rather than editing them by hand.
## Skills: shape the steps Traversal takes
A **skill** is how your team's own way of working gets into an investigation — the steps you take and the order you take them in. It's where your preferences and rituals live: how you work a stuck worker pool, what you always pull first when the payments service alerts, which check settles the argument. Traversal reaches for the right skill when the situation calls for it.
Open **Add → Add skill**.
**Upload** brings in a skill file you already have. **Create new** lets you write one here.
Give it a name, a one-line description of what it does, and the instructions themselves. Optionally drop it in a folder.
Traversal saves it as a `SKILL.md` file and can use it on the next investigation.
Write the description for the *trigger*, not the content — it's what Traversal reads when deciding whether a skill applies. "When the payments service alerts" beats "payments runbook."
## Uploaded documentation: give Traversal more to work with
Skills shape *how* Traversal works. Documentation expands *what it knows*. Use it for the information your systems don't carry: the business constraint, the exception nobody wrote down, the workaround that only works on Tuesdays.
Open **Add → Add documentation**.
Upload files, import from a URL, or write a document directly in Traversal.
Drop it into an existing folder, or create one as you go.
Uploads accept Markdown, text, CSV, TSV, JSON, and YAML, up to 10 MB per file. You can also drag files straight onto the Skills or documentation list.
Make each document specific enough to act on. "During a payment-provider incident, check the provider's status page before restarting checkout" is useful. "Payments is important" is not.
## Memories: Traversal getting better as it works
Traversal learns from working with you. When something durable comes up mid-investigation — you correct it, or tell it to remember something — it drafts a memory and asks before keeping it. The proposal appears in the session. If the investigation started in Slack, a review card is also posted in the same thread so you can approve or reject it without leaving Slack.
**Approve** files it into Memories, where it's available to future investigations. **Reject** discards it. **Download** gets you the file first if you'd rather read it in full. Traversal never applies its own proposal on its own — a person decides every time.
Approved memories keep a link back to the investigation they came from, so you can trace how Traversal's understanding of your systems has grown and where each piece of it came from.
Memories are created during investigations — they can't be added by hand. To capture something on the spot, just tell Traversal to remember it in a session.
## FAQ
No — and that's the point. Traversal learns your environment from the integrations you've connected and investigates without any Knowledge Bank setup at all. What you add here is last-mile refinement on a foundation that's already there.
Something that regularly changes how your team responds: a runbook you follow often, an alert everyone knows to ignore, a service whose downtime is a business emergency. Start there rather than trying to document everything.
Comment on it. Traversal regenerates these pages as your systems change, so a note is what carries forward — Traversal weighs it against everything else it knows and folds it into the next refresh, where a hand edit would just be overwritten.
Skills, documentation, and approved memories are available to the next investigation. Comments are visible to your team immediately and are folded into Traversal's understanding as it refreshes.
It can propose a memory during an investigation, but a person approves it before it takes effect. The Auto-created wiki is Traversal's own, regenerated from your connected systems.
The original was a place to upload your team's context. This one starts with everything Traversal already learned on its own, adds skills and memories, and puts it all in a file system you can browse and search.
Knowledge Bank feeds every surface Traversal runs on. The sharper it is, the sharper your [investigations](/using-traversal/investigations) and [Workers](/using-traversal/workers) get.
## Support
Questions or issues? Email [support@traversal.com](mailto:support@traversal.com).
# MCP
Source: https://docs.traversal.com/using-traversal/mcp
Run Traversal from Claude, Cursor, and other AI clients — and let your local agent and Traversal investigate together.
The Traversal MCP server lets any compatible AI client — Claude Code, Cursor, Claude Desktop, and others — run investigations, ask follow-ups, and retrieve results using the [Model Context Protocol](https://modelcontextprotocol.io/). It uses the streamable-http transport with API key authentication.
But it's more than a remote command. Over MCP, Traversal acts as a **peer agent your local agent works alongside**: your agent handles what's local — your code, your terminal, your tools — while Traversal investigates across your whole production stack, and the two hand work back and forth until the incident is understood.
**Endpoint:** `https://api.prod.traversal.com/mcp/`
For [single-tenant SaaS](/architecture/single-tenant-saas) or [BYOC](/architecture/byoc) deployments, use your organization's API URL + `/mcp/`.
## Agent-to-agent mode
When Traversal is called from another agent, it runs in **Lightning mode** — a fast investigation path built for the rapid back-and-forth of agent collaboration. A first pass typically returns in **30 seconds to 2 minutes**, fast enough for a real back-and-forth between your agent and Traversal rather than a single round trip. Your agent can always follow up to go deeper.
The collaboration runs in both directions:
Your agent sends Traversal the problem — along with what it can do locally (your repo, `kubectl`, your other tools). Traversal investigates across your metrics, logs, traces, deployments, and topology — the production-wide view your local agent can't see on its own.
Traversal hands back specific, executable next steps — *check `src/checkout/handler.go:142`* or *run `kubectl logs -n prod checkout-service-xyz`*. Your agent runs them locally and feeds the results back — no orchestration on your part.
The payoff: your local agent's strengths — `kubectl`, SSH, repository access, your other MCP tools — combine with Traversal's cross-stack investigation. Neither agent could close the loop alone.
### The collaboration loop
From your side, you just talk to your agent the way you always do — it handles the rest:
It sends Traversal the problem, what it can access locally, and how focused to stay.
A Lightning-mode investigation runs across your observability stack and returns findings plus next steps your agent can execute.
It reads the file, runs the command, or queries a tool only it has access to.
It passes the new evidence back, using the `session_id` from the first call. Traversal folds it in and refines the diagnosis.
Follow up as many times as you need — each call builds on the same investigation context.
### You don't drive it — your agent does
You don't call Traversal directly or craft a special request. Just describe the problem to your AI agent the way you normally would; it decides when to bring Traversal in and passes along the context that sharpens an investigation:
* **What it can access locally** — your codebase, `kubectl`, your other tools — so Traversal can hand back steps your agent can actually run.
* **How focused to be** — a tight look at one service, or a broad sweep across many.
* **The affected services and time window**, so Traversal targets the right place first.
## Authentication
Generate an API key in the Traversal web app under **[Settings > API keys](https://app.traversal.com/settings/api-keys)**. The key is shown once — copy it immediately.
Pass it in the `Authorization` header:
```
Authorization: Bearer trv_ak_...
```
## Setup
The config is the same across clients — an endpoint URL and an `Authorization` header. Find your client below.
Open **Settings > Developer > Edit Config** and add:
```json theme={null}
{
"mcpServers": {
"Traversal": {
"type": "http",
"url": "https://api.prod.traversal.com/mcp/",
"headers": {
"Authorization": "Bearer YOUR_API_KEY"
}
}
}
}
```
Restart Claude Desktop after saving.
```bash theme={null}
claude mcp add traversal \
--transport http \
--header "Authorization: Bearer YOUR_API_KEY" \
-- https://api.prod.traversal.com/mcp/
```
Add to `.cursor/mcp.json` in your project root or global settings:
```json theme={null}
{
"mcpServers": {
"Traversal": {
"url": "https://api.prod.traversal.com/mcp/",
"headers": {
"Authorization": "Bearer YOUR_API_KEY"
}
}
}
}
```
Add to your `opencode.json` config:
```json theme={null}
{
"$schema": "https://opencode.ai/config.json",
"mcp": {
"traversal": {
"type": "remote",
"url": "https://api.prod.traversal.com/mcp/",
"oauth": false,
"timeout": 300000,
"headers": {
"Authorization": "Bearer {env:TRAVERSAL_API_KEY}"
}
}
}
}
```
Set the `TRAVERSAL_API_KEY` environment variable with your API key before launching OpenCode.
Any client supporting the **streamable-http** transport can connect (ChatGPT, Windsurf, VS Code, etc.). Configure it with the endpoint URL and an `Authorization: Bearer` header containing your API key.
## Available tools
| Tool | Description |
| --------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------- |
| `investigate` | Run a fast first-pass investigation (typically 30s–2min). The agent passes the incident, what it can access, and how focused to be. |
| `follow_up` | Continue an investigation using its `session_id` — drill into findings, verify a hypothesis, or feed back evidence your agent gathered locally. |
| `get_investigation` | Get the status and report for an investigation, with a `ui_url` link to the web app. |
| `list_investigations` | List recent investigations for your organization. |
| `product_feedback` | Send feedback about the MCP experience to the Traversal team. |
A single `investigate` call is a first pass. Following up at least once — to drill into findings, verify hypotheses, or return what your agent found locally — is where agent-to-agent mode earns its keep.
## FAQ
MCP calls run in **Lightning mode**, optimized to return a useful first pass in roughly 30s–2min so your agent can keep moving. The web app can run longer, deeper investigations; over MCP, that depth comes from follow-ups instead.
No. Traversal is [always read-only](/architecture/intro) and never touches your systems. In agent-to-agent mode it *suggests* next steps — a command to run, a file to check — and your local agent decides whether to execute them, with its own permissions and access. The two stay cleanly separated: Traversal investigates, your agent acts.
Verify your API key is correct and hasn't been revoked. If you recently regenerated it, update your client configuration.
Broad investigations can take up to 2 minutes. Try a surgical investigation first and use `follow_up` to expand scope.
Call `get_investigation` — the response includes a `ui_url` link to the full investigation in the Traversal web app.
# Microsoft Teams
Source: https://docs.traversal.com/using-traversal/microsoft-teams
Investigate incidents and explore system behavior directly from Microsoft Teams.
With the Traversal Teams bot, you can start investigations, receive reports, and ask follow-up questions without leaving Microsoft Teams. Mention `@Traversal` in any channel, in a group chat, or in a direct chat with the app.
With Traversal in Teams, engineers can:
* **Start investigations** from any channel, group chat, or direct chat — either by mentioning `@Traversal` manually, or through configured auto-triggers
* **Receive investigation reports** rendered directly in the thread where the investigation was started
* **Ask follow-up questions** in natural language to refine the analysis
* **Work alongside Incident and Alert Workers** that maintain channel context and act proactively
See Traversal investigating an incident from Teams.
Get the Traversal app into your Microsoft Teams workspace.
## Starting an investigation
Mention `@Traversal` with a description of what you're seeing. Include any context that helps — symptoms, affected services, and a timeframe.
```text theme={null}
@Traversal checkout latency spiked around 14:05 UTC — what happened?
```
Traversal runs the investigation against your connected observability tools and posts the report back into the same thread.
## Triggers
Triggers let Traversal automatically start investigations when certain conditions are met. There are two types.
### Message triggers
Message triggers fire when a message in a configured channel matches certain conditions — for example, a keyword match or a message from a specific user. Use `@Traversal register-trigger` to create one. You'll provide:
* The **channel** to monitor
* **Conditions** such as keyword matches or sender filters
* The **type of context** Traversal should pull into the investigation
### Channel join triggers
Channel join triggers fire when Traversal is added to a channel that matches a pattern you define. The pattern can match against the channel **name** or **description** — for example, channels whose name starts with `inci-`. If the condition is met when Traversal is added to a channel, we will automatically run an investigation.
You can also specify the context source Traversal pulls into the investigation:
* Channel description
* Message content
* Message thread content
### Managing triggers
Use `@Traversal register-trigger` to set up a new trigger — a pop-up will surface to configure it. Use `@Traversal show-triggers` to view and delete triggers across your organization. Use `@Traversal show-channel-triggers` to see only the triggers configured for the current channel.
## Traversal Workers
Traversal can also work in a channel continuously as a [Worker](/using-traversal/workers), following what happens and deciding when to act without waiting for an `@Traversal` mention.
* [Incident Workers](/using-traversal/incident-workers) own live incident response from initial investigation through verified recovery and the post-mortem. Incident Workers are generally available.
* [Alert Workers](/using-traversal/alert-workers) continuously triage alerts, reuse relevant prior work, and identify the signals that need action. Alert Workers are in public beta.
[Deploy Workers](/using-traversal/workers-console) by scanning your workspace, picking a channel, or setting an auto-deploy rule. Every Worker comes with a [scratchpad, private chat, schedules, and custom instructions](/using-traversal/worker-features), and the [Workers console](/using-traversal/workers-console) shows the whole fleet at a glance.
## Investigation output
When Traversal completes an investigation, it renders the report directly in Teams. The report includes:
* **Citations** with links to the observability sources referenced in the investigation, bringing you directly to the relevant data in your primary observability tool
* A link to **open the investigation in the Traversal UI** for more detail
## Follow-ups
Mention `@Traversal` in the same thread as an investigation report to ask follow-up questions or trigger additional analysis.
```text theme={null}
@Traversal what services were impacted?
```
```text theme={null}
@Traversal re-run this with the last 6 hours of data
```
## Group chats and direct chats
Traversal also works outside channels. Add the Traversal app to a group chat, or open a direct chat with it, and mention `@Traversal` to start an investigation — no need to be in a specific channel.
## Permissions
Traversal follows the principle of least privilege. The bot runs as a registered Microsoft app that a Teams administrator installs. The Microsoft Graph permissions below are granted by a tenant administrator at install time.
| Permission | Why it's needed |
| :---------------------------------------------- | :--------------------------------------------------------------------------------------------------------------------------------------- |
| Bot registration (Azure AD app + Azure Bot) | Receive `@Traversal` mentions and post investigation results back |
| `User.Read.All` | Map Teams users to your Traversal account for a consistent experience |
| `Team.ReadBasic.All`, `Channel.ReadBasic.All` | Read team and channel names to anchor investigations |
| `ChannelMessage.Read.All` | Read channel messages to build investigation context — thread replies for follow-ups, and messages evaluated against configured triggers |
| `ChannelMember.Read.All`, `TeamMember.Read.All` | Read channel and team membership |
| `Chat.Read.All` | Build context from group and direct chats where the bot is used |
For the full breakdown of what data the integration captures and how it's handled, see [Microsoft Teams — data privacy and security](/integrations/microsoft-teams-data-privacy-and-security).
## Security and data handling
* **Data usage.** Teams data is used only to deliver the features described on this page. We do not use Teams data to train LLMs.
* **Scoped engagement.** Your organization decides where Traversal participates — it acts only on mentions, configured triggers, or Workers your team has enabled.
* **Transport and storage.** All traffic is encrypted in transit, credentials are stored securely, and inbound requests are validated against Microsoft-signed tokens.
* **Least privilege.** Traversal requests only the minimum permissions needed for the functionality described here.
* **Retention.** See the [privacy policy](https://www.traversal.com/legal/privacy-policy) for retention and deletion practices.
Traversal uses generative AI and may produce inaccurate or incomplete outputs. Validate recommendations before acting.
## Support
Questions or issues? Email [support@traversal.com](mailto:support@traversal.com).
## FAQ
If Traversal replies with a prompt to connect your workspace, an admin needs to click **Connect to Traversal** to pair it with your organization. If there's no response at all, the app may not be installed correctly — see the [installation guide](/integrations/microsoft-teams) or reach out to a Traversal team member.
Run `@Traversal show-channel-triggers` to confirm the trigger is enabled in the intended channel. Note that trigger conditions are case-sensitive — the message content must match your configured conditions exactly.
# Post-mortems
Source: https://docs.traversal.com/using-traversal/post-mortems
Generate a post-mortem from a Slack incident channel or the Traversal web app — automatically or on demand.
A proper post-incident review takes hours — reconstructing the timeline, writing up what happened, drafting follow-ups. Traversal produces a first draft so your team can skip the assembly work and focus on what actually drives improvement: reviewing, documenting, and shipping the fixes and process changes that make the org more resilient.
Post-mortems can be generated from Slack or the Traversal web app.
## In Slack
For many teams, Slack is the primary place where incident response happens — so it's a natural place to produce post-mortems. There are two ways to get one:
* **Proactively** — Traversal generates and posts a post-mortem on its own once the incident resolves.
* **Manually** — you ask for one by tagging Traversal in the channel.
### Proactive
When an [Incident Worker](/using-traversal/incident-workers) is active in an incident channel, it follows the incident in real time and — the moment the incident resolves — automatically generates and posts a post-mortem in the channel. No `@Traversal` mention required.
#### How Traversal assembles it
Because Traversal investigated the incident in real time, it has two sources to draw from when the incident closes:
1. **Its initial investigation** — the timeline, evidence, and hypothesized root cause Traversal produced at the start of the incident.
2. **The channel conversation** — everything the team said, shared, and concluded in the channel: decisions, handoffs, fixes applied, and the final resolution.
Traversal reconciles the two and synthesizes a single narrative. It explicitly calls out where its initial analysis was correct, where it diverged from what the team ultimately found, and where it lacked visibility to make a call.
#### What to expect
Proactive post-mortems include:
* **Summary** — severity, duration, date, and a paragraph describing customer and system impact.
* **Timeline** — a chronological sequence of alerts firing, log patterns emerging, user actions, and remediation steps, built from both the investigation and the channel.
* **5 Whys root cause analysis** — a layered walk from symptom to underlying cause.
* **Did Traversal get it right?** — an honest reconciliation of Traversal's initial analysis against the final root cause. This flags where Traversal was correct, where it was off, and where it couldn't see enough to be sure.
* **Suggested alerts and tests** — concrete, copy-pasteable alert conditions, log queries, or test ideas that would have caught this incident earlier, with rationale tied to the data seen during the incident.
### Manual
You can also request a post-mortem directly. Once an incident is resolved, tag `@Traversal` in the channel and ask:
> `@Traversal can you write a post-mortem for this incident?`
Traversal scans the channel — pulling in the timeline, key decisions, relevant alerts, and investigation findings — and posts a structured post-mortem directly in the channel.
Traversal reads the full channel either way, so a plain `@Traversal write a post-mortem` is enough. If you want to steer the output, add context — e.g., `@Traversal write a post-mortem focusing on the database layer — the root cause was a connection pool exhaustion`.
## In the web app
You can generate a post-mortem from the Traversal web app by providing context on the incident — either a description, a link to the Slack channel, or both:
> `Write a post-mortem for the checkout service outage on April 14 starting at 14:32 UTC. Here's the thread: [link]`
Traversal pulls from the thread, its own investigation findings, and any other context you provide to produce a complete post-mortem.
## Following up
The post-mortem Traversal posts is a starting point, not a final answer. Traversal is a chat product — you can keep the conversation going in the same Slack thread or in the web app to refine the draft, clarify details, or kick off new analysis. A few things you might ask for:
* **Rewrite a section** — `make the summary more concise` or `expand the timeline with anything that happened before 10:15 UTC`.
* **Clarify reasoning** — `why do you think the DB was the bottleneck and not the upstream service?`
* **Go deeper** — `run a new investigation on the database connection pool during this window` or `what else was deployed that day?`
* **Draft follow-ups** — `turn the suggested alerts into a list of Jira tickets` or `write up the action items for the next review meeting`.
Traversal's post-mortem is a draft, not a replacement for your post-incident review process.
# Prompt library
Source: https://docs.traversal.com/using-traversal/prompt-library
Example questions to ask Traversal — from active incident investigation to onboarding into unfamiliar systems.
These examples show the range of questions you can ask Traversal. You can type naturally — no special syntax required.
## Active incident investigation
Describe what you're seeing and when it started. The more context you provide — time, service, identifiers — the faster and more precise the results.
> "We're seeing elevated 500s in `auth-service` since around 14:30 UTC. What's going on?"
> "Investigate latency spikes in `payments-api` starting 14:32 UTC. Trace ID: `abc-123`, error: `connection pool exhausted`."
> "Deployment `a3f9c21` went out at 15:45 UTC. Did it cause the error spike we're seeing in `checkout-service`?"
> "There's a Jira ticket INC-8823 open for this — can you investigate?"
Include a start time whenever possible — it's the single most important input for an accurate investigation. See [best practices](/using-traversal/best-practices).
## Follow-up questions
After an initial investigation, ask follow-up questions to drill deeper, challenge a hypothesis, or expand the scope. Traversal retains the context of the investigation, so you don't need to repeat yourself.
> "Can you focus on the database layer specifically?"
> "What happened in the 15 minutes before the spike?"
> "What's the blast radius — which other services are affected?"
## Service health
Use Traversal to check in on a service or system without a specific incident in mind. Useful for routine health checks, pre-deploy sanity checks, or investigating a vague concern before it becomes a page.
> "What's the health of `auth-service` over the last 24 hours?"
> "Are there any anomalies in the data pipeline right now?"
> "What typically goes wrong with the database cluster?"
## Dashboards
Point Traversal at a specific dashboard and ask it to interpret what it sees. Useful when you notice something unusual but aren't sure how to read it, or want a second opinion on whether a pattern is significant.
> "Take a look at the payments dashboard and tell me if anything looks off."
> "Walk me through what you see in the `prod-us-east` latency dashboard from this morning."
> "Is the spike on the error rate dashboard related to the deployment that went out at 15:45?"
## System exploration and onboarding
Traversal can map dependencies, explain service relationships, and surface historical incident patterns — making it a powerful tool for engineers ramping up on an unfamiliar system or preparing for on-call.
> "What does `payments-api` depend on, and has anything changed recently?"
> "Walk me through the checkout flow and its dependencies."
> "I'm new to the payments team — what are the most common failure modes in this service?"
> "What should I know about this service before going on-call?"
## Code and pull requests
When Traversal identifies a root cause tied to a code issue, it can look at the relevant commits, identify the change that introduced the regression, and open a pull request with a fix.
> "The root cause is a missing null check in `payments-api` — can you open a PR to fix it?"
> "Create a pull request with a fix for the connection pool exhaustion we identified."
> "Look at recent changes to `checkout-service` and tell me if any of them could have caused this."
> "Which commit introduced this regression?"
## Alert trends and recommendations
Traversal can analyze patterns across your alert channels to surface which alerts are noisy, which are flapping, and which thresholds need adjustment.
> "Which alerts are firing most frequently this week?"
> "Are there any alerts that keep firing and resolving on their own? I want to identify what's flapping."
> "Give me recommendations for which alerts we should tune or suppress."
> "This alert has fired 47 times in the last 7 days. Is it signaling anything real, or should we adjust the threshold?"
## Post-mortems
Traversal can generate a blameless post-mortem from an incident channel or from context you provide directly. See [Post-mortems](/using-traversal/post-mortems) for the full options, including automatic post-mortem generation in Slack.
> "@Traversal write a post-mortem for this incident."
> "Create a blameless post-mortem for today's auth outage starting at 09:12 UTC."
# Slack
Source: https://docs.traversal.com/using-traversal/slack
Investigate incidents, triage alerts, and explore system behavior directly from Slack.
With the Traversal Slack bot, you can start investigations, receive reports, and ask follow-up questions without leaving Slack. Mention `@Traversal` in any channel the app has been added to, or DM it directly.
With Traversal in Slack, engineers can:
* **Start investigations** from any channel or DM — either by invoking Traversal manually, or through configured auto-triggers
* **See investigations** for incidents or alerts shared directly in their Slack channels or DMs
* **Ask follow-up questions** in natural language to refine the analysis
* **Work alongside Incident and Alert Workers** that maintain channel context and act proactively
See Traversal investigating an incident from Slack.
Get the Traversal Slack app into your workspace, including SaaS, single-tenant SaaS, and BYOC install paths.
Want to shape how Traversal investigates or formats its responses in a channel? Set [channel custom instructions](/using-traversal/customization) — they apply to every investigation in that channel.
## Commands
Traversal provides the following slash commands in Slack:
| Command | Description |
| :--------------------------------- | :--------------------------------------------------------------------------------- |
| `/traversal help` | Get documentation and instructions on how to use Traversal in Slack. |
| `/traversal register-trigger` | Create a new message trigger or channel join trigger for automatic investigations. |
| `/traversal show-triggers` | View and delete all triggers configured for your organization. |
| `/traversal show-channel-triggers` | View all triggers configured for the current channel. |
## Triggers
Triggers let Traversal automatically start investigations when certain conditions are met. There are two types.
### Message triggers
Message triggers fire when a message in a configured channel matches certain conditions — for example, a keyword match or a message from a specific user. Use `/traversal register-trigger` to create one. You'll provide:
* A **channel ID** for the channel to monitor
To find a channel ID, click on the channel name in Slack to open channel details. The channel ID is near the bottom.
* **Conditions** such as keyword matches or sender filters
* The **type of context** Traversal should pull into the investigation
### Channel join triggers
Channel join triggers fire when Traversal is added to a channel that matches a pattern you define. The pattern can match against the channel **name**, **topic**, or **description** — for example, channels whose name starts with `inci-` or whose topic contains `alert`. If the condition is met when Traversal is added to a channel, we will automatically run an investigation.
You can also specify the context source Traversal pulls into the investigation:
* Channel topic
* Channel description
* Message content
* Message thread content
### Managing triggers
Use `/traversal show-triggers` to view and delete triggers across your organization. Use `/traversal show-channel-triggers` to see only the triggers configured for the current channel.
## Traversal Workers
Traversal can also work in a channel continuously as a [Worker](/using-traversal/workers), following what happens and deciding when to act without waiting for an `@Traversal` mention.
* [Incident Workers](/using-traversal/incident-workers) own live incident response from initial investigation through verified recovery and the post-mortem. Incident Workers are generally available.
* [Alert Workers](/using-traversal/alert-workers) continuously triage alerts, reuse relevant prior work, and identify the signals that need action. Alert Workers are in public beta.
[Deploy Workers](/using-traversal/workers-console) by scanning your workspace, picking a channel, or setting an auto-deploy rule. Every Worker comes with a [scratchpad, private chat, schedules, and custom instructions](/using-traversal/worker-features), and the [Workers console](/using-traversal/workers-console) shows the whole fleet at a glance.
## DM Traversal
You can message Traversal directly to start an investigation. Open the Traversal app in Slack and send it a message — no need to be in a specific channel.
## Investigation output
When Traversal completes an investigation, it renders the report directly in Slack. The report includes:
* **Citations** with links to the observability sources referenced in the investigation, bringing you directly to the relevant data in your primary observability tool
* A link to **open the investigation in the Traversal UI** for more detail
* A **feedback** option so Traversal can learn about your system over time
* A **memory proposal** card in the thread when Traversal wants to remember something — approve or reject it there, or in the [Knowledge Bank](/using-traversal/knowledge-bank)
## Follow-ups
Tag `@Traversal` in the same thread as an investigation report to ask follow-up questions or trigger additional analysis.
```text theme={null}
@Traversal what services were impacted?
```
```text theme={null}
@Traversal re-run this with the last 6 hours of data
```
## Cross-channel context
Traversal can pull context from any channel it has been added to. You can reference other channels to enrich an investigation or summarize activity across channels — both in channels and in DMs with Traversal.
```text theme={null}
@Traversal please summarize the alerts in #alerts-prod and #alerts-staging over the last 24 hours
```
```text theme={null}
@Traversal summarize the alerts over the last 24 hours in this channel
```
```text theme={null}
@Traversal check #incidents-payments for related issues
```
Traversal can only read channels it has been added to. Make sure Traversal is a member of any channel you want it to reference.
## Permissions
Traversal follows the principle of least privilege and requests only the Slack scopes needed for the features described above.
| Scope | Why it's needed |
| :----------------------------------------------------------------- | :------------------------------------------------------------------------------------------------------------------------------- |
| `commands` | Enable the Traversal slash commands |
| `chat:write` | Post updates, summaries, and investigation results |
| `app_mentions:read` | Respond when someone mentions `@Traversal` |
| `channels:read`, `groups:read`, `im:read`, `mpim:read` | Read basic channel/DM metadata to anchor investigations |
| `channels:history`, `groups:history`, `im:history`, `mpim:history` | When added to a channel or DM, read recent history to build context (errors, timelines) without asking humans to re-type details |
| `channels:join` | Automatically join configured incident channels so Traversal is available the moment one is created |
| `files:read`, `files:write` | Ingest helpful artifacts (logs, traces) and upload images in investigation reports |
| `links:read`, `links:write` | Detect links in messages and unfurl them with relevant context |
| `pins:read`, `pins:write` | Pull context from pinned messages; pin significant milestones during incidents |
| `reactions:read` | Interpret emoji reactions on bot messages for feedback |
| `reactions:write` | React to messages — for example, to signal [Alert Worker](/using-traversal/alert-workers) verdicts |
| `users:read`, `users:read.email` | Map Slack users to your Traversal account for a consistent experience |
| `incoming-webhook` | Post asynchronous alerts when no active WebSocket is available |
## Security and data handling
* **Data usage.** Slack data is used only to deliver the features described on this page. We do not use Slack data to train LLMs.
* **Transport and storage.** All traffic is encrypted in transit, tokens are stored securely, and Slack signature verification is enforced on every request.
* **Least privilege.** Traversal requests only the minimum scopes needed for the functionality described here.
* **Retention.** See the [privacy policy](https://www.traversal.com/legal/privacy-policy) for retention and deletion practices.
Traversal uses generative AI and may produce inaccurate or incomplete outputs. Validate recommendations before acting.
## Support
Questions or issues? Email [support@traversal.com](mailto:support@traversal.com).
## FAQ
Run `/traversal show-channel-triggers` to confirm the trigger is enabled in the intended channel. Note that trigger conditions are case-sensitive — the message content must match your configured conditions exactly.
Only organization admins can register, view, or delete triggers. If you run `/traversal register-trigger`, `/traversal show-triggers`, or `/traversal show-channel-triggers` and see a message that the command is restricted, ask your admin to configure the trigger for you.
# Understanding results
Source: https://docs.traversal.com/using-traversal/understanding-results
Understand how Traversal structures its root cause analysis output, confidence ratings, evidence citations, and recommendations.
Traversal's output is designed to be transparent, explainable, and audit-friendly, so you can understand exactly why Traversal reached its conclusions.
## Root cause analysis
Traversal root causes your incidents automatically. Each root cause analysis includes:
* **Summary statement** — a concise description of the identified root cause
* **Confidence level** — High, Medium, or Low
* **Evidence citations** — the signals and data that support the conclusion
* **Relevant anomalies** — detected deviations from baseline behavior
* **Timeline context** — the sequence of events leading to and during the incident
* **Impact estimation** — scope and severity of the failure
## Confidence levels
Every root cause analysis and alert assessment includes a confidence rating. Use this to calibrate how much human review the conclusion needs.
| Confidence | Meaning |
| ---------- | ------------------------------------------------------------------------- |
| **High** | Strong correlation patterns, consistent evidence across multiple signals. |
| **Medium** | Theory is plausible and likely but missing full visibility or traces. |
| **Low** | Weak or partial signals. Requires human confirmation. |
Hover over any confidence label in the web app to see an inline explanation of what it means for that specific investigation.
Traversal's confidence rating reflects **data completeness**, not certainty. Lower confidence often means missing information — such as absent logs or traces — or ambiguous symptoms. Incomplete data leads to lower confidence, and humans remain in the loop for high-impact decisions.
## Evidence citations
Traversal always shows its work. Every conclusion is backed by cited evidence you can inspect and validate.
Evidence can include:
* Metrics anomalies
* Log lines
* Error rate spikes
* Deployment events
* Code diffs (read-only)
* Upstream and downstream failures
* Topology relationships
* Pull requests
Citations are interactive — you can engage with them directly from the investigation view without losing your place:
* **Hover** over a citation to preview it inline
* **Click** to expand it to a full detail view
* **Follow the source link** to jump to the underlying data in your observability platform (Datadog, Elasticsearch, etc.) whenever available
Source links let you verify the raw signal behind any finding — useful when you want to validate Traversal's interpretation or dig deeper than the summary.
## Timelines
The timeline shows the ordered sequence of events so you can understand what happened first and how failures cascaded.
Timelines include:
* When symptoms began
* When anomalies were detected
* When deployments occurred
* When related alerts fired
* When dependent services began failing
## Recommended next steps
Traversal may offer guidance alongside its root cause analysis:
* Diagnostic suggestions
* Remediation steps
* Runbook-style guidance
* Suggested team escalations
These are **recommendations, not actions**. Traversal will not make changes to your system without your explicit permission.
# Worker features
Source: https://docs.traversal.com/using-traversal/worker-features
The scratchpad, activity feed, private chat, schedules, and custom instructions that every Worker has.
When you open a Worker in the Traversal web app, you can expect a common set of features regardless of the use case.
Reach a Worker from the [Workers console](/using-traversal/workers-console), or from the link it posts in the channel it's working in.
## Scratchpad
The scratchpad is the Worker's up-to-date synthesis of its channel. It is designed to help engineers easily catch up on key context, without having to read dozens or hundreds of Slack threads.
For an **Incident Worker**, that's a synthesis of the incident: current impact, the conclusions so far, mitigation status, remaining risk, and the evidence behind each. It keeps evolving until it becomes the post-mortem.
For an **Alert Worker**, it's a synthesis of the channel: what needs action now, what's recurring and why, known causes and prior fixes, and where the alerts themselves are worth improving.
The Worker keeps its scratchpad current as it works and tells you when there's an update — click **Refresh** to pull it in. Everything in it carries citations you can check, and you can highlight any line to ask about it in [private chat](#private-chat) without touching the channel.
## Activity feed
The activity feed is everything the Worker is seeing and doing. For a Slack or Microsoft Teams Worker, that means every message arriving in the channel alongside the tool calls the Worker makes as it investigates — a mirrored, searchable view of the channel.
Slack and Teams feeds are read-only here; use **View in Slack** or **View in Teams** to reply at the source. For a shared web Worker, the feed is also where you talk to it.
## Private chat
A personal conversation with the Worker, using the same channel context and evidence, that never appears in the shared channel.
Use it to catch up after joining an incident late, test a theory, work through alerts on call, or ask for a summary aimed at your role — without adding a word of noise for the people responding.
Select any text in the scratchpad and pull it straight into the chat as context, so you can ask "what's the current state here, and what should I be watching?" about one specific line rather than re-explaining it.
## Schedules
A schedule sends a Worker back to a task at a specific time or on a recurring cadence, waking it even when it's otherwise dormant. It runs with the Worker's full channel context, so a handoff covers what actually happened and what's still open — not a list of subject lines.
Ask for one in plain English, in the channel or in private chat:
```text theme={null}
Recheck the error rate in 15 minutes. Post only if recovery stalls.
```
```text theme={null}
Every weekday at 9:00 AM America/New_York, post an on-call handoff with unresolved alerts and owners.
```
The Worker can create, list, update, and delete its own schedules — just ask. If a time or timezone is ambiguous it asks rather than guessing.
Or open **Schedules** from the Worker's detail view to see what's queued and what's already run, edit or delete anything active, and create a schedule without going through chat — a one-time date and time, or a recurring cadence in minutes, hours, or days. Each schedule shows who created it, so a shared Worker's queue stays legible to the team.
## Custom instructions
Custom instructions are how you tailor the way a Worker behaves. If you want to adjust its default behavior — where it looks first, what it prioritizes, when it engages, who it notifies, how it formats what it writes — this is how.
A Worker works without them. Add them when you have a standing preference you'd otherwise repeat.
Set them in three places:
| Where | Applies to |
| :---------------------------------------------------------------------------------------- | :--------------------------------------------------------------------------------------------------- |
| When you deploy a Worker | That one Worker |
| On an [auto-deploy rule](/using-traversal/workers-console#auto-deploy-to-future-channels) | Every future Worker the rule creates — so `inci-` channels behave differently from `retro-` channels |
| **Edit Worker** on an existing Worker | That one Worker, from now on |
```text theme={null}
Check the deployment dashboard before investigating application code.
Post major status changes in the main channel rather than in a thread.
```
Use **custom instructions** for durable behavior, a **schedule** for work that should happen later, and a plain message for something you want now.
See [Customization](/using-traversal/customization) for how Worker instructions relate to personal and channel-level instructions.
## Stop a Worker
Stopping a Worker halts new proactive work and clears its active schedules. Its activity and scratchpad stay available to read. Use **Edit Worker** in the web app, or the stop control on the Worker's arrival message in the channel.
Traversal stays available in the channel either way — mention `@Traversal` whenever you have a question and it answers. Stopping only ends the acting-on-its-own part.
# Workers
Source: https://docs.traversal.com/using-traversal/workers
Proactive AI SREs that take on the work that burdens your on-call team, and act without being asked.
**Traversal Workers** are proactive AI SREs. Rather than waiting for you to reach for Traversal, a Worker joins a channel and works alongside your team like a teammate — reading the situation and doing what's worth doing on its own. You never have to catch the right moment or learn how to prompt it. **There's no way to use it wrong. It just works.**
A Worker runs on Traversal's Causal Search Engine and Production World Model, the same engines behind every Traversal investigation. It belongs to the channel rather than to you, so everyone sees the same investigation and the same running picture of where things stand — nobody re-explains the incident to whoever just joined.
Read the [launch announcement](http://traversal.com/blog/traversal-workers-are-now-generally-available).
## Three kinds of Worker
**[Incident Workers](/using-traversal/incident-workers)** are for high-severity incidents. One joins the moment the incident is declared and helps the team all the way through to resolution, driving down MTTR.
**[Alert Workers](/using-traversal/alert-workers)** are for the alert channels a team works out of on an ongoing basis. They help you catch incidents sooner, cut the time you spend triaging alerts, and improve your system over time.
**Custom Workers** are for the work that is neither an incident nor an alert feed. You give one a mission in plain language when you deploy it, and it follows that channel and acts on your direction — watching a rollout in a deployment channel, surfacing the recurring problem behind a run of customer reports in a support channel, or keeping a running picture of ongoing work in a team or project channel.
In every case, a Worker follows the full channel including threads, grounds every conclusion in evidence from your connected systems, and takes plain-language [custom instructions](/using-traversal/worker-features#custom-instructions) and [schedules](/using-traversal/worker-features#schedules).
## Where Workers run
* **Slack and Microsoft Teams channels**, where they work alongside responders.
* **The Traversal web app**, where you can start a shared Worker when the work isn't tied to an existing channel.
You never have to summon a Worker — but you can always talk to one. Mention `@Traversal` or reply in a thread and it answers, with everything it already knows about the channel behind the response.
## Running a fleet from the web
The **[Workers console](/using-traversal/workers-console)** gives you a single pane of glass into all your Workers, so you don't need to open every channel and read dozens of threads. It's synthesized for you.
Each Worker reports a one-line summary of what it's doing and a status it sets for itself: **Urgent**, **Attention**, or **Stable**. Sort by status and go straight to the situations that are deteriorating. The console is also where you deploy Workers and set auto-deploy rules.
Opening a Worker adds what the channel doesn't hold: a [scratchpad](/using-traversal/worker-features#scratchpad) with its working synthesis of the channel, a searchable [activity feed](/using-traversal/worker-features#activity-feed), and [private chat](/using-traversal/worker-features#private-chat) for questions that shouldn't add noise to the response. [Schedules](/using-traversal/worker-features#schedules) and [custom instructions](/using-traversal/worker-features#custom-instructions) are set there too.
Deploy Workers, set auto-deploy rules, and track the fleet.
Scratchpad, activity feed, private chat, schedules, and instructions.
## Getting started
Open **Workers** in the Traversal web app and [scan your workspace](/using-traversal/workers-console#scan-your-workspace) — Traversal reviews the channels it can already see and recommends where to deploy. You can also deploy to a specific channel, or set an [auto-deploy rule](/using-traversal/workers-console#auto-deploy-to-future-channels) that covers every future `inci-` channel.
## FAQ
The channel it's deployed to, and the observability, code, ticketing, and knowledge context you've already connected to Traversal. Workers operate within the same Slack scopes described on the [Slack](/using-traversal/slack#permissions) page — read-only wherever they can be. Traversal can also be configured to follow the transcript of a live incident call, giving a Worker the same real-time context as the people responding.
No. A Worker reads, reasons, explains, and recommends. It never modifies your alerting, paging, or routing, and it does not take destructive action. You stay in control of what happens next.
Restraint is the design. In an incident channel a Worker holds back until it has something evidence-backed and worth the room's attention, and threads its replies so the main channel stays for findings everyone needs.
In an alert channel it answers every fire, but quietly — each response goes in that alert's own thread, so the channel reads exactly as it did before, just with a verdict on each alert.
You shouldn't need to — a Worker is good out of the box, and most teams never change a thing. But when you have an opinion, [custom instructions](/using-traversal/worker-features#custom-instructions) let you set it in plain English.
Set them on a single Worker to shape how it behaves in that one channel, or on a channel pattern so every future channel matching it inherits them — which is how you give a Worker in an `inci-` channel different marching orders from one in a `retro-` channel, without configuring either by hand.
Yes. Click **Stop proactive mode** on the message it posts when it joins, or use **Edit Worker** in the web app. Stopping it halts new proactive work; its activity and scratchpad stay available for review.
You don't lose Traversal in the channel by doing this. Mention `@Traversal` any time you have a question and it still answers — it just stops acting on its own.
The [Workers console](/using-traversal/workers-console) — the fleet view, each Worker's scratchpad and activity feed, private chat, schedules, and custom instructions. Workers also run in Microsoft Teams now, you can start one directly from the web when there's no channel for it, and [Alert Workers](/using-traversal/alert-workers) are new.
## Support
Questions or issues? Email [support@traversal.com](mailto:support@traversal.com).
# Workers console
Source: https://docs.traversal.com/using-traversal/workers-console
Deploy Workers, and see every one of them across Slack, Microsoft Teams, and the web in a single view.
The **Workers console** is where you deploy Workers and keep track of the ones you have. It covers Slack, Microsoft Teams, and web Workers in one view, so you can see the state of the fleet without opening each channel.
Open [Workers](https://app.traversal.com/workers) in the Traversal web app.
## Deploy Workers
Three ways to get a Worker running.
### Scan your workspace
**Workspace scan** deploys an initial fleet in one pass, and is available to org admins. Traversal reviews the channels it can already see, detects your channel-naming conventions, and returns a proposal: which channels should get Alert Workers, which should get Incident Workers, and what rule covers the rest.
Click **Scan workspace**. Traversal looks at operational activity and recurring channel-name patterns across the channels it can access.
Proposed Alert Worker channels are listed individually. Proposed Incident Workers come with the conventions Traversal detected in your channel names — for example, `inci-` and `retro-`.
Remove any channel you don't want, edit a proposed pattern, add another, and attach optional [custom instructions](/using-traversal/worker-features#custom-instructions) to any rule.
Nothing is deployed until you approve it. On confirmation the Workers come online and begin catching up on channel history.
Traversal cannot join private channels on its own. To put a Worker in one, invite `@Traversal` to that channel in Slack or Microsoft Teams first, then deploy the Worker.
### Deploy to a channel
Click **Deploy new Worker**, pick a Slack or Microsoft Teams channel, and choose the Worker's use case — Incident, Alerts, or Custom. A Custom Worker takes the mission you write for it. You can add custom instructions before it deploys.
### Start a shared web Worker
Choose **Traversal** as the location to create a Worker in the web app itself, for work that isn't tied to an existing Slack or Teams channel. Like any other Worker it belongs to the team: anyone with access sees the same investigation and can post to it.
## Auto-deploy to future channels
An auto-deploy rule brings an **Incident Worker** online in every channel whose name matches a pattern you define, so a Worker is present from the moment an incident channel opens. Auto-deploy settings are available to org admins.
Open **Auto-deploy settings** and match on the channel name. Most teams set a **Prefix** rule on their incident convention — `inci-`, `sev-`, `incident-`. Suffix, contains, and exact-match options are there too, each with a negation.
Each rule can carry [custom instructions](/using-traversal/worker-features#custom-instructions) that are copied to every Worker it deploys, so a Worker in an `inci-` channel can behave differently from one in a `retro-` channel without configuring either by hand.
Auto-deploy covers **public channels only**. A new rule deploys to matching channels that are open now and to every future match, but it doesn't reach back into old, closed ones.
## Track the fleet
Each row shows the Worker and where it lives, when it last updated, its current status, and a one-line summary of what it's doing.
| Status | Meaning |
| :------------ | :-------------------------------------------------------------- |
| **Urgent** | The Worker found something that needs your immediate attention. |
| **Attention** | The Worker is tracking something that could escalate. |
| **Stable** | No action required. |
A Worker sets its own status as it works and re-evaluates it continuously, so a row reflects the Worker's current read rather than its last post. A Worker you just deployed shows a rotating **Working… / Deploying… / Configuring… / Traversing…** state while it catches up on channel history.
* **Search** Workers and their recent activity.
* **Filter** by Worker type, status, and last update. [Stopped](/using-traversal/worker-features#stop-a-worker) Workers are hidden by default.
* **Pin** the Workers you watch most, so they sit at the top.
* See recent investigations and who's participating.
* Open any Worker for its [scratchpad, activity feed, and private chat](/using-traversal/worker-features).
## Worker Impact
**Worker Impact** reports what the fleet has done for you over the current month, against the month before it.
It covers two measures:
* **Engineering hours saved** — the time your team didn't spend doing what the Workers did. Broken out by Incident and Alert Workers, so you can see which is contributing what.
* **Downtime averted** — time your systems stayed up that they otherwise wouldn't have. This is wall-clock time rather than engineering time, so it's reported on its own and never added into hours saved.
Toggle the panel from **Worker Impact** in the console header.
### How it's calculated
Both figures are high-level estimates. Traversal counts what your Workers did, then applies a fixed set of assumptions to turn those counts into time. The assumptions are the same for every organization.
For incidents, each responder — the people who posted in the channel shortly after the Worker joined — is credited with the share of their time Traversal removed. For alerts, each response the Worker posts is credited with the time a person didn't spend reading the alert, deciding whether it mattered, and working out what caused it.
| Assumption | Value |
| :------------------------------------------- | :----- |
| Incident length | 4 h |
| Share of an incident a responder is hands-on | 30% |
| Share of that effort Traversal removes | 10% |
| Responders never pulled in at all | 10% |
| Time saved per alert response | 10 min |
Downtime averted applies that same 10% to incident length. Month-over-month comparisons use the same number of elapsed days, so on the 3rd you're seeing three days against the first three days of the month before.