> ## Documentation Index
> Fetch the complete documentation index at: https://docs.traversal.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Redaction

> Scrub sensitive information from your data before it leaves your environment, with rules applied by the Connector and the Processor.

Redaction is a regex-based pipeline that rewrites sensitive text before your data is forwarded to Traversal. Both the Connector and the Processor apply redaction rules, but they apply them to different data and read **different rule file formats**:

| | Connector | Processor |
| :- | :- | :- |
| Redacts | Upstream responses to Traversal's requests | The values in log records |
| Rule format | Top-level `version` and `[[rules]]` | A `[redaction]` table with `[[redaction.rules]]` |
| How rules are supplied | A rules file that you mount | Configuration that Traversal publishes to the Processor |
| Rule `type` | Required: `regex` or `regex-structured-data` | Optional: only `regex-structured-data` |
| Host filtering | Yes (`hosts`) | No |

Don't copy a rules file from one component to the other. A Processor-shaped file loaded by the Connector silently loads zero rules, and a Connector-shaped rule in the Processor's configuration (for example, one with `hosts`) rejects the whole configuration.

Rules are applied in order, and each rule's output feeds the next rule, so order matters when patterns could overlap. Each rule is a named regex pattern (Go RE2 syntax in the Connector, the Rust `regex` crate in the Processor) with a replacement string. Neither engine supports lookahead, lookbehind, or backreferences. In a replacement, `$1` or `${name}` expands a capture group. In the Processor, write `$$` for a literal `$`.

## Connector rules

The Connector redacts the **responses** it receives from your upstream services, before they go back to Traversal. It doesn't redact requests. For how to supply the rules file with Helm or Docker, see [Redaction in the Connector setup](https://docs.traversal.com/setup/connector.md#redaction).

```toml theme={null}
version = "v1"

# Optional. Fallback replacement for rules that omit their own.
# Defaults to [REDACTED].
default_replacement = "[REDACTED]"

[[rules]]
name    = "email"
type    = "regex"
pattern = '[a-zA-Z0-9._%+\-]+@[a-zA-Z0-9.\-]+\.[a-zA-Z]{2,6}'
hosts   = ['api\.example\.com']

[[rules]]
name          = "card-number"
type          = "regex-structured-data"
pattern       = '\b\d{16}\b'
replacement   = "[CARD]"
hosts         = ['api\.example\.com']
redact_fields = ["payment|card|number"]
```

### Connector fields

| Field | Required | Description |
| :- | :- | :- |
| `version` | No | Informational only. Use `"v1"`. |
| `default_replacement` | No | Fallback for rules that omit `replacement`. Defaults to `[REDACTED]`. |
| `rules[].name` | Yes | Label for the rule. Appears in logs and metrics. |
| `rules[].type` | Yes | `regex` or `regex-structured-data`. There is no default. A missing or unknown `type` rejects the whole file. |
| `rules[].pattern` | Yes | RE2 regular expression. All matches are replaced. |
| `rules[].replacement` | No | Text substituted for each match. Falls back to `default_replacement`. |
| `rules[].hosts` | No | Allowlist of hostname regexes. See [Host filtering](#host-filtering). |
| `rules[].redact_fields` | No | Allowlist of field paths. `regex-structured-data` rules only. See [Field filtering](#field-filtering). |
| `rules[].skip_fields` | No | Denylist of field paths. `regex-structured-data` rules only. |

> [!WARNING]
>
> The Connector ignores unknown keys in a local rules file without an error. A typo such as `host =` instead of `hosts =` makes the rule apply to every host.

### How the two rule types work

* **`regex`** runs over the raw bytes of the response body and ignores any JSON structure.
* **`regex-structured-data`** parses a JSON response and matches your pattern against object keys and string values. Numbers are matched on their text and become strings after a match. Booleans and `null` are left untouched. It applies only to responses with a JSON content type. For any other content type, the rule is skipped and the response isn't refused. If a JSON response isn't one complete JSON document, the Connector refuses it.

Within each type, rules run in file order and each rule works on the output of the one before it. For a JSON response, the `regex-structured-data` rules run first, then the `regex` rules. For any other content type, only the `regex` rules run.

### Which responses are redacted

The Connector buffers and redacts only responses from hosts that at least one rule covers. Responses from other hosts stream through unchanged.

The Connector **refuses** a covered response, rather than forwarding it unredacted, when it:

* Uses an encoding other than `identity` or `gzip`, or a content type that isn't text-based (for example, an image or a binary download). Text, JSON, XML, YAML, JavaScript, form data, and NDJSON are supported, and so is a missing content type.
* Is a server-sent events (SSE) stream, a gRPC response, or a WebSocket upgrade.
* Is a `206` partial response.
* Exceeds `MAX_RESPONSE_BODY_SIZE_MB` or `MAX_DECODED_RESPONSE_BODY_SIZE_MB`.
* Has a JSON content type but isn't one complete JSON document, when a `regex-structured-data` rule covers its host.

### Host filtering

Use `hosts` on a rule to limit redaction to responses from specific upstream hostnames. The Connector checks the hostname in the request URL, not a `host` or `hostname` field in the response body. The filter applies to both rule types.

Each entry in `hosts` is a regular expression, not a glob. A rule applies when **any** entry matches the **entire hostname**:

| `hosts` | Matches |
| :- | :- |
| `['api\.example\.com']` | Only `api.example.com`, not `other.api.example.com` or `api.example.com.other.com`. |
| `['api\.example\.com', 'logs\.example\.com']` | Either of the two listed hostnames. |
| `['([^.]+\.)*example\.com']` | `example.com` and its subdomains, but not `otherexample.com`. |
| `['.*']` | Every hostname. |

Use single-quoted TOML strings to preserve regex backslashes. Escape literal dots as `\.`. Write hostname patterns without a URL scheme, port, or path.

Matching is case-insensitive, ignores the request port, and removes a trailing dot from the requested hostname. Write patterns without a trailing dot. For internationalized hostnames, use their IDNA ASCII (punycode) form in patterns.

Omitting `hosts` or setting `hosts = []` also matches every hostname. An empty list does **not** disable a rule. The Connector logs a warning and buffers every response. Use `hosts = ['.*']` to make all-host scope explicit.

> [!WARNING]
>
> To leave a destination unredacted, ensure **no rule** matches its hostname. Removing a hostname from one rule does not prevent another rule from applying. In particular, any rule with `hosts = ['.*']` still matches that destination. `hosts` is an allowlist, not a denylist.

### Field filtering

On `regex-structured-data` rules, `redact_fields` and `skip_fields` restrict which JSON fields a rule applies to. On a `regex` rule, the Connector ignores them and logs a warning.

* Each entry is an exact field path with nested keys separated by `|`. For example, `payment|card|number` is the `number` key inside `card` inside `payment`. Entries are not regexes or globs.
* An entry matches that field and everything beneath it.
* `redact_fields` is an allowlist. The rule fires only on matching fields. An empty list matches nothing, so the rule never fires.
* `skip_fields` is a denylist. The rule never fires on matching fields. If a field matches both lists, `skip_fields` wins.

### Reload behavior

The Connector checks the rules file for changes every 10 seconds by default (`redaction.reloadInterval` in Helm, `REDACTION_RELOAD_INTERVAL` as an environment variable). It reloads when the content changes, and no restart is required.

* If a reload finds an invalid file, the Connector keeps the last valid rules and logs a warning.
* After three consecutive failed reloads, the Connector exits.
* At startup, an unreadable or invalid file stops the Connector from starting instead of running unredacted.

## Processor rules

The Processor applies rules to the values in each log record. Traversal publishes redaction rules to the Processor as part of its configuration, so there is no rules file to mount. Contact Traversal to add or change rules. Redaction is off until rules are configured.

The Processor fetches that configuration only when `traversalProcessor.dynamicConfig.deploymentId` is set. Without it, the Processor receives no rules. See [Traversal Processor](https://docs.traversal.com/setup/processor.md).

Rules are defined in a `[redaction]` table:

```toml theme={null}
[redaction]
# Optional. Fallback replacement for rules that omit their own.
# Defaults to [REDACTED].
default_replacement = "[REDACTED]"

[[redaction.rules]]
name    = "email"
pattern = '[a-zA-Z0-9._%+\-]+@[a-zA-Z0-9.\-]+\.[a-zA-Z]{2,6}'

[[redaction.rules]]
name          = "card-number"
pattern       = '\b\d{16}\b'
replacement   = "[CARD]"
redact_fields = ["body", "attributes|message"]
```

### Processor fields

| Field | Required | Description |
| :- | :- | :- |
| `default_replacement` | No | Fallback for rules that omit `replacement`. Defaults to `[REDACTED]`. |
| `rules[].name` | Yes | Label for the rule. Appears in logs and metrics. |
| `rules[].pattern` | Yes | Regular expression. All matches in a value are replaced. |
| `rules[].replacement` | No | Text substituted for each match. Falls back to `default_replacement`. |
| `rules[].type` | No | The only valid value is `regex-structured-data`, which is also the default. `regex` is rejected. |
| `rules[].redact_fields` | No | Allowlist of field paths. |
| `rules[].skip_fields` | No | Denylist of field paths. |

The Processor doesn't support host-based rule selection. Don't include `hosts` in Processor rules.

The Processor redacts string and number **values** in a log record. It doesn't redact keys. Booleans and `null` are left untouched.

### Field filtering in the Processor

`redact_fields` and `skip_fields` select fields by their full path, with nested keys separated by `|`. For example, `log|data|message` is the `message` key inside `data` inside `log`. OTLP attributes sit under `attributes|...`.

Paths match exactly, with no wildcards or prefix matching. When both filters are set, both must pass for the rule to fire. Rules with neither filter apply to every value.

### Invalid rules

The Processor rejects the **entire configuration** if any rule has an unrecognized key (such as `hosts`), an unsupported `type`, or a pattern that doesn't compile. A new pod then fails to start. A running Processor keeps its last good configuration. A misspelled key on the `[redaction]` table itself, such as `default_replacment`, is ignored.

Rule changes take effect at the next configuration refresh, which is every 60 seconds plus up to 10 seconds of jitter.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.