Skip to main content
Redaction is a regex-based pipeline that rewrites sensitive text before your data is forwarded to Traversal. Both the Connector and the Processor apply redaction rules, but they apply them to different data and read different rule file formats: Don’t copy a rules file from one component to the other. A Processor-shaped file loaded by the Connector silently loads zero rules, and a Connector-shaped rule in the Processor’s configuration (for example, one with hosts) rejects the whole configuration. Rules are applied in order, and each rule’s output feeds the next rule, so order matters when patterns could overlap. Each rule is a named regex pattern (Go RE2 syntax in the Connector, the Rust regex crate in the Processor) with a replacement string. Neither engine supports lookahead, lookbehind, or backreferences. In a replacement, $1 or ${name} expands a capture group. In the Processor, write $$ for a literal $.

Connector rules

The Connector redacts the responses it receives from your upstream services, before they go back to Traversal. It doesn’t redact requests. For how to supply the rules file with Helm or Docker, see Redaction in the Connector setup.

Connector fields

The Connector ignores unknown keys in a local rules file without an error. A typo such as host = instead of hosts = makes the rule apply to every host.

How the two rule types work

  • regex runs over the raw bytes of the response body and ignores any JSON structure.
  • regex-structured-data parses a JSON response and matches your pattern against object keys and string values. Numbers are matched on their text and become strings after a match. Booleans and null are left untouched. It applies only to responses with a JSON content type. For any other content type, the rule is skipped and the response isn’t refused. If a JSON response isn’t one complete JSON document, the Connector refuses it.
Within each type, rules run in file order and each rule works on the output of the one before it. For a JSON response, the regex-structured-data rules run first, then the regex rules. For any other content type, only the regex rules run.

Which responses are redacted

The Connector buffers and redacts only responses from hosts that at least one rule covers. Responses from other hosts stream through unchanged. The Connector refuses a covered response, rather than forwarding it unredacted, when it:
  • Uses an encoding other than identity or gzip, or a content type that isn’t text-based (for example, an image or a binary download). Text, JSON, XML, YAML, JavaScript, form data, and NDJSON are supported, and so is a missing content type.
  • Is a server-sent events (SSE) stream, a gRPC response, or a WebSocket upgrade.
  • Is a 206 partial response.
  • Exceeds MAX_RESPONSE_BODY_SIZE_MB or MAX_DECODED_RESPONSE_BODY_SIZE_MB.
  • Has a JSON content type but isn’t one complete JSON document, when a regex-structured-data rule covers its host.

Host filtering

Use hosts on a rule to limit redaction to responses from specific upstream hostnames. The Connector checks the hostname in the request URL, not a host or hostname field in the response body. The filter applies to both rule types. Each entry in hosts is a regular expression, not a glob. A rule applies when any entry matches the entire hostname: Use single-quoted TOML strings to preserve regex backslashes. Escape literal dots as \.. Write hostname patterns without a URL scheme, port, or path. Matching is case-insensitive, ignores the request port, and removes a trailing dot from the requested hostname. Write patterns without a trailing dot. For internationalized hostnames, use their IDNA ASCII (punycode) form in patterns. Omitting hosts or setting hosts = [] also matches every hostname. An empty list does not disable a rule. The Connector logs a warning and buffers every response. Use hosts = ['.*'] to make all-host scope explicit.
To leave a destination unredacted, ensure no rule matches its hostname. Removing a hostname from one rule does not prevent another rule from applying. In particular, any rule with hosts = ['.*'] still matches that destination. hosts is an allowlist, not a denylist.

Field filtering

On regex-structured-data rules, redact_fields and skip_fields restrict which JSON fields a rule applies to. On a regex rule, the Connector ignores them and logs a warning.
  • Each entry is an exact field path with nested keys separated by |. For example, payment|card|number is the number key inside card inside payment. Entries are not regexes or globs.
  • An entry matches that field and everything beneath it.
  • redact_fields is an allowlist. The rule fires only on matching fields. An empty list matches nothing, so the rule never fires.
  • skip_fields is a denylist. The rule never fires on matching fields. If a field matches both lists, skip_fields wins.

Reload behavior

The Connector checks the rules file for changes every 10 seconds by default (redaction.reloadInterval in Helm, REDACTION_RELOAD_INTERVAL as an environment variable). It reloads when the content changes, and no restart is required.
  • If a reload finds an invalid file, the Connector keeps the last valid rules and logs a warning.
  • After three consecutive failed reloads, the Connector exits.
  • At startup, an unreadable or invalid file stops the Connector from starting instead of running unredacted.

Processor rules

The Processor applies rules to the values in each log record. Traversal publishes redaction rules to the Processor as part of its configuration, so there is no rules file to mount. Contact Traversal to add or change rules. Redaction is off until rules are configured. The Processor fetches that configuration only when traversalProcessor.dynamicConfig.deploymentId is set. Without it, the Processor receives no rules. See Traversal Processor. Rules are defined in a [redaction] table:

Processor fields

The Processor doesn’t support host-based rule selection. Don’t include hosts in Processor rules. The Processor redacts string and number values in a log record. It doesn’t redact keys. Booleans and null are left untouched.

Field filtering in the Processor

redact_fields and skip_fields select fields by their full path, with nested keys separated by |. For example, log|data|message is the message key inside data inside log. OTLP attributes sit under attributes|.... Paths match exactly, with no wildcards or prefix matching. When both filters are set, both must pass for the rule to fire. Rules with neither filter apply to every value.

Invalid rules

The Processor rejects the entire configuration if any rule has an unrecognized key (such as hosts), an unsupported type, or a pattern that doesn’t compile. A new pod then fails to start. A running Processor keeps its last good configuration. A misspelled key on the [redaction] table itself, such as default_replacment, is ignored. Rule changes take effect at the next configuration refresh, which is every 60 seconds plus up to 10 seconds of jitter.