Skip to main content
The Traversal Processor is a telemetry ingestion and processing service that receives telemetry data from your infrastructure, then extracts and indexes structured data into the Traversal platform. Today, it accepts log data only, through HTTP endpoints, OpenTelemetry Protocol (OTLP) over gRPC and HTTP, and a Splunk HTTP Event Collector (HEC) endpoint. The Traversal Processor ships as a container image that you deploy with a Helm chart. It scales horizontally and does not require coordination across instances.
If your environment restricts outbound traffic, see Network allowlisting for the IPs to allow before deploying.

Prerequisites

  • A Kubernetes cluster and Helm.
  • An existing traversal namespace. The install command from the Traversal web app doesn’t create it:
  • openssl, to generate the processor’s private key and certificate request.
  • The organization admin role in Traversal.
  • Outbound network access from the cluster to your Traversal endpoints (see the note above), and access to the registries below. Set up image pull access for the private processor image before you deploy.

Container image

Traversal publishes the processor image to the traversalext/traversal-processor repository on Docker Hub. The image is private. The Helm chart (oci://registry-1.docker.io/traversalext/traversal-processor-charts) and the traversalext/alloy telemetry forwarder image are public.
The processor image is multi-platform (linux/amd64 and linux/arm64).
If your nodes need credentials to pull the image, create a kubernetes.io/dockerconfigjson Secret in the release namespace and reference it from the chart:
image.pullSecrets attaches to the pod, so it covers the processor, the telemetry forwarder, and the SQS adapter. To pull from an internal mirror instead, set image.repository and, if the mirror requires authentication, image.pullSecrets. You are responsible for setting up image pull access before you deploy.

Set up in the Traversal web app

You create the processor in the web app. You generate the private key and certificate request on your machine, so the private key never leaves your environment.
1

Open the Processors tab

In the left navigation, click Integrations, open the Processors tab, and add a processor. Setup has four steps: Details, Certificate, Size, and Deploy.
2

Details

Enter a name for the processor. The app shows its Processor ID, which the install command sets as traversalProcessor.dynamicConfig.deploymentId.
3

Certificate

Generate a private key and a certificate signing request (CSR) with the commands the app shows. They look like this:
The CSR must use a NIST P-256 key, have an empty subject (-subj "/"), and contain exactly one extension: a single SAN URI. The app fills in the URI for your organization.Upload the .csr file. The app signs it and returns the certificate. Save the certificate next to the .key file, because the install command needs both.
4

Size

Choose normal or large. The app recommends one. See Sizing and throughput.
5

Deploy

Copy the helm install command the app shows and run it from the directory that contains the certificate and key. See Install and upgrade. The app then shows whether the processor is Online or Offline, and lets you re-check the status.

Install and upgrade

The app prints an install command pinned to a chart version. Use the version it shows in place of <version> below:
The app also shows an upgrade command, which runs helm upgrade with --reuse-values so your existing settings carry over. Run it when you move to a newer chart version. See the chart changelog for what changed in each version and any upgrade notes. The chart is generic and holds no customer-specific configuration. To see every supported field, pull it and read values.yaml, which also marks the required fields:
You can also install the chart through a GitOps pipeline such as Argo CD or Flux, or from an internal Helm registry mirror. The pod exposes a health endpoint at /health on a dedicated probe port (healthPort, default 3001). The readiness probe gates traffic until the service is ready.
If Traversal provided you a values file: pass it with -f <your-values-file>.yaml instead of the --set flags in the command above. The required values below still apply.

Ports and connectivity

Choose the endpoint that matches your sender’s protocol. These are the default backend ports exposed by the chart’s Service; an ingress or load balancer can expose a different external port, such as 443 for HTTPS. Port 3000 accepts HTTP requests, not gRPC or WebSocket connections. Processor TLS enables HTTPS on both HTTP ingest ports. The health server stays on plain HTTP, and the gRPC listener stays on plaintext HTTP/2.

Test connectivity

Run these checks from a machine that can reach the processor’s Service or load balancer. Set PROCESSOR_HOST to its hostname, without a scheme or port, and adjust the ports if your deployment overrides them. Run the following snippets in the same shell to reuse that hostname. The first commands connect directly to a processor with TLS disabled:
Expect HTTP 200 for each request. /ping returns {"service":"traversal-processor"}. /health returns {"status":"healthy","service":"traversal-processor"}. The OTLP request returns an export response with no rejected records. To test gRPC, you need grpcurl and the OpenTelemetry proto files. Run this download on the machine or inside the pod where you’ll run grpcurl. It uses curl and tar to extract a release into a temporary directory, including the files imported by logs_service.proto. OTEL_PROTO_VERSION defaults to v1.7.0; to use another release, export its tag (including the leading v) before running the snippet:
In the same shell, use that directory as the import path. grpcurl reads the proto files directly; you don’t need to install a protobuf compiler or copy each imported file separately:
Expect a successful export response with no rejected records and exit status 0. The processor doesn’t enable server reflection, so supply the proto files instead of using grpcurl list. For HTTPS on the processor or ingress, change the HTTP ingest URLs to https:// and use the endpoint’s external port when connecting through ingress. If its certificate uses a private CA, add --cacert <ca-certificate.pem> to curl. For example, with Processor TLS enabled:
Use a hostname covered by the server certificate. When testing a TLS-terminating gRPC ingress, remove -plaintext, use its hostname and external port, and add -cacert <ca-certificate.pem> if it uses a private CA. Direct gRPC connections to the processor still use -plaintext, including when Processor TLS is enabled. These requests send no log records. They check endpoint reachability and protocol handling; they don’t verify processing or delivery to Traversal. A successful health check alone doesn’t confirm that the ingest ports are reachable.

Configure ingress

The chart creates a ClusterIP Service by default and doesn’t create an Ingress resource. Senders inside the cluster can use the Service directly. If senders need ingress or a load balancer, configure it for the protocols you use: Expose only the ingestion protocols your senders use and restrict access to those senders. The processor relies on network controls to restrict ingestion. For gRPC, a TCP passthrough load balancer doesn’t add TLS to the plaintext backend; use a TLS-terminating proxy or load balancer when TLS is required. These inbound listeners are separate from the processor’s outbound connections to Traversal, including its operational telemetry export.

Required values

Most keys sit under traversalProcessor. The exceptions are top-level keys: replicaCount, image.*, resources, service, healthPort, the probes, scheduling, pdb, and extraEnv, extraEnvFrom, extraVolumes, extraVolumeMounts, and envFiles. This page uses full key paths. The chart doesn’t require traversalProcessor.dynamicConfig.deploymentId at render time, but without it the processor doesn’t fetch its configuration, and the helm install output notes the missing configuration source. With the default telemetry forwarder enabled, the chart fails to render unless you supply an mTLS source.
Supply the mTLS material in exactly one of these ways:
  • Inline PEM: traversalProcessor.mtls.certPEM and traversalProcessor.mtls.keyPEM, set to the PEM content. Use --set-file to read them from disk. The chart base64-encodes the content and renders a Kubernetes Secret.
  • Inline base64: traversalProcessor.mtls.certBase64 and traversalProcessor.mtls.keyBase64.
  • Existing Secret: traversalProcessor.mtls.existingSecret, the name of a Secret that contains the keys cert.pem and key.pem. A cert-manager Certificate writes tls.crt and tls.key, so copy those values to cert.pem and key.pem first.
  • Files: traversalProcessor.mtls.certFile and traversalProcessor.mtls.keyFile, PEM paths already present in the container. Mount them with extraVolumes and extraVolumeMounts, a CSI driver, or an init container. The telemetry forwarder reads the same files, so mount them into it through extraVolumeMounts too, or the chart fails to render.
With inline PEM, inline base64, and existingSecret, the credentials are mounted read-only into the container. The private key isn’t placed in the pod’s environment.If a *Base64 field contains PEM content, or a *PEM field contains a base64-encoded value, the install fails with an error that names the correct field.The telemetry forwarder reuses this identity.
All other keys are optional.

Optional configuration

Endpoints

The processor sends processed log clusters to the ingest endpoint, fetches its configuration from the configuration endpoint, and sends its own telemetry to the telemetry endpoint. The chart derives all three from traversalProcessor.baseDomain: The chart derives the configuration endpoint only when both baseDomain and dynamicConfig.deploymentId are set. Specify the domain without a scheme, port, path, or ingest. prefix. The chart adds these and rejects values that include them. For a dedicated Traversal deployment, set baseDomain to the domain Traversal gives you. To reach its ingest endpoint over AWS PrivateLink, see PrivateLink. An explicit endpoint overrides the derived value. Set endpoints explicitly to:
  • Send telemetry to your own collector. Set all three of traversalProcessor.observability.otel.metricsEndpoint, traversalProcessor.observability.otel.tracesEndpoint, and traversalProcessor.observability.otel.logsEndpoint.
  • Reach the ingest endpoint through a hostname that isn’t derived from baseDomain. Set traversalProcessor.traversalIngestion.endpoint.
The processor resolves telemetry endpoints independently of the ingest endpoint. Setting only traversalProcessor.traversalIngestion.endpoint leaves telemetry on Traversal’s hosted endpoints. To control where telemetry goes, do one of the following:
  • Set baseDomain to move the ingest, configuration, and telemetry endpoints together.
  • Set the three traversalProcessor.observability.otel endpoints to send telemetry to your own collector.
  • Set traversalProcessor.observability.otel.forwardProxy to reach Traversal’s hosted telemetry through a forward proxy.
The install output lists every resolved endpoint.

Sizing and throughput

traversalProcessor.size selects a validated resource-and-tuning preset. The chart sets CPU, memory, and internal ingest tuning for the tier you choose. Size presets require chart 0.4.0 or later. To scale up, change normal to large. To scale out, raise the top-level replicaCount. If your cluster mixes node families, pin pods to the recommended instance type with nodeSelector. Throughput figures were measured with roughly 4 KB average log records, about 40% cluster compression, and redaction disabled. Actual throughput depends on record size, field cardinality, redaction, and downstream latency. Validate against your own traffic.

TLS for the ingestion endpoint

By default the processor serves plain HTTP on ports 3000 (native ingest) and 4318 (OTLP HTTP). If your environment requires end-to-end encryption, for example when a load balancer or service mesh terminates TLS and re-encrypts to the backend, enable TLS so both HTTP ingest listeners serve HTTPS directly. Configure TLS in traversalProcessor.tls. Provide exactly one of the following credential sources. PEM files. Provide paths to certificate and key files already mounted in the container, for example from a Kubernetes Secret volume:
Inline PEM. Set certPEM and keyPEM to the PEM content. The chart base64-encodes the content, creates a Kubernetes Secret, and mounts it read-only into the container:
You can also read the files from disk with --set-file:
Inline base64. Set certB64 and keyB64 to base64-encoded PEM content:
The install fails if more than one source is set, if only half of a pair is set, if certB64 or keyB64 contains PEM content, or if certPEM or keyPEM contains a base64-encoded value. This setting applies to both HTTP ingest listeners. The separate probe port (healthPort) stays on plain HTTP, so the liveness and readiness probes work whether or not ingest TLS is enabled. It doesn’t enable TLS on the OTLP gRPC listener. See Configure ingress for gRPC transport requirements.

Log ingestion from Amazon SQS

Instead of sending logs to the processor’s HTTP or OTLP endpoints, you can have the processor pull them from an Amazon SQS queue. An optional sidecar container runs in the same pod, polls the queue, and hands each batch to the processor over the pod’s loopback interface. No inbound network path to the processor is needed, and there is no separate forwarder service to deploy. The SQS adapter requires chart 0.5.0 or later. It is off by default:
Every replica polls the same queue and SQS divides messages between them, so raising replicaCount scales ingest with no further configuration.
The sidecar ships as its own image, traversalext/traversal-processor-sqs-adapter, so deployments that don’t use SQS carry no AWS dependencies. Like the processor image, it is private. image.pullSecrets covers both.
The sidecar adds to the pod’s resource footprint, on top of the size preset. Size the node for the sum: normal with the adapter needs 6 cores / 5 GiB (the chart sizes this for a c8g.2xlarge), and large with the adapter needs 11 cores / 11 GiB (c8g.4xlarge).

Message format

Message bodies must be NDJSON, with one JSON object per line. A single message can carry many lines, and the sidecar reads all of them. If your pipeline wraps the payload instead of sending NDJSON directly, set traversalProcessor.sqsAdapter.ndjsonPayloadPath to a |-delimited path to the string that holds it. For example, content|lines_str reads the NDJSON from {"content": {"lines_str": "..."}}. To control how each record’s index and timestamp are derived, set indexStrategy and timestampStrategy under traversalProcessor.sqsAdapter. They accept the same strategies as the JSON ingest API. Leave them empty to use the processor’s defaults.

AWS credentials

The sidecar needs permission to call sqs:ReceiveMessage, sqs:DeleteMessage, sqs:GetQueueAttributes, and sqs:ChangeMessageVisibility on the queue. Grant it by exactly one of three routes: To tell which mechanism your cluster uses, look for a running eks-pod-identity-agent DaemonSet in kube-system. If it’s there, you have Pod Identity.

Networking

Check how SQS traffic leaves your VPC before you enable this. Without an SQS interface endpoint, calls from a private subnet leave through your NAT gateway, which bills data processing per GB in both directions.
If you reach SQS through a VPC or PrivateLink endpoint whose private DNS isn’t enabled, set traversalProcessor.sqsAdapter.endpointUrl to that endpoint. Leave it empty for standard SQS, which the processor resolves from region.

Splunk HEC

The processor serves a Splunk HTTP Event Collector (HEC) endpoint on the main ingest port. The endpoint is always on and needs no chart configuration. It requires chart 0.8.2 or later. To send logs to the processor, set the sender’s HEC destination to http://<processor>:3000/services/collector. No other sender configuration changes are required.

Supported senders

The endpoint accepts any sender that emits the HEC event format, including Vector’s splunk_hec_logs sink and Splunk Edge Processor or Ingest Processor. Splunk universal and heavy forwarders aren’t supported. Forwarders send data over S2S, raw TCP, or syslog, none of which carry the HEC envelope. To ingest data from forwarders, route it through Cribl Stream or Vector.

Authentication

The endpoint accepts all three Splunk token forms: the Authorization: Splunk <token> header, basic authentication with the token as the password, and the token query parameter. It also accepts requests without a token.
The endpoint doesn’t validate tokens and doesn’t authenticate senders. Like the processor’s other ingest endpoints, it relies on network controls. Restrict access to the ingest port.

Batches and responses

The endpoint accepts all Splunk-documented batch framings: concatenated JSON objects, JSON objects separated by whitespace or newlines, and a single top-level JSON array. A batch can contain up to 10,000 events. Responses use the Splunk HEC response format: If a batch contains both valid and invalid events, the processor ingests the valid events and returns a success response. The invalid_events field reports the number of events dropped. Senders shouldn’t retry these batches, because a retry duplicates the events that were already accepted.

Field mapping

The processor reads HEC metadata keys from the top level of each event. The event value can be a string or an object. If a metadata key and a key inside event have the same name, the metadata value takes precedence.

Indexes and timestamps

The index is taken from the event’s index field. Events without an index field are assigned to the splunk-hec index. The timestamp is taken from time, a Unix epoch value in seconds, milliseconds, microseconds, or nanoseconds. Fractional values are supported. Events with a missing or unreadable time are timestamped on arrival, and an unreadable value is preserved in the time field. Index and timestamp handling is configured per deployment, in the configuration that Traversal publishes to the processor rather than in your Helm values. Changes take effect on the next configuration refresh without a restart. For example, Traversal can include sourcetype in the index, which changes how logs are clustered. Contact Traversal to change these settings.

Unsupported endpoints

The processor doesn’t serve /services/collector/raw or /services/collector/ack. Requests to either path return 404. Contact Traversal if you need either endpoint.

Redaction

The processor can redact sensitive text in log fields before data leaves your environment. The processor has no rules file to mount. It receives redaction rules only through the configuration it fetches from Traversal, so redaction needs traversalProcessor.dynamicConfig.deploymentId. It isn’t available with static traversalProcessor.fields configuration. Contact Traversal to add or change rules. Redaction is off until rules are configured. Rule changes take effect on the next configuration refresh, without a restart. Each rule’s type must be regex-structured-data, which is also the default. A bad key, an unsupported type such as regex, or a pattern that doesn’t compile rejects the whole configuration revision: a new pod fails to start, and a running pod keeps its last good configuration. See Redaction for the rule format and field filtering. If you upgrade from a chart earlier than 0.8.0 and set traversalProcessor.redaction, read the 0.8.0 changelog entry first.

Telemetry forwarding

The processor reports its own operational telemetry (metrics, traces, and logs) to Traversal over OTLP. Traversal uses it to support your deployment. It is separate from the log data you send the processor. By default the chart adds a second container to the processor pod: a forwarder that buffers this telemetry and retries delivery, so a brief interruption reaching Traversal doesn’t lose it. It authenticates with the processor’s mTLS identity and needs no credentials of its own. The forwarder requires chart 0.4.1 or later. The traversalProcessor.observability.otel endpoints derive from baseDomain (see Endpoints). To send telemetry to your own collector, override all three.

Disabling telemetry

Setting traversalProcessor.observability.otel.enabled: false stops all telemetry export and removes the forwarder.
Don’t disable telemetry. It is Traversal’s only view into a processor running in your network, so without it Traversal can provide only limited support.

Startup logging

Each pod logs its effective configuration when it starts. Use these lines to check how a pod is configured, for example which endpoints it sends to, where it gets its configuration, and whether a setting you changed took effect, without access to the pod’s shell or metrics. Startup logging requires chart 0.9.1 or later.

Configuration source

If configuration published by Traversal doesn’t take effect, check config_source on the config line. remote_endpoint means the processor fetches its configuration from Traversal. local_args means it uses the static traversalProcessor.fields instead. To fetch configuration from Traversal, set traversalProcessor.dynamicConfig.deploymentId to your Processor ID:
A failed first configuration fetch is fatal. A pod that crash-loops usually has a configuration-fetch problem. Check baseDomain, dynamicConfig.deploymentId, the mTLS certificate, and outbound access to the configuration endpoint.

Overridden settings

The overrides line lists every setting supplied through the environment, by variable name and source. It never logs the values. If a setting you changed doesn’t appear there, it didn’t reach the processor. Helm doesn’t catch every mistake in your values:
  • Since chart 0.7.0, helm install and helm upgrade print an advisory note that lists unrecognized keys placed directly under traversalProcessor. The note never fails the install.
  • Nested keys, top-level typos, and misspelled extraEnv variable names aren’t checked. The chart renders without errors and the processor uses its defaults.
  • Some mistakes do fail the render, such as a malformed baseDomain, conflicting mTLS or TLS sources, or an unknown size.
Startup lines never include credentials. They report certificates and keys only by source, and proxies only as set or unset.

Log verbosity

The processor logs at the info level by default. To increase verbosity, for example when troubleshooting with Traversal, set traversalProcessor.logLevel. The key requires chart 0.9.1 or later:
At debug, the processor logs each received batch and per-request details for each ingest path. This significantly increases log volume. Revert to the default level after troubleshooting.

Throughput logging

Throughput logging shows whether a processor is receiving data and keeping up with it, and where the data comes from, using only kubectl logs. Use it when you can’t access the processor’s metrics. It is off by default. To enable it:
Each pod then logs a processor throughput line every minute:
Each throughput line is followed by lines that list the systems that sent the processor the most data during the interval, by IP address.
Throughput logging records throughput and health values, and the IP addresses of the systems sending data. It doesn’t include account, deployment, configuration, or log content. It adds a small, continuous amount of log volume while it is on.

Troubleshooting

If the processor shows Offline or isn’t receiving data, see Troubleshooting the Traversal Processor.

Learn more

Redaction

Rule format and field filtering.

Chart changelog

What changed in each Helm chart version, with upgrade notes.

Troubleshooting

Diagnose common processor problems from its status in the web app.

Helm OCI registries

Helm’s native OCI support for chart distribution.