When your organization runs Zscaler Private Access (ZPA) for zero trust network access, you inevitably need visibility into what’s happening: who’s connecting, what they’re accessing, and whether your app connectors are healthy. ZPA’s Log Streaming Service (LSS) gets those logs out of Zscaler’s cloud — but getting them into your SIEM reliably and cheaply is a design problem worth talking about. It was not as simple as the documentation makes it out to be.

This post walks through how I built a fully Terraform-managed log streaming pipeline that delivers ZPA logs to Splunk Cloud at a fraction of the typical cost, using a dual-component architecture on AWS.

The Problem

ZPA generates four categories of logs I care about:

ZPA’s LSS delivers these as newline-delimited JSON (NDJSON) over TCP syslog. It does not support pushing directly to a Splunk HTTP Event Collector (HEC) endpoint — you need something in the middle to receive the syslog stream and relay it to HEC. The conventional answer is a Splunk Heavy Forwarder, but that comes with baggage: an m5.2xlarge instance minimum, Java dependencies, complex .conf file management, and Splunk licence implications.

I wanted something simpler.

The Architecture

The pipeline has two compute layers bridged by an internal Network Load Balancer, all deployed as a single Terraform module:

Zscaler Cloud (ZPA)
  │  pull-based, persistent HTTPS
  ▼
ZPA LSS App Connectors (t3.medium, Zscaler AMI)
  │  syslog TCP/514
  ▼
Internal Network Load Balancer
  │  distributes across targets
  ▼
SC4S Forwarder Instances (t4g.small, Amazon Linux 2, Docker)
  │  HTTPS, HEC format
  ▼
Splunk Cloud

Each layer runs in its own Auto Scaling Group within a dedicated VPC. The ZPA connectors receive logs from Zscaler’s cloud, emit them as syslog, and the SC4S containers convert and push them to Splunk HEC. Let’s break down why each piece exists and the decisions behind it.

SC4S Over Heavy Forwarder

The single biggest design decision was choosing https://splunk.github.io/splunk-connect-for-syslog/main/ over a Splunk Heavy Forwarder. SC4S is a containerised syslog-ng deployment maintained by Splunk with pre-built vendor integrations — including native Zscaler ZPA log parsing.

The cost difference is stark:

Solution: Splunk Heavy Forwarder

• Instance Type: m5.2xlarge (8 vCPU, 32GB RAM)
• Monthly Cost (approx.): ~$280 USD

Solution: SC4S

• Instance Type: t4g.small (2 vCPU, 2GB RAM)
• Monthly Cost (approx.): ~$15 USD

That’s roughly a 95% reduction in compute cost for the relay layer.

The heavy forwarder’s capabilities — index-time field extractions, SPL-based transformations, complex routing — are genuine features. I just don’t need any of them. Our use case is: receive JSON, forward to HEC. SC4S does exactly that, and its built-in Zscaler parser handles sourcetype assignment and field normalisation without custom configuration.

The operational difference matters too. Upgrading a Heavy Forwarder means planning a Splunk version upgrade with all the ceremony that entails. Upgrading SC4S means pulling a new container image.

The Dual-ASG Module

Everything deploys through a single Terraform module (modules/log_streaming_app_connector/) that creates both compute layers and the networking betIen them.

ZPA Connector Layer

The ZPA connectors use Zscaler’s official AMI and enroll themselves via a provisioning key injected through user data at boot:

resource "aws_launch_template" "zpa_connector" {
  name_prefix   = "${var.name_prefix}-lt-"
  image_id      = data.aws_ami.zpa_connector.id
  instance_type = var.instance_type  # t3.medium

  metadata_options {
    http_tokens = "required"  # IMDSv2
  }

  user_data = base64encode(templatefile("${path.module}/app_connector_user_data.sh", {
    zpa_provisioning_key = var.zpa_provisioning_key
  }))
}

The ASG runs with min_size = 1, max_size = 5, and rolling instance refresh at 50% minimum healthy — enough headroom to absorb a connector replacement without dropping logs. Since the ZPA app connector is only forwarding logs, it does not need multiple instances; however, having it in an auto-scaling group means that it can scale out as needed to handle the load. In addition, having a desired_capacity = 1 means that the app connector is self-healing in the event of hardware failure (what AWS called “EC2 retirement”).

SC4S Forwarder Layer

The forwarders run on t4g.small instances — ARM64 Graviton processors, chosen purely for cost efficiency. SC4S runs as a Docker container managed by systemd:

ExecStart=/usr/bin/docker run \
  --name sc4s \
  --env-file /opt/sc4s/env_file \
  -p 514:514/tcp \
  -p 514:514/udp \
  -p 8080:8080/tcp \
  -v /opt/sc4s/local:/etc/syslog-ng/conf.d/local:z \
  -v /opt/sc4s/archive:/var/lib/syslog-ng/archive:z \
  -v /opt/sc4s/tls:/etc/syslog-ng/tls:z \
  ghcr.io/splunk/splunk-connect-for-syslog/container:latest

The entire SC4S configuration is environment variables and a single CSV file. Compare that to a Heavy Forwarder’s inputs.conf, outputs.conf, props.conf, and transforms.conf.

One subtlety worth calling out: SC4S defaults to routing all syslog traffic to Splunk’s netproxy index, which will silently reject events if that index doesn’t exist in your environment. A one-line override in splunk_metadata.csv fixes this:

zscaler_lss,index,zscaler_zpa

This routes all Zscaler LSS events to a zscaler_zpa index. It’s the kind of thing that costs you hours of debugging if you don’t know about it — the logs appear to flow through SC4S successfully, but nothing shows up in Splunk.

The NLB in Between

An internal Network Load Balancer (NLB) bridges the two layers. All four ZPA log types flow through a single port (514) — SC4S auto-detects the log type from the JSON content.

resource "aws_lb" "forwarder" {
  name               = "${var.name_prefix}-fwd-nlb"
  internal           = true
  load_balancer_type = "network"
  subnets            = aws_subnet.public[*].id
  security_groups    = [aws_security_group.nlb.id]

  enable_cross_zone_load_balancing = true
}

Health checks target port 8080, where SC4S exposes a health endpoint. The target group uses preserve_client_ip = true so SC4S can see the originating connector IP in its logs, which helps with debugging.

The NLB is present here because the SC4S servers exist in an ASG. This is done for the purposes of supporting scale-out (in the event there is a lot of log activity) and for self-healing (again, for the eventual hardware failure). Because the servers are in an ASG, their IP addresses will change, so putting an NLB in front of it means that the ZPA app connector has a static domain name to send logs to in its configuration.

No NAT Gateway

ZPA uses a pull-based model: the app connectors maintain persistent outbound HTTPS connections to Zscaler’s cloud. Zscaler pushes log data down those existing connections. The connectors never need to accept inbound traffic.

This means the connectors can live in public subnets with an Internet Gateway — no NAT Gateway required. The security group is egress-only:

# Primary channel — Zscaler cloud communication
resource "aws_security_group_rule" "zpa_connector_https" {
  type        = "egress"
  from_port   = 443
  to_port     = 443
  protocol    = "tcp"
  cidr_blocks = ["0.0.0.0/0"]
  security_group_id = aws_security_group.zpa_connector.id
}

# Certificate revocation / OCSP checks
resource "aws_security_group_rule" "zpa_connector_http" {
  type        = "egress"
  from_port   = 80
  to_port     = 80
  protocol    = "tcp"
  cidr_blocks = ["0.0.0.0/0"]
  security_group_id = aws_security_group.zpa_connector.id
}

# Syslog to forwarders via NLB
resource "aws_security_group_rule" "zpa_connector_to_nlb" {
  type                     = "egress"
  from_port                = 514
  to_port                  = 514
  protocol                 = "tcp"
  security_group_id        = aws_security_group.zpa_connector.id
  source_security_group_id = aws_security_group.nlb.id
}

No inbound rules at all. The only lateral traffic is egress on port 514 to the internal NLB, scoped to the NLB’s security group. Dropping the NAT Gateway saves ~$30-45 USD/month per region — not transformative on its own, but it adds up alongside the SC4S savings, and it’s one less component to monitor.

Managing ZPA Configuration as Code

One of the less obvious wins is that the ZPA-side configuration — the LSS controllers themselves — is fully managed in Terraform alongside the AWS infrastructure. The zpa_lss_config_controller resource wires the Zscaler cloud to the AWS-side NLB:

resource "zpa_lss_config_controller" "user_activity" {
  config {
    name        = "Splunk User Activity"
    description = "Sends events to Splunk"
    enabled     = true

    lss_host        = module.site_us_Ist_2.log_streaming_nlb_dns_name
    lss_port        = "514"
    source_log_type = "zpn_trans_log"
    format          = data.zpa_lss_config_log_type_formats.zpn_trans_log.json
  }
  connector_groups {
    id = [zpa_app_connector_group.usw2_log_streaming.id]
  }
}

Four of these controllers exist — one per log type — all pointing to the same NLB endpoint. The lss_host value is an output from the site module, so the ZPA configuration automatically picks up the correct NLB DNS name. Change the infrastructure, and the ZPA config follows.

This matters because ZPA is a SaaS product. Without Terraform, you’d be configuring LSS controllers through the ZPA admin console, manually copying NLB DNS names betIen AWS and Zscaler’s UI. Having it in code means the entire pipeline — from Zscaler cloud to Splunk index — is reviewable, auditable, version-controlled, and reproducible. Furthermore, I believe in an internal open-source model at companies, and now anyone in the organization can submit a pull request to modify the Zscaler system without needing the read/write permissions to the Zscaler administration console.

Secrets at Boot Time

The forwarder instances need a Splunk HEC token to authenticate with Splunk Cloud. Rather than baking it into the launch template or passing it as a Terraform variable that ends up in state, I store it in AWS Secrets Manager and retrieve it during instance bootstrap (i.e. user data):

SPLUNK_TOKEN=$(aws secretsmanager get-secret-value \
  --secret-id "${splunk_hec_token_secret_arn}" \
  --query 'SecretString' \
  --output text \
  --region $(ec2-metadata --availability-zone | cut -d' ' -f2 | sed 's/[a-z]$//'))

The forwarder’s IAM role has a scoped policy granting read access to exactly one secret. The token is written to the SC4S environment file with chmod 600 and never touches disk in any other form.

Isolation from Traffic Connectors

A deliberate design choice is that log streaming connectors are completely separated from the VPN-style traffic connectors that handle user access:

• Separate VPCs — log streaming gets its own /24 network per region
• Separate app connector groups — with staggered upgrade schedules (log streaming upgrades on Tuesdays, traffic connectors on Mondays)
• Separate provisioning keys — each connector group has its own enrolment credential
• Separate server groups — global_log_streaming_traffic vs global_zpa_traffic_vpn_style

This means a bad upgrade or misconfiguration in log streaming can’t affect user access, and vice versa. The upgrade staggering is a small detail that matters operationally: if a Zscaler connector update introduces a regression, you find out on Monday with traffic connectors before it hits your logging pipeline on Tuesday.

What It Costs

Putting it all together for a single active region:

Compare that to a Heavy Forwarder approach at ~$280 USD just for the relay instance, before you add the NLB. The entire pipeline costs less than a quarter of the Heavy Forwarder instance alone.

Lessons Learned

Right-size your tooling. The instinct when you hear “forward logs to Splunk” is to reach for a Splunk forwarder. But if your use case is “receive JSON, send to HEC,” a purpose-built tool like SC4S is a better fit by every measure — cost, complexity, and operational burden.

Document your decisions. I maintain Architecture Decision Records (ADR) in the project codebase for choices like SC4S over Heavy Forwarder and the index override configuration. When someone asks “why don’t I just use a Heavy Forwarder?” six months from now, the ansIr is written down with context and cost comparisons, not trapped in someone’s memory.

Treat SaaS config as code. Managing ZPA’s LSS controllers in Terraform alongside the AWS infrastructure eliminates an entire category of copy-paste errors and configuration drift. The ZPA Terraform provider makes this possible, and it’s worth the investment in learning it.

Separate your concerns. Keeping log streaming connectors isolated from traffic connectors costs almost nothing in terms of additional infrastructure, but it buys meaningful operational safety. When your logging breaks, your users don’t notice. When your users can’t connect, your logs still flow for debugging.

Always Threat model. The app connector and the SC4S servers are each in their own ASG because that means they will self-heal in the event of hardware failure. Not all threats come from humans, sometimes they are just part of residing in a cloud environment. Making sure to look at every point in the architecture and thinking about how to prevent attack or failure of that node is key.