Prerequisites
| Requirement | Minimum | Recommended |
|---|---|---|
| Linux kernel | 5.10 | 6.1+ |
| Rust toolchain | 1.75 | latest stable |
| Redis (optional) | 6.0 | 7.x |
Building
Build thearc-gateway binary in release mode:
./target/release/arc-gateway
Systemd (bare metal or VM)
Install
Run the install script as root:- Copies
./target/release/arc-gateway→/usr/local/bin/arc-gateway - Creates
/etc/arc/ - Copies
arc.example.json→/etc/arc/arc.json - Installs
deploy/arc-gateway.service→/etc/systemd/system/ - Runs
systemctl daemon-reload && systemctl enable arc-gateway
Service unit settings
The service unit is installed at/etc/systemd/system/arc-gateway.service.
| Directive | Value | Purpose |
|---|---|---|
ExecStart | /usr/local/bin/arc-gateway --config /etc/arc/arc.json | Launch binary |
ExecReload | /bin/kill -HUP $MAINPID | Trigger hot reload |
Restart | on-failure | Auto-restart on crash |
RestartSec | 5s | Backoff between restarts |
TimeoutStopSec | 35s | Drain window before SIGKILL |
WatchdogSec | 60s | systemd watchdog interval |
LimitNOFILE | 1048576 | Max open file descriptors |
LimitNPROC | 65536 | Max processes |
NoNewPrivileges=yesPrivateTmp=yesProtectSystem=strictProtectHome=yesProtectKernelTunables=yesProtectControlGroups=yesRestrictAddressFamilies=AF_INET AF_INET6 AF_UNIX
Hot reload
After changing/etc/arc/arc.json:
Useful commands
Kubernetes
All Kubernetes manifests are underdeploy/kubernetes/. Apply them in order:
Resources created
| Resource | Details |
|---|---|
| Namespace | arc-gateway |
| ConfigMap | arc-gateway-config — contains arc.toml |
| Secret | arc-gateway-secret — contains auth-token |
| Deployment | replicas=2, rolling update |
| Service (ClusterIP) | Ports 8080, 8443, 9090 |
| Service (NodePort) | Node ports 30080 (HTTP), 30443 (HTTPS) |
| HPA | CPU-based autoscaling |
Before you apply: set the auth token
The Secret must be created with an actual token before applying the Deployment:control_plane.auth_token in the ConfigMap consistent with this value.
ConfigMap key fields to override
| Key | Default | Notes |
|---|---|---|
workers | 0 (auto-detect) | Set to your pod’s CPU limit for predictable performance |
control_plane.auth_token | __ARC_TOKEN__ | Must replace |
io_uring.sqpoll | false | Enable for bare-metal nodes |
upstreams[0].addr | 127.0.0.1:18080 | Point to your backend |
Deployment spec
| Field | Value |
|---|---|
replicas | 2 |
strategy | RollingUpdate, maxSurge=1, maxUnavailable=0 |
terminationGracePeriodSeconds | 35 |
| HTTP port | 8080 |
| HTTPS port | 8443 |
| Admin port | 9090 |
| CPU request/limit | 500m / 2000m |
| Memory request/limit | 256Mi / 1Gi |
GET /healthz on port 9090, initialDelaySeconds=5, periodSeconds=10
Readiness probe: GET /healthz on port 9090, initialDelaySeconds=3, periodSeconds=5
Graceful shutdown
ApreStop hook runs sleep 5 before the container receives SIGTERM, giving the load balancer time to drain. terminationGracePeriodSeconds=35 matches the drain window.
XDP/eBPF in Kubernetes
XDP requiresNET_ADMIN and SYS_ADMIN capabilities plus privileged: true. This is disabled by default in the manifests (commented out YAML). It is not compatible with most managed Kubernetes platforms.
Startup flow (arc-daemon)
Arc’s startup sequence when you runarc-gateway --config arc.yaml:
node.workers. If set to 0, it uses std::thread::available_parallelism().
Production checklist
Before going live:- Set
node.workersto0(auto-detect) or pin to the number of CPU cores available to the process - Set
max_connectionsto match your expected concurrent connection count - Configure
LimitNOFILEin the systemd unit (default:1048576) - Enable
SO_REUSEPORTon listeners — Arc uses it to distribute connections across worker threads without kernel lock contention - Configure upstream
pool.max_idlebased on your peak concurrent request rate - Set
control_plane.auth_tokento a strong random value if the control plane port is reachable from outside loopback - Bind the metrics server to loopback only (
observability.metrics_bind: "127.0.0.1:9090") or protect port 9090 with a firewall rule — the endpoint has no built-in authentication - Configure log rotation (
logging.output.rotation.max_sizeandmax_files) - Point Prometheus to
<host>:9090/metrics - Verify hot reload works:
kill -HUP <pid>, then checkarc_config_reload_totalincrements in/metrics
Troubleshooting
systemd: Failed to start arc-gateway.service
systemd: Failed to start arc-gateway.service
Check
journalctl -xe -u arc-gateway for the error. The most common causes are: the binary path in ExecStart does not exist, or the config file path is wrong. Verify with arc-gateway --config /etc/arc/arc.json --check (if --check is supported) or inspect startup logs.Arc restarts in a loop with io_uring: Operation not supported
Arc restarts in a loop with io_uring: Operation not supported
The kernel may not support the requested io_uring operations. Try disabling SQPOLL (
io_uring.sqpoll: false) and reducing uring_entries. Required minimum: Linux 5.10.Kubernetes pod stuck in CrashLoopBackOff
Kubernetes pod stuck in CrashLoopBackOff
Run
kubectl logs -n arc-system <pod>. Common causes are: arc-gateway-secret does not exist or auth-token key is missing; ConfigMap arc.toml references an upstream address that is not reachable at pod start; resource limits are too low and you need to increase resources.limits.memory.bind: address already in use
bind: address already in use
Another process is using the port. Check with
ss -tlnp | grep 8080. If this is a pod restart, the previous pod may not have fully terminated yet — increase terminationGracePeriodSeconds.Hot reload (SIGHUP) is not applying changes
Hot reload (SIGHUP) is not applying changes
Fields such as
node.workers, listener bind addresses, and io_uring.uring_entries require a full process restart. All others apply live. Check the arc_config_reload_total metric to confirm a reload fired.
