Prerequisites
Kernel 6.1+ is recommended for multishot accept and SQPOLL stability. Verify your kernel:
Install the binary
Option A (recommended): install latest release with install.sh
arc-gateway to /usr/local/bin/arc-gateway.
Current release assets are arc-gateway-linux-x86_64 and arc-gateway-linux-arm64.
Systemd (bare metal or VM)
Install
Install the service unit and default config files:- Binary:
/usr/local/bin/arc-gateway - Config file:
/etc/arc/arc.json
Service unit settings
The service unit is installed at/etc/systemd/system/arc-gateway.service.
Security hardening enabled by default:
NoNewPrivileges=yesPrivateTmp=yesProtectSystem=strictProtectHome=yesProtectKernelTunables=yesProtectControlGroups=yesRestrictAddressFamilies=AF_INET AF_INET6 AF_UNIX
Hot reload
After changing/etc/arc/arc.json:
Useful commands
Kubernetes
All Kubernetes manifests are underdeploy/kubernetes/. Apply them in order:
Resources created
Before you apply: set the auth token
The Secret must be created with an actual token before applying the Deployment:control_plane.auth_token in the ConfigMap consistent with this value.
ConfigMap key fields to override
Deployment spec
Liveness probe:
GET /healthz on port 9090, initialDelaySeconds=5, periodSeconds=10
Readiness probe: GET /healthz on port 9090, initialDelaySeconds=3, periodSeconds=5
Graceful shutdown
ApreStop hook runs sleep 5 before the container receives SIGTERM, giving the load balancer time to drain. terminationGracePeriodSeconds=35 matches the drain window.
XDP/eBPF in Kubernetes
XDP requiresNET_ADMIN and SYS_ADMIN capabilities plus privileged: true. This is disabled by default in the manifests (commented out YAML). It is not compatible with most managed Kubernetes platforms.
Startup flow (arc-daemon)
Arc’s startup sequence when you runarc-gateway --config arc.yaml:
node.workers. If set to 0, it uses std::thread::available_parallelism().
Production checklist
Before going live:- Set
node.workersto0(auto-detect) or pin to the number of CPU cores available to the process - Set
max_connectionsto match your expected concurrent connection count - Configure
LimitNOFILEin the systemd unit (default:1048576) - Enable
SO_REUSEPORTon listeners — Arc uses it to distribute connections across worker threads without kernel lock contention - Configure upstream
pool.max_idlebased on your peak concurrent request rate - Set
control_plane.auth_tokento a strong random value if the control plane port is reachable from outside loopback - Bind the metrics server to loopback only (
observability.metrics_bind: "127.0.0.1:9090") or protect port 9090 with a firewall rule — the endpoint has no built-in authentication - Configure log rotation (
logging.output.rotation.max_sizeandmax_files) - Point Prometheus to
<host>:9090/metrics - Verify hot reload works:
kill -HUP <pid>, then checkarc_config_reload_totalincrements in/metrics
Troubleshooting
systemd: Failed to start arc-gateway.service
systemd: Failed to start arc-gateway.service
Check
journalctl -xe -u arc-gateway for the error. The most common causes are: the binary path in ExecStart does not exist, or the config file path is wrong. Verify with arc-gateway --config /etc/arc/arc.json --check (if --check is supported) or inspect startup logs.Arc restarts in a loop with io_uring: Operation not supported
Arc restarts in a loop with io_uring: Operation not supported
The kernel may not support the requested io_uring operations. Try disabling SQPOLL (
io_uring.sqpoll: false) and reducing uring_entries. Required minimum: Linux 5.10.Kubernetes pod stuck in CrashLoopBackOff
Kubernetes pod stuck in CrashLoopBackOff
Run
kubectl logs -n arc-system <pod>. Common causes are: arc-gateway-secret does not exist or auth-token key is missing; ConfigMap arc.toml references an upstream address that is not reachable at pod start; resource limits are too low and you need to increase resources.limits.memory.bind: address already in use
bind: address already in use
Another process is using the port. Check with
ss -tlnp | grep 8080. If this is a pod restart, the previous pod may not have fully terminated yet — increase terminationGracePeriodSeconds.Hot reload (SIGHUP) is not applying changes
Hot reload (SIGHUP) is not applying changes
Fields such as
node.workers, listener bind addresses, and io_uring.uring_entries require a full process restart. All others apply live. Check the arc_config_reload_total metric to confirm a reload fired.
