HA-WAF Handbook

English translation of the public sections of the HA-WAF handbook. Pilot scope: Introduction, Installation, WAF. More parts are added as they're translated — see docs/handbook.md for the full Russian original.

Part I: Introduction

1. What is HA-WAF

HA-WAF is a single executable (Go binary) that bundles:

• HAProxy 3.0 — a high-performance reverse proxy and load balancer
• Coraza WAF — an OWASP-compliant web application firewall with full CRS v4 support
• REST API — a full-featured configuration management API
• Web UI — a built-in React admin interface

HA-WAF manages HAProxy as a child process: it generates the config, starts it, reloads it seamlessly (via SIGUSR2, without dropping connections), and talks to it through the admin socket. The WAF runs over the SPOE protocol on a local TCP port.

2. Key features

3. Architecture (brief)

Standard edition (HA-WAF):

                        ┌──────────────────────────────────────────┐
                        │              ha-waf process               │
  Client ──── :80/443 ─►│                                          │
                        │  HAProxy 3.0 (child) ◄──SPOE──► Coraza  │
  Admin  ─── :8080 ────►│         REST API + Web UI                │
                        │         SQLite / PostgreSQL               │
                        └──────────────────────────────────────────┘

On every configuration change, HA-WAF:

1. Regenerates the HAProxy config from templates
2. Atomically reloads the WAF instances (no dropped connections)
3. Performs a seamless HAProxy reload via SIGUSR2

Part II: Installation

4. Docker Compose (quick start)

git clone https://github.com/ha-waf/ha-waf
cd ha-waf
docker compose up -d

The default stack starts:

• HA-WAF — ports 80, 443, 8080 (API + UI), 9091 (Prometheus)
• otel-collector — receives OTLP, routes to VictoriaMetrics / VictoriaLogs
• VictoriaMetrics — metrics storage
• VictoriaLogs — structured log storage
• Grafana — dashboards (pre-provisioned)

Web UI: http://localhost:8080

Default credentials: admin / admin

First thing to do: change the admin password under Settings → Users.

Overriding the config

Mount your own config.yaml:

# docker-compose.override.yml
services:
  ha-waf:
    volumes:
      - ./my-config.yaml:/etc/ha-waf/config.yaml:ro

5. Kubernetes / Helm

The chart lives in the repository at deploy/ha-waf/ (not charts/ha-waf) and is published to an OCI registry:

helm install ha-waf oci://ghcr.io/ha-waf/charts --version 0.4.1 \
  --set image.tag=v0.3.0 \
  --set config.database.driver=postgres \
  --set config.database.dsn="host=pg-primary ..." \
  --set ingress.enabled=true

Or locally from source: helm install ha-waf deploy/ha-waf/.

The chart creates:

• A Deployment with a readinessProbe on /readyz
• Service objects (ha-waf-api — ClusterIP for management, ha-waf-proxy — LoadBalancer on ports 80/443, optional extra ports via service.proxy.extraPorts)
• A ServiceAccount (automountServiceAccountToken is enabled only when k8sNetworkBan.enabled)
• ClusterRole/ClusterRoleBinding — only when k8sNetworkBan.enabled (RBAC on ciliumclusterwidenetworkpolicies)
• An Ingress (classic, ingress.enabled) and/or HTTPRoute (Gateway API, gateway.enabled) for the UI/API
• PodDisruptionBudget, HorizontalPodAutoscaler
• A PersistentVolumeClaim for SQLite (or use PostgreSQL instead)
• A ConfigMap for config.yaml

For horizontal scaling, use PostgreSQL with dsn_read for a read replica. SQLite does not support multi-replica deployments.

Part V: WAF

18. WAF policy

Every site has exactly one WAF policy (1:1). The policy controls Coraza + OWASP CRS behavior.

API:

GET  /api/v1/sites/{siteId}/waf
PUT  /api/v1/sites/{siteId}/waf
{
  "enabled": true,
  "mode": "prevention",
  "paranoia_level": 1,
  "anomaly_threshold": 5,
  "response_body_check": false,
  "custom_directives": ""
}

19. WAF modes

Recommended workflow: start in detection, review WAF events in the UI, add exclusions for false positives, then switch to prevention.

20. Paranoia Level

The Paranoia Level (PL) determines how many CRS rules are active. Higher level = better protection, but more false positives.

21. Rule Exclusions

Exclusions let you disable specific CRS rules, or narrow their scope.

API:

POST /api/v1/sites/{siteId}/exclusions
{
  "rule_id": 942100,
  "type": "disable",
  "comment": "SQL injection false positive in search form"
}

Exclusion types

Example: skip checking the q field for rule 942100:

{
  "rule_id": 942100,
  "type": "exclude_arg",
  "target": "q"
}

Example: disable a rule only for /api/search:

{
  "rule_id": 942100,
  "type": "exclude_url",
  "url_pattern": "/api/search"
}

You can find the rule number in the WAF Events section — it shows the Rule ID of the triggered rule.

22. Custom Rules

Custom Rules let you write arbitrary Coraza/ModSecurity directives.

Global rules (site_id = "") apply to all sites. Per-site rules apply only to a specific site.

POST /api/v1/sites/{siteId}/rules
{
  "name": "Block suspicious UA",
  "enabled": true,
  "priority": 10100,
  "directives": "SecRule REQUEST_HEADERS:User-Agent \"@contains evil-bot\" \"id:10100,phase:1,deny,status:403,msg:'Blocked UA'\""
}

Global rules:

POST /api/v1/rules
{
  "name": "Global: block tor exit nodes",
  "priority": 9000,
  "directives": "..."
}

Custom Rule fields

Rule application order

1. SecRuleRemoveById (disable exclusions)
2. SecRuleUpdateTargetById (exclude_arg/header exclusions)
3. URL-scoped exclusions (exclude_url)
4. Global custom rules (sorted by priority)
5. Per-site custom rules (sorted by priority)
6. WAFPolicy.custom_directives (inline field)

23. WAF Events

UI section: WAF Events (available globally or via a site's tab).

Every WAF trigger records an event:

Use the Rule ID from events to build targeted exclusions.

Part III: Configuration File

6. Structure of config.yaml

server:    { ... }   # REST API server
database:  { ... }   # SQLite or PostgreSQL
haproxy:   { ... }   # HAProxy parameters
spoa:      { ... }   # Coraza SPOE address
auth:      { ... }   # JWT settings
telemetry: { ... }   # Prometheus + OTLP

Startup flag: -config /path/to/config.yaml (default /etc/ha-waf/config.yaml).

7. The server section

server:
  addr: ":8080"   # REST API and Web UI address

8. The database section

database:
  driver: sqlite
  dsn: /var/lib/ha-waf/ha-waf.db

SQLite

database:
  driver: sqlite
  dsn: /var/lib/ha-waf/ha-waf.db

PostgreSQL (single server)

database:
  driver: postgres
  dsn: "host=localhost port=5432 user=hawaf password=secret dbname=hawaf sslmode=disable"

PostgreSQL with a read replica (Patroni, Citus, RDS)

database:
  driver: postgres
  dsn:      "host=pg-primary port=5432 user=hawaf password=secret dbname=hawaf sslmode=disable"
  dsn_read: "host=pg-replica port=5433 user=hawaf password=secret dbname=hawaf sslmode=disable"

Writes always go to dsn (primary). Read-only requests (GET) go to dsn_read.

9. The haproxy section

haproxy:
  binary:      /usr/local/sbin/haproxy
  config_dir:  /etc/ha-waf/haproxy
  config_file: /etc/ha-waf/haproxy/haproxy.cfg
  pid_file:    /var/run/ha-waf/haproxy.pid
  socket_path: /var/run/ha-waf/haproxy-admin.sock

  maxconn:   50000
  ulimit_n:  200000
  ssl_options: "no-sslv3 no-tlsv10 no-tlsv11"

  http_port:  80
  https_port: 443

  ssl_cert_dir: /etc/ha-waf/certs/
  error_dir:    /etc/ha-waf/errors

  geoip_map_file: /etc/ha-waf/geoip.txt

  metrics_bind: ":8405"
  metrics_acls:
    - "127.0.0.1"
    - "192.168.0.0/16"

  internal_rate_table_port: 19999

10. The spoa section

spoa:
  addr: "127.0.0.1:9000"

Never expose SPOA externally. It's an internal protocol between HAProxy and Coraza.

11. The auth section

auth:
  jwt_secret: "your-very-secret-key-here"
  token_ttl:  "24h"

In multi-replica mode you must set the same jwt_secret on every node.

12. The telemetry section

telemetry:
  prometheus_addr: ":9091"

  otlp:
    endpoint: "otel-collector:4317"
    insecure: true
    logs:
      enabled: true
    metrics:
      enabled: true
    traces:
      enabled: true

Part IV: Sites and Backends

13. Creating an HTTP/HTTPS site

Web UI: Sites section → Add Site button.

API:

POST /api/v1/sites
Content-Type: application/json

{
  "name": "My App",
  "enabled": true,
  "mode": "http",
  "domains": ["app.example.com"],
  "domain_suffixes": [".app.example.com"],
  "http_enabled": true,
  "https_enabled": true,
  "redirect_http_to_https": true,
  "acme_enabled": true,
  "lb_algorithm": "roundrobin"
}

Site fields

Request routing

Every site gets an ACL in HAProxy. A request routes to the site if:

• The Host header exactly matches one of domains, or
• The Host header ends with one of domain_suffixes, or
• The site matched via a DomainList (domain list), or
• The site has no domains, domain_suffixes, or domain_lists — always matches (catch-all)

14. Creating a TCP site

TCP sites operate in L4 pass-through mode: HAProxy does not terminate TLS, it forwards traffic based on the SNI hostname.

POST /api/v1/sites
{
  "name": "NetBird",
  "enabled": true,
  "mode": "tcp",
  "tcp_port": 443,
  "domains": ["vpn.example.com"],
  "domain_suffixes": [],
  "lb_algorithm": "roundrobin"
}

How SNI routing works:

If a TCP port has multiple sites with different domains/domain_suffixes, HAProxy:

1. Enables tcp-request inspect-delay 5s (waits for the TLS ClientHello)
2. Reads the SNI from req.ssl_sni
3. Routes to the matching backend

If a port has only one site, SNI inspection isn't enabled (no overhead).

A site with no domains or domain_suffixes becomes the default_backend for that port.

TCP mode limitations: WAF, GeoIP, IP lists, Path Rules, and Rate Limits do not work in TCP mode — HAProxy doesn't see the HTTP content.

15. Backends

Each site can have multiple backend servers.

API:

POST /api/v1/sites/{siteId}/backends
{
  "name": "web-1",
  "address": "10.0.0.10",
  "port": 8080,
  "weight": 10,
  "enabled": true,
  "ssl_enabled": false,
  "ssl_verify_none": false,
  "ssl_sni_from_host": false,
  "send_proxy": 0
}

Backend fields

16. Load balancing algorithms

17. Domain Lists

Domain Lists let you load domain lists from an external URL and route traffic to a dedicated backend.

Use cases:

• Routing traffic for a group of client domains
• Dynamic updates (e.g. from an external management system)

POST /api/v1/sites/{siteId}/domainlists
{
  "name": "Client Domains",
  "enabled": true,
  "source_url": "https://config.example.com/domains.txt",
  "fetch_interval": 300,
  "auth_user": "user",
  "auth_password": "secret",
  "backend_id": "uuid-of-backend"
}

Domain file format

One domain per line:

client1.example.com
client2.example.com
# comments are ignored

Domain List fields

Part VI: GeoIP Filtering

24. Configuring GeoIP

A GeoIP policy lets you allow traffic only from specific countries.

API:

GET /api/v1/sites/{siteId}/geoip
PUT /api/v1/sites/{siteId}/geoip
{
  "enabled": true,
  "map_file": "/etc/ha-waf/geoip.txt",
  "allowed_countries": ["RU", "BY", "KZ"],
  "deny_status": 403,
  "add_country_header": true
}

Logic: if the IP is found in the map and its country is NOT in allowed_countries — block it. If the IP isn't found in the map — allow it (unknown IPs are never blocked).

25. GeoIP map format

The map file: one CIDR per line, space-separated from the country code:

# GeoIP map format: CIDR ISO-3166-1
1.0.0.0/24 AU
1.0.1.0/24 CN
5.8.18.0/23 RU
77.88.0.0/18 RU

HA-WAF does not download GeoIP data automatically — you provide the map yourself. Recommended sources:

• MaxMind GeoLite2 (free with registration)
• ip-api.com bulk export
• DB-IP free editions

After updating the file on disk, click Reload in the UI or call POST /api/v1/reload.

Part VII: IP Lists

26. Whitelists

A whitelist allows traffic from the given CIDRs, bypassing all checks.

POST /api/v1/sites/{siteId}/iplists
{
  "name": "Internal Networks",
  "type": "whitelist",
  "enabled": true,
  "cidrs": ["10.0.0.0/8", "192.168.0.0/16", "172.16.0.0/12"],
  "bypass_waf": true
}

If bypass_waf = false, traffic is allowed, but the WAF still inspects requests.

27. Blacklists

A blacklist blocks traffic from the given CIDRs with a 403.

POST /api/v1/sites/{siteId}/iplists
{
  "name": "Blocked IPs",
  "type": "blacklist",
  "enabled": true,
  "cidrs": ["1.2.3.4/32", "5.6.7.0/24"]
}

Check order in HAProxy:

1. Blacklist: is the IP blacklisted? → 403
2. GeoIP: is the country blocked? → 403
3. Whitelist + bypass_waf: is the IP whitelisted? → allow (skip WAF)
4. WAF check
5. Whitelist without bypass_waf: is the IP whitelisted? → allow (after WAF)

Part VIII: Rate Limiting

28. Request rate limit rules

Rate limiting uses a shared HAProxy stick-table. Limits apply per client IP.

POST /api/v1/sites/{siteId}/ratelimits
{
  "name": "Global rate limit",
  "enabled": true,
  "http_req_rate": 100,
  "conn_rate": 50,
  "bytes_in_rate": 1048576,
  "deny_status": 429
}

You can define multiple rules — they all apply (OR logic: any exceeded limit → block).

29. Path-prefix and regex filters

A rate limit can be scoped to a specific path:

POST /api/v1/sites/{siteId}/ratelimits
{
  "name": "API rate limit",
  "http_req_rate": 30,
  "path_prefixes": ["/api/", "/graphql"],
  "deny_status": 429
}

Or by regular expression:

{
  "path_regex": "^/api/v[0-9]+/",
  "http_req_rate": 20
}

path_prefixes and path_regex are mutually exclusive. If both are empty, the rule applies to all requests on the site.

Part IX: Path Rules (nginx location blocks)

30. Path Rules overview

Path Rules are the equivalent of nginx location blocks. They let you configure special behavior for specific URL paths on an L7 site:

• Proxy to a different upstream (e.g. API → a different service)
• Return a static response
• Perform a redirect
• Apply a rate limit only to this path
• Disable WAF rules for this path
• Configure a WebSocket or gRPC tunnel
• Modify request/response headers

UI tab: Site → Path Rules

31. Path match type

POST /api/v1/sites/{siteId}/pathrules
{
  "name": "API Proxy",
  "enabled": true,
  "path": "/api/",
  "match_type": "prefix",
  "priority": 10,
  "action": "proxy",
  ...
}

32. Actions

33. Proxying (proxy)

{
  "action": "proxy",
  "upstream_addr": "10.0.0.20:3000",
  "upstream_scheme": "http",
  "websocket": false,
  "grpc": false,
  "ssl_enabled": false,
  "ssl_verify_none": false,
  "ssl_sni": "",
  "timeout_connect": 5,
  "timeout_server": 60,
  "lb_algorithm": "roundrobin",
  "retries": 2,
  "check_enabled": false
}

34. Static response (return)

{
  "action": "return",
  "return_status": 200,
  "return_type": "application/json",
  "return_body": "{\"status\":\"ok\"}"
}

Use cases: health-check endpoints, stubs for temporarily disabled routes, static JSON responses.

35. Redirect

{
  "action": "redirect",
  "redirect_url": "https://new-site.example.com/path",
  "redirect_code": 301
}

36. Conditions

Conditions let a Path Rule apply only when additional conditions (beyond the path match) are met.

{
  "conditions": [
    {
      "type": "header",
      "name": "X-Internal",
      "value": "true",
      "negate": false
    },
    {
      "type": "src_cidr",
      "value": "10.0.0.0/8",
      "negate": false
    }
  ]
}

All conditions are combined with AND logic — the rule applies only if every condition is met.

Condition types

The negate: true field inverts the condition (NOT).

37. Rate limit for a Path Rule

{
  "rate_limit": {
    "enabled": true,
    "http_req_rate": 50,
    "conn_rate": 20,
    "bytes_in_rate": 524288,
    "window": 15,
    "deny_status": 429
  }
}

A Path Rule's rate limit uses a separate counter (sc2), independent of the site's global rate limit (sc0).

38. WAF exclusions for a Path Rule

{
  "waf_exclude_ids": [942100, 942200, 941100]
}

A list of CRS rule IDs disabled only for requests matching this Path Rule. Generates a Coraza SecRule with REQUEST_URI @beginsWith {path}.

Use case: API endpoints where the request body contains SQL-like data or a specific format (e.g. GraphQL queries).

39. Header modification

{
  "request_headers": [
    {"action": "set",  "name": "X-Forwarded-Prefix", "value": "/api"},
    {"action": "del",  "name": "X-Internal-Token"}
  ],
  "response_headers": [
    {"action": "set",  "name": "X-Frame-Options", "value": "DENY"},
    {"action": "add",  "name": "X-Content-Type-Options", "value": "nosniff"}
  ]
}

40. WebSocket

{
  "action": "proxy",
  "path": "/ws",
  "match_type": "prefix",
  "upstream_addr": "10.0.0.5:3001",
  "websocket": true,
  "timeout_tunnel": 3600
}

With websocket: true, HAProxy enables timeout tunnel to support long-lived connections. No special proto configuration is needed — the WebSocket upgrade is transparent over HTTP/1.1.

timeout_tunnel is the tunnel's idle timeout in seconds (default 3600 = 1 hour). Set 0 for an unlimited tunnel.

41. gRPC

{
  "action": "proxy",
  "path": "/grpc.",
  "match_type": "prefix",
  "upstream_addr": "10.0.0.5:9090",
  "upstream_scheme": "h2c",
  "grpc": true,
  "timeout_tunnel": 3600
}

With grpc: true or upstream_scheme: "h2"/"h2c", HAProxy adds proto h2 to the server line, enabling gRPC over cleartext HTTP/2 (h2c).

For gRPC with TLS to the upstream, use upstream_scheme: "https" + ssl_enabled: true.

42. Multiple upstream servers

Instead of upstream_addr you can provide a list of servers (servers):

{
  "action": "proxy",
  "lb_algorithm": "leastconn",
  "servers": [
    {"name": "api-1", "address": "10.0.0.10", "port": 3000, "weight": 10},
    {"name": "api-2", "address": "10.0.0.11", "port": 3000, "weight": 10},
    {"name": "api-3", "address": "10.0.0.12", "port": 3000, "weight": 5}
  ]
}

Server fields within a Path Rule mirror a regular backend's fields, plus SSL and PROXY protocol options.

43. Priority and processing order

Path Rules are sorted by the priority field (lower = higher priority). If several match, the one with the lowest priority wins.

Recommended ranges:

• 1–9 — critical routes (blocks, health checks)
• 10–99 — main routes
• 100–999 — fallback routes

Order in the HAProxy config:

1. All http-request return (static responses)
2. All http-request redirect
3. All http-request track-sc2 (rate limits)
4. use_backend be_ha_waf_api if acme_challenge
5. All use_backend be_pr_* (path rule proxy)
6. use_backend be_* (site's main backend)

Part X: TLS and Certificates

44. Certificate upload

UI: Certificates section → Upload PEM button

Accepts a PEM file containing the certificate (chain) + private key:

-----BEGIN CERTIFICATE-----
...
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
... (intermediate CA)
-----END CERTIFICATE-----
-----BEGIN PRIVATE KEY-----
...
-----END PRIVATE KEY-----

API:

POST /api/v1/certs
Content-Type: application/json
{
  "name": "example.com.pem",
  "pem_data": "-----BEGIN CERTIFICATE-----\n..."
}

The certificate is saved to the database and written to ssl_cert_dir on the next reload. HAProxy loads every .pem file from that directory.

45. Bulk archive import

Upload a ZIP or TAR.GZ archive containing many certificates. HA-WAF automatically:

• Finds .crt + .key pairs (matched by CN)
• Recognizes self-contained .pem bundles
• Imports everything into the database

UI: Certificates → Import Archive

API:

POST /api/v1/certs/upload-archive?auto_renew=true
Content-Type: application/zip

<body — raw archive bytes, not multipart>

The import runs asynchronously; the endpoint immediately responds 202 Accepted with {"job_id": "..."}. Check status via:

GET /api/v1/certs/upload/{jobId}

The response includes: status (pending/running/done/failed), done/total, and an items list with the result for each certificate.

46. ACME accounts — multi-account model

HA-WAF supports an unlimited number of ACME accounts. This lets you:

• Use different Let's Encrypt accounts for different teams/clients
• Mix HTTP-01 and DNS-01 challenges within a single installation
• Connect different DNS providers for wildcard certificates

One account is marked default — it's used when issuing a certificate without an explicit account.

Managing accounts via the UI

Settings → ACME Accounts — the account management page. Direct link: /#/settings?tab=acme.

Available actions:

• Add account — create a new account (HTTP-01 or DNS-01)
• ✏️ Edit — change name, email, challenge, credentials
• ★ Set as default — make this the default account
• 🗑 Delete — delete (you can't delete the default or the last remaining account)

Creating an account (API)

POST /api/v1/acme/accounts
{
  "name": "Production LE",
  "email": "ops@example.com",
  "challenge_type": "http01",
  "staging": false,
  "is_default": true
}

Response: 201 Created with the account object (without the private key).

Staging URL: https://acme-staging-v02.api.letsencrypt.org/directory

Fetching and updating an account

GET    /api/v1/acme/accounts           # list all accounts
GET    /api/v1/acme/accounts/{id}      # a single account
PUT    /api/v1/acme/accounts/{id}      # update
DELETE /api/v1/acme/accounts/{id}      # delete
POST   /api/v1/acme/accounts/{id}/set-default  # make default

PUT semantics:

• Changing name does not reset the ACME registration (saves Let's Encrypt API budget)
• Changing email, directory_url, or staging resets the registration; on the next issuance, HA-WAF re-registers the account with the ACME server
• DNS credentials: an empty {} in the request body means "don't change"; to update individual keys, send only the ones that changed

Setting the default account

POST /api/v1/acme/accounts/{id}/set-default

Response: the updated account object with "is_default": true. The previous default is unset atomically.

47. Issuing a certificate via ACME

Step 1: Make sure an account is configured

UI: Certificates — the ACME Accounts section at the top of the page shows the current default account. The Manage button leads to Settings → ACME Accounts.

Step 2: Make sure the site is reachable (for HTTP-01)

• Port 80 must be publicly reachable
• HA-WAF automatically responds on /.well-known/acme-challenge/*
• If the backend also requests its own LE certificate (e.g. Caddy), make sure it accepts the PROXY protocol — HA-WAF forwards the challenge request with a PROXY v1 header if the backend is configured with send_proxy

Step 3: Issue the certificate

UI: Issue via ACME button → enter domains → pick an account → Issue.

POST /api/v1/acme/issue
{
  "cert_name": "example.com.pem",
  "domains": ["example.com", "www.example.com"],
  "auto_renew": true,
  "account_id": ""
}

Response: 202 Accepted with an ACMEJob object (see §50). You can poll issuance status:

GET /api/v1/acme/jobs/{jobId}

Issuing for a whole site (all of the site's domains automatically):

POST /api/v1/sites/{siteId}/acme/issue

48. DNS-01 challenge

DNS-01 lets you issue wildcard certificates and works without public access to port 80.

Supported DNS providers

Creating a DNS-01 account

POST /api/v1/acme/accounts
{
  "name": "Wildcard via Cloudflare",
  "email": "admin@example.com",
  "challenge_type": "dns01",
  "dns_provider": "cloudflare",
  "dns_credentials": {
    "CF_API_TOKEN": "your-cloudflare-token"
  }
}

Security: credentials are stored in the database. The UI never returns saved values in GET requests — when editing an account, enter only the keys that changed.

Issuing a wildcard certificate

POST /api/v1/acme/issue
{
  "cert_name": "example.com-wildcard.pem",
  "domains": ["*.example.com", "example.com"],
  "auto_renew": true,
  "account_id": "<dns01 account id>"
}

Yandex Cloud: preparing the key

# Create a service account with the dns.editor role
# Download the JSON key and encode it:
cat key.json | base64 -w 0
# Paste the resulting string into YANDEX_CLOUD_IAM_TOKEN

49. Certificate auto-renewal

Every 12 hours, HA-WAF checks all certificates flagged auto_renew: true:

• If fewer than 30 days remain until expiry — it triggers renewal via ACME
• The same account used for the original issuance is used again
• If that account was deleted — the current default is used instead
• After renewal — the new certificate is applied automatically (HAProxy reload)

Enable/disable auto_renew for a certificate:

PATCH /api/v1/certs/{name}
{"auto_renew": true}

In the UI — the 🔄 button in the certificate's row (blue = enabled, gray = disabled).

Manual renewal:

POST /api/v1/acme/renew/{name}

50. Asynchronous issuance jobs (JobRunner)

Issuing and renewing certificates (/acme/issue, /sites/{siteId}/acme/issue, /acme/renew/{name}) run asynchronously: the endpoint immediately responds 202 Accepted with a job object, and the actual work with the ACME server happens in the background (internal/acme/jobs.go, JobRunner). This protects the API request from timing out on a slow DNS-01 challenge or an unreachable ACME server.

GET /api/v1/acme/jobs           # all jobs in memory, newest first
GET /api/v1/acme/jobs/{jobId}   # status of a specific job

Statuses: pending → running → done | failed. Types: issue (manual issuance), renew (renewal), site_issue (bulk issuance for all of a site's domains). JobRunner won't let a second job run for the same cert_name/site in parallel — a repeat request returns 409 Conflict.

The same JobRunner also drives the background auto-renewal (§49) — manual API calls and the automatic timer share the same queueing mechanism.

51. Tracking issuance failures (Cert Failures)

If an issuance/renewal job fails, a record is kept separately from the job itself — with an attempt history and configurable auto-retry (internal/certimport, the cert_failures table).

GET    /api/v1/acme/failures                  # list
GET    /api/v1/acme/failures/{id}
PATCH  /api/v1/acme/failures/{id}             # change auto_retry / next_retry_at
DELETE /api/v1/acme/failures/{id}
POST   /api/v1/acme/failures/{id}/retry       # retry immediately

Record statuses: pending (awaiting retry) → resolved (certificate successfully issued) | cancelled (retries disabled manually). The error_code field classifies the cause (e.g. dns_challenge_failed, rate_limited) — used by the UI to group errors under Certificates → Failures.

52. Per-site certificate directory override

By default HAProxy loads certificates from the global ssl_cert_dir. For a specific site you can point it elsewhere:

PUT /api/v1/sites/{siteId}
{
  "ssl_cert_dir": "/etc/ha-waf/certs/tenant-a/"
}

This lets you isolate certificates between different tenants.

Part XI: Users and API Keys

53. User management

UI: Settings → Users section

GET    /api/v1/users              # list users
POST   /api/v1/users              # create
PUT    /api/v1/users/{id}         # update (role and/or password)
DELETE /api/v1/users/{id}         # delete
POST   /api/v1/auth/password      # change YOUR OWN password (self-service)

POST /api/v1/users
{
  "username": "operator",
  "password": "secure-password",
  "role": "viewer"
}

An administrator changes another user's password via PUT /api/v1/users/{id} with the password field (there's no separate endpoint for it).

Users pending approval. Accounts created through OIDC self-registration (see Part XII: OIDC / SSO) get a pending status and can't sign in until an administrator assigns them a role:

GET /api/v1/users/pending           # list of pending users
PUT /api/v1/users/{id}/role         # assign a role → moves to active
{ "role": "viewer" }

54. Roles and permissions

55. API keys

API keys are meant for automation (scripts, CI/CD, monitoring).

Create a key:

POST /api/v1/users/{userId}/api-keys
{
  "name": "Deployment script",
  "expires_at": "2027-01-01T00:00:00Z"
}

The response includes the raw key — shown once and never stored anywhere:

{
  "id": "uuid",
  "name": "Deployment script",
  "key": "hwaf_a3f8d2...",
  "expires_at": "2027-01-01T00:00:00Z"
}

Usage:

X-API-Key: hwaf_a3f8d2...

List a user's keys:

GET /api/v1/users/{userId}/api-keys

Delete a key:

DELETE /api/v1/users/{userId}/api-keys/{keyId}

Part XII: OIDC / SSO

Enterprise feature: requires a license with the oidc feature (see Part XIII: Licensing). Without a license, this part doesn't apply — the public login routes aren't registered, and sign-in stays username/password only.

56. Configuring a provider

The provider is configured via Settings → SSO in the UI, or directly through the API (admin role):

POST /api/v1/auth/providers
{
  "name": "Corporate SSO",
  "client_id": "ha-waf",
  "client_secret": "...",
  "discovery_url": "https://idp.example.com/.well-known/openid-configuration",
  "enabled": true
}

discovery_url must point to the provider's OpenID Connect discovery document (Keycloak, Authentik, Okta, Azure AD, etc. — any standard OIDC IdP). client_secret is encrypted before storage with the same key used for domain lists' source_pass, and is never returned via the API. Updating a provider without including client_secret in the request body keeps the existing secret.

57. Sign-in flow

1. The user sees the provider's button on the login page (the list comes from GET /auth/oidc/providers — a public route that returns only {id, name} for enabled providers).
2. Navigating to GET /auth/oidc/{providerId}/login — HA-WAF generates a PKCE code verifier/challenge and state, stores them in an HMAC-signed cookie (key derived from the JWT secret), and redirects to the provider's authorization endpoint.
3. After sign-in, the provider redirects back to GET /auth/oidc/{providerId}/callback — HA-WAF exchanges the code for tokens, validates the id_token (issuer, audience, signature), and upserts the user by sub/email/name.
4. If the user is new, they're created with pending status and sign-in is rejected (redirect to /login?pending=true) until an administrator assigns a role (see §53). An already-approved user gets a normal HA-WAF JWT and cookie session.
5. The active session is stored in the OIDCSession table (keyed by the token's jti) — needed for backchannel logout.

Both public routes (/auth/oidc/providers — unrated, /auth/oidc/{id}/login — 10 requests/min per IP) live outside /api/v1, unlike the rest of the API.

58. Backchannel Logout

HA-WAF implements OpenID Connect Back-Channel Logout 1.0: if the IdP initiates a logout (e.g. an IdP administrator force-ends a user's session), the provider sends a POST with a logout_token to:

POST /auth/backchannel-logout    (no auth, 5 requests/min per IP, outside /api/v1)

HA-WAF extracts the issuer from the token (without full validation — just to find the provider; the signature is verified separately), finds the matching OIDCProvider by issuer, and revokes all of that user's active OIDCSession records. A revoked session stops passing the JWT middleware on the next request, even if the JWT itself hasn't expired yet. If no provider is found, the endpoint still responds 200 OK, so as not to leak the list of configured providers.

Part XIII: Licensing

59. Licensing model

A license is an offline-verifiable JWT token (Ed25519 signature) that doesn't require contacting an external activation server. The public key used to verify the signature is embedded in the HA-WAF binary. Set it in config.yaml:

license_key: "eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9...."

or via the HAWAF_LICENSE_KEY environment variable (handy for a k8s Secret — keeps the key out of the ConfigMap in plaintext).

Without a key (or with an empty license_key), HA-WAF runs in Community mode — all core features (WAF, sites, certificates, rate limiting, GeoIP, k8sban, etc.) are available without restriction; a license is only needed for select enterprise features.

60. Status and features

GET /api/v1/license

{
  "status": "valid",
  "customer": "Acme Corp",
  "features": ["oidc"],
  "expires_at": "2027-01-01T00:00:00Z",
  "days_until_expiry": 180
}

As of this writing, the only licensable feature is oidc (SSO providers, see Part XII). Calling its endpoints without a valid license returns 402 Payment Required.

Part XIV: k8sban — Network-level Bans via Cilium

Kubernetes with the Cilium CNI only. Not available on self-hosted (Docker Compose) installs.

61. Why it exists

The built-in "WAF IP Ban" (config.haproxy.waf_ban_enabled) bans an IP at the HAProxy level: after N WAF blocks, the IP gets 429 on every subsequent request. This works, but every banned request still reaches HAProxy — spending a TCP/TLS handshake and a request-processing cycle. k8sban mirrors the same bans into a CiliumClusterwideNetworkPolicy, so Cilium drops the banned IP's traffic at the network level (eBPF), before it ever reaches the HA-WAF pod — HAProxy never sees the packet at all.

62. Enabling it

Two independent switches — both are required:

1. Chart-level (deploy-time, grants RBAC):

   # values.yaml
   k8sNetworkBan:
     enabled: true   # default true since chart version 0.4.1
   
   config:
     k8s_network_ban:
       enabled: true
       policy_name: ha-waf-ip-ban   # optional
       poll_interval: "10s"          # optional
       ports: ["80", "443"]          # optional
   

   Enabling this creates a ClusterRole/ClusterRoleBinding with get/list/watch/create/update/patch on ciliumclusterwidenetworkpolicies (a cluster-scoped resource) and mounts the pod's ServiceAccount token — this is a real privilege expansion, see SECURITY.md.
2. Runtime-level (the same toggle used for the L7 ban): config.haproxy.waf_ban_enabled: true — without it k8sban won't ban anything, even with the chart flag enabled.

63. How it works

Every poll_interval (default 10s), each HA-WAF pod independently:

1. Reads its local HAProxy stick-table (show table ha_waf_ban_table via the admin socket) — the list of IPs whose WAF-block counter has reached the threshold.
2. Merges those IPs into a shared CiliumClusterwideNetworkPolicy object named policy_name (default ha-waf-ip-ban), setting spec.ingressDeny.fromCIDR.
3. Simultaneously sets spec.ingress with fromEntities: ["all"] on the same ports — without this rule, ingressDeny would put the pod into default-deny mode for all inbound traffic, not just the banned IPs (a real outage, caught and fixed while building the feature — see the project's issue tracker).

Why there's no leader election. Each pod's stick-table is local and isn't synchronized between replicas (the peers section isn't configured in HAProxy). Instead of figuring out which pod "owns" a given ban, each entry's lifetime is stored in a JSON annotation on the object itself (ha-waf.io/ban-expiry: {"1.2.3.4/32": "2026-07-28T22:00:00Z"}). On every cycle, any pod: adds its current bans with a fresh expiry time, and removes any entry from the shared list whose expiry has passed — regardless of which pod originally added it. After a pod crashes or is recreated, its bans simply stop being renewed and get "washed out" by the other replicas the next time the TTL expires.

View the current bans:

kubectl get ciliumclusterwidenetworkpolicies.cilium.io ha-waf-ip-ban -o yaml

What this doesn't protect against. The drop happens on the node the packet has already reached (before userspace HAProxy) — this relieves load on the pod/node, but doesn't help against uplink saturation during a volumetric attack (that needs an external anti-DDoS layer — Cloud Armor/Shield or RTBH at a transit provider, outside HA-WAF's scope).

Part XV: Metrics and Telemetry

64. Prometheus metrics

HA-WAF exports metrics through two endpoints:

Key HAProxy metrics

To filter by a specific site, use the label:

proxy="be_<site_id_with_underscores>"

Example PromQL for requests to the site my-app (UUID abc-def):

sum(rate(haproxy_backend_http_responses_total{proxy="be_abc_def"}[5m]))

WAF metric

Per-domain (sni) breakdowns are not metric labels of waf_*_total — labels are intentionally limited (site_id/mode) to avoid unbounded cardinality and service memory risk. Build per-domain graphs from logs: HAProxy access logs carry sni (all requests), the WAF-event log carries sni for blocked requests — at the log→metric transformation layer (Loki/Prometheus recording rules, etc.).

65. OpenTelemetry (OTLP)

HA-WAF supports sending traces, metrics, and logs via OTLP gRPC.

telemetry:
  otlp:
    endpoint: "otel-collector:4317"
    insecure: true
    logs:
      enabled: true
    metrics:
      enabled: true
    traces:
      enabled: true

Additional OTLP exporters can be configured via the UI (Telemetry section) or the API:

POST /api/v1/telemetry/exporters
{
  "name": "Main collector",
  "type": "otlp",
  "enabled": true,
  "endpoint": "collector.monitoring.svc:4317",
  "insecure": true,
  "signals": ["logs", "metrics", "traces"]
}

66. The metrics dashboard in the UI

UI section: Activity

Shows PromQL charts for the selected time range:

• Requests/sec (inbound)
• HTTP Error Rate (4xx, 5xx)
• Active Connections
• Bandwidth (bytes in/out)
• WAF blocks/sec
• P95 Response Time

Site filter: picking a specific site from the dropdown hidden in the dashboard header automatically adds {proxy="be_<id>"} to every PromQL query.

The dashboard needs Prometheus/VictoriaMetrics with HAProxy metrics on :8405.

Part XVI: Config Revisions

67. What a revision is

On every reload, HA-WAF takes a configuration snapshot:

• A full JSON dump of every database entity
• The generated haproxy.cfg text
• The time it was applied and the user who applied it

UI: Config → Revisions section

68. Viewing and rolling back

List revisions:

GET /api/v1/config/revisions

View a specific revision:

GET /api/v1/config/revisions/{id}

Diff between the current DB state and the latest revision:

GET /api/v1/config/diff

Returns a JSON diff with added, removed, changed fields for each entity.

Roll back to a revision:

POST /api/v1/config/revisions/{id}/rollback

Restores the database state from the revision and performs a reload. Creates a new revision.

Unsaved changes status:

GET /api/v1/config/status
# {"has_pending_changes": true}

Discard unsaved changes:

POST /api/v1/config/discard

69. Config export and import

Export (a full config backup):

GET /api/v1/export

Returns JSON with every entity. Use it for backups and migrating config between instances.

Import:

POST /api/v1/import
Content-Type: application/json

{"sites": [...], "certs": [...], ...}

Import replaces the existing configuration. Export first.

Part XVII: Error Pages

70. Custom error pages

HA-WAF lets you set custom HTML pages for standard HTTP errors.

UI: Error Pages section

Supported codes: 400, 403, 404, 429, 500, 502, 503, 504

GET /api/v1/error-pages                     # list
GET /api/v1/error-pages/{code}              # get a page
PUT /api/v1/error-pages/{code}              # save
DELETE /api/v1/error-pages/{code}           # reset to default

PUT /api/v1/error-pages/403
{
  "code": 403,
  "content": "<!DOCTYPE html><html>...</html>"
}

Pages are stored in error_dir (/etc/ha-waf/errors/) and wired into HAProxy via the errorfile directive.

Part XVIII: System Settings

71. System Config via the UI

UI: Settings → System section

Lets you override config.yaml parameters through the web interface without restarting the service. Changes are saved to the database and applied on the next reload.

Manageable parameters:

• HAProxy maxconn, timeouts, SSL options
• HTTP/HTTPS ports
• SPOA address
• Metrics bind address
• GeoIP parameters

GET /api/v1/config
PUT /api/v1/config
{
  "maxconn": 100000,
  "timeout_connect": 5,
  "timeout_client": 30,
  "timeout_server": 60
}

Appendix A: REST API

The full reference with every endpoint, request/response body field, and example lives in docs/api.md. There's no point duplicating it here — that copy would drift out of sync with the code just like the rest of the documentation did; one source of truth is easier to keep than two.

Authentication — three interchangeable methods: Authorization: Bearer <JWT>, the auth_token cookie (+ X-CSRF-Token for unsafe methods), X-API-Key: hwaf_<key>. Details in docs/api.md → Authentication.

Appendix B: config.yaml Reference

# ──────────────────────────────────────────────
# REST API server
# ──────────────────────────────────────────────
server:
  addr: ":8080"                          # API + Web UI bind address

# ──────────────────────────────────────────────
# Database
# ──────────────────────────────────────────────
database:
  driver: sqlite                         # sqlite | postgres
  dsn: /var/lib/ha-waf/ha-waf.db        # path (sqlite) or connstring (postgres)
  dsn_read: ""                           # read-only replica (postgres only)

# ──────────────────────────────────────────────
# HAProxy
# ──────────────────────────────────────────────
haproxy:
  binary:      /usr/local/sbin/haproxy
  config_dir:  /etc/ha-waf/haproxy
  config_file: /etc/ha-waf/haproxy/haproxy.cfg
  pid_file:    /var/run/ha-waf/haproxy.pid
  socket_path: /var/run/ha-waf/haproxy-admin.sock

  # Performance
  maxconn:   50000
  ulimit_n:  200000

  # TLS
  ssl_options: "no-sslv3 no-tlsv10 no-tlsv11"
  ssl_cert_dir: /etc/ha-waf/certs/

  # Ports
  http_port:  80
  https_port: 443

  # Paths
  error_dir:      /etc/ha-waf/errors
  geoip_map_file: /etc/ha-waf/geoip.txt

  # Prometheus endpoint (HAProxy native)
  metrics_bind: ":8405"
  metrics_acls:
    - "127.0.0.1"

  # Internal rate-limit stick-table
  internal_rate_table_port: 19999

# ──────────────────────────────────────────────
# Coraza WAF SPOE
# ──────────────────────────────────────────────
spoa:
  addr: "127.0.0.1:9000"

# ──────────────────────────────────────────────
# Authentication
# ──────────────────────────────────────────────
auth:
  jwt_secret: ""          # empty = random on startup (set explicitly in production)
  token_ttl: "24h"        # Go duration: 1h, 24h, 168h

# ──────────────────────────────────────────────
# Telemetry
# ──────────────────────────────────────────────
telemetry:
  prometheus_addr: ":9091"   # empty = disabled

  # otlp:
  #   endpoint: "otel-collector:4317"
  #   insecure: true
  #   logs:    { enabled: true }
  #   metrics: { enabled: true }
  #   traces:  { enabled: true }

# ──────────────────────────────────────────────
# k8sban — network-level bans via Cilium (k8s only, see Part XIV)
# ──────────────────────────────────────────────
k8s_network_ban:
  enabled: false          # false on self-hosted (Docker Compose) — this section is ignored
  policy_name: ha-waf-ip-ban
  poll_interval: "10s"
  ports: ["80", "443"]

# ──────────────────────────────────────────────
# Top-level fields
# ──────────────────────────────────────────────
license_key: ""            # see Part XIII; can also be set via the HAWAF_LICENSE_KEY env var
log_level: info             # debug | info | warn | error

Appendix C: Environment Variables

HA-WAF reads its main config only from a file (the -config flag) — environment variables are not templated into config.yaml. The one exception:

Startup flag:

ha-waf -config /etc/ha-waf/config.yaml

Version check:

ha-waf -version

Health endpoints:

GET /healthz   → 200 immediately after startup (liveness)
GET /readyz    → 200 once fully initialized (readiness)

Part XX: Bot Protection (Anti-Bot)

Chapter numbering in the English mirror continues from the Russian original's content (this part is chapter-for-chapter translation of Russian Part XX, chapters 75–83).

72. Anti-bot stack overview

HA-WAF ships a built-in anti-bot stack — a set of mechanisms wired into the generated HAProxy config and the Coraza SPOA agent. The stack landed in two stages: stage 1 (#157) — quick wins in pure HAProxy config; stage 2 (#158) — IP reputation, verified bots, challenge and honeypot.

Request processing order (fixed by the config template):

whitelist (bypass WAF) → WAF domain bypass → GeoIP → IP-reputation →
IP-blacklist → verified bots → Bot Policy (UA filters) →
Coraza/SPOE (CRS + honeypot/challenge detection) → WAF-deny →
honeypot reaction → challenge → rate limits (conn_cur etc.) →
throttle → path rules → backend

The principle: cheap checks run before expensive ones. A bot blocked by a UA filter or a reputation feed never burns the WAF pipeline; the challenge stands after SPOE because its verdicts come from there.

Threat-to-mechanism matrix:

Deep documentation per mechanism lives in docs/anti-bot/ (stack README + stage docs).

UI: the mechanisms are configured on site tabs — Bot Policy, IP Reputation, Challenge, Honeypot (traps), and conn_cur under Rate Limits.

73. Per-site Bot Policy (UA filters)

Per-site filtering by the User-Agent header. Runs as HAProxy ACLs before Coraza/SPOE — a blocked bot never reaches the WAF.

UI tab: Site → Bot Policy

PUT /api/v1/sites/{siteId}/bot-policy
{
  "enabled": true,
  "entries": [
    {"pattern": "python-requests", "match_type": "sub",   "action": "block",  "comment": "scripts"},
    {"pattern": "scrapy",          "match_type": "sub",   "action": "tarpit", "comment": "scraper"},
    {"pattern": "UptimeRobot",     "match_type": "sub",   "action": "allow",  "comment": "monitoring"},
    {"pattern": "",                "match_type": "empty", "action": "block"}
  ]
}

Processing order: allow entries evaluate first, then reject actions. Rejects additionally skip: the ACME path /.well-known/acme-challenge/ (a pattern like bot cannot break certificate issuance), verified bots (txn.verified_bot, ch. 77) and the WAF whitelist (txn.waf_bypass).

Pattern validation. Patterns are interpolated into HAProxy config lines unquoted, so the API rejects spaces, newlines and # (directive injection) plus a trailing \ (line continuation). For match_type=reg the regex is smoke-checked by compiling it with Go's RE2 — some PCRE-only constructs (lookaheads, backreferences) are valid for HAProxy yet rejected by the API.

Tarpit holds a maxconn slot for the whole hold time — use it for surgical entries only. Decoy answers with a neutral 200 OK and does not reveal the block.

Matching is case-insensitive for all types; empty catches both a missing header and User-Agent: of zero length.

Typical patterns: curl, python-requests, scrapy, wget, Go-http-client (sub + block); an empty UA (empty + block); a monitoring agent (UptimeRobot etc. — sub + allow). Do not block the UAs of honest crawlers (Googlebot, bingbot) — their spoofing is handled by verified bots (ch. 77), and a block can cut off the real search crawler.

74. Slowloris protection (conn_cur)

Slow L7 attacks (Slowloris: the client keeps connections open, feeding headers drop by drop) are covered by two settings.

UI tab: Site → Rate Limits, the “Current connections (0 = disabled)” block, field “Max concurrent”.

POST /api/v1/sites/{siteId}/ratelimits
{
  "name": "Conn cap",
  "enabled": true,
  "conn_cur": 15
}

Counters live in the shared stick-table ha_waf_rate_table (key sc0, shared with the site's other rate metrics). A conservative threshold is 10–20; the UI pre-fills 10.

CGNAT. Behind a shared NAT (mobile carriers, corporate networks) hundreds of legitimate users share one IP — do not set the threshold aggressively low. For mobile-heavy sites raise it based on actual traffic (tens to hundreds).

The second line is the global timeout http-request 10s (in the defaults of the generated config): a client that fails to finish sending its request within 10 seconds is dropped instead of holding the slot forever.

Limitations: per-site rate rules (conn_cur included) render on the HTTPS frontend and custom HTTP/HTTPS ports but not on the shared :80 frontend — enable the site-level “Redirect HTTP → HTTPS” so plain-HTTP traffic cannot bypass the limits. The stick-table is type ip — counters are IPv4-only.

75. Tarpit and decoy

The two “quiet” actions of the anti-bot arsenal. Available as Bot Policy actions (ch. 73) and as the honeypot reaction (ch. 79).

Tarpit occupies a maxconn slot for the whole hold. Mass tarpit rules (hundreds of patterns, a busy public site) eat the connection budget — keep tarpit for a narrow list. For bulk filtering use block or decoy.

Both actions inherit the Bot Policy reject guards: they never fire on the ACME path, for verified bots, allow entries or the WAF whitelist.

76. IP-Reputation feeds

Blocking (or logging) IPs from external reputation feeds: Spamhaus DROP, FireHOL, blocklist.de. HA-WAF downloads the feeds itself into map files; no external services or API keys are needed.

UI tab: Site → IP Reputation (feed snapshot status is on the same tab).

PUT /api/v1/sites/{siteId}/ip-reputation
{
  "enabled": true,
  "mode": "log",
  "sources": ["spamhaus_drop", "firehol_l1"],
  "exceptions": ["203.0.113.0/24"]
}

There is no global switch: feeds are always downloaded (~6 small requests per hour, conditional GET with ETag); control is per-site policies only. The policy sits in the pipeline right after GeoIP, before the IP blacklist and bot rules.

Feed status (bottom of the tab / GET /api/v1/iprep/status): per source — snapshot age, IP count (ips), flags stale (snapshot older than 24 h or missing) and held (the new snapshot differed from the previous one by more than ±50 % of entries — the update is postponed, the source stays on the old list). In enforce mode stale sources do not block (log behavior); a site whose every selected source is stale falls back entirely to log mode.

Start with log. Keep observation mode for 1–2 weeks before switching to enforce: a feed can list your corporate range — put it into exceptions up front.

blocklist.de is built from fail2ban reports and carries a high false-positive risk for CGNAT neighbors. Do not use it as the sole source. spamhaus_asndrop currently parses into an empty list (the feed carries ASNs without prefixes) — the status honestly shows ips: 0.

Legal notice. HA-WAF does not redistribute the lists — it downloads them on the fly for its own instance and refers to the original publishers. Each list's terms live on the Spamhaus / FireHOL / blocklist.de sites; Spamhaus requires its own agreement for commercial use.

77. Verified bots

The real Googlebot must not be caught by UA filters and other anti-bot mechanisms — while a spoofer impersonating Googlebot should be. Verified bots solve both with a two-factor UA-claim × IP check:

1. Publisher ranges (map file verified_ranges.map): HA-WAF downloads the publishers' official JSON prefix lists once a day (Google Googlebot/special-crawlers/user-triggered-fetchers, Bing, OpenAI GPTBot/OAI-SearchBot/ChatGPT-User, Applebot, Anthropic, PerplexityBot). An unreachable source keeps the prefixes of the last successful fetch (fail-open).
2. FCrDNS (stick-table): for publishers without published ranges (Yandex, Amazon, Meta) — forward-confirmed reverse DNS: the PTR name must sit under the publisher's suffix (with a leading dot — evilgooglebot.com does not pass .googlebot.com), and the forward lookup must return the original IP. Checks happen off the hot path: HAProxy registers UA candidates into the table, and an HA-WAF worker verifies them via the runtime CLI every 10 s.

A client whose IP is in a publisher range or has a positive FCrDNS verdict gets txn.verified_bot — and only Bot Policy blocks (deny/tarpit/decoy) are lifted for it.

Fail-open semantics: “unverifiable ≠ spoofed”. The mechanism only lifts blocks, never adds them — bots without published ranges and without FCrDNS are not cut off by it.

Configuration (yaml, defaults):

bots:
  enabled: true            # on by default; false → byte-identical config without the feature
  map_file: /etc/ha-waf/verified_ranges.map
  sync_interval: 24h
  verify_interval: 10s
  dns_timeout: 2s
  workers: 4
haproxy:
  verified_table_port: 19996

Port 19996 is the FCrDNS stick-table listener on 127.0.0.1. If the port is taken on the host, HAProxy will not start — override haproxy.verified_table_port. With the feature enabled HAProxy also raises tune.stick-counters to 4 (sc0–sc3).

Limitation: the verified-bot stick-table is type ip — the FCrDNS factor works for IPv4 only (the map factor covers publishers' IPv6 prefixes too).

78. Challenge (redirect → PoW)

A two-stage challenge for suspicious clients (the Anubis approach): instead of a block — a proof of work.

• Stage 1 — a redirect challenge without JS: the client gets a 302 to /_waf/challenge, the API sets a signed pending cookie (HMAC) and returns the client to the original URL.
• Stage 2 — a proof-of-work interstitial (SHA-256, leading zero nibbles) rendered by HAProxy itself. A solution yields a passed cookie for the policy TTL; the cookie is HMAC-signed and bound to the IP prefix (/24 v4, /56 v6) + UA class.

UI tab: Site → Challenge

PUT /api/v1/sites/{siteId}/challenge
{
  "enabled": true,
  "mode": "shadow",
  "paths": ["/login", "/api/"],
  "cookie_ttl_hours": 168,
  "rechallenge_pct": 2,
  "pow_difficulty": 5,
  "require_pow": true
}

Automatically not challenged (exemption chain by priority): verified bots (ch. 77), robots.txt and /.well-known/*, *.xml/*.atom feeds, non-Mozilla UAs (curl/git/RSS — they cannot run JS), the WAF whitelist. The __Host-hw_chal cookie (Secure, HttpOnly, SameSite=Lax) requires HTTPS.

Enforce requires HTTPS on the site. The challenge redirect targets an absolute https:// origin — on an HTTP-only site that is a dead endpoint that locks Mozilla-UA clients out. The API rejects mode: enforce for a site without https_enabled (#172). Enable HTTPS or stay in shadow.

The challenge rides the SPOE pipeline: a site with the WAF policy disabled gets no challenge. When SPOA degrades (agent down or overloaded), enforce policies automatically downgrade to shadow (the challenge_degraded metric, ch. 80) — no redirect loops on a live site.

The cookie secret is the yaml challenge.secret (or the env HAWAF_CHALLENGE_SECRET); empty → a random secret at startup plus a warning (cookies leak across restarts) — set it explicitly for multi-replica setups. Changing any policy setting invalidates all of the site's cookies (policy hash in the signature).

SEO: leave paths empty only deliberately — a blanket 302 hurts crawling; challenge selected prefixes (/search, /login, /api). robots.txt and feeds are already in the exemption chain — do not duplicate them.

79. Honeypot

Hidden trap fields in forms: a field a human never sees or fills, but a bot does. Detection runs before CRS in Coraza (via SPOE); the reaction is quiet, and bans happen only through repeat hits on the shared WAF gpc0 counter. The rollout starts in shadow (counters only).

UI tab: Site → Honeypot (traps)

Site-owner workflow:

1. Enable the policy (shadow first!):

   PUT /api/v1/sites/{siteId}/honeypot
   {"enabled": true, "mode": "shadow", "action": "silent200"}
   

2. Create a field — POST /api/v1/sites/{siteId}/honeypot/fields. The backend generates the name: hp_ + 12 hex (crypto-random).
3. Embed the snippet into the form (the “Show snippet” button on the tab):

   <input type="hidden" name="hp_0123456789ab" value="">
   <input type="hidden" name="hp_0123456789ab_token" value="1737050000.9f2c…">
   <style>
     input[name="hp_0123456789ab"] { position:absolute; left:-9999px; top:-9999px; }
   </style>
   

   The first input is the trap (always empty for a human), the second is the backend's embed token. Hide the field with off-screen positioning (not display:none — some bots skip literally invisible fields), plus tabindex="-1" and aria-hidden="true".
4. Apply the configuration (POST /api/v1/reload).
5. Watch: the counters on the tab (GET /api/v1/honeypot/status) and the honeypot_hits_total metric. Once enough data accumulates, switch mode to enforce.

Signals:

Reaction (enforce): silent200 — a neutral 200 OK (“the bot sees success”, the trap stays hidden) or tarpit (ch. 75). Never a 4xx, and never a direct ban on the first hit: every hit increments the shared WAF gpc0 ban counter; 429 arrives only on repeats.

Third-party forge protection: the token is public (it is baked into the HTML), so the protection is not its secrecy but the SPOA Origin gate: a browser must send Origin on a cross-origin POST and cannot forge it from a page. No Origin / same-origin → the normal hit path (a non-browser client = a bot); a foreign/null Origin → the forged signal — the victim is neither tar-pitted nor banned. The query vector is excluded by the rule structure (only ARGS_POST + POST method match).

robots.txt is not needed. The trap is a hidden field in a real form, not a separate trap page: honest crawlers never submit forms and never fall into the trap. (Classic page-traps — decoy links disallowed in robots.txt — are not implemented in HA-WAF.)

CSS refactoring is the trap's worst enemy. If a refactor makes the field visible (broken off-screen positioning, a renamed field), humans start filling it. Symptom: hit counters suddenly grow after a frontend release while bot traffic is unchanged. Do not rename the field to a “human” name — browser autofill will fill it (the hp_+hex name is chosen deliberately).

The /honeypot/status counters are in-memory and reset on restart (the history lives in the honeypot_hits_total metric). Tokens never expire; changing challenge.secret invalidates them (old ones become notoken — a weak signal, not a block).

80. Anti-bot monitoring and operations

Anti-bot stack metrics (control-plane Prometheus endpoint / OTel):

SPOA degradation and auto-downgrade. Enforce challenge depends on SPOE verdicts. The “HAProxy alive, no verdicts” scenario (agent overload, control-plane restart) produces an infinite redirect loop. The control-plane monitor probes SPOA TCP availability every 5 s (the same address HAProxy's health check uses) and the SPOE processing failure ratio: a 30 s window, >50 % failures with ≥20 processed messages → the probe counts as failed (low traffic is not overload). Hysteresis of 3 failures/3 successes (15 s) → challenge_degraded=1, all enforce policies render as shadow; recovery is automatic, nothing is written to the DB.

What to alert on:

• challenge_degraded == 1 longer than 5 minutes — SPOA overloaded or down;
• rising rate(spoa_processing_failures_total) while the agent port is alive — overload with a live TCP (the monitor's second signal);
• a spike of honeypot_hits_total{signal=hit} after a frontend release — a CSS regression un-hiding the field (ch. 79);
• sustained challenge_verdicts_total{verdict=challenge} on one site — a redirect loop (client cookies disabled) or paths too broad;
• iprep stale: true / held: true — a feed has not updated for over a day.

Typical issues: