Architecture Overview

DHCP DPI is a hybrid system: deep packet inspection runs in userspace while enforcement happens at kernel level via nftables.

This section covers the packet flow, system components, parallel data paths, and performance characteristics. Understanding the architecture helps you make informed decisions about deployment, tuning, and troubleshooting.

Packet Flow

Every DHCP packet passes through nftables first, then userspace inspection, then back to nftables for enforcement.

ASCII fallback

                        DHCP Client
                             |
                        [Network Interface]
                             |
                             v
                  +-----------------------+
                  |  nftables prerouting  |
                  |  priority -100        |
                  |  (dhcp_inspect chain) |
                  +-----------+-----------+
                              |
                       NFQueue 1-4 (inbound) / 1-8 (outbound)
                              |
                              v
                  +-----------------------+
                  |   Go Application      |
                  |                       |
                  |  1. Parse DHCP packet |
                  |  2. Match rules       |
                  |  3. Assign mark       |
                  |  4. Return verdict    |
                  +-----------+-----------+
                              |
                        verdict + mark
                              |
                              v
                  +-----------------------+
                  |  nftables chains      |
                  |                       |
                  |  route_by_type  (-98) |  -> per-msg-type chains
                  |  mac_actions    (90)  |
                  |  dhcp_block    (100)  |
                  +-----------+-----------+
                              |
                     accept / drop
                              |
                              v
                        DHCP Server

The application runs between two nftables stages. The first stage (dhcp_inspect, priority -100) captures packets and sends them to userspace via NFQueue. After inspection, the application returns a verdict with a mark value. Subsequent nftables chains at higher priorities use that mark to enforce throttling or blocking entirely in kernel space.

Why Hybrid?

Kernel-level enforcement handles the high packet rate; userspace handles the complex inspection logic.

Pure userspace solutions bottleneck at the application layer. Pure kernel solutions cannot do deep DHCP field inspection. The hybrid approach gives you both: the Go application parses Option 82, vendor classes, hostnames, and client identifiers, while nftables handles the simple “is this mark over its rate limit?” decision at wire speed.

Why Not Pure nftables Matching?

nftables alone cannot reliably classify DHCP traffic because the interesting fields live in Layer 7 and many deployments hide the client MAC behind a relay.

A tempting alternative is to skip userspace entirely and write a fat nftables ruleset that matches DHCP options directly with @offset,length raw payload rules. In practice this is brittle and quickly becomes unmaintainable:

L7 matching by byte offset is fragile. DHCPv4 options are a flat TLV stream with variable length and no fixed positions; DHCPv6 options nest inside relay-forw / relay-repl wrappers, and the order of OPTION_IA_NA, OPTION_CLIENTID, OPTION_VENDOR_CLASS, OPTION_RELAY_MSG, and friends is not guaranteed to be stable between clients, relay agents, or even firmware revisions of the same device. A static rule that works in your lab can silently stop matching after a vendor firmware bump.
Relay-only deployments hide the client MAC at L2. When DHCP traffic arrives via an isc-dhcp-relay (or any L3 relay), the client’s MAC is gone from the Ethernet header by the time it reaches the appliance — it survives only inside DHCP Option 61 (client identifier) or the relay’s Option 82 sub-options. Matching the client at L2 is no longer possible; you have to parse the application payload. Many real deployments (service-provider access networks, segmented enterprise networks, the simulated topology this project tests against) are relay-only by design.
Hostnames, vendor classes, and Option 82 sub-options are strings of varying length. Matching them against regex or wildcards is straightforward in the application and absent in nftables.
Composite per-message-type decisions need state nftables doesn’t have. Cross-referencing a SOLICIT against later REQUEST/RENEW traffic from the same DUID, or tracking that a MAC has just hit its analyze threshold, belongs in a process with memory and a scheduler — not a stateless packet matcher.

So the architecture lifts parsing and classification into the application (where regex, structured option parsing, and proper DHCPv6 relay-message walking are easy) and pushes the enforcement decision back into nftables as deterministic mark-based set membership tests. That is the smallest piece of work nftables has to do per packet, and it is the piece nftables is genuinely good at.

Components

NFQueue Mechanism

NFQueue transfers packets from kernel to userspace for inspection, then returns them with a verdict.

The queue num 1-4 directive in the inbound dhcp_inspect chain sends matching packets to queues 1 through 4; the outbound dhcp_inspect_output chain uses queue num 1-8. The Go application binds to these queues (nfqueue.queue_count in config.yaml defaults to 8) and processes packets in parallel across multiple worker goroutines. The bypass fanout flag ensures that if the application is not running, packets pass through unimpeded — the system fails open, not closed.

The queue count in nft-v2.sh must match the queue_count value in config.yaml.

System Components

Eight functional components work together: processor, output manager, API server, action manager, automation scheduler (which owns the LLM integration), alarms & notifications, support/feedback system, and packet capture. They are implemented across a small set of binaries that are described first, then in detail below.

Shipped Binaries

All shipped binaries live under /opt/dhcp-dpi/bin/ (core deb) or /opt/dhcp-dpi-tileserver/bin/ (tileserver deb). Daemons run as systemd services; CLIs are invoked on demand.

Binary	Type	Package	Role
`dhcp-processor`	Daemon (systemd: `dhcp-processor.service`)	`dhcp-dpi`	The main appliance: NFQueue worker pool, DHCP parser, action manager, automation scheduler, alarms engine, REST/WebSocket API, GUI. Everything in System Components below except packet capture lives here.
`geomock`	Daemon (systemd: `dhcp-dpi-geomock.service`, optional)	`dhcp-dpi` (built only with `--include-geomock`)	Geolocation mock service used by the flow visualiser when a real geolocation database is not licensed.
`dpictl`	CLI	`dhcp-dpi`	Operator/support CLI for system management and scripted actions.
`user-manager`	CLI	`dhcp-dpi`	One-off user-administration utility (create/reset/disable accounts from the shell).
`dhcp-collector`	Daemon (systemd: `dhcp-collector.service`)	`dhcp-dpi`	Central-site TZSP receiver for distributed deployments: accepts TZSP datagrams from remote `dhcp-agent` instances and re-injects packets into local DHCP ports so the processor sees them. See chapter 04, TZSP Mirrored from Remote Sites.
`dhcp-agent`	Daemon (systemd: `dhcp-agent.service`)	`dhcp-dpi`	Remote-site capture-and-forward agent: sniffs DHCP traffic via libpcap, wraps each frame in TZSP, ships it to the central `dhcp-collector`. Single binary, lightweight, runs on anything from an x86 VM to an ARM SoC.
`tileserver`	Daemon (systemd: `dhcp-dpi-tileserver.service`)	`dhcp-dpi-tileserver` (optional)	Self-hosted map tile server that powers the flow-visualiser map. Map tile data is sideloaded separately under `/opt/dhcp-dpi-tileserver/data/`. See chapter 21.
`simulator`	CLI / one-shot	`dhcp-dpi-simulator` (optional)	DHCP traffic generator used for load testing, demos, and reproducing customer issues.

The deb packages and how to install them are covered in chapter 05 — Installation. The functional descriptions below explain what each piece of dhcp-processor (and its sibling daemons) actually does.

Processor

The processor receives packets from NFQueue, coordinates parsing and rule matching, and returns verdicts.

The processor binds to the configured NFQueue numbers and spawns worker goroutines. Each worker receives a raw packet, parses it as a DHCPv4 or DHCPv6 message, derives a mark (message type encoded in the high byte, client identifier in the low 24 bits), and returns the verdict to the kernel. Processing is non-blocking — a slow worker does not stall other queues.

For every DHCP packet the userspace verdict is always Accept — the application has no ability to drop a packet on its own. All blocking and throttling is performed by nftables in the chains that run after inspection, using the mark the processor attached. The one exception is non-DHCP traffic that leaks into the queue: if drop_non_dhcp_packets is enabled, those packets are dropped directly by the processor, but they were never DHCP to begin with.

Note: Earlier versions also ran each packet through a configurable custom rule engine that matched fields like client_mac, vendor_class, hostname, or client_identifier and assigned custom marks. This path is deprecated — the rules section in config.yaml ships empty and the recommended way to classify traffic is the Automation Scheduler, where rule visibility, history, and conflict resolution are far better. The rule engine can still be restored as a last resort by uncommenting entries under rules.rules in config.yaml.

Output Manager

The output manager sends parsed events to ClickHouse, Vector, and WebSocket clients in non-blocking mode.

All output operations are asynchronous. ClickHouse inserts are batched on two thresholds — a row count (batch_size) and a wall-clock interval (flush_duration_secs) — whichever fires first. Both are configurable in config.yaml. The recommended target is one flush every one to two minutes, with batch_size sized to roughly match the number of DHCP messages expected in that window. A small site running ~4,000 req/s lands around 24,000 events per minute-long batch; a 50,000 req/s deployment with a 30-second flush window lands around 1.5M — tune both knobs to whatever fits your traffic profile. Vector output uses HTTP. The output manager never blocks the packet processing pipeline — if an output destination is slow or unavailable, events are buffered or dropped gracefully.

API Server

The API server provides a REST API and WebSocket endpoint for the GUI and external integrations.

Built on Gin, the API server handles JWT authentication, session management, configuration CRUD, historical queries against ClickHouse, and real-time event streaming via WebSockets. The WebSocket endpoint supports configurable rate limiting to prevent browser overload during high-traffic periods.

Action Manager

The action manager executes enforcement actions by adding marks to nftables sets.

The action manager supports six action types, all of which can be triggered by an automation rule, an LLM recommendation, or an operator from the GUI:

Action	Effect	Default duration
`block`	Drop packets at the kernel via `llm_blocked_marks`	2 hours
`throttle`	Rate-limit packets via `llm_throttled_marks`	24 hours
`monitor`	Tag the client for observation via `llm_monitored_marks`; traffic is unaffected	7 days
`allow`	Mark the client as trusted via `llm_allowed_marks`; bypasses throttling	30 days
`deny`	Permanent block plus shared deny set (see below)	indefinite
`analyze`	Queue the device for the optional AI backend to examine; it then emits one of the five enforcement actions above	—

Each enforcement set has its own configurable timeout. When entries expire, the kernel automatically stops applying the action — no userspace cleanup required.

The action manager also maintains a shared deny set that is checked by the packet processor. When a device is denied, the processor skips event generation entirely — no ClickHouse writes, no WebSocket broadcasts. This reduces the resource cost of persistent offenders to near zero while still dropping their packets at the kernel level.

Automation Scheduler (and LLM Integration)

The automation scheduler runs periodic queries against aggregated data to detect anomalies. Five of its six actions are deterministic; the sixth (analyze) routes a device to the LLM.

Rules define thresholds (e.g., “more than 1000 requests per hour”) and an action. The scheduler queries Stage 1 materialized views in ClickHouse, identifies MACs exceeding thresholds, and either takes a direct enforcement action (block, throttle, monitor, allow, deny) or chooses analyze to hand the device off to the LLM. Rules are priority-ordered; the highest-priority match wins when multiple rules fire for the same MAC.

LLM path. The LLM never scans raw events on its own. When a rule chooses analyze (or an operator requests analysis from the GUI), the device is queued for the optional AI backend, which reviews a summary of recent activity and recommends one of the deterministic actions (block, throttle, monitor, allow, or deny). That recommendation is then dispatched through the same action manager as a deterministic rule. The AI backend is optional; the system enforces fully without it.

The optional AI backend is configured separately; its full configuration ships with the install package.

Alarms & Notifications

The alarms engine generates state-tracked alarms from events; the notifications dispatcher routes them to configured sinks.

The alarms engine evaluates events against alarm rules, manages alarm state transitions (raised, acknowledged, cleared), and persists alarm history in ClickHouse. The notifications dispatcher subscribes to state changes and routes messages through pluggable sinks. Rule matching uses pre-compiled regex on source keys for efficient routing.

Two sinks ship out of the box: the in-app Bell sink (shown in the GUI header) and the Vector sink. The Vector sink is the wide-reach option — it forwards notifications to a local Vector agent, which can then fan out to 50+ downstream destinations including syslog, Kafka, AWS S3, ElasticSearch, MQTT, NATS, Splunk, Datadog, Loki, Prometheus Remote Write, and many others. Adding a new external destination is a Vector configuration change on the operator’s side — no appliance code change required.

Support/Feedback System

The support system provides an optional, consent-based support channel between the appliance and the vendor — there is no standing inbound path.

The appliance never exposes an inbound port for support. A session is established only when an operator explicitly starts it, runs over an encrypted outbound connection, and is fully auditable. It is opt-in, time-boxed, revocable at any time, tier-gated (more privileged access requires an explicit operator approval prompt), and can be disabled entirely. Closing the session terminates every channel at once and nothing reconnects on its own.

Packet Capture

The packet capture module provides on-demand DHCP packet capture with GUI integration.

Operators can capture live DHCP traffic matching configurable filters, manage capture sessions from the GUI, and download standard capture files for offline analysis. This is useful for troubleshooting specific devices, verifying that filtering rules behave as expected, and collecting evidence for network issue reports.

Data Paths

Events flow in parallel through three independent paths after packet processing; the LLM sits downstream of Automation, not on the event stream.

ASCII fallback

  Parsed Event
       |
       +---> ClickHouse (batch insert, configurable size + flush interval)
       |
       +---> WebSocket Manager ---> Connected GUI clients
       |
       +---> Automation Scheduler (queries ClickHouse aggregates on a schedule)
                   |
                   +---> Direct action (block / throttle / monitor / allow / deny)
                   |     via Action Manager ---> nftables sets
                   |
                   +---> Queue device for the optional AI backend
                                  |
                                  v
                         AI Analyzer (reviews a summary
                         of recent activity)
                                  |
                                  v
                         Action Manager ---> nftables sets

Each path is independent. A ClickHouse outage does not affect packet processing or WebSocket streaming. The LLM analysis path is optional — the system operates fully without it, just without AI-driven enforcement. Operators can also queue a specific device for analysis from the GUI, which uses the same analysis queue.

Note: Denied devices are an exception. The processor checks an in-memory deny set before entering the output pipeline. If a device’s mark is in the deny set, the event is never created — none of the four paths receive it. The packet still receives a verdict and mark so nftables drops it, but the device produces zero downstream load.

Performance

The system sustains 50k+ requests/second on commodity hardware with minimal CPU usage.

Metric	Value
Sustained throughput	50,000 req/s toward the DHCP server
Peak shaping	nftables rate-limit chains absorb DHCP storms in-kernel, so spikes do not translate into matching userspace CPU spikes
NFQueue workers	Configurable (default 8 queues, scales with available CPU cores)
ClickHouse batch size	Configurable (sized to ~1-2 min of traffic)
nftables set capacity	~2.5M entries per set
Set entry timeout	1 min (rate limiting) to 30 days (trusted)
WebSocket rate limit	Configurable events/second with burst

Performance scales linearly with queue count up to the number of available CPU cores. The 50k req/s figure covers both steady-state traffic (large networks with high renewal volume) and DHCP storms or attack spikes — because rate-limiting and throttling are enforced in nftables, surges are shaped at wire speed before they reach the userspace inspector. The main bottleneck at the highest rates is ClickHouse batch insertion, which is asynchronous and does not affect packet processing latency.

In practical terms, this translates to protecting networks ranging from hundreds of thousands to multi-millions of DHCP clients — service-provider access networks, campus and enterprise environments, large-scale IoT deployments — on a single commodity appliance.