Files
YoungReckless4 56ac8d0fc7 feat(amneziawg): add opt-in Linux kernel datapath (#6877)
* feat(amneziawg): add opt-in Linux kernel datapath

Add the kernel backend as an explicit per-inbound option while keeping the embedded backend as the default. Include host preflight, UI/API support, the bundled installer CLI, lifecycle and accounting integration, regression tests, and upstream-facing documentation. Kernel mode is NAT-only and rejects unsupported routing, IPv6, forwarding, and pinned-listen configurations. Local saves check the local host; remote nodes check their own host during apply.

Preserve adopted interface accounting across panel restarts, retain manager state after failed teardown so the job can retry, prevent embedded startup while a kernel interface remains live, and prune collector state for removed peers. Tested the restart accounting path on Debian 13 with XanMod 7.1.11 on x86_64; Docker and cross-kernel coverage are not claimed.

* fix(amneziawg): align readiness and endpoint reporting

Expire endpoint sightings using the latest handshake and sweep stale addresses for the polled interface. Keep the readiness API schema, generated client types, and ready-host empty blocker list consistent; watch the nested form field and defer local readiness checks for remote targets. Document safe rollback before using a build without kernel support.

Regression evidence: the idle endpoint test failed before the collector change because the old address remained visible after 31 minutes; endpoint tests pass after the fix. The form component suite passes 3 tests; restoring the invalid array watch made its off-mode severity test fail, while the corrected dot-path watch passes. Targeted Go regressions across amneziawgkern, job, and service pass.

* style(amneziawg): format readiness test

* test: reproduce kernel AWG client validation gaps

* test(service): cover kernel AWG shared client fanout

* fix(amneziawg): validate kernel client fanout and config values

* test(amneziawgkern): cover owned interface lifecycle recovery

* fix(amneziawgkern): recover owned interfaces on restart

* test(amneziawgkern): retain visibility after recovered cleanup failure

* fix(amneziawgkern): report recovered interfaces while cleanup retries

* style(amneziawgkern): satisfy static analysis

* test(amneziawgkern): cover long rendered peer comments

* fix(amneziawgkern): accept long generated config comments

* test(amneziawgkern): refuse foreign occupied interfaces

Capture the current Ensure ordering failure with real temporary config paths and manager side-effect boundaries.

* test(amneziawgkern): cover stale interface collisions

Extend the Ensure regression across absent links, mismatched live interfaces, cached manager state, and Remove teardown.

* fix(amneziawgkern): verify interface ownership before mutation

Refuse to replace or tear down links unless the canonical config is a valid panel-owned file and its key matches the live interface. Preserve normal adoption and key rotation, and document migration of legacy unmarked configs.

Validation: go test ./internal/amneziawgkern -count=1; go test -race ./internal/amneziawgkern -count=1; golangci-lint run ./internal/amneziawgkern/...; git diff --check.

* build: update Go toolchain to 1.27.3

* test(amneziawg): reproduce unsafe key rotation recovery

Keep the previous owned config authoritative until teardown succeeds, and require fresh managers to rebuild structural key changes. These regressions expose the stuck live-key/disk-key mismatch after teardown failure.

* fix(amneziawg): recover interrupted key rotations

Keep the old owned config on disk until a structural rebuild has confirmed the previous interface is down. Fresh managers only adopt when the live config and desired structural identity match, so retries and removals retain the key needed to prove ownership.

* fix(scanner): align Go image with module version

* fix(docs): pin pnpm release with bootstrap fix

pnpm 12.9.1 restores a Node launcher before its postinstall replaces the target with ELF; pnpm 11 then runs that binary as JavaScript. Pin 12.10.0, which includes the shim fix, so the docs CI bootstrap can switch versions safely.

* fix(xray): build core from pinned module

The configured Xray release asset tag is unpublished, and the latest published binary lacks GetUsersStats used by the panel. Build the packaged core from the exact Xray module version already selected in go.mod for Docker and release targets.

* fix(tests): stabilize local network fixtures

Keep the TUIC stream fragmentation check on a working UDP loopback path,
allocate the AmneziaWG fixture ports dynamically, and wait for a real
Xray stats response before starting API mutation assertions. Recreate
failed readiness channels so an early connection cannot stall the probe.

Verified host and offline parallel Go suites, repeated occupied-port and
UDP tests, race/shuffle runs, and golangci-lint. Production behavior and
dependency versions are unchanged.

* fix(i18n): add the kernel-mode AmneziaWG keys to sk-SK

main gained the sk-SK locale after this branch added its kernel-mode keys
to the other 13, so i18n-dead-keys.test.ts failed on the merged tree.

* refactor(amneziawgkern): apply kernel interfaces with direct commands, no awg-quick

awg-quick evaluates PostUp/PostDown through a root shell, so every value the
panel rendered into its config was one validation slip away from root command
execution. The interface is now built with direct ip/awg/nft execs from a
setconf-only file: link add, awg setconf, address, mtu, an atomically replaced
nftables table per inbound, and the IPv4 forwarding sysctl written directly.

- Panel-owned names and paths: xui-awg<id> interfaces, xui_awg<id> NAT tables
  and configs under the panel's data dir, so nothing collides with an
  operator's own awg0/awg1 awg-quick tunnels in /etc/amnezia/amneziawg.
- FORWARD accepts (comment 3x-ui:xui-awg<id>) when iptables is present: Docker
  and ufw set the FORWARD policy to DROP, which silently black-holed the tunnel.
- A full panel stop records the final counters and tears the interfaces down,
  like the embedded datapath; a stopped panel enforces no quota or expiry.
  Crash survivors are still adopted without a rebuild.
- The job reads counters before reconciling, and from every managed interface,
  so a client disabled for quota, or a removed inbound, is charged for its last
  interval instead of losing it with the dropped peer.
- An unchanged reconcile tick forks no process (it probed the live key with
  `awg show` every 10s); readiness no longer loads the module from a GET.

Verified on the real module (WSL2 6.6 kernel, v3.1.20260906): create, peer and
egress changes in place, SIGKILL adoption with the same ifindex, SIGTERM
teardown leaving no link, table, FORWARD rule or config behind.

* fix(amneziawg): gate kernel host readiness only when a save starts the datapath

Every save of a kernel-mode inbound re-ran the host preflight, so a host that
lost its module (a kernel upgrade without a DKMS rebuild) refused even
disabling the inbound or switching it back to embedded mode. Host blockers now
apply only when the save starts the kernel datapath: a new inbound, a switch
from embedded, or enabling a disabled one. Feature blockers (IPv6, forwarded
ports, pinned listen) still apply to every save.

* feat(nodes): refuse kernel-mode AmneziaWG pushes to nodes that predate it

A node built before kernel mode drops the unknown kernelMode field and serves
the inbound on the embedded datapath while the master's form shows kernel
mode on. Nodes now advertise awg-kernel in X-3x-Node-Caps, and Remote refuses
to push a kernel-mode inbound to a node that has not advertised it, learning
the caps with an inbound list first when this Remote has not heard from it.

* fix(amneziawg): pin the kernel module installer to verified commits

The installer ran as root and built whatever upstream tag sorted newest at
install time. It now clones fixed tags (module v3.1.20260906, tools
v3.1.20260812) and refuses unless each resolves to its recorded commit. The
udp_tunnel perl patch is gone: those releases carry the kernel 7.1.5 compat
upstream, so the patch never matched.

* fix(i18n): translate the kernel-mode AmneziaWG strings

All thirteen non-English locales carried the English text verbatim.

* fix(amneziawg): skip the kernel traffic flush when no kernel interface runs

Shutdown listed the desired AmneziaWG inbounds from the database even on a
panel with no kernel interface, an unneeded query on every stop. A server
stopped before its database was opened panicked on it, which turned
TestServerStopWhenXrayNotRunningSucceeds red whenever it ran first.

---------

Co-authored-by: MHSanaei <ho3ein.sanaei@gmail.com>
2026-10-25 13:24:50 +01:00

415 lines
10 KiB
Go

package amneziawgkern
import (
"fmt"
"strconv"
"strings"
"sync"
"time"
)
// PeerStat is one row of `awg show <iface> dump`.
type PeerStat struct {
PublicKey string
// Endpoint is "ip:port", or "(none)" while the peer has never been seen.
Endpoint string
// LatestHandshake is a unix timestamp; 0 means never.
LatestHandshake int64
// RxBytes is what the server received from the peer -- the client's
// upload. TxBytes is what the server sent -- the client's download.
RxBytes int64
TxBytes int64
}
// Delta is one peer's traffic since the previous poll, already corrected for
// counter resets.
type Delta struct {
PublicKey string
Up int64
Down int64
}
// Dump reads the live per-peer counters for one interface.
func Dump(iface string) ([]PeerStat, error) {
out, err := run("", "awg", "show", iface, "dump")
if err != nil {
return nil, fmt.Errorf("amneziawgkern: awg show %s dump: %w (%s)", iface, err, out)
}
return ParseDump(out)
}
// ParseDump parses the tab-separated output from `awg show <iface> dump`.
func ParseDump(out string) ([]PeerStat, error) {
lines := strings.Split(strings.TrimSpace(out), "\n")
if len(lines) == 0 || strings.TrimSpace(lines[0]) == "" {
return nil, nil
}
var stats []PeerStat
// lines[0] is the interface itself (private key, public key, listen port,
// fwmark) and carries no per-peer counters.
for _, line := range lines[1:] {
f := strings.Split(strings.TrimRight(line, "\r"), "\t")
if len(f) < 7 {
continue
}
handshake, err := strconv.ParseInt(f[4], 10, 64)
if err != nil {
return nil, fmt.Errorf("amneziawgkern: parsing handshake %q: %w", f[4], err)
}
rx, err := strconv.ParseInt(f[5], 10, 64)
if err != nil {
return nil, fmt.Errorf("amneziawgkern: parsing rx %q: %w", f[5], err)
}
tx, err := strconv.ParseInt(f[6], 10, 64)
if err != nil {
return nil, fmt.Errorf("amneziawgkern: parsing tx %q: %w", f[6], err)
}
stats = append(stats, PeerStat{
PublicKey: f[0],
Endpoint: f[2],
LatestHandshake: handshake,
RxBytes: rx,
TxBytes: tx,
})
}
return stats, nil
}
type counters struct {
rx, tx int64
}
type endpointSighting struct {
addr string
seen time.Time
}
// Collector converts kernel counters to traffic deltas and tracks recent peer
// endpoints; adopted interfaces are primed once before deltas resume.
type Collector struct {
mu sync.Mutex
last map[string]counters
endpoints map[string][]endpointSighting
// emails maps iface/publicKey to the owning client, so observations can be
// reported against the identity the rest of the panel keys on.
emails map[string]string
// handshakes is each peer's latest handshake, the only online signal a
// kernel interface exposes.
handshakes map[string]int64
}
func NewCollector() *Collector {
return &Collector{
last: map[string]counters{},
endpoints: map[string][]endpointSighting{},
emails: map[string]string{},
handshakes: map[string]int64{},
}
}
func key(iface, publicKey string) string { return iface + "/" + publicKey }
// Delta records the current counters and returns what changed since the last call.
func (c *Collector) Delta(iface string, stats []PeerStat) []Delta {
c.mu.Lock()
defer c.mu.Unlock()
out := make([]Delta, 0, len(stats))
for _, s := range stats {
k := key(iface, s.PublicKey)
prev, seen := c.last[k]
up, down := s.RxBytes, s.TxBytes
if seen {
up = s.RxBytes - prev.rx
down = s.TxBytes - prev.tx
if up < 0 {
up = s.RxBytes
}
if down < 0 {
down = s.TxBytes
}
}
c.last[k] = counters{rx: s.RxBytes, tx: s.TxBytes}
if up == 0 && down == 0 {
continue
}
out = append(out, Delta{PublicKey: s.PublicKey, Up: up, Down: down})
}
return out
}
// Prime sets an adopted interface's current counters as the starting point.
func (c *Collector) Prime(iface string, stats []PeerStat) {
c.mu.Lock()
defer c.mu.Unlock()
prefix := iface + "/"
for k := range c.last {
if strings.HasPrefix(k, prefix) {
delete(c.last, k)
}
}
for _, s := range stats {
c.last[key(iface, s.PublicKey)] = counters{rx: s.RxBytes, tx: s.TxBytes}
}
}
// ObserveEndpoints tracks recent source addresses used by kernel-mode peers.
func (c *Collector) ObserveEndpoints(iface string, stats []PeerStat, now time.Time, window time.Duration) {
c.mu.Lock()
defer c.mu.Unlock()
cutoff := now.Add(-window)
prefix := iface + "/"
for k, sightings := range c.endpoints {
if !strings.HasPrefix(k, prefix) {
continue
}
kept := sightings[:0]
for _, sighting := range sightings {
if !sighting.seen.Before(cutoff) {
kept = append(kept, sighting)
}
}
if len(kept) == 0 {
delete(c.endpoints, k)
} else {
c.endpoints[k] = kept
}
}
for _, s := range stats {
addr := endpointHost(s.Endpoint)
if addr == "" || s.LatestHandshake <= 0 {
continue
}
seen := time.Unix(s.LatestHandshake, 0)
if seen.After(now) || seen.Before(cutoff) {
continue
}
k := key(iface, s.PublicKey)
kept := c.endpoints[k][:0]
found := false
for _, e := range c.endpoints[k] {
if e.addr == addr {
if seen.After(e.seen) {
e.seen = seen
}
found = true
}
kept = append(kept, e)
}
if !found {
kept = append(kept, endpointSighting{addr: addr, seen: seen})
}
if len(kept) == 0 {
delete(c.endpoints, k)
} else {
c.endpoints[k] = kept
}
}
}
// DistinctEndpoints reports how many different addresses a peer has connected
// from inside the window last passed to ObserveEndpoints.
func (c *Collector) DistinctEndpoints(iface, publicKey string) int {
c.mu.Lock()
defer c.mu.Unlock()
return len(c.endpoints[key(iface, publicKey)])
}
// EndpointsFor returns the tracked addresses for a peer, for surfacing in the
// UI next to the limit that acted on them.
func (c *Collector) EndpointsFor(iface, publicKey string) []string {
c.mu.Lock()
defer c.mu.Unlock()
seen := c.endpoints[key(iface, publicKey)]
out := make([]string, 0, len(seen))
for _, e := range seen {
out = append(out, e.addr)
}
return out
}
// Forget drops all state for an interface that is going away, so a later
// inbound reusing the name cannot inherit stale counters or sightings.
func (c *Collector) Forget(iface string) {
c.mu.Lock()
defer c.mu.Unlock()
prefix := iface + "/"
for k := range c.last {
if strings.HasPrefix(k, prefix) {
delete(c.last, k)
}
}
for k := range c.endpoints {
if strings.HasPrefix(k, prefix) {
delete(c.endpoints, k)
}
}
for k := range c.emails {
if strings.HasPrefix(k, prefix) {
delete(c.emails, k)
}
}
for k := range c.handshakes {
if strings.HasPrefix(k, prefix) {
delete(c.handshakes, k)
}
}
}
// endpointHost strips the port so a peer behind a NAT that renumbers its
// source port on every rekey is not mistaken for a peer in a new location.
func endpointHost(endpoint string) string {
e := strings.TrimSpace(endpoint)
if e == "" || e == "(none)" {
return ""
}
if i := strings.LastIndex(e, "]:"); i >= 0 { // [v6]:port
return e[:i+1]
}
if i := strings.LastIndex(e, ":"); i >= 0 && strings.Count(e, ":") == 1 {
return e[:i]
}
return e
}
var (
collectorOnce sync.Once
collector *Collector
)
// GetCollector returns the process-wide counter/endpoint collector.
func GetCollector() *Collector {
collectorOnce.Do(func() { collector = NewCollector() })
return collector
}
// EmailFor returns the client a peer key was last recorded against, so a peer
// that just left the desired set is still charged for its final counters.
func (c *Collector) EmailFor(iface, publicKey string) string {
c.mu.Lock()
defer c.mu.Unlock()
return c.emails[key(iface, publicKey)]
}
// SetEmails records which client each peer key belongs to, so observations can
// be reported against the identity the rest of the panel uses.
func (c *Collector) SetEmails(iface string, byPublicKey map[string]string) {
c.mu.Lock()
defer c.mu.Unlock()
active := make(map[string]string, len(byPublicKey))
for pk, email := range byPublicKey {
if pk != "" && email != "" {
active[key(iface, pk)] = email
}
}
prefix := iface + "/"
for k := range c.last {
if strings.HasPrefix(k, prefix) {
if _, ok := active[k]; !ok {
delete(c.last, k)
}
}
}
for k := range c.endpoints {
if strings.HasPrefix(k, prefix) {
if _, ok := active[k]; !ok {
delete(c.endpoints, k)
}
}
}
for k := range c.handshakes {
if strings.HasPrefix(k, prefix) {
if _, ok := active[k]; !ok {
delete(c.handshakes, k)
}
}
}
if c.emails == nil {
c.emails = map[string]string{}
}
for k := range c.emails {
if strings.HasPrefix(k, prefix) {
if _, ok := active[k]; !ok {
delete(c.emails, k)
}
}
}
for k, email := range active {
c.emails[k] = email
}
}
// ClientIPs returns recent peer endpoints as email -> address -> last-seen Unix timestamp.
func (c *Collector) ClientIPs() map[string]map[string]int64 {
c.mu.Lock()
defer c.mu.Unlock()
out := map[string]map[string]int64{}
for k, sightings := range c.endpoints {
email := c.emails[k]
if email == "" {
continue
}
for _, s := range sightings {
addr := strings.Trim(s.addr, "[]")
if _, ok := out[email]; !ok {
out[email] = map[string]int64{}
}
if prev, seen := out[email][addr]; !seen || s.seen.Unix() > prev {
out[email][addr] = s.seen.Unix()
}
}
}
return out
}
// onlineHandshakeAge is how recently a peer must have completed a handshake to count as online.
const onlineHandshakeAge = 3 * time.Minute
// ObserveHandshakes records each peer's latest handshake so online status can be derived.
func (c *Collector) ObserveHandshakes(iface string, stats []PeerStat) {
c.mu.Lock()
defer c.mu.Unlock()
if c.handshakes == nil {
c.handshakes = map[string]int64{}
}
for _, s := range stats {
if s.LatestHandshake > 0 {
c.handshakes[key(iface, s.PublicKey)] = s.LatestHandshake
}
}
}
// OnlineEmails returns the clients whose peers handshook recently enough to be
// considered connected.
func (c *Collector) OnlineEmails(now time.Time) []string {
c.mu.Lock()
defer c.mu.Unlock()
cutoff := now.Add(-onlineHandshakeAge).Unix()
seen := make(map[string]struct{}, len(c.handshakes))
out := make([]string, 0, len(c.handshakes))
for k, ts := range c.handshakes {
if ts < cutoff {
continue
}
email := c.emails[k]
if email == "" {
continue
}
if _, dup := seen[email]; dup {
continue
}
seen[email] = struct{}{}
out = append(out, email)
}
return out
}