status: proposed

Arch tools-VM on Chatsubo bhyve, per-user 9p homes, ansible tooling manifest

Chatsubo gains a persistent Arch Linux bhyve guest (ADR-0056's bare-metal substrate provides the VT-x/VT-d bhyve needs) hosting open-terminal (open-webui/open-terminal) as the per-user compute surface for Open WebUI. Each OWUI user maps to a Linux account whose home is the host FreeBSD /usr/home/<uid> shared into the guest via virtio-9p (p9), so the terminal/file APIs operate on the user's own folder. The guest is provisioned from the official arch-boxes cloud image (qcow2 + cloud-init + SHA256) and carries an ansible-managed tooling manifest as the "add more tooling" mechanism.

Context

The Open Terminal feature needs a Linux environment with a real kernel for LaTeX+beamer, LibreOffice, ffmpeg, playwright, JVM+Clojure+etaoin, and the rest of the tooling. FreeBSD jails and Linuxulator jails cannot run all of it (the same reason kanban got a bhyve guest instead of a jail). Chatsubo is the only host allowed (the "Chatsubo only" constraint), and bhyve there requires the bare-metal upgrade of ADR-0056.

open-terminal's multi-user mode is the folder-switching mechanism natively: X-User-Id → sanitized Linux account (useradd -m -s /bin/bash, /home/<name>, chmod 2770, sudo -u for all I/O), auto-provisioned on first use. The OWUI 0.11.0 fork stamps X-User-Id + X-Session-Id on every proxied and tool call, so identity flows automatically. The repo's old "no 9p" note predates FreeBSD 13.1's native virtio-9p (VirtFS), which works with Linux guests and is exposed by vm-bhyve via bhyve_options.

Decision

1. One Arch VM, per-user homes via 9p. A single bhyve guest on bridge7 /

172.31.107.0/24 (host .1, guest .2). Each user's home is the host /usr/home/<uid> shared in through virtio-9p. Guest accounts are pre-created with uid-mapped deterministic USER_PREFIX naming; open- terminal's sanitizer yields deterministic names (e.g. ctni.teste → ctnitest).

1. Sizing. 16 GB / 4 vCPU / 200 GB thin-provisioned disk image on ZFS —

headroom for the tooling manifest without starving the host (which keeps its jails + ZFS ARC). User homes live on the host via 9p, not in the image.

1. open-terminal run mode. A pip venv (sha-pinned tarball) + a systemd

unit, OPEN_TERMINAL_MULTI_USER=true, running as root in the VM — the VM *is* the sandbox. OPEN_TERMINAL_API_KEY is vault-held. One package authority (ansible/pacman manifest), one service to health-check, no nested Docker daemon.

1. Cloud-image provisioning. arch-boxes official qcow2 pinned by SHA256,

NoCloud seed (user-data/meta-data/network-config), UEFI loader with a BIOS/grub-bhyve fallback (the ADR-0035 lesson). The install/reconcile script runs guest-side and is idempotent.

1. Ansible tooling manifest. arch_tools_packages / _pip_packages /

_npm_packages lists in the role defaults are the add-tooling mechanism; runtime install vars are the playground only. First set: texlive+beamer, LibreOffice, ffmpeg, ImageMagick+GIMP (headless), playwright, JVM+Clojure+etaoin, csv/xlsx, jq/yq/edn, reveal.js (npm), curl, jupyter.

1. Role placement. New role arch-tools-vm at

deployer/services/arch-tools-vm/role with a byte-identical mirror at ansible/roles/arch-tools-vm, enforced by scripts/tests/test-role-tree-parity.sh.

1. Network. pf NAT for 172.31.107.0/24 in access/pf.conf.j2, plus an

entry in small_pine_egress_peers so small-pine (the proxy of ADR-0034) can reach the VM.

Consequences

• Chatsubo hosts a real Linux kernel, unblocking the tooling surface that

FreeBSD jails and Linuxulator could not run.

• Per-user isolation is workspace separation inside one guest (shared

kernel/net/processes), not a security boundary between users — the same posture kanban already accepts for trusted operators. The VM boundary is what protects Chatsubo.

• The ansible tooling manifest makes adding a tool a one-line, reviewable,

rebuild-surviving change.

• The 9p home mapping means user files live on the host, so the 200 GB image

stays thin and user data is backed up with the host's ZFS.

Future work should not re-litigate: the one-VM/per-user-9p-home shape, the 16 GB/4 vCPU/200 GB sizing, the venv+systemd MULTI_USER=true run mode, the cloud-image + NoCloud + UEFI-with-BIOS-fallback provisioning, the ansible tooling manifest as the add-tooling mechanism, and the mirrored role trees with enforced parity.