eric
ba31d79368
Run 10536 reached pytest but every connect was reset: the simulator died at
import with 'Tkinter is needed' — lib/gui.py calls matplotlib.use("TkAgg")
unconditionally (an explicit use() overrides MPLBACKEND=Agg), and the port
probe passed instantly because it was accepting the localhost forwarders, not
the nodes.
- run_e2e.sh: patch meshtasticator lib/gui.py TkAgg->Agg before boot;
probe readiness on the upstream host (docker daemon) when forwards are
active; docker pull meshtastic/meshtasticd up front so the pull is off the
simulator's node-boot path.
- e2e tests: cancel leftover loop tasks (watchdog) at loop teardown.
142 lines
6.6 KiB
Markdown
142 lines
6.6 KiB
Markdown
# Meshtasticator e2e — radio-level validation for hermes-meshtastic
|
|
|
|
This directory is the **radio-level counterpart** to the contract suite
|
|
(`plugin/tests/contract`). The contract suite guarantees the adapter matches
|
|
the Hermes gateway API; it cannot prove the adapter actually talks to a
|
|
Meshtastic node. These e2e tests boot the
|
|
[Meshtasticator](https://meshtastic.org/docs/software/meshtasticator/)
|
|
interactive simulator — which runs the **real MeshtasticD device software**
|
|
per node with a simulated LoRa PHY — and drive the adapter against a live
|
|
simulated node.
|
|
|
|
| Layer | Validated by |
|
|
|---|---|
|
|
| Adapter ↔ Hermes gateway API | `plugin/tests/contract` (no deployment) |
|
|
| Adapter ↔ real MeshtasticD TCP/protobuf session | **this suite** |
|
|
| Inbound text across the (simulated) LoRa mesh | **this suite** |
|
|
| Outbound chunked sends arriving at another node | **this suite** |
|
|
| True RF (silicon LoRa, real propagation, duty cycle) | physical radio only |
|
|
|
|
## Layout
|
|
|
|
```
|
|
scripts/meshtasticator_e2e/
|
|
run_e2e.sh orchestrator: boots N nodes, runs pytest, tears down
|
|
tcp_forward.py localhost->daemon TCP forward (remote docker daemons)
|
|
README.md this file
|
|
plugin/tests/e2e/
|
|
test_meshtasticator_e2e.py the e2e assertions (skip-gated)
|
|
.gitea/workflows/
|
|
meshtasticator-e2e.yaml on-demand CI job (docker-enabled runner)
|
|
```
|
|
|
|
## Prerequisites
|
|
|
|
* **uv** + Python 3.13.
|
|
* A contract venv with the real gateway **and** the plugin (with deps) installed:
|
|
```bash
|
|
scripts/contract-test.sh # creates plugin/.venv-contract (hermes-agent pinned)
|
|
uv pip install --python plugin/.venv-contract/bin/python -e plugin
|
|
```
|
|
(`-e plugin` installs the meshtastic client library the tests also use.)
|
|
* A clone of Meshtasticator: `git clone https://github.com/meshtastic/meshtasticator`
|
|
* Runtime for the simulator — pick one:
|
|
* **Docker** (simplest, works on macOS/Windows): the simulator pulls the
|
|
`meshtastic/meshtasticd` image on first run (slow once). Node TCP API ports
|
|
are published on your docker host's localhost, which is what the simulator
|
|
itself expects.
|
|
* **Linux native**: build MeshtasticD's `native` PlatformIO target and pass
|
|
the build dir with `--program`.
|
|
* Optional but recommended: an isolated simulator venv, because Meshtasticator
|
|
pins old client libs (`meshtastic~=2.6.1` in its requirements.txt):
|
|
```bash
|
|
uv venv --python 3.13 /tmp/sim-venv
|
|
uv pip install --python /tmp/sim-venv/bin/python \
|
|
-r <meshtasticator>/requirements.txt docker
|
|
export SIM_VENV=/tmp/sim-venv # else the script uses plain `python3`
|
|
```
|
|
|
|
## Run
|
|
|
|
```bash
|
|
# Docker mode (default)
|
|
MESHTASTICATOR_DIR=/path/to/meshtasticator \
|
|
scripts/meshtasticator_e2e/run_e2e.sh
|
|
|
|
# Linux native mode
|
|
MESHTASTICATOR_DIR=/path/to/meshtasticator \
|
|
scripts/meshtasticator_e2e/run_e2e.sh \
|
|
--mode native --program /path/to/firmware/.pio/build/native
|
|
|
|
# Tune the scenario
|
|
... --nodes 3 --channel-index 0 --host 127.0.0.1
|
|
```
|
|
|
|
The orchestrator boots the simulator, waits until `N` node TCP API ports accept
|
|
connections (Meshtasticator assigns node *n* → port `4404 + n`), then runs
|
|
`pytest plugin/tests/e2e` with `MESHTASTICATOR_E2E=1`. Simulator output lands in
|
|
`meshtasticator-sim.log` at the repo root.
|
|
|
|
**Remote docker daemon** (`DOCKER_HOST=tcp://host:2375`): the simulator's own
|
|
control connections assume nodes listen on localhost, so the script starts a
|
|
`tcp_forward.py` per node port (localhost → daemon host) before booting the
|
|
simulator. Nothing is forwarded when the daemon is local (unset/unix
|
|
`DOCKER_HOST`). Readiness is probed on the *upstream* host (the daemon), never
|
|
on the forwarders themselves.
|
|
|
|
Other headless adaptations the script makes automatically:
|
|
* patches `lib/gui.py` `matplotlib.use("TkAgg")` → `"Agg"` in the pinned
|
|
Meshtasticator checkout — the TkAgg call crashes headless runs even under
|
|
`MPLBACKEND=Agg` (only the interactive GUI mode is lost),
|
|
* warms the `meshtastic/meshtasticd` image (`docker pull`) before booting the
|
|
simulator so the pull is not on the simulator's node-boot critical path.
|
|
|
|
Without `MESHTASTICATOR_E2E=1` the e2e module skips, so ordinary unit/CI runs
|
|
stay fast and gateway-free.
|
|
|
|
## CI
|
|
|
|
`.gitea/workflows/meshtasticator-e2e.yaml` runs the whole flow on a
|
|
docker-enabled runner (on demand via `workflow_dispatch` until validated, then
|
|
flip on the commented `pull_request` trigger): it builds the Hermes contract
|
|
env (pinned like `release.yaml`), installs the plugin with deps, clones
|
|
Meshtasticator at a pinned commit, installs the simulator's own requirements in
|
|
an isolated venv, and calls `run_e2e.sh --mode docker`. Simulator logs print on
|
|
failure. Only one e2e run at a time (the sim hardcodes the `Meshtastic`
|
|
container/volume name on the shared daemon).
|
|
|
|
## What the tests do
|
|
|
|
1. **Inbound**: peer node 1 broadcasts text; the adapter (connected to node 0)
|
|
must receive it as a real gateway `MessageEvent` (`host/port/channel_index`
|
|
come from `E2E_PORTS`/`E2E_HOST`/`E2E_CHANNEL_INDEX`).
|
|
2. **Outbound**: the adapter sends a >180-char message; it must be chunked and
|
|
the final chunk (`(i/n) … word59`) must arrive at peer node 1.
|
|
|
|
A `_ProbeAdapter` subclass records inbound events instead of dispatching to a
|
|
Hermes agent handler (no agent runs in these tests; dispatch is already covered
|
|
by the contract suite).
|
|
|
|
## Status / open items (v1 scaffold)
|
|
|
|
This is an initial, structurally validated scaffold. A full green run still
|
|
needs to happen on a Linux/Docker host with a MeshtasticD build, and these
|
|
items are expected to need iteration there:
|
|
|
|
* **Node adjacency**: the simulator's default random placement may put nodes out
|
|
of range. For deterministic tests, seed close-together coordinates (the sim
|
|
supports `--from-file` with an `out/nodeConfig.yaml`).
|
|
* **Channel semantics**: tests default to channel index 0 (stock primary
|
|
channel). Mirroring production (channel 1 / private channel) requires
|
|
configuring matching channels on the simulated nodes.
|
|
* **Reconnect flow**: killing/restarting a single simulated node needs per-node
|
|
lifecycle control (native mode subprocesses; docker mode currently runs all
|
|
nodes in one container), so the adapter's watchdog reconnect is **not** yet
|
|
exercised here. See the watchdog unit coverage in the adapter for now.
|
|
* **CI validation run**: `.gitea/workflows/meshtasticator-e2e.yaml` is drafted
|
|
and on-demand only; it still needs a first green run on the docker-enabled
|
|
runner (topology assumptions: docker reachable from the job, node ports on
|
|
the job's localhost via the automatic tcp forwards). Trigger it via
|
|
`workflow_dispatch`, watch the sim log step, and only then enable the
|
|
`pull_request` trigger.
|