From 80ccf671f322916d8662f81a723deb2756281b7f Mon Sep 17 00:00:00 2001 From: Sleeper85 Date: Wed, 5 Aug 2026 21:03:35 +0200 Subject: [PATCH] Create Understanding_ESPHome_Modbus.md --- .../README/Understanding_ESPHome_Modbus.md | 218 ++++++++++++++++++ .../YamBMS_multi-node_modbus_settings.md | 205 ---------------- 2 files changed, 218 insertions(+), 205 deletions(-) create mode 100644 documents/README/Understanding_ESPHome_Modbus.md delete mode 100644 documents/README/YamBMS_multi-node_modbus_settings.md diff --git a/documents/README/Understanding_ESPHome_Modbus.md b/documents/README/Understanding_ESPHome_Modbus.md new file mode 100644 index 0000000..42282fd --- /dev/null +++ b/documents/README/Understanding_ESPHome_Modbus.md @@ -0,0 +1,218 @@ +# YamBMS - Understanding ESPHome Modbus + +[![Badge License: GPLv3](https://img.shields.io/badge/License-GPLv3-brightgreen.svg)](https://www.gnu.org/licenses/gpl-3.0) +[![Badge Version](https://img.shields.io/github/v/release/Sleeper85/esphome-yambms?include_prereleases&color=yellow&logo=DocuSign&logoColor=white)](https://github.com/Sleeper85/esphome-yambms/releases/latest) +![GitHub stars](https://img.shields.io/github/stars/Sleeper85/esphome-yambms) +![GitHub forks](https://img.shields.io/github/forks/Sleeper85/esphome-yambms) +![GitHub watchers](https://img.shields.io/github/watchers/Sleeper85/esphome-yambms) + +> **Work in progress.** The ESPHome Modbus stack is being reworked by +> [@exciton](https://github.com/exciton) through a long series of pull requests. +> The first of them landed in ESPHome 2026.7, and more are on the way — including +> a full overhaul of `modbus_controller` +> ([#11781](https://github.com/esphome/esphome/pull/11781)) that will change how +> commands are queued and how offline devices are handled. This page tracks that +> work and will be updated progressively as the PRs are merged. + +Starting with **ESPHome 2026.7**, the Modbus client timing defaults have changed +(`send_wait_time` 250 ms → 2000 ms, `turnaround_time` 100 ms → 600 ms). +These new defaults are safe for slow third-party hardware, but they are far too +conservative for a YamBMS setup where many fast ESP32-based servers share one bus. + +YamBMS therefore ships its own defaults: + +```yaml +substitutions: + # UART + modbus_baud_rate: '19200' # 9600 / 19200 / 115200 + # Modbus + modbus_id: '1' + modbus_send_wait_time: '200ms' # Timeout: max time to wait for the FIRST byte of a response + modbus_turnaround_time: '0ms' # Pause after a completed response before the next command + # Modbus Controller + modbus_update_interval: '5s' # Frequency with which BMS/Balancer/Shunt (modbus servers) are queried + +modbus_controller: + max_cmd_retries: 1 # Retransmissions after a timeout (default is 4) + offline_skip_updates: 5 # Server re-polled every 30s with an update_interval of 5s +``` + +> **The one idea to remember:** what costs time on the bus is the **number of +> Modbus commands per cycle**, not the number of servers. A BMS whose register +> map has gaps generates several commands instead of one. Reducing the command +> count is usually worth more than raising the baud rate. + +--- + +## What each parameter does + +### `send_wait_time` — response timeout + +Maximum time the client waits for the **first byte** of a response. If nothing +has started arriving within this time, the command is treated as failed. + +* It is a **latency** budget, not a transmission budget: it does not depend on + the baud rate or on the response length. Once the first byte arrives, the + client waits for the whole response however long it takes. +* It only costs time **when a server does not answer**. Lowering it has no effect + on a healthy bus — it just bounds the price of a failure. + +An ESPHome server on ESP32 answers within one pass of its main loop (16 ms by +default). `200ms` leaves ample margin for an occasional slow loop (WiFi burst, +API reconnect, flash write) without ever mistaking a busy server for a dead one. + +### `turnaround_time` — inter-command pause + +Pause inserted after a completed response, before the next command. It exists to +give slow devices time to get ready for the next frame. + +**It is charged per command, not per server** — including between two commands +sent to the same BMS. With 24 BMS generating 4 commands each, a +`turnaround_time` of 100 ms would add 96 × 100 ms = **9.6 s** to every cycle. + +`0ms` is the right value for an all-ESPHome bus. It never produces truly +back-to-back frames: the Modbus specification silence (3.5 characters) is always +enforced, and the ESPHome main loop adds its own scheduling delay on both sides. + +### `update_interval` — polling frequency + +How often each `modbus_controller` reads **all** its register ranges. All +controllers fire at roughly the same time and the resulting commands are +serialized on the bus, one conversation at a time. + +### `max_cmd_retries` — retransmissions + +Retransmissions of a timed-out command before giving up (not counting the initial +transmission). ESPHome defaults to `4`; YamBMS uses `1`. One retry absorbs a +transient glitch, and a genuinely dead server is handed to `offline_skip_updates` +quickly instead of blocking the bus. + +### `offline_skip_updates` — offline back-off + +Once a server has exhausted its retries it is marked **offline**, and this many +update cycles are skipped before it is probed again. With `5`, a silent server is +re-polled every `(5 + 1) × update_interval` = **30 s**. As soon as it answers a +probe it is marked online and normal polling resumes — no reboot or manual action +needed. + +--- + +## Reducing the number of commands + +ESPHome merges **consecutive** registers into a single range, and reads each +range with one Modbus command. A single unmapped register in the middle splits +the range in two — and every extra command costs a request frame, a response +frame, and a fixed scheduling overhead. + +### Fill the gaps with `register_count` + +`register_count` can span a gap. Set it on the sensor **before** the gap, sized +to reach the next register you actually want. The registers in between are read +and discarded. + +Example — a PACE BMS map with gaps at register 8 and registers 13–14: + +| Sensor to modify | Value | Effect | +|------------------|-------|--------| +| last sensor before the gap (address 7) | `register_count: 2` | absorbs register 8 | +| last sensor before the gap (address 12) | `register_count: 3` | absorbs registers 13–14 | + +Ranges `0–7`, `9–12` and `15–36` then merge into a **single command covering +0–36**, taking that BMS from 4 commands per cycle down to 2 — roughly a third off +the total cycle time, for the cost of 3 useless registers read (6 bytes). + +> Check first that the skipped registers are actually **readable** on your +> device. An invalid register anywhere inside a range makes the whole command +> fail with a Modbus exception. + +### `force_new_range` — use sparingly + +`force_new_range: true` (default `false`, available on every `modbus_controller` +platform) prevents a sensor from joining the previous range, forcing a separate +command. It works in the *opposite* direction to the optimisation above, so use +it only to isolate genuinely static registers that can then carry a high +`skip_updates`. + +Two things to know: + +* `skip_updates` applies to the **whole range**, and ESPHome keeps the lowest + non-zero value among its sensors. One fast sensor inside a large block forces + the entire block to the fast rate. +* Sensors marked `force_new_range` are sorted **before** all others, ahead of the + address sort. Marking one sensor in the middle of a block does not split it in + two — it pulls that sensor to the front and lets the rest merge. To split a + block, mark the first sensor of each intended segment. + +It is also set automatically for any sensor using `custom_command`. + +### Sensors can share a range for free + +Several sensors may read the same registers without any extra bus traffic — they +all decode the same response buffer. A `delta cell voltage` sensor declared at the +same address as `cell voltage 1`, with a lambda walking all 16 cells, costs +**zero** additional commands. Two declarations at the same address produce one +transaction, not two. + +### YAML order: when it matters + +Sensors are sorted internally (by register type, `force_new_range`, address, then +offset), so their order in the file is normally irrelevant. + +**Except when two sensors share the same address and offset.** The final +tie-break is allocation order — that is, YAML order — and the sensor that comes +first defines the range's `register_count`. + +With `cell voltage 1` (1 register) and `delta cell voltage` +(`register_count: 16`) both at address 15: + +* `cell voltage 1` **first** → the range opens with count 1, cells 2–16 extend it + normally → **one command**. ✅ +* `delta` **first** → the range opens with count 16, cell 2 no longer fits the + extension test → **a second range starts**, and registers 16–30 are read twice. ❌ + +Leave a comment in your YAML so nobody reorders these by accident. + +--- + +## Tuning guide + +* **`send_wait_time`**: set it to the slowest observed "time to first byte" of any + server on the bus, with generous margin. `200ms` suits all-ESPHome/ESP32 buses. + Raise it only if a device that is genuinely alive produces spurious timeouts. + Do **not** raise it "to be safe": it is the unit price of every dead-server + event. +* **`turnaround_time`**: keep `0ms` on an all-ESPHome bus. For a slow third-party + device try `10–30ms` first; `100ms` (the old ESPHome default) is the safe + fallback for unknown hardware, and `200–600ms` only for VFDs and gateways. + Remember the cost is per command. +* **Mixed buses**: `turnaround_time` belongs to the *hub*, so one slow device + imposes its value on all your BMS. Put slow third-party hardware on a **second + UART** with its own `modbus:` hub rather than raising the global value. +* **`modbus_baud_rate`**: `19200` is a good default. Gains flatten out above it, + because per-command scheduling overhead starts to dominate transmission time — + merge ranges before reaching for a higher baud rate. +* **`update_interval`**: leave enough headroom that the full cycle still fits when + a server is dead. If some entities stop refreshing while others keep updating, + the cycle is overrunning the interval — reduce the command count, raise the + interval, or split the bus. +* **`offline_skip_updates`**: `(value + 1) × update_interval` is the re-probe + period. `5` → every 30 s at a 5 s interval. Lower it to `3` for faster recovery + detection. + +--- + +## Checking your configuration + +Set the logger to `DEBUG` (or `VERBOSE` for the range details) and watch a few +cycles: + +```yaml +logger: + level: DEBUG +``` + +* Ranges are logged as they are built — the quickest way to confirm that a + `register_count` gap fill actually merged what you expected. +* `Modbus command timed out` on a device that is alive means `send_wait_time` is + too low, or the bus is saturated. +* `Modbus device=N set offline` / `back online` traces the offline back-off. diff --git a/documents/README/YamBMS_multi-node_modbus_settings.md b/documents/README/YamBMS_multi-node_modbus_settings.md deleted file mode 100644 index 2467a36..0000000 --- a/documents/README/YamBMS_multi-node_modbus_settings.md +++ /dev/null @@ -1,205 +0,0 @@ -# YamBMS - Multi-node Modbus settings - -[![Badge License: GPLv3](https://img.shields.io/badge/License-GPLv3-brightgreen.svg)](https://www.gnu.org/licenses/gpl-3.0) -[![Badge Version](https://img.shields.io/github/v/release/Sleeper85/esphome-yambms?include_prereleases&color=yellow&logo=DocuSign&logoColor=white)](https://github.com/Sleeper85/esphome-yambms/releases/latest) -![GitHub stars](https://img.shields.io/github/stars/Sleeper85/esphome-yambms) -![GitHub forks](https://img.shields.io/github/forks/Sleeper85/esphome-yambms) -![GitHub watchers](https://img.shields.io/github/watchers/Sleeper85/esphome-yambms) - -Starting with **ESPHome 2026.7**, the Modbus client timing defaults have changed -(`send_wait_time` 250 ms → 2000 ms, `turnaround_time` 100 ms → 600 ms). -These new defaults are safe for slow third-party hardware, but they are far too -conservative for a YamBMS setup where many fast ESP32-based servers share one bus. - -YamBMS therefore ships its own defaults: - -```yaml -substitutions: - # UART - modbus_baud_rate: '19200' # 9600 / 19200 / 115200 - # Modbus - modbus_id: '1' - modbus_send_wait_time: '200ms' # Timeout: max time to wait for the FIRST byte of a response - modbus_turnaround_time: '0ms' # Pause after a completed response before the next command - # Modbus Controller - modbus_update_interval: '5s' # Frequency with which BMS/Balancer/Shunt (modbus servers) are queried - -modbus_controller: - max_cmd_retries: 1 # Retransmissions after a timeout (default is 4) - offline_skip_updates: 5 # Server re-polled every 30s with an update_interval of 5s -``` - -This page explains what each parameter does, what actually happens on the bus -every `update_interval`, and how to tune the values for your hardware. - -## What each parameter does - -### `send_wait_time` — response timeout - -Maximum time the client waits for the **first byte** of a response after sending -a command. If nothing has started arriving within this time, the command is -considered failed (timeout). - -Two important properties: - -* It is a **latency** budget, not a transmission budget. It does not depend on - the baud rate or on the response length. Once the first byte arrives, the - client waits for the complete response no matter how long it takes. -* It only costs time **when a server does not answer**. Lowering it has zero - effect on a healthy bus — it only bounds the price of a failure. - -An ESPHome server on ESP32 normally starts answering within 20–30 ms (one pass -of the main loop). `200ms` leaves ample margin for the occasional slow loop -(WiFi burst, API reconnect, flash write) without ever mistaking a busy server -for a dead one. - -### `turnaround_time` — inter-command pause - -Pause inserted after a **completed** response before the next command is sent. -Its purpose is to give slow devices time to get ready for the next frame. - -With ESPHome firmware on ESP32 on both sides of the bus, this can safely be set -to `0ms`: the silent interval required by the Modbus specification (3.5 -characters, ≈ 2 ms at 19200 baud) is still enforced automatically. Only raise -this value if you mix in third-party devices that need extra recovery time. - -### `update_interval` — polling frequency - -How often each `modbus_controller` schedules a read of its registers. All -controllers fire at (roughly) the same time; the commands are then serialized -on the bus one by one (see below). - -### `max_cmd_retries` — retransmissions - -How many times a timed-out command is retransmitted before giving up -(not counting the initial transmission). The ESPHome default is `4`, which -makes a dead server cost `(1 + 4) × send_wait_time` per cycle. YamBMS lowers -it to `1`: one retry is enough to ride out a transient glitch, and a truly -dead server is handed over to `offline_skip_updates` quickly. - -### `offline_skip_updates` — offline back-off - -When a server exhausts its retries, it is marked **offline** and this many -update cycles are skipped before it is probed again. With `5`, a silent server -is re-polled every `(5 + 1) × update_interval` = **30 s**. As soon as it -answers a probe, it is marked online again and normal polling resumes — -no reboot or manual action needed. - -## How the bus is shared - -There is exactly **one wire and one conversation at a time**. ESPHome enforces -this with a two-level design: - -1. Each `modbus_controller` (one per BMS/Balancer/Shunt) keeps its own small - command queue and submits **one command at a time** to the shared `modbus` - hub, waiting for the response (or the final failure) before submitting the - next one. -2. The `modbus` hub owns the bus: a single FIFO transmit queue and a single - "waiting for response" slot. It sends one frame, waits for the response or - the `send_wait_time` timeout, applies `turnaround_time`, then sends the - next frame in the queue. - -A read of 33 consecutive registers is a **single Modbus transaction**: -one request frame (8 bytes) and one response frame (71 bytes). - -## What happens every 5 seconds — all 24 servers responding - -At each `update_interval` tick, all 24 controllers queue their read command. -The hub then works through them strictly one at a time: - -``` -tick (t=0) idle until next tick - │ │ - ▼ ▼ - ├─ Server 1 ──┬─ Server 2 ──┬─ Server 3 ── ... ──┬─ Server 24 ──────────┤ - │ │ │ - │ request (8 bytes ≈ 4.6 ms at 19200 baud) │ - │ latency (first byte after ≈ 10–30 ms) │ - │ response (71 bytes ≈ 40.7 ms at 19200 baud) │ - │ turnaround_time = 0 → next command immediately (spec silence ≈ 2 ms) │ - │ │ - │ ≈ 55–75 ms per server │ - └── 24 servers ≈ 1.4–1.8 s total ────────────────────────────────────────┘ -``` - -The whole cycle completes in well under 2 seconds, then the bus sits idle for -about 3 seconds until the next tick. Every controller drains its queue long -before its next update — there is no backlog and no starvation. - -Approximate full-cycle duration for 24 servers × 33 registers: - -| Baud rate | Per transaction | Full cycle (24 servers) | Headroom in 5 s | -|-----------|----------------|-------------------------|-----------------| -| 9600 | ≈ 100–120 ms | ≈ 2.4–2.9 s | ≈ 2× | -| **19200** | **≈ 55–75 ms** | **≈ 1.4–1.8 s** | **≈ 3×** | -| 115200 | ≈ 15–35 ms | ≈ 0.4–0.8 s | ≈ 6× | - -## What happens when one server does not respond - -Say Server 7 is disconnected or has crashed: - -**First cycle after the failure** - -``` -... ─ Server 6 ─┬────────── Server 7 ──────────┬─ Server 8 ─ ... - │ │ - │ request → silence │ - │ wait send_wait_time (200 ms)│ - │ retry → silence │ - │ wait send_wait_time (200 ms)│ - │ → give up, mark OFFLINE │ - │ │ - └── cost: (1+1) × 200 ms = 400 ms, then move on -``` - -The other 23 servers are read normally. Total cycle: -23 × ~65 ms + 400 ms ≈ **1.9 s** — still far below the 5 s budget, so nothing -piles up and every other controller keeps refreshing on time. - -**Subsequent cycles** - -Server 7 is offline: the next 5 update cycles skip it entirely (zero cost on -the bus). Every 30 s it is probed once (400 ms worst case). When it answers -again, it is marked online and rejoins normal polling from the next cycle. - -**Why this matters — the same scenario with ESPHome 2026.7 defaults** - -With `send_wait_time: 2000ms` and `max_cmd_retries: 4`, a single dead server -would cost `(1 + 4) × 2000 ms = 10 s` per cycle — **twice the entire -update_interval on its own**. The command queues fill faster than they drain, -and since bus access is not fairly arbitrated, the controllers at the end of -the line may **never** get their turn (some entities stop refreshing -indefinitely). This starvation issue is a long-standing architectural -limitation, exposed (not caused) by the new defaults, and is queued to be -fixed in a future release. Until then, the YamBMS values above keep the -worst-case cycle short enough that the problem cannot occur. - -## Tuning guide - -* **`send_wait_time`**: set it to the slowest observed "time to first byte" of - any server on your bus, with generous margin. `200ms` is right for - all-ESPHome/ESP32 buses. Raise it only if you see spurious timeouts on a - device that is actually alive (check the logs for `Modbus command timed - out`). Do **not** raise it "to be safe" — every dead-server event costs - `(1 + max_cmd_retries) × send_wait_time`. -* **`turnaround_time`**: keep `0ms` for all-ESPHome devices. If a third-party - device occasionally misses requests, try `5–20ms` before anything larger. -* **`modbus_baud_rate`**: higher is better for cycle time; `19200` gives a - comfortable 3× headroom with 24 servers. Use `9600` only if your wiring is - long or noisy. -* **`update_interval`**: with the timings above you could go down to 2–3 s if - you want faster refresh; make sure the full cycle (table above) stays well - below the interval, dead-server penalty included. -* **`offline_skip_updates`**: `(value + 1) × update_interval` is the re-probe - period. `5` → every 30 s at a 5 s interval. Since a failed probe only costs - 400 ms, you can lower it to `3` for faster recovery detection with no - measurable impact on the bus. - -**Rule of thumb:** - -``` -worst-case cycle ≈ (N_online × per-transaction time) - + N_offline_being_probed × (1 + max_cmd_retries) × send_wait_time - -→ keep this comfortably below update_interval -```