Create Understanding_ESPHome_Modbus.md

This commit is contained in:
Sleeper85 committed 2026-11-08 23:08:32 +01:00
1 parent 9b2787ecca
commit 80ccf671f3
2 files changed
+218 -205

No files matched your search

@@ -0,0 +1,218 @@
# YamBMS - Understanding ESPHome Modbus
[![Badge License: GPLv3](https://img.shields.io/badge/License-GPLv3-brightgreen.svg)](https://www.gnu.org/licenses/gpl-3.0)
[![Badge Version](https://img.shields.io/github/v/release/Sleeper85/esphome-yambms?include_prereleases&color=yellow&logo=DocuSign&logoColor=white)](https://github.com/Sleeper85/esphome-yambms/releases/latest)
![GitHub stars](https://img.shields.io/github/stars/Sleeper85/esphome-yambms)
![GitHub forks](https://img.shields.io/github/forks/Sleeper85/esphome-yambms)
![GitHub watchers](https://img.shields.io/github/watchers/Sleeper85/esphome-yambms)
> **Work in progress.** The ESPHome Modbus stack is being reworked by
> [@exciton](https://github.com/exciton) through a long series of pull requests.
> The first of them landed in ESPHome 2026.7, and more are on the way — including
> a full overhaul of `modbus_controller`
> ([#11781](https://github.com/esphome/esphome/pull/11781)) that will change how
> commands are queued and how offline devices are handled. This page tracks that
> work and will be updated progressively as the PRs are merged.
Starting with **ESPHome 2026.7**, the Modbus client timing defaults have changed
(`send_wait_time` 250 ms → 2000 ms, `turnaround_time` 100 ms → 600 ms).
These new defaults are safe for slow third-party hardware, but they are far too
conservative for a YamBMS setup where many fast ESP32-based servers share one bus.
YamBMS therefore ships its own defaults:
```yaml
substitutions:
# UART
modbus_baud_rate: '19200' # 9600 / 19200 / 115200
# Modbus
modbus_id: '1'
modbus_send_wait_time: '200ms' # Timeout: max time to wait for the FIRST byte of a response
modbus_turnaround_time: '0ms' # Pause after a completed response before the next command
# Modbus Controller
modbus_update_interval: '5s' # Frequency with which BMS/Balancer/Shunt (modbus servers) are queried
modbus_controller:
max_cmd_retries: 1 # Retransmissions after a timeout (default is 4)
offline_skip_updates: 5 # Server re-polled every 30s with an update_interval of 5s
```
> **The one idea to remember:** what costs time on the bus is the **number of
> Modbus commands per cycle**, not the number of servers. A BMS whose register
> map has gaps generates several commands instead of one. Reducing the command
> count is usually worth more than raising the baud rate.
---
## What each parameter does
### `send_wait_time` — response timeout
Maximum time the client waits for the **first byte** of a response. If nothing
has started arriving within this time, the command is treated as failed.
* It is a **latency** budget, not a transmission budget: it does not depend on
the baud rate or on the response length. Once the first byte arrives, the
client waits for the whole response however long it takes.
* It only costs time **when a server does not answer**. Lowering it has no effect
on a healthy bus — it just bounds the price of a failure.
An ESPHome server on ESP32 answers within one pass of its main loop (16 ms by
default). `200ms` leaves ample margin for an occasional slow loop (WiFi burst,
API reconnect, flash write) without ever mistaking a busy server for a dead one.
### `turnaround_time` — inter-command pause
Pause inserted after a completed response, before the next command. It exists to
give slow devices time to get ready for the next frame.
**It is charged per command, not per server** — including between two commands
sent to the same BMS. With 24 BMS generating 4 commands each, a
`turnaround_time` of 100 ms would add 96 × 100 ms = **9.6 s** to every cycle.
`0ms` is the right value for an all-ESPHome bus. It never produces truly
back-to-back frames: the Modbus specification silence (3.5 characters) is always
enforced, and the ESPHome main loop adds its own scheduling delay on both sides.
### `update_interval` — polling frequency
How often each `modbus_controller` reads **all** its register ranges. All
controllers fire at roughly the same time and the resulting commands are
serialized on the bus, one conversation at a time.
### `max_cmd_retries` — retransmissions
Retransmissions of a timed-out command before giving up (not counting the initial
transmission). ESPHome defaults to `4`; YamBMS uses `1`. One retry absorbs a
transient glitch, and a genuinely dead server is handed to `offline_skip_updates`
quickly instead of blocking the bus.
### `offline_skip_updates` — offline back-off
Once a server has exhausted its retries it is marked **offline**, and this many
update cycles are skipped before it is probed again. With `5`, a silent server is
re-polled every `(5 + 1) × update_interval` = **30 s**. As soon as it answers a
probe it is marked online and normal polling resumes — no reboot or manual action
needed.
---
## Reducing the number of commands
ESPHome merges **consecutive** registers into a single range, and reads each
range with one Modbus command. A single unmapped register in the middle splits
the range in two — and every extra command costs a request frame, a response
frame, and a fixed scheduling overhead.
### Fill the gaps with `register_count`
`register_count` can span a gap. Set it on the sensor **before** the gap, sized
to reach the next register you actually want. The registers in between are read
and discarded.
Example — a PACE BMS map with gaps at register 8 and registers 13–14:
| Sensor to modify | Value | Effect |
|------------------|-------|--------|
| last sensor before the gap (address 7) | `register_count: 2` | absorbs register 8 |
| last sensor before the gap (address 12) | `register_count: 3` | absorbs registers 13–14 |
Ranges `0–7`, `9–12` and `15–36` then merge into a **single command covering
0–36**, taking that BMS from 4 commands per cycle down to 2 — roughly a third off
the total cycle time, for the cost of 3 useless registers read (6 bytes).
> Check first that the skipped registers are actually **readable** on your
> device. An invalid register anywhere inside a range makes the whole command
> fail with a Modbus exception.
### `force_new_range` — use sparingly
`force_new_range: true` (default `false`, available on every `modbus_controller`
platform) prevents a sensor from joining the previous range, forcing a separate
command. It works in the *opposite* direction to the optimisation above, so use
it only to isolate genuinely static registers that can then carry a high
`skip_updates`.
Two things to know:
* `skip_updates` applies to the **whole range**, and ESPHome keeps the lowest
non-zero value among its sensors. One fast sensor inside a large block forces
the entire block to the fast rate.
* Sensors marked `force_new_range` are sorted **before** all others, ahead of the
address sort. Marking one sensor in the middle of a block does not split it in
two — it pulls that sensor to the front and lets the rest merge. To split a
block, mark the first sensor of each intended segment.
It is also set automatically for any sensor using `custom_command`.
### Sensors can share a range for free
Several sensors may read the same registers without any extra bus traffic — they
all decode the same response buffer. A `delta cell voltage` sensor declared at the
same address as `cell voltage 1`, with a lambda walking all 16 cells, costs
**zero** additional commands. Two declarations at the same address produce one
transaction, not two.
### YAML order: when it matters
Sensors are sorted internally (by register type, `force_new_range`, address, then
offset), so their order in the file is normally irrelevant.
**Except when two sensors share the same address and offset.** The final
tie-break is allocation order — that is, YAML order — and the sensor that comes
first defines the range's `register_count`.
With `cell voltage 1` (1 register) and `delta cell voltage`
(`register_count: 16`) both at address 15:
* `cell voltage 1` **first** → the range opens with count 1, cells 2–16 extend it
normally → **one command**. ✅
* `delta` **first** → the range opens with count 16, cell 2 no longer fits the
extension test → **a second range starts**, and registers 16–30 are read twice. ❌
Leave a comment in your YAML so nobody reorders these by accident.
---
## Tuning guide
* **`send_wait_time`**: set it to the slowest observed "time to first byte" of any
server on the bus, with generous margin. `200ms` suits all-ESPHome/ESP32 buses.
Raise it only if a device that is genuinely alive produces spurious timeouts.
Do **not** raise it "to be safe": it is the unit price of every dead-server
event.
* **`turnaround_time`**: keep `0ms` on an all-ESPHome bus. For a slow third-party
device try `10–30ms` first; `100ms` (the old ESPHome default) is the safe
fallback for unknown hardware, and `200–600ms` only for VFDs and gateways.
Remember the cost is per command.
* **Mixed buses**: `turnaround_time` belongs to the *hub*, so one slow device
imposes its value on all your BMS. Put slow third-party hardware on a **second
UART** with its own `modbus:` hub rather than raising the global value.
* **`modbus_baud_rate`**: `19200` is a good default. Gains flatten out above it,
because per-command scheduling overhead starts to dominate transmission time —
merge ranges before reaching for a higher baud rate.
* **`update_interval`**: leave enough headroom that the full cycle still fits when
a server is dead. If some entities stop refreshing while others keep updating,
the cycle is overrunning the interval — reduce the command count, raise the
interval, or split the bus.
* **`offline_skip_updates`**: `(value + 1) × update_interval` is the re-probe
period. `5` → every 30 s at a 5 s interval. Lower it to `3` for faster recovery
detection.
---
## Checking your configuration
Set the logger to `DEBUG` (or `VERBOSE` for the range details) and watch a few
cycles:
```yaml
logger:
level: DEBUG
```
* Ranges are logged as they are built — the quickest way to confirm that a
`register_count` gap fill actually merged what you expected.
* `Modbus command timed out` on a device that is alive means `send_wait_time` is
too low, or the bus is saturated.
* `Modbus device=N set offline` / `back online` traces the offline back-off.
@@ -1,205 +0,0 @@
# YamBMS - Multi-node Modbus settings
[![Badge License: GPLv3](https://img.shields.io/badge/License-GPLv3-brightgreen.svg)](https://www.gnu.org/licenses/gpl-3.0)
[![Badge Version](https://img.shields.io/github/v/release/Sleeper85/esphome-yambms?include_prereleases&color=yellow&logo=DocuSign&logoColor=white)](https://github.com/Sleeper85/esphome-yambms/releases/latest)
![GitHub stars](https://img.shields.io/github/stars/Sleeper85/esphome-yambms)
![GitHub forks](https://img.shields.io/github/forks/Sleeper85/esphome-yambms)
![GitHub watchers](https://img.shields.io/github/watchers/Sleeper85/esphome-yambms)
Starting with **ESPHome 2026.7**, the Modbus client timing defaults have changed
(`send_wait_time` 250 ms → 2000 ms, `turnaround_time` 100 ms → 600 ms).
These new defaults are safe for slow third-party hardware, but they are far too
conservative for a YamBMS setup where many fast ESP32-based servers share one bus.
YamBMS therefore ships its own defaults:
```yaml
substitutions:
# UART
modbus_baud_rate: '19200' # 9600 / 19200 / 115200
# Modbus
modbus_id: '1'
modbus_send_wait_time: '200ms' # Timeout: max time to wait for the FIRST byte of a response
modbus_turnaround_time: '0ms' # Pause after a completed response before the next command
# Modbus Controller
modbus_update_interval: '5s' # Frequency with which BMS/Balancer/Shunt (modbus servers) are queried
modbus_controller:
max_cmd_retries: 1 # Retransmissions after a timeout (default is 4)
offline_skip_updates: 5 # Server re-polled every 30s with an update_interval of 5s
```
This page explains what each parameter does, what actually happens on the bus
every `update_interval`, and how to tune the values for your hardware.
## What each parameter does
### `send_wait_time` — response timeout
Maximum time the client waits for the **first byte** of a response after sending
a command. If nothing has started arriving within this time, the command is
considered failed (timeout).
Two important properties:
* It is a **latency** budget, not a transmission budget. It does not depend on
the baud rate or on the response length. Once the first byte arrives, the
client waits for the complete response no matter how long it takes.
* It only costs time **when a server does not answer**. Lowering it has zero
effect on a healthy bus — it only bounds the price of a failure.
An ESPHome server on ESP32 normally starts answering within 20–30 ms (one pass
of the main loop). `200ms` leaves ample margin for the occasional slow loop
(WiFi burst, API reconnect, flash write) without ever mistaking a busy server
for a dead one.
### `turnaround_time` — inter-command pause
Pause inserted after a **completed** response before the next command is sent.
Its purpose is to give slow devices time to get ready for the next frame.
With ESPHome firmware on ESP32 on both sides of the bus, this can safely be set
to `0ms`: the silent interval required by the Modbus specification (3.5
characters, ≈ 2 ms at 19200 baud) is still enforced automatically. Only raise
this value if you mix in third-party devices that need extra recovery time.
### `update_interval` — polling frequency
How often each `modbus_controller` schedules a read of its registers. All
controllers fire at (roughly) the same time; the commands are then serialized
on the bus one by one (see below).
### `max_cmd_retries` — retransmissions
How many times a timed-out command is retransmitted before giving up
(not counting the initial transmission). The ESPHome default is `4`, which
makes a dead server cost `(1 + 4) × send_wait_time` per cycle. YamBMS lowers
it to `1`: one retry is enough to ride out a transient glitch, and a truly
dead server is handed over to `offline_skip_updates` quickly.
### `offline_skip_updates` — offline back-off
When a server exhausts its retries, it is marked **offline** and this many
update cycles are skipped before it is probed again. With `5`, a silent server
is re-polled every `(5 + 1) × update_interval` = **30 s**. As soon as it
answers a probe, it is marked online again and normal polling resumes —
no reboot or manual action needed.
## How the bus is shared
There is exactly **one wire and one conversation at a time**. ESPHome enforces
this with a two-level design:
1. Each `modbus_controller` (one per BMS/Balancer/Shunt) keeps its own small
command queue and submits **one command at a time** to the shared `modbus`
hub, waiting for the response (or the final failure) before submitting the
next one.
2. The `modbus` hub owns the bus: a single FIFO transmit queue and a single
"waiting for response" slot. It sends one frame, waits for the response or
the `send_wait_time` timeout, applies `turnaround_time`, then sends the
next frame in the queue.
A read of 33 consecutive registers is a **single Modbus transaction**:
one request frame (8 bytes) and one response frame (71 bytes).
## What happens every 5 seconds — all 24 servers responding
At each `update_interval` tick, all 24 controllers queue their read command.
The hub then works through them strictly one at a time:
```
tick (t=0) idle until next tick
│ │
▼ ▼
├─ Server 1 ──┬─ Server 2 ──┬─ Server 3 ── ... ──┬─ Server 24 ──────────┤
│ │ │
│ request (8 bytes ≈ 4.6 ms at 19200 baud) │
│ latency (first byte after ≈ 10–30 ms) │
│ response (71 bytes ≈ 40.7 ms at 19200 baud) │
│ turnaround_time = 0 → next command immediately (spec silence ≈ 2 ms) │
│ │
│ ≈ 55–75 ms per server │
└── 24 servers ≈ 1.4–1.8 s total ────────────────────────────────────────┘
```
The whole cycle completes in well under 2 seconds, then the bus sits idle for
about 3 seconds until the next tick. Every controller drains its queue long
before its next update — there is no backlog and no starvation.
Approximate full-cycle duration for 24 servers × 33 registers:
| Baud rate | Per transaction | Full cycle (24 servers) | Headroom in 5 s |
|-----------|----------------|-------------------------|-----------------|
| 9600 | ≈ 100–120 ms | ≈ 2.4–2.9 s | ≈ 2× |
| **19200** | **≈ 55–75 ms** | **≈ 1.4–1.8 s** | **≈ 3×** |
| 115200 | ≈ 15–35 ms | ≈ 0.4–0.8 s | ≈ 6× |
## What happens when one server does not respond
Say Server 7 is disconnected or has crashed:
**First cycle after the failure**
```
... ─ Server 6 ─┬────────── Server 7 ──────────┬─ Server 8 ─ ...
│ │
│ request → silence │
│ wait send_wait_time (200 ms)│
│ retry → silence │
│ wait send_wait_time (200 ms)│
│ → give up, mark OFFLINE │
│ │
└── cost: (1+1) × 200 ms = 400 ms, then move on
```
The other 23 servers are read normally. Total cycle:
23 × ~65 ms + 400 ms ≈ **1.9 s** — still far below the 5 s budget, so nothing
piles up and every other controller keeps refreshing on time.
**Subsequent cycles**
Server 7 is offline: the next 5 update cycles skip it entirely (zero cost on
the bus). Every 30 s it is probed once (400 ms worst case). When it answers
again, it is marked online and rejoins normal polling from the next cycle.
**Why this matters — the same scenario with ESPHome 2026.7 defaults**
With `send_wait_time: 2000ms` and `max_cmd_retries: 4`, a single dead server
would cost `(1 + 4) × 2000 ms = 10 s` per cycle — **twice the entire
update_interval on its own**. The command queues fill faster than they drain,
and since bus access is not fairly arbitrated, the controllers at the end of
the line may **never** get their turn (some entities stop refreshing
indefinitely). This starvation issue is a long-standing architectural
limitation, exposed (not caused) by the new defaults, and is queued to be
fixed in a future release. Until then, the YamBMS values above keep the
worst-case cycle short enough that the problem cannot occur.
## Tuning guide
* **`send_wait_time`**: set it to the slowest observed "time to first byte" of
any server on your bus, with generous margin. `200ms` is right for
all-ESPHome/ESP32 buses. Raise it only if you see spurious timeouts on a
device that is actually alive (check the logs for `Modbus command timed
out`). Do **not** raise it "to be safe" — every dead-server event costs
`(1 + max_cmd_retries) × send_wait_time`.
* **`turnaround_time`**: keep `0ms` for all-ESPHome devices. If a third-party
device occasionally misses requests, try `5–20ms` before anything larger.
* **`modbus_baud_rate`**: higher is better for cycle time; `19200` gives a
comfortable 3× headroom with 24 servers. Use `9600` only if your wiring is
long or noisy.
* **`update_interval`**: with the timings above you could go down to 2–3 s if
you want faster refresh; make sure the full cycle (table above) stays well
below the interval, dead-server penalty included.
* **`offline_skip_updates`**: `(value + 1) × update_interval` is the re-probe
period. `5` → every 30 s at a 5 s interval. Since a failed probe only costs
400 ms, you can lower it to `3` for faster recovery detection with no
measurable impact on the bus.
**Rule of thumb:**
```
worst-case cycle ≈ (N_online × per-transaction time)
+ N_offline_being_probed × (1 + max_cmd_retries) × send_wait_time
→ keep this comfortably below update_interval
```