Commit graph

22 commits

Author SHA1 Message Date
cjkas
cfa298f053 fix: review fixes for step 4a
- e2e: the heap check uses a socket slot now, so it waits for the device
  to free it, and test_every_client_reconnecting_at_once closes its
  clients before the check runs.
- Comments and docs that still named removed things (the /controller read,
  the event buffer, memStatus, SocketEmitter, ModelSnapshot).
- ESPNetwork::emitSockets is emitStatus: one socket, and it sends the status.
- Unused includes, whitespace-only lines left by the toJSON removal.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-27 22:11:54 +01:00
cjkas
2a954aa158 feat!: remove the old API, ports 8081 and 8080 (step 4a)
Step 4a of docs/SECURITY_PLAN.md. Port 80 keeps the eleven HTTP routes of
ADR 0006 and the socket at /ws; everything else moved to protocol 2.

- Port 8081 (ApiRoutes) and the socket on 8080 (SocketEmitter) are gone.
  Protocol 2 owns its frame buffer and loop-task check.
- Port 80 drops every route the old web UI called: reads, edits,
  settings, scans, reboot, /shades.cfg, /discovery. HttpRoutes.cpp holds
  what stays.
- ModelSnapshot, SnapshotSlots and the read documents are gone. The item
  snapshot types stay, as ItemSnapshot, for events, MQTT and the command
  replies.
- Model edits reply only to their protocol 2 op: requestResult goes.
- The settings' toJSON writers, used only by the settings routes, go.
- The _espsomfy_rts._tcp mDNS record moves to port 80, with path=/ws.
- The memory event reports the loop-rate window it closes.
- A deleted room no longer re-publishes its retained MQTT topics.
- e2e: the heap check reads the sync's status; the sync is checked on
  its own instead of against /controller.
- README gets an API and ports section.

BREAKING CHANGE: ports 8081 and 8080 and the old HTTP routes are gone.
Home Assistant needs integration 3.0.0 or newer.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-27 21:49:38 +01:00
cjkas
330d8fa0de fix: e2e failures on the device: edit replies before their events, /ws closed on reboot
- runModelEdit sent an edit's events (room, roomRemoved, the orders,
  repeaters) before its reply, where PROTOCOL.md says they follow it. A
  client waiting for the reply read past them (test_adding_and_deleting_a_room).
- A restart dropped the /ws connections without a close, so clients never
  saw them go (test_reboot). ESPNetwork::end() now closes them too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-25 12:43:06 +01:00
cjkas
096eef0581 fix: settings ops review findings
- The frequency scan ops and reboot run on loop() like every op (ADR 0005).
  endFrequencyScan goes to the front of the loop queue, so it never waits
  behind an unpair that waits for the scan to end.
- A security save that leaves the device Secured, or changes its mode,
  closes every client, the saver too once its reply has gone. None to None
  closes nobody.
- proto must be a radio_proto value; frequency, rxBandwidth and deviation
  are checked against what the CC1101 can be set to.
- wifiNetworks sends fewer networks instead of none when they don't fit.
- A client WiFi scan doesn't start while the device joins a network.
- rebootTime is set before reboot; cancelSettingsSave is freeSettingsSave.
- PROTOCOL.md: pins that may be -1, and the fields the routes sent that
  moved elsewhere. e2e: the WiFi scan test goes by the connection in use.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-25 10:27:53 +01:00
cjkas
2517f40709 feat: settings ops, the WiFi and frequency scans, and reboot for socket protocol 2
Step 1d of the protocol 2 core (docs/SECURITY_PLAN.md).

- getSettings/saveSettings by section: general, network, mqtt, security,
  radio. A save is the settings route's request, so loop() applies it with
  the settings' own fromJSON(). ProtoSettings checks every field first:
  kind, range, string length, IP addresses.
- Secrets are only reported as has*. A security save replies with the
  client's new token and closes the other sockets; other saves send
  settingsChanged to the other clients.
- scanWifi runs an async scan from ESPNetwork, kept apart from the roaming
  scan, and wifiNetworks goes to the clients that asked.
- beginFrequencyScan, endFrequencyScan and reboot run at once on async_tcp,
  as their routes do: a scan's end mustn't wait in the loop queue behind an
  unpair that waits for the scan to end (new op kind 'a').
- e2e: test_settings.py and a --reboot flag.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-25 09:52:09 +01:00
cjkas
482e495678 feat: socket protocol 2 core on ws://host/ws
The /ws endpoint runs next to the old socket on port 8080.  The
handshake checks the token under security, the device sends hello,
then a sync of the schema (unless the client has it), header, rooms,
shades, groups, repeaters and status, paced by the client's next.
Events go out with the old ones: shadeState or the whole shade on a
config change, groups, rooms, removals, orders, repeaters,
shadeCommand, memory, wifi, ethernet, and the frames and
frequencyScan feeds.

The table of types, fields, errors and feeds is defined once and is
sent as the schema.  The writer, item writers, sync and client frame
parser are pure and natively tested.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 11:03:07 +02:00
cjkas
25d548ef07 fix: send socket events only from loop()
Every socket event is formatted into one static buffer by one shared
formatter.  setConnected() runs on the WiFi/Ethernet event task and sent
memStatus, wifiStrength and ethernet, then ran sockEmit.loop(), while
loop() could be building its own event.  _safecat() checks the room and
then writes at _len, so another task moving _len in between could write
past the end of g_response.

setConnected() now sends nothing.  loop() broadcasts the status every
STATUS_EMIT_MS (30 s): memStatus, then ethernet, or wifiStrength when it
moved 3 dBm or changed channel.  A client that connects still gets all
of it at once from initClients(), so needsBroadcast and the 1.5 s tick
go, and so does the 1.5 s memStatus for the config panel.

SocketEmitter records loop()'s task in begin().  Off it, beginEmit()
logs and hands out a formatter that writes nothing, and endEmit(),
endEmitRoom() and loop() do nothing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 15:27:21 +02:00
cjkas
609cd455c6 fix: only touch the task watchdog from tasks that are subscribed to it
Only loop() is subscribed.  setConnected() runs on the WiFi event task
and reached SocketEmitter::endEmit(), whose plain esp_task_wdt_reset()
logged "esp_task_wdt_reset(707): task not found" on every boot.  The web
and socket handlers on async_tcp did the same.

/scanaps on async_tcp called esp_task_wdt_add(NULL) after the scan,
subscribing a task that nothing resets, so the board could panic 15 s
after a scan from the UI.

TaskWdt.h resets only a subscribed task and re-adds a task after a long
call only when it was watched before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 13:45:04 +02:00
cjkas
a1622e652d build: move to pioarduino 55.03.312 (Arduino 3.3.12, ESP-IDF 5.5)
IDF 4.4 gets no more security fixes, and the planned RMT radio needs
IDF 5's RMT driver.  See docs/adr/0003-pioarduino-idf55.md.

- Every env on one pinned pioarduino release.  C6 drops its own
  library list and flags but stays a debug build on huge_app.
- Release envs build at CORE_DEBUG_LEVEL=2 so the ESP32 image fits its
  1.5 MB OTA slot (17 KB free); debug envs stay at 3.
- Drop the C3: its image is 76.5 KB over the OTA slot on IDF 5.5.
- CI installs pioarduino's Core and checks firmware.bin against the app
  partition, failing under 8 KB free.  PlatformIO's "Flash:" figure
  reads ~45 KB low and passed a C3 image that doesn't fit.
- esp_task_wdt_reconfigure(): on 3.x the core already starts the watchdog
  and esp_task_wdt_init() fails, so the C6 build kept the 5 s default.
- An updated firmware is kept only once it has run 60 s with the
  network up; a reset before then rolls back to the previous one.
- ETH.begin() takes the 3.x argument order; Arduino 2.x branches gone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-19 22:05:22 +02:00
cjkas
e1a62fe5be feat: keep Ethernet settings independent of the framework
Prepares the move to Arduino 3.x / ESP-IDF 5.5, which renumbers
eth_phy_type_t (ETH_PHY_GENERIC is now 0) and only has the RMII MAC
API on the original ESP32.

- Store the PHY chip as our own id (eth_phy_id_t, the 2.x values) and
  translate it by name at ETH.begin(), so NVS, backups and the web UI
  keep their numbers on either framework.
- Build Ethernet only where CONFIG_ETH_USE_ESP32_EMAC is set instead of
  everywhere but the C6.  C3 and S3 could never use it.  A stored
  Ethernet connection falls back to Wi-Fi on those chips.
- /networksettings reports hasEthernet; the UI hides the Ethernet
  options without it.
- Fix the UI PHY list: 5 is the KSZ8041, add the KSZ8081 (6), drop the
  DM9051, an SPI chip the MAC can't drive.
- SSDP takes its address from WiFi/ETH.localIP() instead of the
  tcpip_adapter API IDF 5 removed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-19 21:13:49 +02:00
cjkas
6d1e9064c2 fix: join the AP our scan picked without rescanning every channel
Every WiFi connect took about 17.6 s.  Our scan already picks the AP and
passes its BSSID and channel to WiFi.begin(), but the connect still ran
with WIFI_ALL_CHANNEL_SCAN.  So the driver swept every channel again
before it sent the first auth frame.  A capture of the join showed the AP
silent for 17.6 s, the radio on ch7 while the AP is on ch6, and then
auth, association and the handshake all done in 80 ms.

When the BSSID is known (the normal connect, the strongest-AP path and
roaming), use WIFI_FAST_SCAN: the driver starts on that channel and joins
the first match, and with bssid_set only that AP matches.  The hidden-SSID
path still uses the all-channel scan, since there it is what finds the
strongest AP (rstrouse/ESPSomfy-RTS#294).  Soft reboots now connect in
0.3-1.5 s.

Also log how long each join took and the AP's security mode, on the
connect line and on the roaming-scan lines, so a slow connect shows up.

One boot in the old logs connected in 2.47 s, and nothing in its log
explains it.  After a power cut the first attempts can still fail on the
AP side; a soft reboot sends a deauth first and doesn't have that problem.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-19 16:00:12 +02:00
cjkas
ce505e93f6 feat: report main-loop health in memStatus
memStatus now carries loopRate (loop() passes per second) and loopMaxUs
(the longest pass, in microseconds) over the window since the last
broadcast. Only a broadcast closes the window, so a client connecting
does not wipe a slow pass before Home Assistant has seen it.

/discovery is formatted on the web server's task, so its memory block
reports the last closed window through atomics instead of reading the
counters the loop task is still filling.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-18 13:53:30 +02:00
cjkas
a0ad301779 refactor: split the transports into the modules GLOSSARY.md names
Code moves unchanged; only files and names change.

- entryPoints/WebUiRoutes.cpp: registerWebUiRoutes(), the port 80 routes
  and the UI's files, with the helpers only they use.
- entryPoints/ApiRoutes.cpp: registerApiRoutes(), the port 8081 routes.
- entryPoints/Web.cpp keeps the servers' lifecycle, auth and the handlers
  both route sets call; entryPoints/WebRoutes.h is what the three share.
- entryPoints/MQTT.* -> MqttConnection.*, class MQTTClass -> MqttConnection.
- outputs/SomfyMqtt.cpp -> outputs/MqttPublisher.cpp.
- outputs/Sockets.* -> outputs/SocketEmitter.*.

The subscribe and unsubscribe lists (the latter repeated
groups/+/direction/set) become MqttCommands' MQTT_COMMAND_TOPICS, and a
native test checks the parser takes every topic in it. GLOSSARY.md gains
MqttConnection and Change fan-out.

328 native tests. Flash: esp32devdbg +3,112 bytes (the routes are a second
ArduinoJson file), esp32dev -2,336 bytes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 13:47:56 +02:00
cjkas
e624a4177d refactor: move the transports into entryPoints and outputs
The HTTP servers and MQTT command handling go to src/entryPoints/, the
socket emitter and the MQTT and socket output of shades, groups and rooms
to src/outputs/. GLOSSARY.md names what they become once split:
WebUiRoutes, ApiRoutes and MqttCommands in, MqttPublisher and
SocketEmitter out.

Only files and include paths change. esp32devdbg flash goes from
1,546,405 to 1,546,257 bytes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 19:41:49 +02:00
cjkas
293e013aa2 test: cover the roam threshold and who gets memStatus
Move the roam check into worthRoaming() and the memStatus routing into
heapEmitTarget() in Utils so the native tests reach them.

worthRoaming() roams for a gain of at least ROAM_MIN_GAIN_DB, as the
comment and the 6 dB commit said. The old check needed one dB more.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 15:37:12 +02:00
cjkas
e47381b19e fix: only let wifiStrength broadcasts move the RSSI baseline
The emit to a newly connected client also stored the RSSI and time, which
restarted the 10s hold-off and hid a real change from the clients already
connected.  The baseline moves into RssiBroadcast, which ignores
per-client emits, with unit tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 18:34:45 +02:00
cjkas
b4d480caa0 fix: send memStatus once per network broadcast
emitSockets() sent memStatus and then emitSockets(255) sent it again, so
room 0 got two identical events in one pass after every /discovery.
Only the per-client path sends it from emitSockets(num).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 18:00:52 +02:00
cjkas
66d264e4b5 fix: broadcast memStatus every 30s
It went out whenever free heap moved by 1.5 KB and 7s had passed,
which with normal traffic was about 240 times an hour. Send it every
30s instead. It is also the one event always sent, so 30s keeps Home
Assistant well inside its 60s receive timeout. Per-client sends and
the web UI's room are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 20:50:47 +02:00
cjkas
bd7ed17a97 feat: log AP signal levels and roam for a 6 dB gain
The roaming scan found two APs for the SSID every 10 minutes but the log
never showed their signal, so there was no way to tell whether the
device at -70 dBm had a better AP to move to. Log each AP on the
roaming scans, show roams at info level, and switch for a 6 dB gain
instead of 10.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 20:50:46 +02:00
cjkas
05c3d309a6 fix: send wifiStrength only for real signal changes
At -70 dBm the RSSI moves a dBm or two constantly, so the >1 dBm check
every 1.5s sent wifiStrength to every client about 270 times an hour.
That is extra traffic on an already lossy link and a state write in
Home Assistant for each one.

Broadcast on a channel change, or for a change of at least 3 dBm no
more than every 10s. A newly connected client still gets the current
value straight away.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 20:35:42 +02:00
Shailen Sobhee
b2e402e20e Fixed task_wdt: esp_task_wdt_reset(707): task not found 2026-04-11 23:48:19 +02:00
cjkas
72bfd26443 Added c6 support 2026-03-29 21:53:57 +02:00
Renamed from src/Network.cpp (Browse further)