Commit graph

614 commits

Author SHA1 Message Date
cjkas
108b2960c7 build: vendor SmartRC-CC1101-Driver-Lib 2.5.7 unchanged
Copied from the registry package lsatan/SmartRC-CC1101-Driver-Lib@2.5.7
(upstream unmaintained since 2021) so it can be fixed for Arduino 3.x in
the next commit. This commit is the untouched copy, so that diff shows
every change. Licence: LICENSE.txt ("permission to modify and publish").

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 12:08:42 +02:00
cjkas
a1622e652d build: move to pioarduino 55.03.312 (Arduino 3.3.12, ESP-IDF 5.5)
IDF 4.4 gets no more security fixes, and the planned RMT radio needs
IDF 5's RMT driver.  See docs/adr/0003-pioarduino-idf55.md.

- Every env on one pinned pioarduino release.  C6 drops its own
  library list and flags but stays a debug build on huge_app.
- Release envs build at CORE_DEBUG_LEVEL=2 so the ESP32 image fits its
  1.5 MB OTA slot (17 KB free); debug envs stay at 3.
- Drop the C3: its image is 76.5 KB over the OTA slot on IDF 5.5.
- CI installs pioarduino's Core and checks firmware.bin against the app
  partition, failing under 8 KB free.  PlatformIO's "Flash:" figure
  reads ~45 KB low and passed a C3 image that doesn't fit.
- esp_task_wdt_reconfigure(): on 3.x the core already starts the watchdog
  and esp_task_wdt_init() fails, so the C6 build kept the 5 s default.
- An updated firmware is kept only once it has run 60 s with the
  network up; a reset before then rolls back to the previous one.
- ETH.begin() takes the 3.x argument order; Arduino 2.x branches gone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-19 22:05:22 +02:00
cjkas
e1a62fe5be feat: keep Ethernet settings independent of the framework
Prepares the move to Arduino 3.x / ESP-IDF 5.5, which renumbers
eth_phy_type_t (ETH_PHY_GENERIC is now 0) and only has the RMII MAC
API on the original ESP32.

- Store the PHY chip as our own id (eth_phy_id_t, the 2.x values) and
  translate it by name at ETH.begin(), so NVS, backups and the web UI
  keep their numbers on either framework.
- Build Ethernet only where CONFIG_ETH_USE_ESP32_EMAC is set instead of
  everywhere but the C6.  C3 and S3 could never use it.  A stored
  Ethernet connection falls back to Wi-Fi on those chips.
- /networksettings reports hasEthernet; the UI hides the Ethernet
  options without it.
- Fix the UI PHY list: 5 is the KSZ8041, add the KSZ8081 (6), drop the
  DM9051, an SPI chip the MAC can't drive.
- SSDP takes its address from WiFi/ETH.localIP() instead of the
  tcpip_adapter API IDF 5 removed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-19 21:13:49 +02:00
cjkas
b74f382650
Merge pull request #69 from cjkas/fix/boot-clock-restore-logs
fix: stop the 5 s boot wait, tell the client about a bad restore
2026-10-19 20:01:57 +02:00
cjkas
953d4dc6f7 fix: stop the 5 s boot wait, tell the client about a bad restore
NTPSettings::apply() waited up to 5 s in getLocalTime() for a clock that
can't be set before the network is up, then returned before it set TZ.
Every cold boot was 5 s slower and ran without its time zone.  It now
calls configTzTime(), which sets the zone and starts SNTP without
waiting, and save() goes through apply().  /setgeneral checked
ntpServer twice, so a request that changed only the zone was ignored.

A restore ignored the result of ShadeConfigFile::restore() and rebooted
either way, so a bad file failed with no word to the user.  The /restore
reply now validates the file and answers 400, which the restore dialog
shows, and an incomplete upload gets a 400 instead of no reply.  A
failed restore changes nothing, so loop() no longer reboots after one.
/shades.tmp, a full copy of the backup with its rolling codes, is
deleted after every restore and after /updateShadeConfig.

The 6 extra bytes in the 2026-10-18 backup: TRANS_REC_SIZE stayed 74
when noiseDetection was added to the 80-byte transceiver record, so
every backup since declares it short.  It is 80 now.  The size check
stays a warning so those backups still restore, and the sum moves to
expectedFileBytes() with tests.  The repeater record length is checked
as a warning only, since validate() also runs on shades.cfg at boot.

Commands for an unknown shade, tilt or group now log a warning, and a
socket that never pinged says so.  KNOWN_ISSUES.md is updated, including
the real cause of the ~365 ms loop stall (radio_tx busy-waits each frame
on loop()'s core).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-19 18:08:16 +02:00
cjkas
4b75e8fb08
Merge pull request #68 from cjkas/docs/known-issues-from-logs
docs: record the issues found in the 2026-10-18/19 serial logs
2026-10-19 16:16:18 +02:00
cjkas
31581b42e8
Merge pull request #66 from cjkas/fix/wifi-fast-scan
fix: join the AP our scan picked without rescanning every channel
2026-10-19 16:16:08 +02:00
cjkas
4ee711f060 docs: record the issues found in the 2026-10-18/19 serial logs
The cold-boot 5 s wait in NTPSettings::apply() that also skips the time
zone, restores that reboot even when validation fails and leave
shades.tmp behind, the ~365 ms each moving shade command holds up
loop(), one unexplained slow /controller read, and three misleading or
missing log lines.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-19 16:14:42 +02:00
cjkas
ea06f1bf53
Merge pull request #67 from cjkas/docs/security-plan
docs: plan auth on every entry point and one HTTP server
2026-10-19 16:13:17 +02:00
cjkas
3a684e863d
Merge pull request #64 from cjkas/fix/secrets-write-only
fix: stop sending stored passwords and PIN back to clients
2026-10-19 16:08:04 +02:00
cjkas
6d1e9064c2 fix: join the AP our scan picked without rescanning every channel
Every WiFi connect took about 17.6 s.  Our scan already picks the AP and
passes its BSSID and channel to WiFi.begin(), but the connect still ran
with WIFI_ALL_CHANNEL_SCAN.  So the driver swept every channel again
before it sent the first auth frame.  A capture of the join showed the AP
silent for 17.6 s, the radio on ch7 while the AP is on ch6, and then
auth, association and the handshake all done in 80 ms.

When the BSSID is known (the normal connect, the strongest-AP path and
roaming), use WIFI_FAST_SCAN: the driver starts on that channel and joins
the first match, and with bssid_set only that AP matches.  The hidden-SSID
path still uses the all-channel scan, since there it is what finds the
strongest AP (rstrouse/ESPSomfy-RTS#294).  Soft reboots now connect in
0.3-1.5 s.

Also log how long each join took and the AP's security mode, on the
connect line and on the roaming-scan lines, so a slow connect shows up.

One boot in the old logs connected in 2.47 s, and nothing in its log
explains it.  After a power cut the first attempts can still fail on the
AP side; a soft reboot sends a deauth first and doesn't have that problem.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-19 16:00:12 +02:00
cjkas
04cb97e3bf docs: plan auth on every entry point and one HTTP server
SECURITY_PLAN.md records the agreed design: an access table for port 80
and the 8081 alias, websocket handshake auth, CORS removal, a random
token key, /login throttling, and moving API and websocket to port 80.
KNOWN_ISSUES.md gets an entry for each open issue it fixes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-18 20:08:20 +02:00
cjkas
9383ebec2e fix: keep stored secrets when an older UI sends them empty
Firmware and UI files update separately, so an old index.js can run
against firmware that no longer returns secrets. It sends "" for the
field it can't fill in, which cleared the WiFi passphrase (reboot into
AP mode) or the MQTT password.

The firmware now decides: "" keeps the stored secret, unless the same
request changes the SSID or MQTT username, then it clears it (open
network). /setNetwork compares the settings after applying them, so a
kept passphrase no longer triggers a reboot. The UI cache-buster moves
to 3.0.14d.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-18 20:07:31 +02:00
cjkas
1fb4c4cfcd fix: stop sending stored passwords and PIN back to clients
The settings routes returned the WiFi passphrase, MQTT password and the
login password and PIN in plain text, and the debug log printed them and
the API key. Responses now carry hasPassphrase/hasPassword/hasPin flags,
logs mask the values, and an omitted secret keeps the stored one.

The UI loads secret fields empty and treats a blank field as "keep" while
the SSID or username is unchanged. Clearing a secret without changing the
SSID or username is listed in docs/KNOWN_ISSUES.md.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-18 14:34:18 +02:00
cjkas
e963a5adb3
Merge pull request #63 from cjkas/feat/loop-health
feat: report main-loop health in memStatus
2026-10-18 14:17:33 +02:00
cjkas
ce505e93f6 feat: report main-loop health in memStatus
memStatus now carries loopRate (loop() passes per second) and loopMaxUs
(the longest pass, in microseconds) over the window since the last
broadcast. Only a broadcast closes the window, so a client connecting
does not wipe a slow pass before Home Assistant has seen it.

/discovery is formatted on the web server's task, so its memory block
reports the last closed window through atomics instead of reading the
counters the loop task is still filling.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-18 13:53:30 +02:00
cjkas
96b00b70f4 Merge branch 'main' of https://github.com/cjkas/ESPSomfy-RTS into deps/update-libs 2026-10-18 13:22:11 +02:00
cjkas
0ad2b24b70 build: update AsyncTCP to 3.5.0 and ESPAsyncWebServer to 3.12.1
Both are minor updates with no API change for us.  The espressif32 7.x
platform and SmartRC-CC1101-Driver-Lib 3.x major updates are left out.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-17 23:56:40 +02:00
cjkas
b08f1c9ad8
Merge pull request #62 from cjkas/docs/review-items-done
docs: mark the review items fixed by #54, #57, #58 and HA#4
2026-10-17 23:44:49 +02:00
cjkas
b9a65e7b5f docs: mark the Home Assistant apikey item fixed by ESPSomfy-RTS-HA#4
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-17 23:43:16 +02:00
cjkas
61f394221f docs: mark the review items fixed by #54, #57 and #58
The restore skip, the short id range and the dropped group members move
from LOOP_QUEUE_PLAN.md's "still open" list to "fixed later". The Home
Assistant apikey item stays open until ESPSomfy-RTS-HA#4 merges.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-17 23:38:30 +02:00
cjkas
e06ee3a060
Merge pull request #61 from cjkas/fix/socket-buffer-4k
fix: make the socket buffer big enough for the largest event
2026-10-17 23:36:19 +02:00
cjkas
2056930005
Merge pull request #60 from cjkas/test/snapshot-worst-case
test: prove no read item can outgrow the snapshot unit
2026-10-17 23:36:08 +02:00
cjkas
fa79d44e70
Merge pull request #59 from cjkas/cleanup/waiting-frame-steps
cleanup: drop the step cases onWaitingFrame never reaches
2026-10-17 23:35:59 +02:00
cjkas
9bef92d754
Merge pull request #58 from cjkas/cleanup/group-json-members
cleanup: drop the group members SomfyGroup::fromJSON threw away
2026-10-17 23:35:50 +02:00
cjkas
0d90544c86 fix: make the socket buffer big enough for the largest event
Socket events that don't fit the static buffer are logged and not sent.
The state events are small (shadeState is about 400 bytes), but
remoteFrame lists up to 300 pulse timings and real frames already came
to 1.8-2.1 KB against the 2 KB buffer. MAX_SOCK_RESPONSE is now 4096 in
SocketEmitter.h, and a static_assert next to remoteFrame fails the build
if 300 pulses of 10 digits no longer fit. KNOWN_ISSUES.md closes the
entry with the bounds of every event.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-17 23:32:50 +02:00
cjkas
89d3713d86 test: prove no read item can outgrow the snapshot unit
The read routes leave out a room, shade, group or linked shade that
doesn't fit the 3 KB unit, without telling the client (KNOWN_ISSUES.md).
test_snapshot_limits fills a snapshot with the largest possible items and
checks every read document written with the real unit is identical to
the same document written with an unlimited one. The largest item, a
shade, is 1,097 bytes. The unit is now SNAPSHOT_UNIT_SIZE in
ModelSnapshot.h, used by Web.cpp and the tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-17 23:29:04 +02:00
cjkas
9df2638f03 cleanup: drop the step cases onWaitingFrame never reaches
onWaitingFrame only acts on a frame that isn't processed yet, and the
step rule in onFrame marks a step frame processed as soon as it arrives;
a repeat or synonym keeps the mark. So the "fixed 1% step of a held step
button" never ran. A new test pins what a held step button does: one
step by the frame's step size. SOMFY_SPLIT_PLAN.md is corrected.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-17 23:25:16 +02:00
cjkas
9ba33c75db cleanup: drop the group members SomfyGroup::fromJSON threw away
fromJSON read linkedShades into a local array and dropped it, so /addGroup
and /saveGroup never changed a group's members; /linkToGroup and
/unlinkFromGroup do, with ModelEditor's checks. The loop also wrote past
the end of that stack array when a request listed more than 32 shades.
No client sends the list: the web UI's group form has no members field.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-17 23:23:13 +02:00
cjkas
29211630f2
Merge pull request #57 from cjkas/fix/full-id-range
fix: give out ids up to the maximum
2026-10-17 23:12:31 +02:00
cjkas
67da75fcf4
Merge pull request #55 from cjkas/build/same-fs-partition
build: give the debug build the release file system size
2026-10-17 23:11:59 +02:00
cjkas
2b2a4f815d
Merge pull request #54 from cjkas/fix/restore-skip-rooms
fix: skip the room records too when a restore leaves the shades out
2026-10-17 23:11:42 +02:00
cjkas
e4ac7a29c6 fix: give out ids up to the maximum
New ids stopped two short of the limits, at 30 shades, 14 groups and 14
rooms instead of 32, 16 and 16, while the web UI showed the full limits.
The original loop ran i < MAX - 1, an off-by-one: nothing indexes by id
or treats the last two ids as special. ModelSnapshot's next-id answers
and SomfyShadeController's allocation now run from 1 to the maximum.
Found during LOOP_QUEUE_PLAN.md's review.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-17 23:03:16 +02:00
cjkas
8fcb0f7d67
Merge pull request #53 from cjkas/land/shade-behaviour
Land the shade behaviour work (#47-#52) on main
2026-10-17 23:00:06 +02:00
cjkas
e3c3472967 build: give the debug build the release file system size
esp32_huge_app.csv had a 0xD0000 file system where the release table has
0xE0000 at the same offset. LittleFS can grow but not shrink, so flashing
the debug build onto a device formatted by a release build failed the
lfs_fs_grow assert on every boot. The debug table now uses 0xE0000 and
moves the coredump into the 64 KB that was unused at the end of flash,
so both builds share one data layout. Devices formatted at 0xD0000 (debug,
esp32c6) grow on their next mount.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-17 22:55:31 +02:00
cjkas
50aa491437 fix: skip the room records too when a restore leaves the shades out
A restore without "shades" skipped the shade and group records but not
the room records, which come first, so it read the repeaters and the
settings from room data. The skip is now shadeModelBytes(), in a
ConfigHeader.h the native tests can reach, which counts rooms, shades
and groups. Found during LOOP_QUEUE_PLAN.md's review.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-17 22:53:18 +02:00
cjkas
a34982a4c7 docs: mark the settingPos and getNext*Id items done
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-17 22:49:58 +02:00
cjkas
a954c5047f docs: close the shade behaviour reconcile (Q24, Q25)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 22:49:17 +02:00
cjkas
38d4ce5058 fix: keep a tilt-only shade's lift at 100 whatever sends the command
The web and group Up/Down put a tilt-only shade's lift back at 100, but a
remote's Up/Down/My, a group's My and the web's My while moving left a
lift that something had moved, and the simulation then ran it. Every
Up/Down/My path now pins the lift at 100 and acts on the tilt only.
Reconcile Q22 in SHADE_BEHAVIOUR_PLAN.md, which also records Q23 (the
integrated tilt turn-back) as deferred.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 22:47:53 +02:00
cjkas
7220f38fcd fix: wait out a remote's long press across the millis() wrap
The 500 ms wait that tells a short press from a long one compared
now > await, so a press just before the wrap acted at once. It now
compares the wrap-safe difference. Reconcile Q21 in
SHADE_BEHAVIOUR_PLAN.md.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 22:41:48 +02:00
cjkas
4eff50645d fix: update every group's flags after a group's sun flag command
A SunFlag or Flag frame makes every group work its flags out again, but
the same command from a group only updated that group, so other groups
holding the shade kept stale sun flags. The group rule now reports the
sensor flags as the frame rule does. Reconcile Q20 in
SHADE_BEHAVIOUR_PLAN.md.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 22:40:32 +02:00
cjkas
2cf3c4b542 fix: step a remote's step by the frame's own step size
The step rule read lastFrame.stepSize, which copy() never updates, so a
remote's step moved by whatever step size this shade last sent. It now
reads the frame's step size, 1 when the frame has none. Reconcile Q19 in
SHADE_BEHAVIOUR_PLAN.md.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 22:39:58 +02:00
cjkas
9e519d4d5f fix: end the positioning when the lift reaches its target
Arriving at a lift target never cleared settingPos (the tilt arrival
clears settingTiltPos), so the next step or group move also sent a stop
when it arrived. A stop to a motor that has already stopped is a My,
which sends it to its favorite. Reconcile Q18 in SHADE_BEHAVIOUR_PLAN.md.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 22:39:02 +02:00
cjkas
bee6a6d60b
Merge pull request #46 from cjkas/refactor/shade-settings
refactor: hold shade settings in ShadeSettings and RemoteSettings
2026-10-16 22:33:14 +02:00
cjkas
4f8eac03ac refactor: move the shade's behaviour into ShadeBehaviour
PR B of SHADE_BEHAVIOUR_PLAN.md. What a shade does with frames, commands,
group commands and radio events moves unchanged from SomfyShade into
ShadeBehaviour, which builds natively and is pinned by test_shadebehaviour.

ShadeBehaviour reads ShadeSettings and RemoteSettings, gets the time on
every input, and holds the motion state that other files now read through
state(). It sends through MotorLink and reports through ShadeEvents
(changes, commands, save wanted, sensor flags) as things happen.
SomfyShade wraps it: its MotorLink is SomfyRemote::sendCommandAfter, which
now leaves the sender out of the echo and radio hold because the module
hears its own frame. lastFrame stays shared with the send path.

ConfigFile writes through restore() and clearTilt(), SomfyShadeOps through
setPosition(). The pinning tests label where the copies of a rule disagree
"today:", for the reconcile phase.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 21:21:38 +02:00
cjkas
fa7aaf0d49 refactor: hold shade settings in ShadeSettings and RemoteSettings
The shade behaviour module (SHADE_BEHAVIOUR_PLAN.md, PR B) reads its
settings through read-only structs, so they get one home each:
RemoteSettings (flipCommands, bitLength, repeats) under SomfyRemote,
shared with groups, and ShadeSettings (types, travel times, step size,
flipPosition, simMy, sunSensor) under SomfyShade. Both are base structs,
so shade->upTime and group->repeats keep working.

A shade's flags byte is split by owner: simMy and sunSensor are settings,
light is a field, and the sun flag, sunny, windy, demo mode and lighted
bits are stateFlags. getFlags()/setFlags() put the byte together and take
it apart for the config file, JSON, MQTT, events and group flags, so
nothing saved or sent changes. Groups keep their own flags byte.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 20:31:39 +02:00
cjkas
8a13e0c3a9 docs: settle the shade behaviour design (rounds 1-3)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 20:14:59 +02:00
cjkas
1e4fa74efa
Merge pull request #44 from cjkas/refactor/split-transports
refactor: split the transports into the modules GLOSSARY.md names
2026-10-16 14:44:33 +02:00
cjkas
e9ebcc1ac7 Merge branch 'refactor/split-transports' of https://github.com/cjkas/ESPSomfy-RTS into refactor/split-transports 2026-10-16 14:40:58 +02:00
cjkas
9fdf8c8783 docs: plan deepening the shade behaviour
Records the design state for candidate #1 of the architecture review so a
new session can pick it up: the drifted Up/Down/My copies in SomfyShade.cpp,
the echo path through SomfyRemote::sendCommandAfter, what steps 1-5 already
did, round 1 of the design questions with the recommended answers, and the
topics waiting for round 2.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 14:40:22 +02:00