The web UI sends the restore options as the form field "data". The
param helpers looked up query parameters only, so /restore never saw
them and restored the shades alone, since 3.0.0. The helpers now take a
form field too.
/restore uses an upload once, and drops the file when the options aren't
JSON.
e2e: test_restore.py, a probe that the field is read and, with --reboot,
a settings-only round trip.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Under Secured every HTTP route but the public ones needs the token
(docs/adr/0006-two-security-modes-eleven-http-routes.md):
- isPublicRoute, isWebAsset and originMatchesHost in src/HttpAccess.h,
with native tests.
- One middleware checks the token. It runs once the body is in, so
/restore, /updateFirmware and /updateApplication check it on their
first chunk and drop a refused upload.
- Static serving answers the web UI's files only: shades.cfg and
shades.tmp are no longer served.
- The wildcard CORS headers go; WEB_CORS turns them back on for UI work
from file://.
- The socket refuses a handshake whose Origin doesn't name its Host.
- "Secure Config Only" goes: a device set to it is Secured, and
/loginContext drops permissions.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- e2e: the heap check uses a socket slot now, so it waits for the device
to free it, and test_every_client_reconnecting_at_once closes its
clients before the check runs.
- Comments and docs that still named removed things (the /controller read,
the event buffer, memStatus, SocketEmitter, ModelSnapshot).
- ESPNetwork::emitSockets is emitStatus: one socket, and it sends the status.
- Unused includes, whitespace-only lines left by the toJSON removal.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Step 4a of docs/SECURITY_PLAN.md. Port 80 keeps the eleven HTTP routes of
ADR 0006 and the socket at /ws; everything else moved to protocol 2.
- Port 8081 (ApiRoutes) and the socket on 8080 (SocketEmitter) are gone.
Protocol 2 owns its frame buffer and loop-task check.
- Port 80 drops every route the old web UI called: reads, edits,
settings, scans, reboot, /shades.cfg, /discovery. HttpRoutes.cpp holds
what stays.
- ModelSnapshot, SnapshotSlots and the read documents are gone. The item
snapshot types stay, as ItemSnapshot, for events, MQTT and the command
replies.
- Model edits reply only to their protocol 2 op: requestResult goes.
- The settings' toJSON writers, used only by the settings routes, go.
- The _espsomfy_rts._tcp mDNS record moves to port 80, with path=/ws.
- The memory event reports the loop-rate window it closes.
- A deleted room no longer re-publishes its retained MQTT topics.
- e2e: the heap check reads the sync's status; the sync is checked on
its own instead of against /controller.
- README gets an API and ports section.
BREAKING CHANGE: ports 8081 and 8080 and the old HTTP routes are gone.
Home Assistant needs integration 3.0.0 or newer.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Load the token key in Web::startup(), before either web server starts,
so no token can be made with the all-zero key; log a failed
pref.begin().
- createAPIToken checks mbedtls' return codes. On a failure it leaves
the token empty, which tokenMatches never accepts, instead of hex
from uninitialised stack.
- API_TOKEN_SIZE in ApiToken.h replaces the five char[65] buffers; the
HMAC's description moves to createAPIToken, with the None form.
- handleLogin builds the candidate token once and compares it once.
- e2e: the serverId test skips on `secured` like its neighbours, names
its values, and says it assumes the device sees this PC's address.
- KNOWN_ISSUES keeps #81, which #108 removes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Tokens were HMAC(serverId, credentials + client IP). serverId is public
(mDNS, MQTT, /loginContext), so anyone who saw a token could test PIN
guesses offline. The key is now 32 random bytes made on first boot in
their own NVS namespace, which no settings save clears and no backup
holds. It is made before WiFi starts, from the bootloader's entropy
source.
Tokens are compared with tokenMatches, which takes the same time
whatever the guess, on the apikey header, the socket upgrade and
/login.
createAPIToken never freed its mbedtls context, so every token it made
leaked: 563 bytes per login on the device, until it ran out of memory
after about 150 logins. It now frees it: 0 bytes over 500 logins.
Every token made before the upgrade is refused once. The web UI asks
for login again, and Home Assistant 3.0 logs in again by itself.
Closes#80
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- The throttle runs on a 64-bit clock in ms (esp_timer_get_time()), so a
slot left alone for 49.7 days isn't brought back by millis() wrapping
round to its old failure.
- Web.cpp turns the client into the throttle's key in one place, and
says that an IPv6 client would share key 0; names are clearer
(retryMs, vacantSlot).
- The shared slot's test now succeeds from the IP counted in it, so a
success that cleared the shared slot would fail it.
- GLOSSARY.md names it the login throttle, with lockouts, and
SECURITY_PLAN.md and ADR 0006 use that name.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
PR C of SECURITY_PLAN.md: after 5 failed /login attempts an IP is refused
for 30 s, and each failure after that refuses it for twice as long, up
to 15 min. A success resets it, as does an hour without a failure.
- LoginThrottle.h holds the schedule and 8 IPs, with the time passed in
so native tests cover it. While all 8 hold recent failures, every
other IP shares one more slot, so many addresses don't each get five
free tries.
- A refused IP gets 429 with Retry-After before its credentials are
checked, so a right guess during a lockout reveals nothing. Home
Assistant retries a 429 instead of asking for new credentials.
- The web UI's login panel shows a 401's or 429's message, where it
popped a service error.
- test/e2e --lockout checks it on a Secured device and waits it out.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
From the review of #99:
- A sync could stall when a client took a slot the moment another left:
loop() used up the new client's sync on the old one. The slots move
to ProtoSlots, a pure module with native tests, and loop() takes a
slot's wants only for the client it read (takeWants).
- Deleting a group sends its shades, whose inGroup changed. Deleting a
shade unlinks it from its groups and sends them, so a new shade given
its id doesn't join them.
- next and sync with an id get an ok reply, and a message in pieces
gets one bad request, not one per piece.
- Feeds check for loop() before they write the event buffer, and log
an event too large, as events do.
- editChanges() (ModelEditor, native tests) says what clients get after
an edit; the edit result carries the shade a group gained or lost.
order() takes a sort_list_t.
- The frame writers move to ProtoFrame. Renames: pf to proto_field_t,
the X-macro letters to TYPE/FIELD/ERR/FEED, proto_request_t to
proto_message_t, REQ_* to WANTS_*, fmt to eventJson. The event
bodies share sendItem(), the connection type name lives in
ESPNetwork, and the loop() task check in SocketEmitter.
- e2e: the heap floor check skips when the low since boot is already
under it; tests for replies to next and sync and a long message.
- SECURITY_PLAN.md splits step 1 into 1a-1d.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The /ws endpoint runs next to the old socket on port 8080. The
handshake checks the token under security, the device sends hello,
then a sync of the schema (unless the client has it), header, rooms,
shades, groups, repeaters and status, paced by the client's next.
Events go out with the old ones: shadeState or the whole shade on a
config change, groups, rooms, removals, orders, repeaters,
shadeCommand, memory, wifi, ethernet, and the frames and
frequencyScan feeds.
The table of types, fields, errors and feeds is defined once and is
sent as the schema. The writer, item writers, sync and client frame
parser are pure and natively tested.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The release check, release list, download, cancel and
/recoverFilesystem go, with the GitHub button and the Auto Check for
Updates setting in the web UI (docs/adr/0007-no-github-update.md).
The HTTPS session it needed failed for want of a 35-40 KB block (#76),
it checked no certificate, and the web UI's buttons had been getting
404. Updates are uploads only (/updateFirmware, /updateApplication).
The old socket still sends fwStatus with the installed versions, which
the web UI and Home Assistant read. checkForUpdate stays in NVS and the
backup file so their formats don't change.
The release image shrinks by 126.5 KB: 142 KB of the OTA slot is free.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every socket event is formatted into one static buffer by one shared
formatter. setConnected() runs on the WiFi/Ethernet event task and sent
memStatus, wifiStrength and ethernet, then ran sockEmit.loop(), while
loop() could be building its own event. _safecat() checks the room and
then writes at _len, so another task moving _len in between could write
past the end of g_response.
setConnected() now sends nothing. loop() broadcasts the status every
STATUS_EMIT_MS (30 s): memStatus, then ethernet, or wifiStrength when it
moved 3 dBm or changed channel. A client that connects still gets all
of it at once from initClients(), so needsBroadcast and the 1.5 s tick
go, and so does the 1.5 s memStatus for the config panel.
SocketEmitter records loop()'s task in begin(). Off it, beginEmit()
logs and hands out a formatter that writes nothing, and endEmit(),
endEmitRoom() and loop() do nothing.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
NTPSettings::apply() waited up to 5 s in getLocalTime() for a clock that
can't be set before the network is up, then returned before it set TZ.
Every cold boot was 5 s slower and ran without its time zone. It now
calls configTzTime(), which sets the zone and starts SNTP without
waiting, and save() goes through apply(). /setgeneral checked
ntpServer twice, so a request that changed only the zone was ignored.
A restore ignored the result of ShadeConfigFile::restore() and rebooted
either way, so a bad file failed with no word to the user. The /restore
reply now validates the file and answers 400, which the restore dialog
shows, and an incomplete upload gets a 400 instead of no reply. A
failed restore changes nothing, so loop() no longer reboots after one.
/shades.tmp, a full copy of the backup with its rolling codes, is
deleted after every restore and after /updateShadeConfig.
The 6 extra bytes in the 2026-10-18 backup: TRANS_REC_SIZE stayed 74
when noiseDetection was added to the 80-byte transceiver record, so
every backup since declares it short. It is 80 now. The size check
stays a warning so those backups still restore, and the sum moves to
expectedFileBytes() with tests. The repeater record length is checked
as a warning only, since validate() also runs on shades.cfg at boot.
Commands for an unknown shade, tilt or group now log a warning, and a
socket that never pinged says so. KNOWN_ISSUES.md is updated, including
the real cause of the ~365 ms loop stall (radio_tx busy-waits each frame
on loop()'s core).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The settings routes returned the WiFi passphrase, MQTT password and the
login password and PIN in plain text, and the debug log printed them and
the API key. Responses now carry hasPassphrase/hasPassword/hasPin flags,
logs mask the values, and an omitted secret keeps the stored one.
The UI loads secret fields empty and treats a blank field as "keep" while
the SSID or username is unchanged. Clearing a secret without changing the
SSID or username is listed in docs/KNOWN_ISSUES.md.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
memStatus now carries loopRate (loop() passes per second) and loopMaxUs
(the longest pass, in microseconds) over the window since the last
broadcast. Only a broadcast closes the window, so a client connecting
does not wipe a slow pass before Home Assistant has seen it.
/discovery is formatted on the web server's task, so its memory block
reports the last closed window through atomics instead of reading the
counters the loop task is still filling.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The read routes leave out a room, shade, group or linked shade that
doesn't fit the 3 KB unit, without telling the client (KNOWN_ISSUES.md).
test_snapshot_limits fills a snapshot with the largest possible items and
checks every read document written with the real unit is identical to
the same document written with an unlimited one. The largest item, a
shade, is 1,097 bytes. The unit is now SNAPSHOT_UNIT_SIZE in
ModelSnapshot.h, used by Web.cpp and the tests.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Code moves unchanged; only files and names change.
- entryPoints/WebUiRoutes.cpp: registerWebUiRoutes(), the port 80 routes
and the UI's files, with the helpers only they use.
- entryPoints/ApiRoutes.cpp: registerApiRoutes(), the port 8081 routes.
- entryPoints/Web.cpp keeps the servers' lifecycle, auth and the handlers
both route sets call; entryPoints/WebRoutes.h is what the three share.
- entryPoints/MQTT.* -> MqttConnection.*, class MQTTClass -> MqttConnection.
- outputs/SomfyMqtt.cpp -> outputs/MqttPublisher.cpp.
- outputs/Sockets.* -> outputs/SocketEmitter.*.
The subscribe and unsubscribe lists (the latter repeated
groups/+/direction/set) become MqttCommands' MQTT_COMMAND_TOPICS, and a
native test checks the parser takes every topic in it. GLOSSARY.md gains
MqttConnection and Change fan-out.
328 native tests. Flash: esp32devdbg +3,112 bytes (the routes are a second
ArduinoJson file), esp32dev -2,336 bytes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The 19 routes that changed rooms, shades, groups and repeaters on the web
server task (add, save, delete, link and unlink, setPaired, the sort
orders) now copy their request into a model_edit_t, queue it on the
LoopQueue and reply {status:accepted,requestId:N}. loop() applies it
(SomfyJson.cpp, next to fromJSON) and broadcasts requestResult {requestId,
ok, id, code, desc}. One edit at a time; a second gets 503, a body over
4 KB gets 413.
ModelEditor.h/.cpp holds the rules, tested natively: the lowest free id,
the next sort order, a free remote address, adding to and removing from
the group-member and repeater lists, and the GPIO pin-conflict check. They
replace the getNext*Id/getMax*Order copies, the remote-address search,
the list code and the inline pin checks. The pin check now uses the
protocol being saved and checks every pin against Ethernet, not only up.
/getNextRoom, /getNextShade, /getNextGroup, /groupOptions and a new
/repeaters (the UI already called it) are ModelSnapshot reads.
The web UI's editJSON() sends an edit, waits for its requestResult, then
reads the item back from /shade, /group, /room or /repeaters, so the
callbacks that used the old reply bodies are unchanged. No result within
5 s offers a reload; reads and edits told Busy are retried. It now also
refreshes its lists on shadeAdded/shadeRemoved/groupAdded/groupRemoved
from other clients, and procRoomRemoved keeps the right rooms.
Also fixed: a shade that fails validation is no longer added; linked
repeaters and sort orders are saved; linking reports a full list.
27 new native tests, 319 in all. Flash: esp32devdbg -11,484 bytes,
esp32dev -7,180 bytes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
/controller, /discovery, /rooms, /shades, /groups, /room, /shade and /group
now take a SnapshotSlots slot, queue a fill on the LoopQueue and send a
chunked response whose body waits (RESPONSE_TRY_AGAIN) until loop() has
copied the shade model into the slot. SnapshotWriter then streams the
document from that copy a unit at a time, so each document describes one
moment. At most two reads run at once; a third gets 503. A slot's 5.6 KB
snapshot is allocated only while a read holds it, and its atomic state
decides whether loop() or the read frees it when the read goes away first.
The JSON of each item is unchanged and now lives in ModelSnapshot.cpp,
where the native tests check it. Replies still written on the web server
task copy their one item with SomfySnapshot.cpp, so serializing a group
no longer calls updateFlags(), which emitted groupState and published MQTT
from async_tcp.
Also: every read route is chunked and leaves out an item too big for its
3 KB unit, like /controller did; /discovery drops its timing log; the
SOMFY_MAX_* limits move to ShadeTypes.h.
21 new native tests, 292 in all. Flash: esp32devdbg +440 bytes, esp32dev
+2,064 bytes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
HTTP routes and MQTT commands now both build a shade request and submit it
to the LoopQueue, which loop() drains into ShadeRequests. ShadeRequests
finds the shade or group, turns client positions into stored ones, checks
ranges and fills in default repeats, and reaches the shades through the
ShadeOps seam: SomfyShadeOps on the device, a recording fake in the native
tests. MqttCommands turns a .../set topic into a shade request.
Behaviour that changes:
- MQTT tiltTarget, mypos, myTiltPos and tiltPosition, and /setPositions,
now honour flipPosition like the other routes.
- MQTT mypos keeps the favorite tilt (was set to 0) and myTiltPos keeps
the favorite lift.
- repeat 0 on /shadeCommand means no repeats, as it did for groups; a
JSON repeat over 127 is capped instead of wrapping to the default.
- Position corrections go through the setters, so MQTT publishes them.
- MQTT commands wait while the radio is full instead of being refused.
- The queue is drained every loop() pass and no longer from inside a
GitHub firmware download.
WebCommand.h is replaced by LoopQueue.h and ShadeRequests.h. 45 new native
tests, 271 in all. Flash: esp32devdbg +976 bytes, esp32dev +1,380 bytes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The HTTP servers and MQTT command handling go to src/entryPoints/, the
socket emitter and the MQTT and socket output of shades, groups and rooms
to src/outputs/. GLOSSARY.md names what they become once split:
WebUiRoutes, ApiRoutes and MqttCommands in, MqttPublisher and
SocketEmitter out.
Only files and include paths change. esp32devdbg flash goes from
1,546,405 to 1,546,257 bytes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
