The routes matched any path below them, and a web asset name there
passed the token check: GET /backup/a.js returned the backup under
Secured. Every route is now an exact match, and a web asset sits in
the root only.
- e2e: those paths answer 401 or 404; the restore test now fails when
the chunks skip the token check (checked with the check removed).
- WEB_CORS lets the socket take Origin: null instead of skipping the
Origin check.
- ADR 0006, GLOSSARY and README: the Origin check in either mode, /ws
as a public route, the Host header behind a reverse proxy.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Under Secured every HTTP route but the public ones needs the token
(docs/adr/0006-two-security-modes-eleven-http-routes.md):
- isPublicRoute, isWebAsset and originMatchesHost in src/HttpAccess.h,
with native tests.
- One middleware checks the token. It runs once the body is in, so
/restore, /updateFirmware and /updateApplication check it on their
first chunk and drop a refused upload.
- Static serving answers the web UI's files only: shades.cfg and
shades.tmp are no longer served.
- The wildcard CORS headers go; WEB_CORS turns them back on for UI work
from file://.
- The socket refuses a handshake whose Origin doesn't name its Host.
- "Secure Config Only" goes: a device set to it is Secured, and
/loginContext drops permissions.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Home Assistant holds a socket slot whenever it runs, so the tests that
needed every slot never ran on a device it uses. They now fill the slots
left and skip only when none is.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- e2e: the heap check uses a socket slot now, so it waits for the device
to free it, and test_every_client_reconnecting_at_once closes its
clients before the check runs.
- Comments and docs that still named removed things (the /controller read,
the event buffer, memStatus, SocketEmitter, ModelSnapshot).
- ESPNetwork::emitSockets is emitStatus: one socket, and it sends the status.
- Unused includes, whitespace-only lines left by the toJSON removal.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Step 4a of docs/SECURITY_PLAN.md. Port 80 keeps the eleven HTTP routes of
ADR 0006 and the socket at /ws; everything else moved to protocol 2.
- Port 8081 (ApiRoutes) and the socket on 8080 (SocketEmitter) are gone.
Protocol 2 owns its frame buffer and loop-task check.
- Port 80 drops every route the old web UI called: reads, edits,
settings, scans, reboot, /shades.cfg, /discovery. HttpRoutes.cpp holds
what stays.
- ModelSnapshot, SnapshotSlots and the read documents are gone. The item
snapshot types stay, as ItemSnapshot, for events, MQTT and the command
replies.
- Model edits reply only to their protocol 2 op: requestResult goes.
- The settings' toJSON writers, used only by the settings routes, go.
- The _espsomfy_rts._tcp mDNS record moves to port 80, with path=/ws.
- The memory event reports the loop-rate window it closes.
- A deleted room no longer re-publishes its retained MQTT topics.
- e2e: the heap check reads the sync's status; the sync is checked on
its own instead of against /controller.
- README gets an API and ports section.
BREAKING CHANGE: ports 8081 and 8080 and the old HTTP routes are gone.
Home Assistant needs integration 3.0.0 or newer.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Load the token key in Web::startup(), before either web server starts,
so no token can be made with the all-zero key; log a failed
pref.begin().
- createAPIToken checks mbedtls' return codes. On a failure it leaves
the token empty, which tokenMatches never accepts, instead of hex
from uninitialised stack.
- API_TOKEN_SIZE in ApiToken.h replaces the five char[65] buffers; the
HMAC's description moves to createAPIToken, with the None form.
- handleLogin builds the candidate token once and compares it once.
- e2e: the serverId test skips on `secured` like its neighbours, names
its values, and says it assumes the device sees this PC's address.
- KNOWN_ISSUES keeps #81, which #108 removes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Tokens were HMAC(serverId, credentials + client IP). serverId is public
(mDNS, MQTT, /loginContext), so anyone who saw a token could test PIN
guesses offline. The key is now 32 random bytes made on first boot in
their own NVS namespace, which no settings save clears and no backup
holds. It is made before WiFi starts, from the bootloader's entropy
source.
Tokens are compared with tokenMatches, which takes the same time
whatever the guess, on the apikey header, the socket upgrade and
/login.
createAPIToken never freed its mbedtls context, so every token it made
leaked: 563 bytes per login on the device, until it ran out of memory
after about 150 logins. It now frees it: 0 bytes over 500 logins.
Every token made before the upgrade is refused once. The web UI asks
for login again, and Home Assistant 3.0 logs in again by itself.
Closes#80
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
PR C of SECURITY_PLAN.md: after 5 failed /login attempts an IP is refused
for 30 s, and each failure after that refuses it for twice as long, up
to 15 min. A success resets it, as does an hour without a failure.
- LoginThrottle.h holds the schedule and 8 IPs, with the time passed in
so native tests cover it. While all 8 hold recent failures, every
other IP shares one more slot, so many addresses don't each get five
free tries.
- A refused IP gets 429 with Retry-After before its credentials are
checked, so a right guess during a lockout reveals nothing. Home
Assistant retries a 429 instead of asking for new credentials.
- The web UI's login panel shows a 401's or 429's message, where it
popped a service error.
- test/e2e --lockout checks it on a Secured device and waits it out.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The web UI now uses /ws, one slot per open tab. The tests that fill all 5
slots failed with a timeout while a tab was open; they now skip and say
to close it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- data-src/js: ProtoSchema (schema from the sync, short keys and names,
defaults, cached by hash), ProtoModel (rooms, shades, groups, repeaters,
header and status from the sync and the events) and ProtoClient (/ws,
hello, sync with next, op() as a Promise of its reply, events, feeds,
reconnect). Nothing uses them yet: 2b moves the UI onto them.
- minify.py joins js/*.js and index.js into index.js, and the three CSS
files into app.css, so a page load opens fewer connections.
- test/js: node --test on the three modules, replaying the recorded
fixtures; a CI job runs it and the firmware build waits for it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- shadeRemoved, groupRemoved and roomRemoved went out from the deletes,
before the edit's reply. They now come from editChanges and go out with
the edit's other events, after it (test_adding_and_deleting_a_room).
- test_reboot reconnected in the moment between the sockets closing and the
restart, and waited for a hello that never came. It now waits, retries a
connect without hello, and checks the uptime went back.
e2e on the device (esp32s3dbg): 72 passed, 3 skipped.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- The frequency scan ops and reboot run on loop() like every op (ADR 0005).
endFrequencyScan goes to the front of the loop queue, so it never waits
behind an unpair that waits for the scan to end.
- A security save that leaves the device Secured, or changes its mode,
closes every client, the saver too once its reply has gone. None to None
closes nobody.
- proto must be a radio_proto value; frequency, rxBandwidth and deviation
are checked against what the CC1101 can be set to.
- wifiNetworks sends fewer networks instead of none when they don't fit.
- A client WiFi scan doesn't start while the device joins a network.
- rebootTime is set before reboot; cancelSettingsSave is freeSettingsSave.
- PROTOCOL.md: pins that may be -1, and the fields the routes sent that
moved elsewhere. e2e: the WiFi scan test goes by the connection in use.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Step 1d of the protocol 2 core (docs/SECURITY_PLAN.md).
- getSettings/saveSettings by section: general, network, mqtt, security,
radio. A save is the settings route's request, so loop() applies it with
the settings' own fromJSON(). ProtoSettings checks every field first:
kind, range, string length, IP addresses.
- Secrets are only reported as has*. A security save replies with the
client's new token and closes the other sockets; other saves send
settingsChanged to the other clients.
- scanWifi runs an async scan from ESPNetwork, kept apart from the roaming
scan, and wifiNetworks goes to the clients that asked.
- beginFrequencyScan, endFrequencyScan and reboot run at once on async_tcp,
as their routes do: a scan's end mustn't wait in the loop queue behind an
unpair that waits for the scan to end (new op kind 'a').
- e2e: test_settings.py and a --reboot flag.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- A group's sunSensor and a shade's flags are refused by name: the routes
don't set them.
- setPaired needs paired, and without a shade id it is a bad request.
- groupOptions' payload is parsed in ProtoEdits (parseReadPayload).
- The assembler's mutex is static, so creating it can't fail.
- protoIsReadOp and protoIsShadeOp are gone; the tests use protoOpKind.
- Native tests for missing(), offeredRemoteAddress and the edit ops
against the TYPE table.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
From the review of 7723711:
- The assembler is locked: a client closed for a full queue is closed
by the task that was sending, loop(), whose disconnect dropped the
buffer while async_tcp could be copying into it. A whole message is
taken out of it, so a disconnect can't free it while it is read.
- loop() expires a message stalled for 10 s, freeing its buffer and
telling its client, instead of holding it until another client came.
A binary message is refused before it can touch another's buffer.
- Each edit takes only the fields its route reads; one it would ignore
is refused by name. linkRemote's rolling code goes in under the
route's name, so applyModelEdit is unchanged again.
- An edit refused for an id that doesn't exist replies not found
(edit_result_t::missing). A reply that doesn't fit its buffer goes
as an internal error, so the op still gets one.
- The TYPE table says what becomes of each op (proto_op_t), which the
socket switches on. The reply address is one proto_reply_to_t. The
routes and the ops start and queue an edit the same way
(newModelEdit, submitModelEdit). The full-list texts and the offered
remote address rule live in ModelEditor.
- e2e: the sort order test skips a list that it would renumber.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The edit routes become ops 26-44 (addRoom ... groupSortOrder): ProtoEdits
turns an op's payload into the route's request, full names and real
booleans, and loop() applies it with applyModelEdit() as the route
does. The reply goes to the client that sent it, with an add's new id,
or the route's reason and a shade's validation code; the HTTP routes
keep sending requestResult.
getNextRoom, getNextShade, getNextGroup and groupOptions (45-48) are
answered by loop() from the shade model, through a new LoopQueue item.
Messages that arrive in pieces, a frame over one TCP segment or several
frames, are put back together by ProtoAssembly: one at a time across
clients, up to 4 KB, in a buffer held only while one is on its way.
A message in pieces is now told from its first frame's opcode, which
also stops a fragmented message's last frame drawing a second reply.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
From the review of #100:
- setMyPosition on a moving shade replies busy ("The shade is moving"),
and setPositions with only a tilt, for a shade without one, replies
bad request ("The shade has no tilt"). Both said ok before.
- PROTOCOL.md lists where the ops differ from the routes, and how
sendRemoteCommand is answered.
- The LoopQueue hold comment covers protocol 2 replies and an unpair
held through a frequency scan.
- One shadeRequestIsForGroup() for the group cases; ProtoOps shares one
reader for "one or both" fields and one for ids, and the radio
refused text.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
shadeCommand, groupCommand, tiltCommand, repeatCommand, setPositions,
setSensor, setMyPosition, unpairShade, setRollingCode (types 16-24) are
read on async_tcp by ProtoOps and handed to loop() through the
LoopQueue, which replies once the request has run: ok, not found or
radio unavailable. A bad payload, a paused radio or a full queue is
answered straight away. sendRemoteCommand (25) touches nothing of the
shade model, so it goes to the radio from async_tcp like its route.
ShadeRequests::run() now says how a request went, not only whether
the radio took it. An op reads only the fields that were sent, so
setPositions can leave the tilt as it is. The shadeCommand event is
renamed shadeCommanded, since the op has the route's name.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
From the review of #99:
- A sync could stall when a client took a slot the moment another left:
loop() used up the new client's sync on the old one. The slots move
to ProtoSlots, a pure module with native tests, and loop() takes a
slot's wants only for the client it read (takeWants).
- Deleting a group sends its shades, whose inGroup changed. Deleting a
shade unlinks it from its groups and sends them, so a new shade given
its id doesn't join them.
- next and sync with an id get an ok reply, and a message in pieces
gets one bad request, not one per piece.
- Feeds check for loop() before they write the event buffer, and log
an event too large, as events do.
- editChanges() (ModelEditor, native tests) says what clients get after
an edit; the edit result carries the shade a group gained or lost.
order() takes a sort_list_t.
- The frame writers move to ProtoFrame. Renames: pf to proto_field_t,
the X-macro letters to TYPE/FIELD/ERR/FEED, proto_request_t to
proto_message_t, REQ_* to WANTS_*, fmt to eventJson. The event
bodies share sendItem(), the connection type name lives in
ESPNetwork, and the loop() task check in SocketEmitter.
- e2e: the heap floor check skips when the low since boot is already
under it; tests for replies to next and sync and a long message.
- SECURITY_PLAN.md splits step 1 into 1a-1d.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
hello, the schema and a whole sync as the device sent them, recorded with pytest test/e2e --record. Remote addresses, rolling codes, names, the server id, hostname and SSID are replaced with fake values while recording, since with an address and its rolling code anyone could forge frames for a real installation's shades.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A pytest suite run by hand against a device (test/e2e/README.md): hello, the schema and its hash, the sync with next and resync, replies to bad frames, feed subscriptions, the status event, every client syncing at once, a sixth client refused, a page load, and the socket under the device's security mode. It never transmits, changes settings or reboots. --record saves hello, the sync and the schema as fixtures.
PROTOCOL.md now describes the protocol as built.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
