- e2e: the heap check uses a socket slot now, so it waits for the device
to free it, and test_every_client_reconnecting_at_once closes its
clients before the check runs.
- Comments and docs that still named removed things (the /controller read,
the event buffer, memStatus, SocketEmitter, ModelSnapshot).
- ESPNetwork::emitSockets is emitStatus: one socket, and it sends the status.
- Unused includes, whitespace-only lines left by the toJSON removal.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Step 4a of docs/SECURITY_PLAN.md. Port 80 keeps the eleven HTTP routes of
ADR 0006 and the socket at /ws; everything else moved to protocol 2.
- Port 8081 (ApiRoutes) and the socket on 8080 (SocketEmitter) are gone.
Protocol 2 owns its frame buffer and loop-task check.
- Port 80 drops every route the old web UI called: reads, edits,
settings, scans, reboot, /shades.cfg, /discovery. HttpRoutes.cpp holds
what stays.
- ModelSnapshot, SnapshotSlots and the read documents are gone. The item
snapshot types stay, as ItemSnapshot, for events, MQTT and the command
replies.
- Model edits reply only to their protocol 2 op: requestResult goes.
- The settings' toJSON writers, used only by the settings routes, go.
- The _espsomfy_rts._tcp mDNS record moves to port 80, with path=/ws.
- The memory event reports the loop-rate window it closes.
- A deleted room no longer re-publishes its retained MQTT topics.
- e2e: the heap check reads the sync's status; the sync is checked on
its own instead of against /controller.
- README gets an API and ports section.
BREAKING CHANGE: ports 8081 and 8080 and the old HTTP routes are gone.
Home Assistant needs integration 3.0.0 or newer.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Load the token key in Web::startup(), before either web server starts,
so no token can be made with the all-zero key; log a failed
pref.begin().
- createAPIToken checks mbedtls' return codes. On a failure it leaves
the token empty, which tokenMatches never accepts, instead of hex
from uninitialised stack.
- API_TOKEN_SIZE in ApiToken.h replaces the five char[65] buffers; the
HMAC's description moves to createAPIToken, with the None form.
- handleLogin builds the candidate token once and compares it once.
- e2e: the serverId test skips on `secured` like its neighbours, names
its values, and says it assumes the device sees this PC's address.
- KNOWN_ISSUES keeps #81, which #108 removes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Tokens were HMAC(serverId, credentials + client IP). serverId is public
(mDNS, MQTT, /loginContext), so anyone who saw a token could test PIN
guesses offline. The key is now 32 random bytes made on first boot in
their own NVS namespace, which no settings save clears and no backup
holds. It is made before WiFi starts, from the bootloader's entropy
source.
Tokens are compared with tokenMatches, which takes the same time
whatever the guess, on the apikey header, the socket upgrade and
/login.
createAPIToken never freed its mbedtls context, so every token it made
leaked: 563 bytes per login on the device, until it ran out of memory
after about 150 logins. It now frees it: 0 bytes over 500 logins.
Every token made before the upgrade is refused once. The web UI asks
for login again, and Home Assistant 3.0 logs in again by itself.
Closes#80
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- The throttle runs on a 64-bit clock in ms (esp_timer_get_time()), so a
slot left alone for 49.7 days isn't brought back by millis() wrapping
round to its old failure.
- Web.cpp turns the client into the throttle's key in one place, and
says that an IPv6 client would share key 0; names are clearer
(retryMs, vacantSlot).
- The shared slot's test now succeeds from the IP counted in it, so a
success that cleared the shared slot would fail it.
- GLOSSARY.md names it the login throttle, with lockouts, and
SECURITY_PLAN.md and ADR 0006 use that name.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
6.2.0.post1 reports itself as 6.2.0-post.1, which the pinned platform's
">=6.2.0" requirement reads as a pre-release, so every ESP32 build
fails at its first pio run.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
PR C of SECURITY_PLAN.md: after 5 failed /login attempts an IP is refused
for 30 s, and each failure after that refuses it for twice as long, up
to 15 min. A success resets it, as does an hour without a failure.
- LoginThrottle.h holds the schedule and 8 IPs, with the time passed in
so native tests cover it. While all 8 hold recent failures, every
other IP shares one more slot, so many addresses don't each get five
free tries.
- A refused IP gets 429 with Retry-After before its credentials are
checked, so a right guess during a lockout reveals nothing. Home
Assistant retries a 429 instead of asking for new credentials.
- The web UI's login panel shows a 401's or 429's message, where it
popped a service error.
- test/e2e --lockout checks it on a Secured device and waits it out.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The web UI now uses /ws, one slot per open tab. The tests that fill all 5
slots failed with a timeout while a tab was open; they now skip and say
to close it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
All three CSS files start with a UTF-8 byte order mark. Joined into app.css,
the marks of the second and third files sat mid-file, where the browser
reads one as part of the next selector and drops that rule: .container's
max-width (the page ran full width) and the base icss- icon rule (icons
drawn wrong). minify.py now reads the parts as utf-8-sig.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
All three CSS files start with a UTF-8 byte order mark. Joined into app.css,
the marks of the second and third files sat mid-file, where the browser
reads one as part of the next selector and drops that rule: .container's
max-width (the page ran full width) and the base icss- icon rule (icons
drawn wrong). minify.py now reads the parts as utf-8-sig.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Saving the security settings threw: the form's values were named
security, hiding the Security object the new token goes to. A save from
None to None no longer reconnects, as the device closes nobody then.
- Each sync redraws the lists and reloads the settings, so a reconnect
shows what changed meanwhile; closing the config panel drops its feeds
even while the socket is down.
- A 401 on backup or an upload opens the login, as a refused socket does
(Security.tokenRefused); a failed firmware upload says so.
- The general and security panels fill in what the device leaves out at
its default, so a save never writes back what the form showed before.
The hostname holds 31 characters, as the device does.
- The views (settings with defaults, a group's shades, frames, scans, the
network and radio panels, op errors) move to data-src/js/views.js, with
tests. runOp is the one op path with the busy retry; repeatOf,
commandArgs, gone(); dead update*List and editedRepeaters removed.
- Tests for ProtoClient.off and a listener removing itself mid-event.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The UI talks to the device over the one socket at /ws: the sync fills the
shade model, its events redraw the lists, and every read, command, edit,
setting, scan and reboot is an op. Only /loginContext, /login and the file
transfers stay on HTTP, so a page load opens no other connection.
- The 8080 socket, editJSON/requestResult and the /controller, /rooms,
/shades, /groups, /room, /shade, /group, settings and command routes are
no longer used by the UI.
- An edit's callback gets the item once its event is in (ProtoClient
waitFor); busy edits and saves are sent again.
- The token is kept in sessionStorage. When /ws is refused and
/loginContext says a login is needed, the client stops retrying and the
login panel opens. Backup and uploads send the token.
- A banner shows when the device speaks another protocol version.
- Removed dead code: the /pairShade call and its unused /unpairShade twin,
loadLogin, login.html (broken, linked from nowhere), Secure Config Only's
checkbox (protocol 2 has two modes) and the HTTP helpers nothing uses.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- One socket at a time: a socket being replaced is let go without a word,
and connect() or a new token while a reconnect waits opens no second one.
A new token reconnects at once, without a refused.
- Events that came before the schema go as soon as it is whole, ahead of
the sync frames formatted after them, not at syncEnd over newer state.
- A schema's hash counts only once it has come whole, so a sync cut short
gets it again.
- Feeds are subscribed again after a reconnect; an unknown feed is refused.
- A long op refused as busy without its id goes again.
- A sync without repeaters clears them; order changes name their list; a
refused resync is called off.
- refused: a listener can close() to stop the retries, which the UI does
once /loginContext says a login is needed. The client itself can't tell a
refused token from a device that is down.
- localStorage keeps the schema by default. minify.py removes its work
folder even when a bundle fails. ADR 0005 records the two bundles.
- Smaller: ProtoSchema.ROWS/withoutRows, ProtoModel.ITEMS, named type
numbers, unused feeds map and FakeSocket.all gone.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- data-src/js: ProtoSchema (schema from the sync, short keys and names,
defaults, cached by hash), ProtoModel (rooms, shades, groups, repeaters,
header and status from the sync and the events) and ProtoClient (/ws,
hello, sync with next, op() as a Promise of its reply, events, feeds,
reconnect). Nothing uses them yet: 2b moves the UI onto them.
- minify.py joins js/*.js and index.js into index.js, and the three CSS
files into app.css, so a page load opens fewer connections.
- test/js: node --test on the three modules, replaying the recorded
fixtures; a CI job runs it and the firmware build waits for it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- shadeRemoved, groupRemoved and roomRemoved went out from the deletes,
before the edit's reply. They now come from editChanges and go out with
the edit's other events, after it (test_adding_and_deleting_a_room).
- test_reboot reconnected in the moment between the sockets closing and the
restart, and waited for a hello that never came. It now waits, retries a
connect without hello, and checks the uptime went back.
e2e on the device (esp32s3dbg): 72 passed, 3 skipped.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- runModelEdit sent an edit's events (room, roomRemoved, the orders,
repeaters) before its reply, where PROTOCOL.md says they follow it. A
client waiting for the reply read past them (test_adding_and_deleting_a_room).
- A restart dropped the /ws connections without a close, so clients never
saw them go (test_reboot). ESPNetwork::end() now closes them too.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The socket has two modes, None and Secured (ADR 0006), and ignores the
ConfigOnly bit, so the security section no longer reads or saves it.
The HTTP routes keep it until step 4 removes the mode.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- The frequency scan ops and reboot run on loop() like every op (ADR 0005).
endFrequencyScan goes to the front of the loop queue, so it never waits
behind an unpair that waits for the scan to end.
- A security save that leaves the device Secured, or changes its mode,
closes every client, the saver too once its reply has gone. None to None
closes nobody.
- proto must be a radio_proto value; frequency, rxBandwidth and deviation
are checked against what the CC1101 can be set to.
- wifiNetworks sends fewer networks instead of none when they don't fit.
- A client WiFi scan doesn't start while the device joins a network.
- rebootTime is set before reboot; cancelSettingsSave is freeSettingsSave.
- PROTOCOL.md: pins that may be -1, and the fields the routes sent that
moved elsewhere. e2e: the WiFi scan test goes by the connection in use.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Step 1d of the protocol 2 core (docs/SECURITY_PLAN.md).
- getSettings/saveSettings by section: general, network, mqtt, security,
radio. A save is the settings route's request, so loop() applies it with
the settings' own fromJSON(). ProtoSettings checks every field first:
kind, range, string length, IP addresses.
- Secrets are only reported as has*. A security save replies with the
client's new token and closes the other sockets; other saves send
settingsChanged to the other clients.
- scanWifi runs an async scan from ESPNetwork, kept apart from the roaming
scan, and wifiNetworks goes to the clients that asked.
- beginFrequencyScan, endFrequencyScan and reboot run at once on async_tcp,
as their routes do: a scan's end mustn't wait in the loop queue behind an
unpair that waits for the scan to end (new op kind 'a').
- e2e: test_settings.py and a --reboot flag.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Init() starts the SPI bus and resets the chip, then every register write
calls SpiStart() again while the bus is still up. SpiStart() ran pinMode()
on SCK, MOSI and MISO. On Arduino 3.x that takes the pins from the SPI bus,
which stops it (spiStopBus) while SPIClass still holds the handle, so the
following SPI.begin() returns early and the bus stays down.
On the S3 the stopped bus also has clk_gate cleared, so spiTransferByte()
waits on cmd.usr forever and the task watchdog aborts async_tcp when the
radio settings are saved. On the ESP32 the transfer completes on detached
pins, so the first register write (FSCTRL1) is silently lost.
SPI.begin() attaches and configures the bus pins itself, so on ESP32
targets SpiStart() now only sets SS, which the driver drives by hand.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- A group's sunSensor and a shade's flags are refused by name: the routes
don't set them.
- setPaired needs paired, and without a shade id it is a bad request.
- groupOptions' payload is parsed in ProtoEdits (parseReadPayload).
- The assembler's mutex is static, so creating it can't fail.
- protoIsReadOp and protoIsShadeOp are gone; the tests use protoOpKind.
- Native tests for missing(), offeredRemoteAddress and the edit ops
against the TYPE table.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
From the review of 7723711:
- The assembler is locked: a client closed for a full queue is closed
by the task that was sending, loop(), whose disconnect dropped the
buffer while async_tcp could be copying into it. A whole message is
taken out of it, so a disconnect can't free it while it is read.
- loop() expires a message stalled for 10 s, freeing its buffer and
telling its client, instead of holding it until another client came.
A binary message is refused before it can touch another's buffer.
- Each edit takes only the fields its route reads; one it would ignore
is refused by name. linkRemote's rolling code goes in under the
route's name, so applyModelEdit is unchanged again.
- An edit refused for an id that doesn't exist replies not found
(edit_result_t::missing). A reply that doesn't fit its buffer goes
as an internal error, so the op still gets one.
- The TYPE table says what becomes of each op (proto_op_t), which the
socket switches on. The reply address is one proto_reply_to_t. The
routes and the ops start and queue an edit the same way
(newModelEdit, submitModelEdit). The full-list texts and the offered
remote address rule live in ModelEditor.
- e2e: the sort order test skips a list that it would renumber.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The edit routes become ops 26-44 (addRoom ... groupSortOrder): ProtoEdits
turns an op's payload into the route's request, full names and real
booleans, and loop() applies it with applyModelEdit() as the route
does. The reply goes to the client that sent it, with an add's new id,
or the route's reason and a shade's validation code; the HTTP routes
keep sending requestResult.
getNextRoom, getNextShade, getNextGroup and groupOptions (45-48) are
answered by loop() from the shade model, through a new LoopQueue item.
Messages that arrive in pieces, a frame over one TCP segment or several
frames, are put back together by ProtoAssembly: one at a time across
clients, up to 4 KB, in a buffer held only while one is on its way.
A message in pieces is now told from its first frame's opcode, which
also stops a fragmented message's last frame drawing a second reply.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
From the review of #100:
- setMyPosition on a moving shade replies busy ("The shade is moving"),
and setPositions with only a tilt, for a shade without one, replies
bad request ("The shade has no tilt"). Both said ok before.
- PROTOCOL.md lists where the ops differ from the routes, and how
sendRemoteCommand is answered.
- The LoopQueue hold comment covers protocol 2 replies and an unpair
held through a frequency scan.
- One shadeRequestIsForGroup() for the group cases; ProtoOps shares one
reader for "one or both" fields and one for ids, and the radio
refused text.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
shadeCommand, groupCommand, tiltCommand, repeatCommand, setPositions,
setSensor, setMyPosition, unpairShade, setRollingCode (types 16-24) are
read on async_tcp by ProtoOps and handed to loop() through the
LoopQueue, which replies once the request has run: ok, not found or
radio unavailable. A bad payload, a paused radio or a full queue is
answered straight away. sendRemoteCommand (25) touches nothing of the
shade model, so it goes to the radio from async_tcp like its route.
ShadeRequests::run() now says how a request went, not only whether
the radio took it. An op reads only the fields that were sent, so
setPositions can leave the tilt as it is. The shadeCommand event is
renamed shadeCommanded, since the op has the route's name.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
From the review of #99:
- A sync could stall when a client took a slot the moment another left:
loop() used up the new client's sync on the old one. The slots move
to ProtoSlots, a pure module with native tests, and loop() takes a
slot's wants only for the client it read (takeWants).
- Deleting a group sends its shades, whose inGroup changed. Deleting a
shade unlinks it from its groups and sends them, so a new shade given
its id doesn't join them.
- next and sync with an id get an ok reply, and a message in pieces
gets one bad request, not one per piece.
- Feeds check for loop() before they write the event buffer, and log
an event too large, as events do.
- editChanges() (ModelEditor, native tests) says what clients get after
an edit; the edit result carries the shade a group gained or lost.
order() takes a sort_list_t.
- The frame writers move to ProtoFrame. Renames: pf to proto_field_t,
the X-macro letters to TYPE/FIELD/ERR/FEED, proto_request_t to
proto_message_t, REQ_* to WANTS_*, fmt to eventJson. The event
bodies share sendItem(), the connection type name lives in
ESPNetwork, and the loop() task check in SocketEmitter.
- e2e: the heap floor check skips when the low since boot is already
under it; tests for replies to next and sync and a long message.
- SECURITY_PLAN.md splits step 1 into 1a-1d.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
hello, the schema and a whole sync as the device sent them, recorded with pytest test/e2e --record. Remote addresses, rolling codes, names, the server id, hostname and SSID are replaced with fake values while recording, since with an address and its rolling code anyone could forge frames for a real installation's shades.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A pytest suite run by hand against a device (test/e2e/README.md): hello, the schema and its hash, the sync with next and resync, replies to bad frames, feed subscriptions, the status event, every client syncing at once, a sixth client refused, a page load, and the socket under the device's security mode. It never transmits, changes settings or reboots. --record saves hello, the sync and the schema as fixtures.
PROTOCOL.md now describes the protocol as built.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The /ws endpoint runs next to the old socket on port 8080. The
handshake checks the token under security, the device sends hello,
then a sync of the schema (unless the client has it), header, rooms,
shades, groups, repeaters and status, paced by the client's next.
Events go out with the old ones: shadeState or the whole shade on a
config change, groups, rooms, removals, orders, repeaters,
shadeCommand, memory, wifi, ethernet, and the frames and
frequencyScan feeds.
The table of types, fields, errors and feeds is defined once and is
sent as the schema. The writer, item writers, sync and client frame
parser are pure and natively tested.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
