diff --git a/docs/KNOWN_ISSUES.md b/docs/KNOWN_ISSUES.md index e939526..003cc61 100644 --- a/docs/KNOWN_ISSUES.md +++ b/docs/KNOWN_ISSUES.md @@ -2,51 +2,36 @@ ## Concurrent mutation during chunked `/controller` response -**Status:** open -**Introduced:** 2026-04-19, alongside the chunked-response conversion of `/controller` (see [`ControllerChunker` in src/entryPoints/Web.cpp](../src/entryPoints/Web.cpp) and the original crash report about `cbuf::resize` aborting `async_tcp`). +**Status:** fixed for the read routes; the web-task writers below are still open. +**Introduced:** 2026-04-19, alongside the chunked-response conversion of `/controller` (the original crash report was about `cbuf::resize` aborting `async_tcp`). +**Fixed:** 2026-10-16, step 2 of [LOOP_QUEUE_PLAN.md](LOOP_QUEUE_PLAN.md). -### Summary +### What it was -The chunked streaming of `/controller` lazily reads `somfy.rooms`, `somfy.shades`, `somfy.groups`, and `somfy.repeaters` across many `async_tcp` callback invocations as the client's TCP send window opens. If another task mutates those arrays mid-stream, the rendered JSON can be internally inconsistent. +The chunked `/controller` response read `somfy.rooms`, `somfy.shades`, `somfy.groups` and `somfy.repeaters` on async_tcp a piece at a time as the client's TCP window opened, for hundreds of ms to seconds. Anything that changed them meanwhile could make the document contradict itself: a deleted shade in `shades` but not in a group's `linkedShades`, two positions for one shade, a linked shade skipped or written twice. `/discovery`, `/shades` and the other reads had the same exposure, only shorter. -### Why this is strictly worse than the pre-chunked code +All eight read routes (`/controller`, `/discovery`, `/rooms`, `/shades`, `/groups`, `/room`, `/shade`, `/group`) now stream a ModelSnapshot that `loop()` fills, so each document describes one moment ([ADR-0002](adr/0002-loop-task-is-the-only-writer-of-the-shade-model.md)). That is fix option (c), taken per read in a heap slot of about 5.6 KB. -The old `handleController` ran the serializers synchronously inside a single request-handler invocation, so the full JSON was built in memory *before* any bytes were sent. Window of exposure: a few milliseconds. +### Still open -The new `ControllerChunker` reads state as the response drains. On a slow link or under backpressure, that window is hundreds of ms to seconds. - -### Concrete failure modes - -1. A shade is deleted mid-stream (e.g. via `/deleteShade`) — it may appear in the `shades` array but be missing from a later group's `linkedShades`, or vice versa. -2. `lastRollingCode` / position fields change between the `shades` pass and a later group's `linkedShades` pass — the client sees two values for the same shade in one document. -3. A group's `linkedShades` list is mutated while the chunker iterates inside `S_GROUPS` — an entry is skipped or emitted twice. - -### Fix options (pick one later) - -- **(a) Document and accept.** In practice users rarely mutate shades while the config UI is loading. Zero code change. -- **(b) FreeRTOS mutex around `somfy` reads/writes.** Acquire for the full duration of the chunked response and in every mutating path (RF RX, MQTT, web handlers). Cleanest but wide-reaching. -- **(c) Up-front snapshot.** At handler start, copy the subset of `somfy` state the response will serialize into the `ControllerChunker`. Defeats part of the memory benefit — a full snapshot of shades + groups is close in size to the old growing cbuf. Could be reduced by snapshotting only minimal fields (IDs, names, rolling codes) and reading the rest live. - -### Related - -- Same exposure exists in any other endpoint converted to chunked responses next (`/discovery`, `/shades`). Resolve this issue before expanding the pattern. -- Other handlers on the web server task still change the shade model that `loop()` owns: +- Command and edit replies (`/shadeCommand`, `/saveShade`, `/linkRemote`, ...) still copy the one item they report from the live shade model on async_tcp, so a reply can show a value from the middle of a change. `/room`, `/shade` and `/group` also check on the live model whether the item exists, because the status goes out before the snapshot is filled. +- Other handlers on the web server task still change the shade model that `loop()` owns (step 3 of the plan): - `/addRoom`, `/saveRoom`, `/deleteRoom`, `/addShade`, `/saveShade`, `/deleteShade`, `/addGroup`, `/saveGroup` and `/deleteGroup` add, change and remove rooms, shades and groups. - `/linkRemote` and `/unlinkRemote` change a shade's linked remotes. `/linkRemote` also sets a linked remote's rolling code, which `loop()` updates whenever it receives a frame from that remote: the last write wins, so a received frame can undo the code just set. - `/linkToGroup` and `/unlinkFromGroup` change a group's linked shades. - `/setPaired` sets `paired`. Shade requests (shade and group commands, set-My, unpair, `/setPositions`, `/setRollingCode`, and every MQTT command), a shades restore and `/updateShadeConfig` already go through the LoopQueue to `loop()`. -- `/controller` leaves out a room, shade, group or linked shade that doesn't fit in its chunk buffer, see [Oversized `/controller` items are left out](#oversized-controller-items-are-left-out). +- The read routes leave out an item that doesn't fit their 3 KB unit, see [Oversized read items are left out](#oversized-read-items-are-left-out). -## Oversized `/controller` items are left out +## Oversized read items are left out **Status:** open -**Location:** `ControllerChunker::overflowed` in [src/entryPoints/Web.cpp](../src/entryPoints/Web.cpp). +**Location:** `SnapshotWriter::overflowed` in [src/ModelSnapshot.cpp](../src/ModelSnapshot.cpp). ### Summary -`ControllerChunker` builds each room, shade, group and linked shade into a 3 KB buffer before sending it. When one doesn't fit, the chunker logs an `ESP_LOGE` line, skips that item and carries on, so the document stays valid JSON. The client gets no sign that anything is missing: the item just isn't in the list. +Every read route builds each room, shade, group and linked shade into a 3 KB unit before sending it. When one doesn't fit, the writer logs an `ESP_LOGE` line, skips that item and carries on, so the document stays valid JSON. The client gets no sign that anything is missing: the item just isn't in the list. Before step 2 only `/controller` did this; `/discovery`, `/shades` and the rest grew their buffer instead. A shade with every linked remote populated is about 1.5 KB today, so nothing overflows yet. It becomes a silent loss as soon as a change to the shade, room or group fields pushes an item past 3 KB. @@ -90,12 +75,12 @@ Unlike the `/controller` HTTP crash, this path does **not** abort — there is n - Not the same code path as the `/controller` crash. Solve independently. - Worth grepping for `JsonSockEvent` usages that iterate collections (see references in `SomfyEvents.cpp`, `Transceiver.cpp`, `ESPNetwork.cpp`, `GitOTA.cpp`) to identify the most at-risk events. -- [Group reads emit events from the web server task](#group-reads-emit-events-from-the-web-server-task): a second task writing the same buffer. +- [Group reads emit events from the web server task](#group-reads-emit-events-from-the-web-server-task) (fixed): a second task writing the same buffer. ## Group reads emit events from the web server task -**Status:** open -**Location:** `serializeGroupRef` in [src/entryPoints/Web.cpp](../src/entryPoints/Web.cpp) calls `SomfyGroup::updateFlags()` in [src/SomfyGroup.cpp](../src/SomfyGroup.cpp). +**Status:** fixed 2026-10-16, step 2 of [LOOP_QUEUE_PLAN.md](LOOP_QUEUE_PLAN.md) (fix option (a)). Read routes write from a ModelSnapshot, and the replies still written on async_tcp copy the group with `snapshotGroup()`, which works its flags out without changing the group or sending anything. +**Location:** `serializeGroupRef` in [src/entryPoints/Web.cpp](../src/entryPoints/Web.cpp) called `SomfyGroup::updateFlags()` in [src/SomfyGroup.cpp](../src/SomfyGroup.cpp). ### Summary diff --git a/docs/LOOP_QUEUE_PLAN.md b/docs/LOOP_QUEUE_PLAN.md index f9fcba3..7a68701 100644 --- a/docs/LOOP_QUEUE_PLAN.md +++ b/docs/LOOP_QUEUE_PLAN.md @@ -1,6 +1,6 @@ # Plan: domain modules behind the LoopQueue -**Status:** steps 0 and 1 built (PRs #39 and #40); steps 2–5 planned. +**Status:** steps 0–2 built (PRs #39, #40 and #41); steps 3–5 planned. **Written:** 2026-10-15, against commit `e624a41` on `refactor/entry-points-outputs`. Terms are defined in [GLOSSARY.md](../GLOSSARY.md); the threading rule is [ADR-0002](adr/0002-loop-task-is-the-only-writer-of-the-shade-model.md). ## Why @@ -130,6 +130,20 @@ Each step is a PR of its own that can be tested on the device. Each one runs `pi Flash: `esp32devdbg` 1,546,405 → 1,547,381 bytes, `esp32dev` 1,396,957 → 1,398,337 bytes. +### Step 2: what changed in behaviour +- **Each read describes one moment.** `/controller`, `/discovery`, `/rooms`, `/shades`, `/groups`, `/room`, `/shade` and `/group` stream a ModelSnapshot that `loop()` fills. The first two KNOWN_ISSUES items about `/controller` are fixed for reads. +- **Every read is chunked.** `/discovery`, `/shades` and the rest used to be built whole into a growing response buffer, the path of the old `cbuf::resize` crash. +- **A read waits for `loop()`.** The headers go out at once; the body follows once the snapshot is filled. That is usually one `loop()` pass, but if the headers' ACK comes back first the library only asks again on its next poll, up to about 500 ms later. A read queued behind a command that waits for the radio waits with it. +- **At most two reads at once.** A third gets 503 "Busy, try the read again". Each read holds a 5.6 KB snapshot and a 3 KB unit on the heap only while it runs. +- **An item too big for the 3 KB unit is left out of every read route**, not only `/controller` (see KNOWN_ISSUES). +- **`/room`, `/shade` and `/group`** still answer 500 for an id that doesn't exist. If the item disappears between that check and the snapshot, the reply is 200 with `{"status":"ERROR","desc":"… Id not found."}`. +- **Group flags in replies are worked out without side effects.** Serializing a group no longer calls `updateFlags()`, which sent `groupState` and published MQTT from the web server task. +- **`/discovery` no longer logs where its time went.** Building it is now spread over the chunks. + +`ModelSnapshot.h/.cpp` holds the snapshot types, the JSON for each item, `SnapshotWriter` (the eight documents, built a unit at a time) and `SnapshotSlots`. `SomfySnapshot.cpp` copies the shade model into it, for `loop()`'s fills and for the replies still written on the web server task. The `SOMFY_MAX_*` limits moved to `ShadeTypes.h` so the native tests can use them. 21 new native tests, 292 in all. + +Flash: `esp32devdbg` 1,547,381 → 1,547,821 bytes, `esp32dev` 1,398,337 → 1,400,401 bytes. + ### Step 0 on a device NVS stays at `0x9000`, so the rolling codes survive. The file system partition shrinks from `0xE0000` to `0xD0000`, and the existing LittleFS won't mount, because `LittleFS.begin()` doesn't format. So: 1. Take a backup through `/backup`. diff --git a/platformio.ini b/platformio.ini index 02d8f13..808e5a7 100644 --- a/platformio.ini +++ b/platformio.ini @@ -113,7 +113,7 @@ lib_deps = platform = native test_framework = unity test_build_src = yes -build_src_filter = -<*> + + + + + + + + + + +build_src_filter = -<*> + + + + + + + + + + + build_flags = -std=gnu++17 -Isrc diff --git a/src/LoopQueue.cpp b/src/LoopQueue.cpp index 36e350a..1f52aa3 100644 --- a/src/LoopQueue.cpp +++ b/src/LoopQueue.cpp @@ -4,12 +4,14 @@ #include "LoopQueue.h" #include "ConfigFile.h" #include "Somfy.h" +#include "SomfySnapshot.h" #include "Utils.h" static const char *TAG = "LoopQueue"; extern SomfyShadeController somfy; extern ShadeRequests shadeRequests; +extern SnapshotSlots snapshotSlots; extern rebootDelay_t rebootDelay; static restore_options_t restoreOptions(uint8_t parts) { @@ -61,6 +63,9 @@ void LoopQueue::drain() { case loop_item_type_t::load_shades: somfy.loadShadesFile("/shades.tmp"); break; + case loop_item_type_t::snapshot_fill: + snapshotSlots.fill(item.slot, fillSnapshot); + break; } } } diff --git a/src/LoopQueue.h b/src/LoopQueue.h index d4551af..9cc545f 100644 --- a/src/LoopQueue.h +++ b/src/LoopQueue.h @@ -22,12 +22,14 @@ enum class loop_item_type_t : uint8_t { shade_request, // run by ShadeRequests restore_shades, // restore from /shades.tmp, then reboot - load_shades // replace the shades with /shades.tmp + load_shades, // replace the shades with /shades.tmp + snapshot_fill // fill a read's ModelSnapshot }; struct loop_item_t { loop_item_type_t type = loop_item_type_t::shade_request; shade_request_t request; // shade_request uint8_t restoreParts = 0; // restore_shades: RESTORE_* bits + uint8_t slot = 0; // snapshot_fill: the SnapshotSlots slot }; // Whether the item sends anything on air. diff --git a/src/ModelSnapshot.cpp b/src/ModelSnapshot.cpp new file mode 100644 index 0000000..2fb2cb8 --- /dev/null +++ b/src/ModelSnapshot.cpp @@ -0,0 +1,394 @@ +#include +#include +#include +#include "esp_log.h" +#include "ModelSnapshot.h" + +static const char *TAG = "Snapshot"; + +const room_snapshot_t *model_snapshot_t::room(uint8_t roomId) const { + for(uint8_t i = 0; i < this->roomCount; i++) + if(this->rooms[i].roomId == roomId) return &this->rooms[i]; + return nullptr; +} +const shade_snapshot_t *model_snapshot_t::shade(uint8_t shadeId) const { + for(uint8_t i = 0; i < this->shadeCount; i++) + if(this->shades[i].shadeId == shadeId) return &this->shades[i]; + return nullptr; +} +const group_snapshot_t *model_snapshot_t::group(uint8_t groupId) const { + for(uint8_t i = 0; i < this->groupCount; i++) + if(this->groups[i].groupId == groupId) return &this->groups[i]; + return nullptr; +} + +/********************************************************************* + * Items + ********************************************************************/ +void writeRoom(JsonFormatter &json, const room_snapshot_t &room) { + json.addElem("roomId", room.roomId); + json.addElem("name", room.name); + json.addElem("sortOrder", room.sortOrder); +} +void writeShadeRef(JsonFormatter &json, const shade_snapshot_t &shade) { + json.addElem("shadeId", shade.shadeId); + json.addElem("roomId", shade.roomId); + json.addElem("name", shade.name); + json.addElem("remoteAddress", shade.remoteAddress); + json.addElem("paired", shade.paired); + json.addElem("shadeType", shade.shadeType); + json.addElem("flipCommands", shade.flipCommands); + json.addElem("flipPosition", shade.flipPosition); + json.addElem("bitLength", shade.bitLength); + json.addElem("proto", shade.proto); + json.addElem("flags", shade.flags); + json.addElem("sunSensor", shade.sunSensor); + json.addElem("hasLight", shade.light); + json.addElem("repeats", shade.repeats); +} +void writeShade(JsonFormatter &json, const shade_snapshot_t &shade) { + json.addElem("shadeId", shade.shadeId); + json.addElem("roomId", shade.roomId); + json.addElem("name", shade.name); + json.addElem("remoteAddress", shade.remoteAddress); + json.addElem("upTime", shade.upTime); + json.addElem("downTime", shade.downTime); + json.addElem("paired", shade.paired); + json.addElem("lastRollingCode", static_cast(shade.lastRollingCode)); + json.addElem("position", shade.position); + json.addElem("tiltType", shade.tiltType); + json.addElem("tiltPosition", shade.tiltPosition); + json.addElem("tiltDirection", shade.tiltDirection); + json.addElem("tiltTime", shade.tiltTime); + json.addElem("stepSize", static_cast(shade.stepSize)); + json.addElem("tiltTarget", shade.tiltTarget); + json.addElem("target", shade.target); + json.addElem("myPos", shade.myPos); + json.addElem("myTiltPos", shade.myTiltPos); + json.addElem("direction", shade.direction); + json.addElem("shadeType", shade.shadeType); + json.addElem("bitLength", shade.bitLength); + json.addElem("proto", shade.proto); + json.addElem("flags", shade.flags); + json.addElem("flipCommands", shade.flipCommands); + json.addElem("flipPosition", shade.flipPosition); + json.addElem("inGroup", shade.inGroup); + json.addElem("sunSensor", shade.sunSensor); + json.addElem("light", shade.light); + json.addElem("repeats", shade.repeats); + json.addElem("sortOrder", shade.sortOrder); + json.addElem("gpioUp", shade.gpioUp); + json.addElem("gpioDown", shade.gpioDown); + json.addElem("gpioMy", shade.gpioMy); + json.addElem("gpioLLTrigger", shade.gpioLLTrigger); + json.addElem("simMy", shade.simMy); + json.beginArray("linkedRemotes"); + for(uint8_t i = 0; i < shade.linkedRemoteCount; i++) { + json.beginObject(); + json.addElem("remoteAddress", shade.linkedRemotes[i].remoteAddress); + json.addElem("lastRollingCode", static_cast(shade.linkedRemotes[i].lastRollingCode)); + json.endObject(); + } + json.endArray(); +} +void writeGroupRef(JsonFormatter &json, const group_snapshot_t &group) { + json.addElem("groupId", group.groupId); + json.addElem("roomId", group.roomId); + json.addElem("name", group.name); + json.addElem("remoteAddress", group.remoteAddress); + json.addElem("lastRollingCode", static_cast(group.lastRollingCode)); + json.addElem("bitLength", group.bitLength); + json.addElem("proto", group.proto); + json.addElem("sunSensor", group.sunSensor); + json.addElem("flipCommands", group.flipCommands); + json.addElem("flags", group.flags); + json.addElem("repeats", group.repeats); + json.addElem("sortOrder", group.sortOrder); +} +void writeGroup(JsonFormatter &json, const group_snapshot_t &group, shade_lookup_t lookup, void *ctx) { + writeGroupRef(json, group); + json.beginArray("linkedShades"); + for(uint8_t i = 0; i < group.linkedShadeCount; i++) { + const shade_snapshot_t *shade = lookup(group.linkedShades[i], ctx); + if(!shade) continue; + json.beginObject(); + writeShadeRef(json, *shade); + json.endObject(); + } + json.endArray(); +} + +/********************************************************************* + * Documents + ********************************************************************/ +enum : uint8_t { + ST_HEADER, // {
,"rooms":[ + ST_ROOMS, + ST_SHADES, + ST_GROUPS, + ST_REPEATERS, + ST_ROOM, + ST_SHADE, + ST_GROUP, + ST_OPEN_ARRAY, // [ + ST_CLOSE_ARRAY, // ] + ST_TO_SHADES, // ],"shades":[ + ST_TO_GROUPS, // ],"groups":[ + ST_TO_REPEATERS, // ],"repeaters":[ + ST_CLOSE_DOC, // ]} + ST_END +}; +static const uint8_t CONTROLLER_STEPS[] = { ST_HEADER, ST_ROOMS, ST_TO_SHADES, ST_SHADES, ST_TO_GROUPS, ST_GROUPS, ST_TO_REPEATERS, ST_REPEATERS, ST_CLOSE_DOC, ST_END }; +static const uint8_t DISCOVERY_STEPS[] = { ST_HEADER, ST_ROOMS, ST_TO_SHADES, ST_SHADES, ST_TO_GROUPS, ST_GROUPS, ST_CLOSE_DOC, ST_END }; +static const uint8_t ROOMS_STEPS[] = { ST_OPEN_ARRAY, ST_ROOMS, ST_CLOSE_ARRAY, ST_END }; +static const uint8_t SHADES_STEPS[] = { ST_OPEN_ARRAY, ST_SHADES, ST_CLOSE_ARRAY, ST_END }; +static const uint8_t GROUPS_STEPS[] = { ST_OPEN_ARRAY, ST_GROUPS, ST_CLOSE_ARRAY, ST_END }; +static const uint8_t ROOM_STEPS[] = { ST_ROOM, ST_END }; +static const uint8_t SHADE_STEPS[] = { ST_SHADE, ST_END }; +static const uint8_t GROUP_STEPS[] = { ST_GROUP, ST_END }; + +static const uint8_t *stepsFor(snapshot_doc_t doc) { + switch(doc) { + case snapshot_doc_t::controller: return CONTROLLER_STEPS; + case snapshot_doc_t::discovery: return DISCOVERY_STEPS; + case snapshot_doc_t::rooms: return ROOMS_STEPS; + case snapshot_doc_t::shades: return SHADES_STEPS; + case snapshot_doc_t::groups: return GROUPS_STEPS; + case snapshot_doc_t::room: return ROOM_STEPS; + case snapshot_doc_t::shade: return SHADE_STEPS; + case snapshot_doc_t::group: return GROUP_STEPS; + } + return SHADES_STEPS; +} +static const shade_snapshot_t *snapshotShade(uint8_t shadeId, void *ctx) { + return static_cast(ctx)->shade(shadeId); +} + +SnapshotWriter::SnapshotWriter(const model_snapshot_t &snap, snapshot_doc_t doc, uint8_t id, snapshot_header_t header, char *unit, size_t unitSize) + : snap(snap), steps(stepsFor(doc)), id(id), header(header), unit(unit), unitSize(unitSize) {} + +size_t SnapshotWriter::generate(uint8_t *out, size_t maxLen) { + size_t written = 0; + while(written < maxLen) { + if(this->consumed < this->unitLen) { + size_t n = this->unitLen - this->consumed; + if(n > maxLen - written) n = maxLen - written; + memcpy(out + written, this->unit + this->consumed, n); + this->consumed += n; + written += n; + } + else { + // A step or item can produce nothing, so go on until something is + // produced or the document is done. + this->consumed = 0; + do this->produceUnit(); while(this->unitLen == 0 && this->steps[this->step] != ST_END); + if(this->unitLen == 0) return written; + } + } + return written; +} +void SnapshotWriter::nextStep() { + this->step++; + this->idx = 0; + this->firstItem = true; + this->groupOpen = false; +} +void SnapshotWriter::literal(const char *text) { + strncpy(this->unit, text, this->unitSize - 1); + this->unit[this->unitSize - 1] = '\0'; + this->unitLen = strlen(this->unit); + this->nextStep(); +} +void SnapshotWriter::resetFmt(size_t offset) { + this->unit[offset] = '\0'; + this->fmt.setBuffer(this->unit + offset, this->unitSize - offset); +} +bool SnapshotWriter::overflowed(const char *what, uint8_t itemId) { + if(!this->fmt.overflowed()) return false; + ESP_LOGE(TAG, "%s %u doesn't fit in %u bytes, leaving it out", what, itemId, (unsigned)this->unitSize); + this->unitLen = 0; + return true; +} +// Starts a list item, after a comma unless it is the first. +size_t SnapshotWriter::itemStart() { + size_t pos = 0; + if(!this->firstItem) this->unit[pos++] = ','; + this->resetFmt(pos); + return pos; +} +// One piece of a group: its ref, then each linked shade, then the close. +// inList is whether it is an item of a list rather than the whole document. +void SnapshotWriter::groupUnit(const group_snapshot_t &group, bool inList) { + auto done = [this, inList]() { + this->groupOpen = false; + if(inList) this->idx++; + else this->nextStep(); + }; + if(!this->groupOpen) { + const size_t pos = inList ? this->itemStart() : 0; + if(!inList) this->resetFmt(0); + this->fmt.beginObject(); + writeGroupRef(this->fmt, group); + this->fmt.beginArray("linkedShades"); + if(this->overflowed("group", group.groupId)) { done(); return; } + this->unitLen = pos + this->fmt.length(); + this->groupOpen = true; + this->lsIdx = 0; + this->firstLinked = true; + if(inList) this->firstItem = false; + return; + } + const shade_snapshot_t *shade = nullptr; + while(this->lsIdx < group.linkedShadeCount && !(shade = this->snap.shade(group.linkedShades[this->lsIdx]))) this->lsIdx++; + if(!shade) { + strcpy(this->unit, "]}"); + this->unitLen = 2; + done(); + return; + } + size_t pos = 0; + if(!this->firstLinked) this->unit[pos++] = ','; + this->resetFmt(pos); + this->fmt.beginObject(); + writeShadeRef(this->fmt, *shade); + this->fmt.endObject(); + this->lsIdx++; + if(this->overflowed("linked shade", shade->shadeId)) return; + this->unitLen = pos + this->fmt.length(); + this->firstLinked = false; +} +void SnapshotWriter::produceUnit() { + this->unitLen = 0; + switch(this->steps[this->step]) { + case ST_HEADER: + this->resetFmt(0); + this->fmt.beginObject(); + if(this->header) this->header(this->fmt, this->snap); + this->fmt.beginArray("rooms"); + // Only fixed size values so this can't happen short of a bug. + if(this->fmt.overflowed()) ESP_LOGE(TAG, "Header doesn't fit in %u bytes", (unsigned)this->unitSize); + this->unitLen = this->fmt.length(); + this->nextStep(); + return; + case ST_ROOMS: { + if(this->idx >= this->snap.roomCount) { this->nextStep(); return; } + const room_snapshot_t &room = this->snap.rooms[this->idx++]; + const size_t pos = this->itemStart(); + this->fmt.beginObject(); + writeRoom(this->fmt, room); + this->fmt.endObject(); + if(this->overflowed("room", room.roomId)) return; + this->unitLen = pos + this->fmt.length(); + this->firstItem = false; + return; + } + case ST_SHADES: { + if(this->idx >= this->snap.shadeCount) { this->nextStep(); return; } + const shade_snapshot_t &shade = this->snap.shades[this->idx++]; + const size_t pos = this->itemStart(); + this->fmt.beginObject(); + writeShade(this->fmt, shade); + this->fmt.endObject(); + if(this->overflowed("shade", shade.shadeId)) return; + this->unitLen = pos + this->fmt.length(); + this->firstItem = false; + return; + } + case ST_GROUPS: + if(this->idx >= this->snap.groupCount) { this->nextStep(); return; } + this->groupUnit(this->snap.groups[this->idx], true); + return; + case ST_REPEATERS: { + if(this->idx >= this->snap.repeaterCount) { this->nextStep(); return; } + size_t pos = 0; + if(!this->firstItem) this->unit[pos++] = ','; + pos += snprintf(this->unit + pos, this->unitSize - pos, "%lu", (unsigned long)this->snap.repeaters[this->idx++]); + this->unitLen = pos; + this->firstItem = false; + return; + } + case ST_ROOM: { + const room_snapshot_t *room = this->snap.room(this->id); + if(!room) { this->literal("{\"status\":\"ERROR\",\"desc\":\"Room Id not found.\"}"); return; } + this->resetFmt(0); + this->fmt.beginObject(); + writeRoom(this->fmt, *room); + this->fmt.endObject(); + if(!this->overflowed("room", room->roomId)) this->unitLen = this->fmt.length(); + this->nextStep(); + return; + } + case ST_SHADE: { + const shade_snapshot_t *shade = this->snap.shade(this->id); + if(!shade) { this->literal("{\"status\":\"ERROR\",\"desc\":\"Shade Id not found.\"}"); return; } + this->resetFmt(0); + this->fmt.beginObject(); + writeShade(this->fmt, *shade); + this->fmt.endObject(); + if(!this->overflowed("shade", shade->shadeId)) this->unitLen = this->fmt.length(); + this->nextStep(); + return; + } + case ST_GROUP: { + const group_snapshot_t *group = this->snap.group(this->id); + if(!group) { this->literal("{\"status\":\"ERROR\",\"desc\":\"Group Id not found.\"}"); return; } + this->groupUnit(*group, false); + return; + } + case ST_OPEN_ARRAY: this->literal("["); return; + case ST_CLOSE_ARRAY: this->literal("]"); return; + case ST_TO_SHADES: this->literal("],\"shades\":["); return; + case ST_TO_GROUPS: this->literal("],\"groups\":["); return; + case ST_TO_REPEATERS: this->literal("],\"repeaters\":["); return; + case ST_CLOSE_DOC: this->literal("]}"); return; + default: + return; + } +} + +/********************************************************************* + * Slots + ********************************************************************/ +int8_t SnapshotSlots::acquire() { + for(uint8_t i = 0; i < SNAPSHOT_SLOTS; i++) { + uint32_t expected = FREE; + if(!this->state[i].compare_exchange_strong(expected, WAITING)) continue; + // Only the holder of a WAITING slot touches its pointer until it is queued. + this->snaps[i] = new (std::nothrow) model_snapshot_t; + if(!this->snaps[i]) { + this->state[i].store(FREE); + return -1; + } + return static_cast(i); + } + return -1; +} +// The snapshot goes before the slot is marked free, so acquire() never sees a +// free slot that still holds one. +void SnapshotSlots::freeSlot(uint8_t slot) { + delete this->snaps[slot]; + this->snaps[slot] = nullptr; + this->state[slot].store(FREE); +} +void SnapshotSlots::fill(uint8_t slot, void (*fill)(model_snapshot_t &snap)) { + uint32_t expected = WAITING; + if(!this->state[slot].compare_exchange_strong(expected, FILLING)) { + // The read went away before it was filled; nobody else will free it. + if(expected == ABANDONED) this->freeSlot(slot); + return; + } + model_snapshot_t &snap = *this->snaps[slot]; + snap.startingAddress = 0; + snap.roomCount = snap.shadeCount = snap.groupCount = snap.repeaterCount = 0; + fill(snap); + expected = FILLING; + // Released while being filled: the read is gone, so free it here. + if(!this->state[slot].compare_exchange_strong(expected, FILLED)) this->freeSlot(slot); +} +bool SnapshotSlots::filled(uint8_t slot) const { return this->state[slot].load() == FILLED; } +void SnapshotSlots::cancel(uint8_t slot) { this->freeSlot(slot); } +void SnapshotSlots::release(uint8_t slot) { + // Still waiting for or being filled by loop(): it frees the slot when it sees + // ABANDONED. Filled: nothing else touches it, so free it now. + if(this->state[slot].exchange(ABANDONED) == FILLED) this->freeSlot(slot); +} diff --git a/src/ModelSnapshot.h b/src/ModelSnapshot.h new file mode 100644 index 0000000..9c036dc --- /dev/null +++ b/src/ModelSnapshot.h @@ -0,0 +1,187 @@ +#ifndef MODEL_SNAPSHOT_H +#define MODEL_SNAPSHOT_H +#include +#include +#include +#include "ShadeTypes.h" +#include "JsonFormatter.h" + +// A copy of the shade model taken on loop() for one read, so the reply +// describes a single moment (docs/adr/0002-loop-task-is-the-only-writer-of-the-shade-model.md), +// and the JSON the read routes write from it. Kept free of the shade model and +// the web server so it can be unit tested on the PC (pio test -e native). + +// Positions are as clients see them: whole percentages, flipped for a shade +// with flipPosition, -1 when unset. +// No default member values: with them the SnapshotSlots would be stored in +// flash as initialized data (11 KB). The snapshot functions set every field; +// anything else starts from {}. +struct room_snapshot_t { + uint8_t roomId; + char name[21]; + int8_t sortOrder; +}; +struct linked_remote_snapshot_t { + uint32_t remoteAddress; + uint16_t lastRollingCode; +}; +struct shade_snapshot_t { + uint8_t shadeId; + uint8_t roomId; + char name[21]; + uint32_t remoteAddress; + uint32_t upTime; + uint32_t downTime; + uint32_t tiltTime; + uint16_t stepSize; + bool paired; + uint16_t lastRollingCode; + int8_t position; + int8_t target; + int8_t myPos; + int8_t direction; + uint8_t tiltType; + int8_t tiltPosition; + int8_t tiltTarget; + int8_t myTiltPos; + int8_t tiltDirection; + uint8_t shadeType; + uint8_t bitLength; + uint8_t proto; + uint8_t flags; + bool flipCommands; + bool flipPosition; + bool inGroup; + bool sunSensor; + bool light; + bool simMy; + uint8_t repeats; + int8_t sortOrder; + uint8_t gpioUp; + uint8_t gpioDown; + uint8_t gpioMy; + bool gpioLLTrigger; + uint8_t linkedRemoteCount; + linked_remote_snapshot_t linkedRemotes[SOMFY_MAX_LINKED_REMOTES]; +}; +struct group_snapshot_t { + uint8_t groupId; + uint8_t roomId; + char name[21]; + uint32_t remoteAddress; + uint16_t lastRollingCode; + uint8_t bitLength; + uint8_t proto; + bool sunSensor; + bool flipCommands; + uint8_t flags; // its linked shades' flags together + uint8_t repeats; + int8_t sortOrder; + uint8_t linkedShadeCount; + uint8_t linkedShades[SOMFY_MAX_GROUPED_SHADES]; +}; +// Only the rooms, shades, groups and repeaters that exist, in model order. +struct model_snapshot_t { + uint32_t startingAddress; + uint8_t roomCount; + uint8_t shadeCount; + uint8_t groupCount; + uint8_t repeaterCount; + room_snapshot_t rooms[SOMFY_MAX_ROOMS]; + shade_snapshot_t shades[SOMFY_MAX_SHADES]; + group_snapshot_t groups[SOMFY_MAX_GROUPS]; + uint32_t repeaters[SOMFY_MAX_REPEATERS]; + const room_snapshot_t *room(uint8_t roomId) const; + const shade_snapshot_t *shade(uint8_t shadeId) const; + const group_snapshot_t *group(uint8_t groupId) const; +}; + +// The fields of one item, as every route has always written them. +void writeRoom(JsonFormatter &json, const room_snapshot_t &room); +void writeShadeRef(JsonFormatter &json, const shade_snapshot_t &shade); +void writeShade(JsonFormatter &json, const shade_snapshot_t &shade); +void writeGroupRef(JsonFormatter &json, const group_snapshot_t &group); +// The shade a group links to, or nullptr when it has gone. +typedef const shade_snapshot_t *(*shade_lookup_t)(uint8_t shadeId, void *ctx); +// writeGroupRef plus linkedShades, the refs of the shades it links to. +void writeGroup(JsonFormatter &json, const group_snapshot_t &group, shade_lookup_t lookup, void *ctx); + +// The documents the read routes send. +enum class snapshot_doc_t : uint8_t { + controller, // /controller + discovery, // /discovery + rooms, // /rooms + shades, // /shades + groups, // /groups + room, // /room?roomId= + shade, // /shade?shadeId= + group // /group?groupId= +}; +// Writes the fields of /controller or /discovery that aren't in the snapshot +// (device, radio and firmware details) into the open top-level object. +typedef void (*snapshot_header_t)(JsonFormatter &json, const model_snapshot_t &snap); + +// Writes a document from a snapshot a piece at a time for a chunked response. +// Each room, shade, group and linked shade is formatted into unit on its own; +// one that doesn't fit is logged and left out so the JSON stays valid. +class SnapshotWriter { + public: + // id picks the item for the single-item documents. header is only called + // for controller and discovery. unit must outlive the writer. + SnapshotWriter(const model_snapshot_t &snap, snapshot_doc_t doc, uint8_t id, snapshot_header_t header, char *unit, size_t unitSize); + // Copies up to maxLen more bytes of the document into out. 0 once it is all + // out. + size_t generate(uint8_t *out, size_t maxLen); + private: + const model_snapshot_t &snap; + const uint8_t *steps; + uint8_t step = 0; + uint8_t id; + snapshot_header_t header; + BufferedJsonFormatter fmt; + char *unit; + size_t unitSize; + size_t unitLen = 0; + size_t consumed = 0; + uint8_t idx = 0; + uint8_t lsIdx = 0; // next linked shade of the group being written + bool groupOpen = false; // the group's ref is out, its linked shades are next + bool firstItem = true; + bool firstLinked = true; + void produceUnit(); + void nextStep(); + void literal(const char *text); + void resetFmt(size_t offset); + bool overflowed(const char *what, uint8_t itemId); + size_t itemStart(); + void groupUnit(const group_snapshot_t &group, bool comma); +}; + +// The snapshots reads are served from. A read takes a slot on the web server +// task and queues a fill; loop() fills it; the read streams it and lets it go. +// The read may go away before loop() gets to it, so the slot's state decides +// who frees it: whichever side is last. A slot's snapshot (about 5.6 KB) is +// only allocated while a read holds it. +#define SNAPSHOT_SLOTS 2 +class SnapshotSlots { + public: + // A free slot with its snapshot allocated, or -1 when every slot is in use + // or there is no memory. Any task. + int8_t acquire(); + // Fills the slot with fill, unless its read has gone. loop() only. + void fill(uint8_t slot, void (*fill)(model_snapshot_t &snap)); + // Whether loop() has filled the slot. The read's task. + bool filled(uint8_t slot) const; + // Valid from acquire() until the read releases the slot. + model_snapshot_t &snapshot(uint8_t slot) { return *this->snaps[slot]; } + // The read is done with the slot, or gone. The read's task. + void release(uint8_t slot); + // Frees a slot whose fill was never queued. The read's task. + void cancel(uint8_t slot); + private: + enum : uint32_t { FREE, WAITING, FILLING, FILLED, ABANDONED }; + model_snapshot_t *snaps[SNAPSHOT_SLOTS] = {}; + std::atomic state[SNAPSHOT_SLOTS] = {}; + void freeSlot(uint8_t slot); +}; +#endif diff --git a/src/ShadeTypes.h b/src/ShadeTypes.h index 60a8930..bdbe7aa 100644 --- a/src/ShadeTypes.h +++ b/src/ShadeTypes.h @@ -2,8 +2,16 @@ #define SHADE_TYPES_H #include -// The kinds of shade, tilt and group. Kept free of the shade model so the -// shade maths can use them in the unit tests on the PC (pio test -e native). +// The kinds of shade, tilt and group, and how many of each the shade model +// holds. Kept free of the shade model so the shade maths can use them in the +// unit tests on the PC (pio test -e native). + +#define SOMFY_MAX_SHADES 32 +#define SOMFY_MAX_GROUPS 16 +#define SOMFY_MAX_LINKED_REMOTES 7 +#define SOMFY_MAX_GROUPED_SHADES 32 +#define SOMFY_MAX_ROOMS 16 +#define SOMFY_MAX_REPEATERS 7 enum class group_types : byte { channel = 0x00 diff --git a/src/Somfy.h b/src/Somfy.h index ba513cc..c508600 100644 --- a/src/Somfy.h +++ b/src/Somfy.h @@ -4,14 +4,6 @@ #include "ConfigSettings.h" #include "WResp.h" -#define SOMFY_MAX_SHADES 32 -#define SOMFY_MAX_GROUPS 16 -#define SOMFY_MAX_LINKED_REMOTES 7 -#define SOMFY_MAX_GROUPED_SHADES 32 -#define SOMFY_MAX_ROOMS 16 -#define SOMFY_MAX_REPEATERS 7 - - #include "SomfyFrame.h" #include "TxBuffer.h" #include "RadioScheduler.h" diff --git a/src/SomfyController.ino b/src/SomfyController.ino index c1a5653..5cb9b40 100644 --- a/src/SomfyController.ino +++ b/src/SomfyController.ino @@ -12,6 +12,7 @@ #include "LoopQueue.h" #include "ShadeRequests.h" #include "SomfyShadeOps.h" +#include "ModelSnapshot.h" #include "GitOTA.h" #include "esp_core_dump.h" #include "esp_system.h" @@ -39,6 +40,7 @@ GitUpdater git; SomfyShadeOps shadeOps; ShadeRequests shadeRequests(shadeOps); LoopQueue loopQueue; +SnapshotSlots snapshotSlots; uint32_t oldheap = 0; diff --git a/src/SomfySnapshot.cpp b/src/SomfySnapshot.cpp new file mode 100644 index 0000000..3fc7167 --- /dev/null +++ b/src/SomfySnapshot.cpp @@ -0,0 +1,102 @@ +#include +#include "SomfySnapshot.h" + +extern SomfyShadeController somfy; + +static void copyName(char *out, const char *name) { + strncpy(out, name, 20); + out[20] = '\0'; +} + +void snapshotRoom(SomfyRoom &room, room_snapshot_t &out) { + out.roomId = room.roomId; + copyName(out.name, room.name); + out.sortOrder = room.sortOrder; +} +void snapshotShade(SomfyShade &shade, shade_snapshot_t &out) { + out.shadeId = shade.getShadeId(); + out.roomId = shade.roomId; + copyName(out.name, shade.name); + out.remoteAddress = shade.getRemoteAddress(); + out.upTime = shade.upTime; + out.downTime = shade.downTime; + out.tiltTime = shade.tiltTime; + out.stepSize = shade.stepSize; + out.paired = shade.paired; + out.lastRollingCode = shade.lastRollingCode; + out.position = shade.transformPosition(shade.currentPos); + out.target = shade.transformPosition(shade.target); + out.myPos = shade.transformPosition(shade.myPos); + out.direction = shade.direction; + out.tiltType = static_cast(shade.tiltType); + out.tiltPosition = shade.transformPosition(shade.currentTiltPos); + out.tiltTarget = shade.transformPosition(shade.tiltTarget); + out.myTiltPos = shade.transformPosition(shade.myTiltPos); + out.tiltDirection = shade.tiltDirection; + out.shadeType = static_cast(shade.shadeType); + out.bitLength = shade.bitLength; + out.proto = static_cast(shade.proto); + out.flags = shade.flags; + out.flipCommands = shade.flipCommands; + out.flipPosition = shade.flipPosition; + out.inGroup = shade.isInGroup(); + out.sunSensor = shade.hasSunSensor(); + out.light = shade.hasLight(); + out.simMy = shade.simMy(); + out.repeats = shade.repeats; + out.sortOrder = shade.sortOrder; + out.gpioUp = shade.gpioUp; + out.gpioDown = shade.gpioDown; + out.gpioMy = shade.gpioMy; + out.gpioLLTrigger = (shade.gpioFlags & static_cast(gpio_flags_t::LowLevelTrigger)) != 0; + out.linkedRemoteCount = 0; + for(uint8_t i = 0; i < SOMFY_MAX_LINKED_REMOTES; i++) { + SomfyLinkedRemote &remote = shade.linkedRemotes[i]; + if(remote.getRemoteAddress() == 0) continue; + linked_remote_snapshot_t &l = out.linkedRemotes[out.linkedRemoteCount++]; + l.remoteAddress = remote.getRemoteAddress(); + l.lastRollingCode = remote.lastRollingCode; + } +} +void snapshotGroup(SomfyGroup &group, group_snapshot_t &out) { + out.groupId = group.getGroupId(); + out.roomId = group.roomId; + copyName(out.name, group.name); + out.remoteAddress = group.getRemoteAddress(); + out.lastRollingCode = group.lastRollingCode; + out.bitLength = group.bitLength; + out.proto = static_cast(group.proto); + out.sunSensor = group.hasSunSensor(); + out.flipCommands = group.flipCommands; + out.repeats = group.repeats; + out.sortOrder = group.sortOrder; + // The same sum as SomfyGroup::updateFlags, without its event. + out.flags = 0; + for(uint8_t i = 0; i < SOMFY_MAX_GROUPED_SHADES && group.linkedShades[i] != 0; i++) { + SomfyShade *shade = somfy.getShadeById(group.linkedShades[i]); + if(shade) out.flags |= shade->flags; + } + out.linkedShadeCount = 0; + for(uint8_t i = 0; i < SOMFY_MAX_GROUPED_SHADES; i++) { + const uint8_t shadeId = group.linkedShades[i]; + if(shadeId > 0 && shadeId < 255) out.linkedShades[out.linkedShadeCount++] = shadeId; + } +} +void fillSnapshot(model_snapshot_t &snap) { + snap.startingAddress = somfy.startingAddress; + for(uint8_t i = 0; i < SOMFY_MAX_ROOMS; i++) + if(somfy.rooms[i].roomId != 0) snapshotRoom(somfy.rooms[i], snap.rooms[snap.roomCount++]); + for(uint8_t i = 0; i < SOMFY_MAX_SHADES; i++) + if(somfy.shades[i].getShadeId() != 255) snapshotShade(somfy.shades[i], snap.shades[snap.shadeCount++]); + for(uint8_t i = 0; i < SOMFY_MAX_GROUPS; i++) + if(somfy.groups[i].getGroupId() != 255) snapshotGroup(somfy.groups[i], snap.groups[snap.groupCount++]); + for(uint8_t i = 0; i < SOMFY_MAX_REPEATERS; i++) + if(somfy.repeaters[i] != 0) snap.repeaters[snap.repeaterCount++] = somfy.repeaters[i]; +} +const shade_snapshot_t *liveShade(uint8_t shadeId, void *ctx) { + SomfyShade *shade = somfy.getShadeById(shadeId); + if(!shade) return nullptr; + shade_snapshot_t *out = static_cast(ctx); + snapshotShade(*shade, *out); + return out; +} diff --git a/src/SomfySnapshot.h b/src/SomfySnapshot.h new file mode 100644 index 0000000..75feb46 --- /dev/null +++ b/src/SomfySnapshot.h @@ -0,0 +1,17 @@ +#ifndef SOMFY_SNAPSHOT_H +#define SOMFY_SNAPSHOT_H +#include "ModelSnapshot.h" +#include "Somfy.h" + +// Copies the shade model into the ModelSnapshot types. + +void snapshotRoom(SomfyRoom &room, room_snapshot_t &out); +void snapshotShade(SomfyShade &shade, shade_snapshot_t &out); +// Works the group's flags out from its shades without changing the group. +void snapshotGroup(SomfyGroup &group, group_snapshot_t &out); +// The whole shade model. loop() only. +void fillSnapshot(model_snapshot_t &snap); +// A shade_lookup_t over the live shade model for writeGroup. ctx is a +// shade_snapshot_t to copy into. +const shade_snapshot_t *liveShade(uint8_t shadeId, void *ctx); +#endif diff --git a/src/entryPoints/Web.cpp b/src/entryPoints/Web.cpp index d5c60b8..fa1c700 100644 --- a/src/entryPoints/Web.cpp +++ b/src/entryPoints/Web.cpp @@ -11,6 +11,8 @@ #include "Somfy.h" #include "WResp.h" #include "LoopQueue.h" +#include "ModelSnapshot.h" +#include "SomfySnapshot.h" #include "Web.h" #include "MQTT.h" #include "GitOTA.h" @@ -30,12 +32,11 @@ extern MQTTClass mqtt; extern GitUpdater git; extern ESPNetwork net; extern LoopQueue loopQueue; +extern SnapshotSlots snapshotSlots; //#define WEB_MAX_RESPONSE 34768 #define WEB_MAX_RESPONSE 4096 static char g_async_content[WEB_MAX_RESPONSE]; -// /discovery logs where its time went when it takes at least this long. -#define DISCOVERY_SLOW_MS 50 // General responses @@ -202,140 +203,42 @@ static bool asyncHasParam(AsyncWebServerRequest *request, const char *name) { return request->hasParam(name); } -// -- Serialization helpers (accept JsonFormatter& so both sync and async can use them) -- +// -- Serialization helpers -- +// Replies that still read the live shade model on this task (command and edit +// replies) copy one item into the ModelSnapshot types and write it the way the +// read routes do. The read routes themselves write from a ModelSnapshot that +// loop() fills (sendSnapshot). static void serializeRoom(SomfyRoom *room, JsonFormatter &json) { - json.addElem("roomId", room->roomId); - json.addElem("name", room->name); - json.addElem("sortOrder", room->sortOrder); + room_snapshot_t s; + snapshotRoom(*room, s); + writeRoom(json, s); } static void serializeShadeRef(SomfyShade *shade, JsonFormatter &json) { - json.addElem("shadeId", shade->getShadeId()); - json.addElem("roomId", shade->roomId); - json.addElem("name", shade->name); - json.addElem("remoteAddress", (uint32_t)shade->getRemoteAddress()); - json.addElem("paired", shade->paired); - json.addElem("shadeType", static_cast(shade->shadeType)); - json.addElem("flipCommands", shade->flipCommands); - json.addElem("flipPosition", shade->flipPosition); - json.addElem("bitLength", shade->bitLength); - json.addElem("proto", static_cast(shade->proto)); - json.addElem("flags", shade->flags); - json.addElem("sunSensor", shade->hasSunSensor()); - json.addElem("hasLight", shade->hasLight()); - json.addElem("repeats", shade->repeats); + shade_snapshot_t s; + snapshotShade(*shade, s); + writeShadeRef(json, s); } // paired and lastRollingCode are passed in for a response that reports the // change the main loop is about to make. An unpair is held until it goes on // air (holdLoopItem), so the answer comes true. static void serializeShade(SomfyShade *shade, JsonFormatter &json, bool paired, uint16_t lastRollingCode) { - json.addElem("shadeId", shade->getShadeId()); - json.addElem("roomId", shade->roomId); - json.addElem("name", shade->name); - json.addElem("remoteAddress", (uint32_t)shade->getRemoteAddress()); - json.addElem("upTime", (uint32_t)shade->upTime); - json.addElem("downTime", (uint32_t)shade->downTime); - json.addElem("paired", paired); - json.addElem("lastRollingCode", (uint32_t)lastRollingCode); - json.addElem("position", shade->transformPosition(shade->currentPos)); - json.addElem("tiltType", static_cast(shade->tiltType)); - json.addElem("tiltPosition", shade->transformPosition(shade->currentTiltPos)); - json.addElem("tiltDirection", shade->tiltDirection); - json.addElem("tiltTime", (uint32_t)shade->tiltTime); - json.addElem("stepSize", (uint32_t)shade->stepSize); - json.addElem("tiltTarget", shade->transformPosition(shade->tiltTarget)); - json.addElem("target", shade->transformPosition(shade->target)); - json.addElem("myPos", shade->transformPosition(shade->myPos)); - json.addElem("myTiltPos", shade->transformPosition(shade->myTiltPos)); - json.addElem("direction", shade->direction); - json.addElem("shadeType", static_cast(shade->shadeType)); - json.addElem("bitLength", shade->bitLength); - json.addElem("proto", static_cast(shade->proto)); - json.addElem("flags", shade->flags); - json.addElem("flipCommands", shade->flipCommands); - json.addElem("flipPosition", shade->flipPosition); - json.addElem("inGroup", shade->isInGroup()); - json.addElem("sunSensor", shade->hasSunSensor()); - json.addElem("light", shade->hasLight()); - json.addElem("repeats", shade->repeats); - json.addElem("sortOrder", shade->sortOrder); - json.addElem("gpioUp", shade->gpioUp); - json.addElem("gpioDown", shade->gpioDown); - json.addElem("gpioMy", shade->gpioMy); - json.addElem("gpioLLTrigger", ((shade->gpioFlags & (uint8_t)gpio_flags_t::LowLevelTrigger) == 0) ? false : true); - json.addElem("simMy", shade->simMy()); - json.beginArray("linkedRemotes"); - for(uint8_t i = 0; i < SOMFY_MAX_LINKED_REMOTES; i++) { - SomfyLinkedRemote &lremote = shade->linkedRemotes[i]; - if(lremote.getRemoteAddress() != 0) { - json.beginObject(); - json.addElem("remoteAddress", (uint32_t)lremote.getRemoteAddress()); - json.addElem("lastRollingCode", (uint32_t)lremote.lastRollingCode); - json.endObject(); - } - } - json.endArray(); + shade_snapshot_t s; + snapshotShade(*shade, s); + s.paired = paired; + s.lastRollingCode = lastRollingCode; + writeShade(json, s); } static void serializeShade(SomfyShade *shade, JsonFormatter &json) { serializeShade(shade, json, shade->paired, shade->lastRollingCode); } static void serializeGroupRef(SomfyGroup *group, JsonFormatter &json) { - group->updateFlags(); - json.addElem("groupId", group->getGroupId()); - json.addElem("roomId", group->roomId); - json.addElem("name", group->name); - json.addElem("remoteAddress", (uint32_t)group->getRemoteAddress()); - json.addElem("lastRollingCode", (uint32_t)group->lastRollingCode); - json.addElem("bitLength", group->bitLength); - json.addElem("proto", static_cast(group->proto)); - json.addElem("sunSensor", group->hasSunSensor()); - json.addElem("flipCommands", group->flipCommands); - json.addElem("flags", group->flags); - json.addElem("repeats", group->repeats); - json.addElem("sortOrder", group->sortOrder); + group_snapshot_t s; + snapshotGroup(*group, s); + writeGroupRef(json, s); } static void serializeGroup(SomfyGroup *group, JsonFormatter &json) { - serializeGroupRef(group, json); - json.beginArray("linkedShades"); - for(uint8_t i = 0; i < SOMFY_MAX_GROUPED_SHADES; i++) { - uint8_t shadeId = group->linkedShades[i]; - if(shadeId > 0 && shadeId < 255) { - SomfyShade *shade = somfy.getShadeById(shadeId); - if(shade) { - json.beginObject(); - serializeShadeRef(shade, json); - json.endObject(); - } - } - } - json.endArray(); -} -static void serializeRooms(JsonFormatter &json) { - for(uint8_t i = 0; i < SOMFY_MAX_ROOMS; i++) { - SomfyRoom *room = &somfy.rooms[i]; - if(room->roomId != 0) { - json.beginObject(); - serializeRoom(room, json); - json.endObject(); - } - } -} -static void serializeShades(JsonFormatter &json) { - for(uint8_t i = 0; i < SOMFY_MAX_SHADES; i++) { - SomfyShade &shade = somfy.shades[i]; - if(shade.getShadeId() != 255) { - json.beginObject(); - serializeShade(&shade, json); - json.endObject(); - } - } -} -static void serializeGroups(JsonFormatter &json) { - for(uint8_t i = 0; i < SOMFY_MAX_GROUPS; i++) { - SomfyGroup &group = somfy.groups[i]; - if(group.getGroupId() != 255) { - json.beginObject(); - serializeGroup(&group, json); - json.endObject(); - } - } + group_snapshot_t s; + snapshotGroup(*group, s); + shade_snapshot_t linked; + writeGroup(json, s, liveShade, &linked); } static void serializeRepeaters(JsonFormatter &json) { for(uint8_t i = 0; i < SOMFY_MAX_REPEATERS; i++) { @@ -401,318 +304,117 @@ static void serializeGitRelease(GitRelease *rel, JsonFormatter &json) { json.endObject(); } -// Memory-bounded streaming builder for /controller. Produces the JSON payload -// one small unit at a time into a fixed 3KB buffer, so a chunked HTTP response -// can drain it as the TCP send window allows — no growing cbuf in the async -// response stream. -class ControllerChunker { -public: - enum : uint8_t { S_HEADER, S_ROOMS, S_SHADES, S_GROUPS, S_REPEATERS, S_DONE }; - static constexpr uint16_t LS_OPEN = 0xFFFF; +// The parts of /controller that aren't in the shade model. +static void controllerHeader(JsonFormatter &json, const model_snapshot_t &snap) { + json.addElem("maxRooms", (uint8_t)SOMFY_MAX_ROOMS); + json.addElem("maxShades", (uint8_t)SOMFY_MAX_SHADES); + json.addElem("maxGroups", (uint8_t)SOMFY_MAX_GROUPS); + json.addElem("maxGroupedShades", (uint8_t)SOMFY_MAX_GROUPED_SHADES); + json.addElem("maxLinkedRemotes", (uint8_t)SOMFY_MAX_LINKED_REMOTES); + json.addElem("startingAddress", snap.startingAddress); + json.beginObject("transceiver"); + json.beginObject("config"); + serializeTransceiverConfig(json); + json.endObject(); + json.endObject(); + json.beginObject("version"); + serializeGitVersion(json); + json.endObject(); +} +// The parts of /discovery that aren't in the shade model. +static void discoveryHeader(JsonFormatter &json, const model_snapshot_t &snap) { + (void)snap; + const char *connType = "Unknown"; + if(net.connType == conn_types_t::ethernet) connType = "Ethernet"; + else if(net.connType == conn_types_t::wifi) connType = "Wifi"; + json.addElem("serverId", settings.serverId); + json.addElem("version", settings.fwVersion.name); + json.addElem("latest", git.latest.name); + json.addElem("model", "ESPSomfyRTS"); + json.addElem("hostname", settings.hostname); + json.addElem("authType", static_cast(settings.Security.type)); + json.addElem("permissions", settings.Security.permissions); + json.addElem("chipModel", settings.chipModel); + json.addElem("connType", connType); + json.addElem("checkForUpdate", settings.checkForUpdate); + json.beginObject("memory"); + json.addElem("max", ESP.getMaxAllocHeap()); + json.addElem("free", ESP.getFreeHeap()); + json.addElem("min", ESP.getMinFreeHeap()); + json.addElem("total", ESP.getHeapSize()); + json.addElem("uptime", (uint64_t)millis()); + json.endObject(); +} - BufferedJsonFormatter fmt; +// A read served from a ModelSnapshot that loop() fills, so the document +// describes one moment (docs/adr/0002-loop-task-is-the-only-writer-of-the-shade-model.md). +// Each item is formatted into a fixed 3 KB unit and streamed as the TCP send +// window allows, so there is no growing response buffer either. +struct SnapshotRead { + uint8_t slot; char unit[3072]; - size_t unitLen = 0; - size_t consumed = 0; - uint8_t section = S_HEADER; - uint16_t idx = 0; - uint16_t lsIdx = LS_OPEN; - bool firstInArray = true; - bool firstInLSArray = true; - - size_t generate(uint8_t *out, size_t maxLen) { - size_t written = 0; - while(written < maxLen) { - if(consumed < unitLen) { - size_t n = unitLen - consumed; - if(n > maxLen - written) n = maxLen - written; - memcpy(out + written, unit + consumed, n); - consumed += n; - written += n; - } else { - produceNext(); - if(unitLen == 0) return written; - } - } - return written; - } - -private: - void resetFmt(size_t offset = 0) { - unit[offset] = 0; - fmt.setBuffer(unit + offset, sizeof(unit) - offset); - } - - // An item too big for the unit buffer is left out so the JSON stays valid. - // Skipping one produces nothing, so go on to the next until something is - // produced or the document is done. - void produceNext() { - consumed = 0; - do produceUnit(); while(unitLen == 0 && section != S_DONE); - } - - // True when the unit just formatted didn't fit. It is logged and skipped. - bool overflowed(const char *what, uint8_t id) { - if(!fmt.overflowed()) return false; - ESP_LOGE(TAG, "/controller: %s %u doesn't fit in %u bytes, leaving it out", what, id, (unsigned)sizeof(unit)); - unitLen = 0; - return true; - } - - void produceUnit() { - unitLen = 0; - switch(section) { - case S_HEADER: { - resetFmt(); - fmt.beginObject(); - fmt.addElem("maxRooms", (uint8_t)SOMFY_MAX_ROOMS); - fmt.addElem("maxShades", (uint8_t)SOMFY_MAX_SHADES); - fmt.addElem("maxGroups", (uint8_t)SOMFY_MAX_GROUPS); - fmt.addElem("maxGroupedShades", (uint8_t)SOMFY_MAX_GROUPED_SHADES); - fmt.addElem("maxLinkedRemotes", (uint8_t)SOMFY_MAX_LINKED_REMOTES); - fmt.addElem("startingAddress", (uint32_t)somfy.startingAddress); - fmt.beginObject("transceiver"); - fmt.beginObject("config"); - serializeTransceiverConfig(fmt); - fmt.endObject(); - fmt.endObject(); - fmt.beginObject("version"); - serializeGitVersion(fmt); - fmt.endObject(); - fmt.beginArray("rooms"); - // Only fixed size values so this can't happen short of a bug. - if(fmt.overflowed()) ESP_LOGE(TAG, "/controller: header doesn't fit in %u bytes", (unsigned)sizeof(unit)); - unitLen = fmt.length(); - section = S_ROOMS; - idx = 0; - firstInArray = true; - return; - } - case S_ROOMS: { - while(idx < SOMFY_MAX_ROOMS && somfy.rooms[idx].roomId == 0) idx++; - if(idx >= SOMFY_MAX_ROOMS) { - strcpy(unit, "],\"shades\":["); - unitLen = strlen(unit); - section = S_SHADES; - idx = 0; - firstInArray = true; - return; - } - size_t pos = 0; - if(!firstInArray) unit[pos++] = ','; - resetFmt(pos); - fmt.beginObject(); - serializeRoom(&somfy.rooms[idx], fmt); - fmt.endObject(); - if(overflowed("room", somfy.rooms[idx].roomId)) { idx++; return; } - unitLen = pos + fmt.length(); - idx++; - firstInArray = false; - return; - } - case S_SHADES: { - while(idx < SOMFY_MAX_SHADES && somfy.shades[idx].getShadeId() == 255) idx++; - if(idx >= SOMFY_MAX_SHADES) { - strcpy(unit, "],\"groups\":["); - unitLen = strlen(unit); - section = S_GROUPS; - idx = 0; - lsIdx = LS_OPEN; - firstInArray = true; - return; - } - size_t pos = 0; - if(!firstInArray) unit[pos++] = ','; - resetFmt(pos); - fmt.beginObject(); - serializeShade(&somfy.shades[idx], fmt); - fmt.endObject(); - if(overflowed("shade", somfy.shades[idx].getShadeId())) { idx++; return; } - unitLen = pos + fmt.length(); - idx++; - firstInArray = false; - return; - } - case S_GROUPS: { - while(idx < SOMFY_MAX_GROUPS && somfy.groups[idx].getGroupId() == 255) idx++; - if(idx >= SOMFY_MAX_GROUPS) { - strcpy(unit, "],\"repeaters\":["); - unitLen = strlen(unit); - section = S_REPEATERS; - idx = 0; - firstInArray = true; - return; - } - SomfyGroup *g = &somfy.groups[idx]; - if(lsIdx == LS_OPEN) { - size_t pos = 0; - if(!firstInArray) unit[pos++] = ','; - resetFmt(pos); - fmt.beginObject(); - serializeGroupRef(g, fmt); - fmt.beginArray("linkedShades"); - if(overflowed("group", g->getGroupId())) { idx++; return; } - unitLen = pos + fmt.length(); - lsIdx = 0; - firstInArray = false; - firstInLSArray = true; - return; - } - SomfyShade *lshade = nullptr; - while(lsIdx < SOMFY_MAX_GROUPED_SHADES) { - uint8_t sid = g->linkedShades[lsIdx]; - if(sid > 0 && sid < 255) { - lshade = somfy.getShadeById(sid); - if(lshade) break; - } - lsIdx++; - } - if(!lshade) { - strcpy(unit, "]}"); - unitLen = 2; - idx++; - lsIdx = LS_OPEN; - return; - } - size_t pos = 0; - if(!firstInLSArray) unit[pos++] = ','; - resetFmt(pos); - fmt.beginObject(); - serializeShadeRef(lshade, fmt); - fmt.endObject(); - if(overflowed("linked shade", lshade->getShadeId())) { lsIdx++; return; } - unitLen = pos + fmt.length(); - lsIdx++; - firstInLSArray = false; - return; - } - case S_REPEATERS: { - while(idx < SOMFY_MAX_REPEATERS && somfy.repeaters[idx] == 0) idx++; - if(idx >= SOMFY_MAX_REPEATERS) { - strcpy(unit, "]}"); - unitLen = 2; - section = S_DONE; - return; - } - size_t pos = 0; - if(!firstInArray) unit[pos++] = ','; - pos += snprintf(unit + pos, sizeof(unit) - pos, "%lu", (unsigned long)somfy.repeaters[idx]); - unitLen = pos; - idx++; - firstInArray = false; - return; - } - case S_DONE: - default: - unitLen = 0; - return; - } - } + SnapshotWriter writer; + SnapshotRead(uint8_t slot, snapshot_doc_t doc, uint8_t id, snapshot_header_t header) + : slot(slot), writer(snapshotSlots.snapshot(slot), doc, id, header, unit, sizeof(unit)) {} + // The response is gone, whether it finished or the client went away. + ~SnapshotRead() { snapshotSlots.release(this->slot); } }; +static void sendSnapshot(AsyncWebServerRequest *request, snapshot_doc_t doc, uint8_t id = 0, snapshot_header_t header = nullptr) { + const int8_t slot = snapshotSlots.acquire(); + if(slot < 0) { + request->send(503, _encoding_json, F("{\"status\":\"ERROR\",\"desc\":\"Busy, try the read again\"}")); + return; + } + loop_item_t item; + item.type = loop_item_type_t::snapshot_fill; + item.slot = static_cast(slot); + if(!loopQueue.submit(item)) { + snapshotSlots.cancel(slot); + sendQueueFull(request); + return; + } + auto read = std::make_shared(static_cast(slot), doc, id, header); + // The headers go now; the body waits until loop() has filled the snapshot. + // The library asks again on the next ACK or poll. + AsyncWebServerResponse *response = request->beginChunkedResponse(_encoding_json, + [read](uint8_t *buffer, size_t maxLen, size_t index) -> size_t { + (void)index; + if(!snapshotSlots.filled(read->slot)) return RESPONSE_TRY_AGAIN; + return read->writer.generate(buffer, maxLen); + }); + request->send(response); +} // -- Async handler implementations -- void Web::handleDiscovery(AsyncWebServerRequest *request) { if(request->method() == HTTP_POST || request->method() == HTTP_GET) { ESP_LOGD(TAG, "Async Discovery Requested"); - char connType[10] = "Unknown"; - if(net.connType == conn_types_t::ethernet) strcpy(connType, "Ethernet"); - else if(net.connType == conn_types_t::wifi) strcpy(connType, "Wifi"); - // This handler runs on the async_tcp task, which also serves the sockets, - // so a slow one is logged with where its time went. - ElapsedTimer total, step; - uint32_t usHead, usMem, usRooms, usShades, usGroups, usSend; - AsyncJsonResp resp; - resp.beginResponse(request, g_async_content, sizeof(g_async_content)); - resp.beginObject(); - resp.addElem("serverId", settings.serverId); - resp.addElem("version", settings.fwVersion.name); - resp.addElem("latest", git.latest.name); - resp.addElem("model", "ESPSomfyRTS"); - resp.addElem("hostname", settings.hostname); - resp.addElem("authType", static_cast(settings.Security.type)); - resp.addElem("permissions", settings.Security.permissions); - resp.addElem("chipModel", settings.chipModel); - resp.addElem("connType", connType); - resp.addElem("checkForUpdate", settings.checkForUpdate); - usHead = step.lap(); - resp.beginObject("memory"); - resp.addElem("max", ESP.getMaxAllocHeap()); - resp.addElem("free", ESP.getFreeHeap()); - resp.addElem("min", ESP.getMinFreeHeap()); - resp.addElem("total", ESP.getHeapSize()); - resp.addElem("uptime", (uint64_t)millis()); - resp.endObject(); - usMem = step.lap(); - resp.beginArray("rooms"); - serializeRooms(resp); - resp.endArray(); - usRooms = step.lap(); - resp.beginArray("shades"); - serializeShades(resp); - resp.endArray(); - usShades = step.lap(); - resp.beginArray("groups"); - serializeGroups(resp); - resp.endArray(); - resp.endObject(); - usGroups = step.lap(); - resp.endResponse(); - usSend = step.us(); + sendSnapshot(request, snapshot_doc_t::discovery, 0, discoveryHeader); net.needsBroadcast = true; - if(total.us() >= DISCOVERY_SLOW_MS * 1000) { - ESP_LOGW(TAG, "discovery: head %.1f mem %.1f rooms %.1f shades %.1f groups %.1f send %.1f ms", - usHead / 1000.0f, usMem / 1000.0f, usRooms / 1000.0f, usShades / 1000.0f, usGroups / 1000.0f, usSend / 1000.0f); - } } else request->send(500, _encoding_text, "Invalid http method"); } void Web::handleGetRooms(AsyncWebServerRequest *request) { if(!this->isAuthenticated(request)) return; - if(request->method() == HTTP_POST || request->method() == HTTP_GET) { - AsyncJsonResp resp; - resp.beginResponse(request, g_async_content, sizeof(g_async_content)); - resp.beginArray(); - serializeRooms(resp); - resp.endArray(); - resp.endResponse(); - } + if(request->method() == HTTP_POST || request->method() == HTTP_GET) sendSnapshot(request, snapshot_doc_t::rooms); else request->send(404, _encoding_text, _response_404); } void Web::handleGetShades(AsyncWebServerRequest *request) { if(!this->isAuthenticated(request)) return; - if(request->method() == HTTP_POST || request->method() == HTTP_GET) { - AsyncJsonResp resp; - resp.beginResponse(request, g_async_content, sizeof(g_async_content)); - resp.beginArray(); - serializeShades(resp); - resp.endArray(); - resp.endResponse(); - } + if(request->method() == HTTP_POST || request->method() == HTTP_GET) sendSnapshot(request, snapshot_doc_t::shades); else request->send(404, _encoding_text, _response_404); } void Web::handleGetGroups(AsyncWebServerRequest *request) { if(!this->isAuthenticated(request)) return; - if(request->method() == HTTP_POST || request->method() == HTTP_GET) { - AsyncJsonResp resp; - resp.beginResponse(request, g_async_content, sizeof(g_async_content)); - resp.beginArray(); - serializeGroups(resp); - resp.endArray(); - resp.endResponse(); - } + if(request->method() == HTTP_POST || request->method() == HTTP_GET) sendSnapshot(request, snapshot_doc_t::groups); else request->send(404, _encoding_text, _response_404); } void Web::handleController(AsyncWebServerRequest *request) { if(!this->isAuthenticated(request)) return; if(request->method() == HTTP_POST || request->method() == HTTP_GET) { settings.printAvailHeap(); - auto state = std::make_shared(); - AsyncWebServerResponse *response = request->beginChunkedResponse(_encoding_json, - [state](uint8_t *buffer, size_t maxLen, size_t index) -> size_t { - return state->generate(buffer, maxLen); - }); - request->send(response); + sendSnapshot(request, snapshot_doc_t::controller, 0, controllerHeader); } else request->send(404, _encoding_text, _response_404); } @@ -923,15 +625,9 @@ void Web::handleRoom(AsyncWebServerRequest *request) { if(request->method() == HTTP_GET) { if(asyncHasParam(request, "roomId")) { int roomId = asyncParam(request, "roomId").toInt(); - SomfyRoom *room = somfy.getRoomById(roomId); - if(room) { - AsyncJsonResp resp; - resp.beginResponse(request, g_async_content, sizeof(g_async_content)); - resp.beginObject(); - serializeRoom(room, resp); - resp.endObject(); - resp.endResponse(); - } + // The status goes out before the snapshot is filled, so whether the room + // exists is checked here, on the live model. + if(somfy.getRoomById(roomId)) sendSnapshot(request, snapshot_doc_t::room, roomId); else request->send(500, _encoding_json, F("{\"status\":\"ERROR\",\"desc\":\"Room Id not found.\"}")); } else request->send(500, _encoding_json, F("{\"status\":\"ERROR\",\"desc\":\"You must supply a valid room id.\"}")); @@ -944,15 +640,8 @@ void Web::handleShade(AsyncWebServerRequest *request) { if(request->method() == HTTP_GET) { if(asyncHasParam(request, "shadeId")) { int shadeId = asyncParam(request, "shadeId").toInt(); - SomfyShade *shade = somfy.getShadeById(shadeId); - if(shade) { - AsyncJsonResp resp; - resp.beginResponse(request, g_async_content, sizeof(g_async_content)); - resp.beginObject(); - serializeShade(shade, resp); - resp.endObject(); - resp.endResponse(); - } + // As for /room, existence is checked on the live model. + if(somfy.getShadeById(shadeId)) sendSnapshot(request, snapshot_doc_t::shade, shadeId); else request->send(500, _encoding_json, F("{\"status\":\"ERROR\",\"desc\":\"Shade Id not found.\"}")); } else request->send(500, _encoding_json, F("{\"status\":\"ERROR\",\"desc\":\"You must supply a valid shade id.\"}")); @@ -965,15 +654,8 @@ void Web::handleGroup(AsyncWebServerRequest *request) { if(request->method() == HTTP_GET) { if(asyncHasParam(request, "groupId")) { int groupId = asyncParam(request, "groupId").toInt(); - SomfyGroup *group = somfy.getGroupById(groupId); - if(group) { - AsyncJsonResp resp; - resp.beginResponse(request, g_async_content, sizeof(g_async_content)); - resp.beginObject(); - serializeGroup(group, resp); - resp.endObject(); - resp.endResponse(); - } + // As for /room, existence is checked on the live model. + if(somfy.getGroupById(groupId)) sendSnapshot(request, snapshot_doc_t::group, groupId); else request->send(500, _encoding_json, F("{\"status\":\"ERROR\",\"desc\":\"Group Id not found.\"}")); } else request->send(500, _encoding_json, F("{\"status\":\"ERROR\",\"desc\":\"You must supply a valid group id.\"}")); diff --git a/test/test_snapshot/test_snapshot.cpp b/test/test_snapshot/test_snapshot.cpp new file mode 100644 index 0000000..54ee004 --- /dev/null +++ b/test/test_snapshot/test_snapshot.cpp @@ -0,0 +1,296 @@ +#include +#include +#include +#include "ModelSnapshot.h" + +static model_snapshot_t snap; +static char unitBuf[3072]; + +static room_snapshot_t room(uint8_t id, const char *name) { + room_snapshot_t r{}; + r.roomId = id; + strcpy(r.name, name); + return r; +} +static shade_snapshot_t shade(uint8_t id, const char *name) { + shade_snapshot_t s{}; + s.shadeId = id; + s.roomId = 2; + strcpy(s.name, name); + s.remoteAddress = 1000 + id; + s.upTime = 10000; + s.downTime = 11000; + s.tiltTime = 7000; + s.stepSize = 100; + s.paired = true; + s.lastRollingCode = 42; + s.position = 30; + s.target = 30; + s.myPos = -1; + s.tiltType = 2; + s.tiltPosition = 40; + s.tiltTarget = 40; + s.myTiltPos = -1; + s.shadeType = 1; + s.bitLength = 56; + s.inGroup = true; + s.repeats = 1; + return s; +} +static group_snapshot_t group(uint8_t id, const char *name) { + group_snapshot_t g{}; + g.groupId = id; + strcpy(g.name, name); + g.remoteAddress = 2000 + id; + g.lastRollingCode = 5; + g.bitLength = 80; + g.repeats = 2; + return g; +} +static std::string json(void (*write)(JsonFormatter &)) { + static char buf[2048]; + BufferedJsonFormatter fmt; + fmt.setBuffer(buf, sizeof(buf)); + fmt.beginObject(); + write(fmt); + fmt.endObject(); + return std::string(buf, fmt.length()); +} +static std::string shadeRefJson(uint8_t id) { + static uint8_t which; + which = id; + return json([](JsonFormatter &f) { writeShadeRef(f, *snap.shade(which)); }); +} +static std::string shadeJson(uint8_t id) { + static uint8_t which; + which = id; + return json([](JsonFormatter &f) { writeShade(f, *snap.shade(which)); }); +} +static std::string roomJson(uint8_t id) { + static uint8_t which; + which = id; + return json([](JsonFormatter &f) { writeRoom(f, *snap.room(which)); }); +} +static std::string groupRefJson(uint8_t id) { + static uint8_t which; + which = id; + return json([](JsonFormatter &f) { writeGroupRef(f, *snap.group(which)); }); +} +static void testHeader(JsonFormatter &json, const model_snapshot_t &s) { + json.addElem("startingAddress", s.startingAddress); +} +// The whole document, taken chunkSize bytes at a time like the web server does. +static std::string document(snapshot_doc_t doc, uint8_t id = 0, size_t chunkSize = 7, size_t unitSize = sizeof(unitBuf)) { + SnapshotWriter writer(snap, doc, id, testHeader, unitBuf, unitSize); + std::string out; + uint8_t chunk[64]; + for(int guard = 0; guard < 100000; guard++) { + const size_t n = writer.generate(chunk, chunkSize); + if(n == 0) break; + out.append(reinterpret_cast(chunk), n); + } + return out; +} + +void setUp() { + snap = model_snapshot_t{}; + snap.startingAddress = 77; + snap.rooms[snap.roomCount++] = room(2, "Living"); + snap.rooms[snap.roomCount++] = room(3, "Bed"); + snap.shades[snap.shadeCount++] = shade(1, "Kitchen"); + snap.shades[snap.shadeCount++] = shade(4, "Porch"); + group_snapshot_t g = group(6, "All"); + g.linkedShades[g.linkedShadeCount++] = 4; + g.linkedShades[g.linkedShadeCount++] = 9; // gone from the model + g.linkedShades[g.linkedShadeCount++] = 1; + snap.groups[snap.groupCount++] = g; + snap.repeaters[snap.repeaterCount++] = 12345; + snap.repeaters[snap.repeaterCount++] = 678; +} +void tearDown() {} + +/********************************************************************* + * Items, as the routes have always written them + ********************************************************************/ +static void test_a_shade() { + snap.shades[0].linkedRemotes[snap.shades[0].linkedRemoteCount++] = { 5555, 7 }; + TEST_ASSERT_EQUAL_STRING( + "{\"shadeId\":1,\"roomId\":2,\"name\":\"Kitchen\",\"remoteAddress\":1001,\"upTime\":10000,\"downTime\":11000," + "\"paired\":true,\"lastRollingCode\":42,\"position\":30,\"tiltType\":2,\"tiltPosition\":40,\"tiltDirection\":0," + "\"tiltTime\":7000,\"stepSize\":100,\"tiltTarget\":40,\"target\":30,\"myPos\":-1,\"myTiltPos\":-1,\"direction\":0," + "\"shadeType\":1,\"bitLength\":56,\"proto\":0,\"flags\":0,\"flipCommands\":false,\"flipPosition\":false," + "\"inGroup\":true,\"sunSensor\":false,\"light\":false,\"repeats\":1,\"sortOrder\":0,\"gpioUp\":0,\"gpioDown\":0," + "\"gpioMy\":0,\"gpioLLTrigger\":false,\"simMy\":false,\"linkedRemotes\":[{\"remoteAddress\":5555,\"lastRollingCode\":7}]}", + shadeJson(1).c_str()); +} +static void test_a_shade_ref() { + TEST_ASSERT_EQUAL_STRING( + "{\"shadeId\":1,\"roomId\":2,\"name\":\"Kitchen\",\"remoteAddress\":1001,\"paired\":true,\"shadeType\":1," + "\"flipCommands\":false,\"flipPosition\":false,\"bitLength\":56,\"proto\":0,\"flags\":0,\"sunSensor\":false," + "\"hasLight\":false,\"repeats\":1}", + shadeRefJson(1).c_str()); +} +static void test_a_room() { + TEST_ASSERT_EQUAL_STRING("{\"roomId\":2,\"name\":\"Living\",\"sortOrder\":0}", roomJson(2).c_str()); +} +static void test_a_group_ref() { + TEST_ASSERT_EQUAL_STRING( + "{\"groupId\":6,\"roomId\":0,\"name\":\"All\",\"remoteAddress\":2006,\"lastRollingCode\":5,\"bitLength\":80," + "\"proto\":0,\"sunSensor\":false,\"flipCommands\":false,\"flags\":0,\"repeats\":2,\"sortOrder\":0}", + groupRefJson(6).c_str()); +} +static void test_names_are_escaped() { + strcpy(snap.rooms[0].name, "A \"big\" room"); + TEST_ASSERT_EQUAL_STRING("{\"roomId\":2,\"name\":\"A \\\"big\\\" room\",\"sortOrder\":0}", roomJson(2).c_str()); +} + +/********************************************************************* + * Documents + ********************************************************************/ +static std::string groupWithLinked() { + std::string g = groupRefJson(6); + g.pop_back(); // reopen it for linkedShades + return g + ",\"linkedShades\":[" + shadeRefJson(4) + "," + shadeRefJson(1) + "]}"; +} +static void test_rooms_list() { + TEST_ASSERT_EQUAL_STRING(("[" + roomJson(2) + "," + roomJson(3) + "]").c_str(), document(snapshot_doc_t::rooms).c_str()); +} +static void test_shades_list() { + TEST_ASSERT_EQUAL_STRING(("[" + shadeJson(1) + "," + shadeJson(4) + "]").c_str(), document(snapshot_doc_t::shades).c_str()); +} +static void test_groups_list_skips_a_linked_shade_that_is_gone() { + TEST_ASSERT_EQUAL_STRING(("[" + groupWithLinked() + "]").c_str(), document(snapshot_doc_t::groups).c_str()); +} +static void test_empty_lists() { + snap = model_snapshot_t{}; + TEST_ASSERT_EQUAL_STRING("[]", document(snapshot_doc_t::rooms).c_str()); + TEST_ASSERT_EQUAL_STRING("[]", document(snapshot_doc_t::shades).c_str()); + TEST_ASSERT_EQUAL_STRING("[]", document(snapshot_doc_t::groups).c_str()); +} +static void test_controller() { + const std::string expected = "{\"startingAddress\":77,\"rooms\":[" + roomJson(2) + "," + roomJson(3) + + "],\"shades\":[" + shadeJson(1) + "," + shadeJson(4) + + "],\"groups\":[" + groupWithLinked() + + "],\"repeaters\":[12345,678]}"; + TEST_ASSERT_EQUAL_STRING(expected.c_str(), document(snapshot_doc_t::controller).c_str()); +} +static void test_discovery_has_no_repeaters() { + const std::string expected = "{\"startingAddress\":77,\"rooms\":[" + roomJson(2) + "," + roomJson(3) + + "],\"shades\":[" + shadeJson(1) + "," + shadeJson(4) + + "],\"groups\":[" + groupWithLinked() + "]}"; + TEST_ASSERT_EQUAL_STRING(expected.c_str(), document(snapshot_doc_t::discovery).c_str()); +} +static void test_controller_of_an_empty_model() { + snap = model_snapshot_t{}; + TEST_ASSERT_EQUAL_STRING("{\"startingAddress\":0,\"rooms\":[],\"shades\":[],\"groups\":[],\"repeaters\":[]}", + document(snapshot_doc_t::controller).c_str()); +} +static void test_chunk_size_does_not_change_the_document() { + const std::string whole = document(snapshot_doc_t::controller, 0, 64); + for(size_t n = 1; n < 20; n++) + TEST_ASSERT_EQUAL_STRING(whole.c_str(), document(snapshot_doc_t::controller, 0, n).c_str()); +} +static void test_single_items() { + TEST_ASSERT_EQUAL_STRING(roomJson(3).c_str(), document(snapshot_doc_t::room, 3).c_str()); + TEST_ASSERT_EQUAL_STRING(shadeJson(4).c_str(), document(snapshot_doc_t::shade, 4).c_str()); + TEST_ASSERT_EQUAL_STRING(groupWithLinked().c_str(), document(snapshot_doc_t::group, 6).c_str()); +} +static void test_a_single_item_gone_by_the_snapshot_is_an_error() { + TEST_ASSERT_EQUAL_STRING("{\"status\":\"ERROR\",\"desc\":\"Shade Id not found.\"}", document(snapshot_doc_t::shade, 9).c_str()); + TEST_ASSERT_EQUAL_STRING("{\"status\":\"ERROR\",\"desc\":\"Group Id not found.\"}", document(snapshot_doc_t::group, 9).c_str()); + TEST_ASSERT_EQUAL_STRING("{\"status\":\"ERROR\",\"desc\":\"Room Id not found.\"}", document(snapshot_doc_t::room, 9).c_str()); +} +static void test_an_item_too_big_for_the_unit_is_left_out() { + // Room JSON is about 40 bytes and a shade's well over 300, so a 200 byte + // unit takes the rooms and leaves the shades out, keeping the JSON valid. + TEST_ASSERT_EQUAL_STRING(("[" + roomJson(2) + "," + roomJson(3) + "]").c_str(), + document(snapshot_doc_t::rooms, 0, 7, 200).c_str()); + TEST_ASSERT_EQUAL_STRING("[]", document(snapshot_doc_t::shades, 0, 7, 200).c_str()); +} +static void test_a_left_out_item_does_not_leave_a_stray_comma() { + // Give the first shade every linked remote so it no longer fits a unit the + // second one does. + snap.shades[0].linkedRemoteCount = SOMFY_MAX_LINKED_REMOTES; + for(uint8_t i = 0; i < SOMFY_MAX_LINKED_REMOTES; i++) snap.shades[0].linkedRemotes[i] = { 4000000000u, 65535 }; + const size_t fits = shadeJson(4).length() + 8; + TEST_ASSERT_TRUE(shadeJson(1).length() > fits); + TEST_ASSERT_EQUAL_STRING(("[" + shadeJson(4) + "]").c_str(), document(snapshot_doc_t::shades, 0, 7, fits).c_str()); +} + +/********************************************************************* + * Slots + ********************************************************************/ +static SnapshotSlots slots; +static int fills = 0; +static void countFill(model_snapshot_t &s) { + fills++; + s.shadeCount = 1; +} + +static void test_two_reads_at_once_and_no_more() { + const int8_t a = slots.acquire(); + const int8_t b = slots.acquire(); + TEST_ASSERT_TRUE(a >= 0 && b >= 0 && a != b); + TEST_ASSERT_EQUAL_INT8(-1, slots.acquire()); + slots.cancel(a); + slots.cancel(b); +} +static void test_a_read_is_filled_then_freed() { + fills = 0; + const int8_t s = slots.acquire(); + TEST_ASSERT_FALSE(slots.filled(s)); + slots.fill(s, countFill); + TEST_ASSERT_TRUE(slots.filled(s)); + TEST_ASSERT_EQUAL_UINT8(1, slots.snapshot(s).shadeCount); + slots.release(s); + TEST_ASSERT_EQUAL_INT8(s, slots.acquire()); + slots.cancel(s); + TEST_ASSERT_EQUAL_INT(1, fills); +} +static void test_a_read_gone_before_its_fill_is_not_filled() { + fills = 0; + const int8_t s = slots.acquire(); + const int8_t other = slots.acquire(); + slots.release(s); + // Not free yet: loop() still has the fill queued. + TEST_ASSERT_EQUAL_INT8(-1, slots.acquire()); + slots.fill(s, countFill); + TEST_ASSERT_EQUAL_INT(0, fills); + TEST_ASSERT_EQUAL_INT8(s, slots.acquire()); + slots.cancel(s); + slots.cancel(other); +} +static void test_a_cancelled_slot_is_free_again() { + const int8_t s = slots.acquire(); + slots.cancel(s); + TEST_ASSERT_EQUAL_INT8(s, slots.acquire()); + slots.cancel(s); +} + +int main(int argc, char **argv) { + (void)argc; + (void)argv; + UNITY_BEGIN(); + RUN_TEST(test_a_shade); + RUN_TEST(test_a_shade_ref); + RUN_TEST(test_a_room); + RUN_TEST(test_a_group_ref); + RUN_TEST(test_names_are_escaped); + RUN_TEST(test_rooms_list); + RUN_TEST(test_shades_list); + RUN_TEST(test_groups_list_skips_a_linked_shade_that_is_gone); + RUN_TEST(test_empty_lists); + RUN_TEST(test_controller); + RUN_TEST(test_discovery_has_no_repeaters); + RUN_TEST(test_controller_of_an_empty_model); + RUN_TEST(test_chunk_size_does_not_change_the_document); + RUN_TEST(test_single_items); + RUN_TEST(test_a_single_item_gone_by_the_snapshot_is_an_error); + RUN_TEST(test_an_item_too_big_for_the_unit_is_left_out); + RUN_TEST(test_a_left_out_item_does_not_leave_a_stray_comma); + RUN_TEST(test_two_reads_at_once_and_no_more); + RUN_TEST(test_a_read_is_filled_then_freed); + RUN_TEST(test_a_read_gone_before_its_fill_is_not_filled); + RUN_TEST(test_a_cancelled_slot_is_free_again); + return UNITY_END(); +}