refactor: serve the read routes from a ModelSnapshot that loop() fills

/controller, /discovery, /rooms, /shades, /groups, /room, /shade and /group
now take a SnapshotSlots slot, queue a fill on the LoopQueue and send a
chunked response whose body waits (RESPONSE_TRY_AGAIN) until loop() has
copied the shade model into the slot. SnapshotWriter then streams the
document from that copy a unit at a time, so each document describes one
moment. At most two reads run at once; a third gets 503. A slot's 5.6 KB
snapshot is allocated only while a read holds it, and its atomic state
decides whether loop() or the read frees it when the read goes away first.

The JSON of each item is unchanged and now lives in ModelSnapshot.cpp,
where the native tests check it. Replies still written on the web server
task copy their one item with SomfySnapshot.cpp, so serializing a group
no longer calls updateFlags(), which emitted groupState and published MQTT
from async_tcp.

Also: every read route is chunked and leaves out an item too big for its
3 KB unit, like /controller did; /discovery drops its timing log; the
SOMFY_MAX_* limits move to ShadeTypes.h.

21 new native tests, 292 in all. Flash: esp32devdbg +440 bytes, esp32dev
+2,064 bytes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
cjkas 2026-10-16 10:05:22 +02:00
parent 395cb2d1c7
commit 77703a9855
14 changed files with 1163 additions and 477 deletions

View file

@ -2,51 +2,36 @@
## Concurrent mutation during chunked `/controller` response ## Concurrent mutation during chunked `/controller` response
**Status:** open **Status:** fixed for the read routes; the web-task writers below are still open.
**Introduced:** 2026-04-19, alongside the chunked-response conversion of `/controller` (see [`ControllerChunker` in src/entryPoints/Web.cpp](../src/entryPoints/Web.cpp) and the original crash report about `cbuf::resize` aborting `async_tcp`). **Introduced:** 2026-04-19, alongside the chunked-response conversion of `/controller` (the original crash report was about `cbuf::resize` aborting `async_tcp`).
**Fixed:** 2026-10-16, step 2 of [LOOP_QUEUE_PLAN.md](LOOP_QUEUE_PLAN.md).
### Summary ### What it was
The chunked streaming of `/controller` lazily reads `somfy.rooms`, `somfy.shades`, `somfy.groups`, and `somfy.repeaters` across many `async_tcp` callback invocations as the client's TCP send window opens. If another task mutates those arrays mid-stream, the rendered JSON can be internally inconsistent. The chunked `/controller` response read `somfy.rooms`, `somfy.shades`, `somfy.groups` and `somfy.repeaters` on async_tcp a piece at a time as the client's TCP window opened, for hundreds of ms to seconds. Anything that changed them meanwhile could make the document contradict itself: a deleted shade in `shades` but not in a group's `linkedShades`, two positions for one shade, a linked shade skipped or written twice. `/discovery`, `/shades` and the other reads had the same exposure, only shorter.
### Why this is strictly worse than the pre-chunked code All eight read routes (`/controller`, `/discovery`, `/rooms`, `/shades`, `/groups`, `/room`, `/shade`, `/group`) now stream a ModelSnapshot that `loop()` fills, so each document describes one moment ([ADR-0002](adr/0002-loop-task-is-the-only-writer-of-the-shade-model.md)). That is fix option (c), taken per read in a heap slot of about 5.6 KB.
The old `handleController` ran the serializers synchronously inside a single request-handler invocation, so the full JSON was built in memory *before* any bytes were sent. Window of exposure: a few milliseconds. ### Still open
The new `ControllerChunker` reads state as the response drains. On a slow link or under backpressure, that window is hundreds of ms to seconds. - Command and edit replies (`/shadeCommand`, `/saveShade`, `/linkRemote`, ...) still copy the one item they report from the live shade model on async_tcp, so a reply can show a value from the middle of a change. `/room`, `/shade` and `/group` also check on the live model whether the item exists, because the status goes out before the snapshot is filled.
- Other handlers on the web server task still change the shade model that `loop()` owns (step 3 of the plan):
### Concrete failure modes
1. A shade is deleted mid-stream (e.g. via `/deleteShade`) — it may appear in the `shades` array but be missing from a later group's `linkedShades`, or vice versa.
2. `lastRollingCode` / position fields change between the `shades` pass and a later group's `linkedShades` pass — the client sees two values for the same shade in one document.
3. A group's `linkedShades` list is mutated while the chunker iterates inside `S_GROUPS` — an entry is skipped or emitted twice.
### Fix options (pick one later)
- **(a) Document and accept.** In practice users rarely mutate shades while the config UI is loading. Zero code change.
- **(b) FreeRTOS mutex around `somfy` reads/writes.** Acquire for the full duration of the chunked response and in every mutating path (RF RX, MQTT, web handlers). Cleanest but wide-reaching.
- **(c) Up-front snapshot.** At handler start, copy the subset of `somfy` state the response will serialize into the `ControllerChunker`. Defeats part of the memory benefit — a full snapshot of shades + groups is close in size to the old growing cbuf. Could be reduced by snapshotting only minimal fields (IDs, names, rolling codes) and reading the rest live.
### Related
- Same exposure exists in any other endpoint converted to chunked responses next (`/discovery`, `/shades`). Resolve this issue before expanding the pattern.
- Other handlers on the web server task still change the shade model that `loop()` owns:
- `/addRoom`, `/saveRoom`, `/deleteRoom`, `/addShade`, `/saveShade`, `/deleteShade`, `/addGroup`, `/saveGroup` and `/deleteGroup` add, change and remove rooms, shades and groups. - `/addRoom`, `/saveRoom`, `/deleteRoom`, `/addShade`, `/saveShade`, `/deleteShade`, `/addGroup`, `/saveGroup` and `/deleteGroup` add, change and remove rooms, shades and groups.
- `/linkRemote` and `/unlinkRemote` change a shade's linked remotes. `/linkRemote` also sets a linked remote's rolling code, which `loop()` updates whenever it receives a frame from that remote: the last write wins, so a received frame can undo the code just set. - `/linkRemote` and `/unlinkRemote` change a shade's linked remotes. `/linkRemote` also sets a linked remote's rolling code, which `loop()` updates whenever it receives a frame from that remote: the last write wins, so a received frame can undo the code just set.
- `/linkToGroup` and `/unlinkFromGroup` change a group's linked shades. - `/linkToGroup` and `/unlinkFromGroup` change a group's linked shades.
- `/setPaired` sets `paired`. - `/setPaired` sets `paired`.
Shade requests (shade and group commands, set-My, unpair, `/setPositions`, `/setRollingCode`, and every MQTT command), a shades restore and `/updateShadeConfig` already go through the LoopQueue to `loop()`. Shade requests (shade and group commands, set-My, unpair, `/setPositions`, `/setRollingCode`, and every MQTT command), a shades restore and `/updateShadeConfig` already go through the LoopQueue to `loop()`.
- `/controller` leaves out a room, shade, group or linked shade that doesn't fit in its chunk buffer, see [Oversized `/controller` items are left out](#oversized-controller-items-are-left-out). - The read routes leave out an item that doesn't fit their 3 KB unit, see [Oversized read items are left out](#oversized-read-items-are-left-out).
## Oversized `/controller` items are left out ## Oversized read items are left out
**Status:** open **Status:** open
**Location:** `ControllerChunker::overflowed` in [src/entryPoints/Web.cpp](../src/entryPoints/Web.cpp). **Location:** `SnapshotWriter::overflowed` in [src/ModelSnapshot.cpp](../src/ModelSnapshot.cpp).
### Summary ### Summary
`ControllerChunker` builds each room, shade, group and linked shade into a 3 KB buffer before sending it. When one doesn't fit, the chunker logs an `ESP_LOGE` line, skips that item and carries on, so the document stays valid JSON. The client gets no sign that anything is missing: the item just isn't in the list. Every read route builds each room, shade, group and linked shade into a 3 KB unit before sending it. When one doesn't fit, the writer logs an `ESP_LOGE` line, skips that item and carries on, so the document stays valid JSON. The client gets no sign that anything is missing: the item just isn't in the list. Before step 2 only `/controller` did this; `/discovery`, `/shades` and the rest grew their buffer instead.
A shade with every linked remote populated is about 1.5 KB today, so nothing overflows yet. It becomes a silent loss as soon as a change to the shade, room or group fields pushes an item past 3 KB. A shade with every linked remote populated is about 1.5 KB today, so nothing overflows yet. It becomes a silent loss as soon as a change to the shade, room or group fields pushes an item past 3 KB.
@ -90,12 +75,12 @@ Unlike the `/controller` HTTP crash, this path does **not** abort — there is n
- Not the same code path as the `/controller` crash. Solve independently. - Not the same code path as the `/controller` crash. Solve independently.
- Worth grepping for `JsonSockEvent` usages that iterate collections (see references in `SomfyEvents.cpp`, `Transceiver.cpp`, `ESPNetwork.cpp`, `GitOTA.cpp`) to identify the most at-risk events. - Worth grepping for `JsonSockEvent` usages that iterate collections (see references in `SomfyEvents.cpp`, `Transceiver.cpp`, `ESPNetwork.cpp`, `GitOTA.cpp`) to identify the most at-risk events.
- [Group reads emit events from the web server task](#group-reads-emit-events-from-the-web-server-task): a second task writing the same buffer. - [Group reads emit events from the web server task](#group-reads-emit-events-from-the-web-server-task) (fixed): a second task writing the same buffer.
## Group reads emit events from the web server task ## Group reads emit events from the web server task
**Status:** open **Status:** fixed 2026-10-16, step 2 of [LOOP_QUEUE_PLAN.md](LOOP_QUEUE_PLAN.md) (fix option (a)). Read routes write from a ModelSnapshot, and the replies still written on async_tcp copy the group with `snapshotGroup()`, which works its flags out without changing the group or sending anything.
**Location:** `serializeGroupRef` in [src/entryPoints/Web.cpp](../src/entryPoints/Web.cpp) calls `SomfyGroup::updateFlags()` in [src/SomfyGroup.cpp](../src/SomfyGroup.cpp). **Location:** `serializeGroupRef` in [src/entryPoints/Web.cpp](../src/entryPoints/Web.cpp) called `SomfyGroup::updateFlags()` in [src/SomfyGroup.cpp](../src/SomfyGroup.cpp).
### Summary ### Summary

View file

@ -1,6 +1,6 @@
# Plan: domain modules behind the LoopQueue # Plan: domain modules behind the LoopQueue
**Status:** steps 0 and 1 built (PRs #39 and #40); steps 2–5 planned. **Status:** steps 0–2 built (PRs #39, #40 and #41); steps 3–5 planned.
**Written:** 2026-10-15, against commit `e624a41` on `refactor/entry-points-outputs`. Terms are defined in [GLOSSARY.md](../GLOSSARY.md); the threading rule is [ADR-0002](adr/0002-loop-task-is-the-only-writer-of-the-shade-model.md). **Written:** 2026-10-15, against commit `e624a41` on `refactor/entry-points-outputs`. Terms are defined in [GLOSSARY.md](../GLOSSARY.md); the threading rule is [ADR-0002](adr/0002-loop-task-is-the-only-writer-of-the-shade-model.md).
## Why ## Why
@ -130,6 +130,20 @@ Each step is a PR of its own that can be tested on the device. Each one runs `pi
Flash: `esp32devdbg` 1,546,405 → 1,547,381 bytes, `esp32dev` 1,396,957 → 1,398,337 bytes. Flash: `esp32devdbg` 1,546,405 → 1,547,381 bytes, `esp32dev` 1,396,957 → 1,398,337 bytes.
### Step 2: what changed in behaviour
- **Each read describes one moment.** `/controller`, `/discovery`, `/rooms`, `/shades`, `/groups`, `/room`, `/shade` and `/group` stream a ModelSnapshot that `loop()` fills. The first two KNOWN_ISSUES items about `/controller` are fixed for reads.
- **Every read is chunked.** `/discovery`, `/shades` and the rest used to be built whole into a growing response buffer, the path of the old `cbuf::resize` crash.
- **A read waits for `loop()`.** The headers go out at once; the body follows once the snapshot is filled. That is usually one `loop()` pass, but if the headers' ACK comes back first the library only asks again on its next poll, up to about 500 ms later. A read queued behind a command that waits for the radio waits with it.
- **At most two reads at once.** A third gets 503 "Busy, try the read again". Each read holds a 5.6 KB snapshot and a 3 KB unit on the heap only while it runs.
- **An item too big for the 3 KB unit is left out of every read route**, not only `/controller` (see KNOWN_ISSUES).
- **`/room`, `/shade` and `/group`** still answer 500 for an id that doesn't exist. If the item disappears between that check and the snapshot, the reply is 200 with `{"status":"ERROR","desc":"… Id not found."}`.
- **Group flags in replies are worked out without side effects.** Serializing a group no longer calls `updateFlags()`, which sent `groupState` and published MQTT from the web server task.
- **`/discovery` no longer logs where its time went.** Building it is now spread over the chunks.
`ModelSnapshot.h/.cpp` holds the snapshot types, the JSON for each item, `SnapshotWriter` (the eight documents, built a unit at a time) and `SnapshotSlots`. `SomfySnapshot.cpp` copies the shade model into it, for `loop()`'s fills and for the replies still written on the web server task. The `SOMFY_MAX_*` limits moved to `ShadeTypes.h` so the native tests can use them. 21 new native tests, 292 in all.
Flash: `esp32devdbg` 1,547,381 → 1,547,821 bytes, `esp32dev` 1,398,337 → 1,400,401 bytes.
### Step 0 on a device ### Step 0 on a device
NVS stays at `0x9000`, so the rolling codes survive. The file system partition shrinks from `0xE0000` to `0xD0000`, and the existing LittleFS won't mount, because `LittleFS.begin()` doesn't format. So: NVS stays at `0x9000`, so the rolling codes survive. The file system partition shrinks from `0xE0000` to `0xD0000`, and the existing LittleFS won't mount, because `LittleFS.begin()` doesn't format. So:
1. Take a backup through `/backup`. 1. Take a backup through `/backup`.

View file

@ -109,7 +109,7 @@ lib_deps =
platform = native platform = native
test_framework = unity test_framework = unity
test_build_src = yes test_build_src = yes
build_src_filter = -<*> +<Utils.cpp> +<SomfyFrame.cpp> +<JsonFormatter.cpp> +<ShadeMotion.cpp> +<ShadeSensors.cpp> +<PulseDecoder.cpp> +<TxBuffer.cpp> +<RadioScheduler.cpp> +<ShadeRequests.cpp> +<entryPoints/MqttCommands.cpp> build_src_filter = -<*> +<Utils.cpp> +<SomfyFrame.cpp> +<JsonFormatter.cpp> +<ShadeMotion.cpp> +<ShadeSensors.cpp> +<PulseDecoder.cpp> +<TxBuffer.cpp> +<RadioScheduler.cpp> +<ShadeRequests.cpp> +<entryPoints/MqttCommands.cpp> +<ModelSnapshot.cpp>
build_flags = build_flags =
-std=gnu++17 -std=gnu++17
-Isrc -Isrc

View file

@ -4,12 +4,14 @@
#include "LoopQueue.h" #include "LoopQueue.h"
#include "ConfigFile.h" #include "ConfigFile.h"
#include "Somfy.h" #include "Somfy.h"
#include "SomfySnapshot.h"
#include "Utils.h" #include "Utils.h"
static const char *TAG = "LoopQueue"; static const char *TAG = "LoopQueue";
extern SomfyShadeController somfy; extern SomfyShadeController somfy;
extern ShadeRequests shadeRequests; extern ShadeRequests shadeRequests;
extern SnapshotSlots snapshotSlots;
extern rebootDelay_t rebootDelay; extern rebootDelay_t rebootDelay;
static restore_options_t restoreOptions(uint8_t parts) { static restore_options_t restoreOptions(uint8_t parts) {
@ -61,6 +63,9 @@ void LoopQueue::drain() {
case loop_item_type_t::load_shades: case loop_item_type_t::load_shades:
somfy.loadShadesFile("/shades.tmp"); somfy.loadShadesFile("/shades.tmp");
break; break;
case loop_item_type_t::snapshot_fill:
snapshotSlots.fill(item.slot, fillSnapshot);
break;
} }
} }
} }

View file

@ -22,12 +22,14 @@
enum class loop_item_type_t : uint8_t { enum class loop_item_type_t : uint8_t {
shade_request, // run by ShadeRequests shade_request, // run by ShadeRequests
restore_shades, // restore from /shades.tmp, then reboot restore_shades, // restore from /shades.tmp, then reboot
load_shades // replace the shades with /shades.tmp load_shades, // replace the shades with /shades.tmp
snapshot_fill // fill a read's ModelSnapshot
}; };
struct loop_item_t { struct loop_item_t {
loop_item_type_t type = loop_item_type_t::shade_request; loop_item_type_t type = loop_item_type_t::shade_request;
shade_request_t request; // shade_request shade_request_t request; // shade_request
uint8_t restoreParts = 0; // restore_shades: RESTORE_* bits uint8_t restoreParts = 0; // restore_shades: RESTORE_* bits
uint8_t slot = 0; // snapshot_fill: the SnapshotSlots slot
}; };
// Whether the item sends anything on air. // Whether the item sends anything on air.

394
src/ModelSnapshot.cpp Normal file
View file

@ -0,0 +1,394 @@
#include <new>
#include <stdio.h>
#include <string.h>
#include "esp_log.h"
#include "ModelSnapshot.h"
static const char *TAG = "Snapshot";
const room_snapshot_t *model_snapshot_t::room(uint8_t roomId) const {
for(uint8_t i = 0; i < this->roomCount; i++)
if(this->rooms[i].roomId == roomId) return &this->rooms[i];
return nullptr;
}
const shade_snapshot_t *model_snapshot_t::shade(uint8_t shadeId) const {
for(uint8_t i = 0; i < this->shadeCount; i++)
if(this->shades[i].shadeId == shadeId) return &this->shades[i];
return nullptr;
}
const group_snapshot_t *model_snapshot_t::group(uint8_t groupId) const {
for(uint8_t i = 0; i < this->groupCount; i++)
if(this->groups[i].groupId == groupId) return &this->groups[i];
return nullptr;
}
/*********************************************************************
* Items
********************************************************************/
void writeRoom(JsonFormatter &json, const room_snapshot_t &room) {
json.addElem("roomId", room.roomId);
json.addElem("name", room.name);
json.addElem("sortOrder", room.sortOrder);
}
void writeShadeRef(JsonFormatter &json, const shade_snapshot_t &shade) {
json.addElem("shadeId", shade.shadeId);
json.addElem("roomId", shade.roomId);
json.addElem("name", shade.name);
json.addElem("remoteAddress", shade.remoteAddress);
json.addElem("paired", shade.paired);
json.addElem("shadeType", shade.shadeType);
json.addElem("flipCommands", shade.flipCommands);
json.addElem("flipPosition", shade.flipPosition);
json.addElem("bitLength", shade.bitLength);
json.addElem("proto", shade.proto);
json.addElem("flags", shade.flags);
json.addElem("sunSensor", shade.sunSensor);
json.addElem("hasLight", shade.light);
json.addElem("repeats", shade.repeats);
}
void writeShade(JsonFormatter &json, const shade_snapshot_t &shade) {
json.addElem("shadeId", shade.shadeId);
json.addElem("roomId", shade.roomId);
json.addElem("name", shade.name);
json.addElem("remoteAddress", shade.remoteAddress);
json.addElem("upTime", shade.upTime);
json.addElem("downTime", shade.downTime);
json.addElem("paired", shade.paired);
json.addElem("lastRollingCode", static_cast<uint32_t>(shade.lastRollingCode));
json.addElem("position", shade.position);
json.addElem("tiltType", shade.tiltType);
json.addElem("tiltPosition", shade.tiltPosition);
json.addElem("tiltDirection", shade.tiltDirection);
json.addElem("tiltTime", shade.tiltTime);
json.addElem("stepSize", static_cast<uint32_t>(shade.stepSize));
json.addElem("tiltTarget", shade.tiltTarget);
json.addElem("target", shade.target);
json.addElem("myPos", shade.myPos);
json.addElem("myTiltPos", shade.myTiltPos);
json.addElem("direction", shade.direction);
json.addElem("shadeType", shade.shadeType);
json.addElem("bitLength", shade.bitLength);
json.addElem("proto", shade.proto);
json.addElem("flags", shade.flags);
json.addElem("flipCommands", shade.flipCommands);
json.addElem("flipPosition", shade.flipPosition);
json.addElem("inGroup", shade.inGroup);
json.addElem("sunSensor", shade.sunSensor);
json.addElem("light", shade.light);
json.addElem("repeats", shade.repeats);
json.addElem("sortOrder", shade.sortOrder);
json.addElem("gpioUp", shade.gpioUp);
json.addElem("gpioDown", shade.gpioDown);
json.addElem("gpioMy", shade.gpioMy);
json.addElem("gpioLLTrigger", shade.gpioLLTrigger);
json.addElem("simMy", shade.simMy);
json.beginArray("linkedRemotes");
for(uint8_t i = 0; i < shade.linkedRemoteCount; i++) {
json.beginObject();
json.addElem("remoteAddress", shade.linkedRemotes[i].remoteAddress);
json.addElem("lastRollingCode", static_cast<uint32_t>(shade.linkedRemotes[i].lastRollingCode));
json.endObject();
}
json.endArray();
}
void writeGroupRef(JsonFormatter &json, const group_snapshot_t &group) {
json.addElem("groupId", group.groupId);
json.addElem("roomId", group.roomId);
json.addElem("name", group.name);
json.addElem("remoteAddress", group.remoteAddress);
json.addElem("lastRollingCode", static_cast<uint32_t>(group.lastRollingCode));
json.addElem("bitLength", group.bitLength);
json.addElem("proto", group.proto);
json.addElem("sunSensor", group.sunSensor);
json.addElem("flipCommands", group.flipCommands);
json.addElem("flags", group.flags);
json.addElem("repeats", group.repeats);
json.addElem("sortOrder", group.sortOrder);
}
void writeGroup(JsonFormatter &json, const group_snapshot_t &group, shade_lookup_t lookup, void *ctx) {
writeGroupRef(json, group);
json.beginArray("linkedShades");
for(uint8_t i = 0; i < group.linkedShadeCount; i++) {
const shade_snapshot_t *shade = lookup(group.linkedShades[i], ctx);
if(!shade) continue;
json.beginObject();
writeShadeRef(json, *shade);
json.endObject();
}
json.endArray();
}
/*********************************************************************
* Documents
********************************************************************/
enum : uint8_t {
ST_HEADER, // {<header>,"rooms":[
ST_ROOMS,
ST_SHADES,
ST_GROUPS,
ST_REPEATERS,
ST_ROOM,
ST_SHADE,
ST_GROUP,
ST_OPEN_ARRAY, // [
ST_CLOSE_ARRAY, // ]
ST_TO_SHADES, // ],"shades":[
ST_TO_GROUPS, // ],"groups":[
ST_TO_REPEATERS, // ],"repeaters":[
ST_CLOSE_DOC, // ]}
ST_END
};
static const uint8_t CONTROLLER_STEPS[] = { ST_HEADER, ST_ROOMS, ST_TO_SHADES, ST_SHADES, ST_TO_GROUPS, ST_GROUPS, ST_TO_REPEATERS, ST_REPEATERS, ST_CLOSE_DOC, ST_END };
static const uint8_t DISCOVERY_STEPS[] = { ST_HEADER, ST_ROOMS, ST_TO_SHADES, ST_SHADES, ST_TO_GROUPS, ST_GROUPS, ST_CLOSE_DOC, ST_END };
static const uint8_t ROOMS_STEPS[] = { ST_OPEN_ARRAY, ST_ROOMS, ST_CLOSE_ARRAY, ST_END };
static const uint8_t SHADES_STEPS[] = { ST_OPEN_ARRAY, ST_SHADES, ST_CLOSE_ARRAY, ST_END };
static const uint8_t GROUPS_STEPS[] = { ST_OPEN_ARRAY, ST_GROUPS, ST_CLOSE_ARRAY, ST_END };
static const uint8_t ROOM_STEPS[] = { ST_ROOM, ST_END };
static const uint8_t SHADE_STEPS[] = { ST_SHADE, ST_END };
static const uint8_t GROUP_STEPS[] = { ST_GROUP, ST_END };
static const uint8_t *stepsFor(snapshot_doc_t doc) {
switch(doc) {
case snapshot_doc_t::controller: return CONTROLLER_STEPS;
case snapshot_doc_t::discovery: return DISCOVERY_STEPS;
case snapshot_doc_t::rooms: return ROOMS_STEPS;
case snapshot_doc_t::shades: return SHADES_STEPS;
case snapshot_doc_t::groups: return GROUPS_STEPS;
case snapshot_doc_t::room: return ROOM_STEPS;
case snapshot_doc_t::shade: return SHADE_STEPS;
case snapshot_doc_t::group: return GROUP_STEPS;
}
return SHADES_STEPS;
}
static const shade_snapshot_t *snapshotShade(uint8_t shadeId, void *ctx) {
return static_cast<const model_snapshot_t *>(ctx)->shade(shadeId);
}
SnapshotWriter::SnapshotWriter(const model_snapshot_t &snap, snapshot_doc_t doc, uint8_t id, snapshot_header_t header, char *unit, size_t unitSize)
: snap(snap), steps(stepsFor(doc)), id(id), header(header), unit(unit), unitSize(unitSize) {}
size_t SnapshotWriter::generate(uint8_t *out, size_t maxLen) {
size_t written = 0;
while(written < maxLen) {
if(this->consumed < this->unitLen) {
size_t n = this->unitLen - this->consumed;
if(n > maxLen - written) n = maxLen - written;
memcpy(out + written, this->unit + this->consumed, n);
this->consumed += n;
written += n;
}
else {
// A step or item can produce nothing, so go on until something is
// produced or the document is done.
this->consumed = 0;
do this->produceUnit(); while(this->unitLen == 0 && this->steps[this->step] != ST_END);
if(this->unitLen == 0) return written;
}
}
return written;
}
void SnapshotWriter::nextStep() {
this->step++;
this->idx = 0;
this->firstItem = true;
this->groupOpen = false;
}
void SnapshotWriter::literal(const char *text) {
strncpy(this->unit, text, this->unitSize - 1);
this->unit[this->unitSize - 1] = '\0';
this->unitLen = strlen(this->unit);
this->nextStep();
}
void SnapshotWriter::resetFmt(size_t offset) {
this->unit[offset] = '\0';
this->fmt.setBuffer(this->unit + offset, this->unitSize - offset);
}
bool SnapshotWriter::overflowed(const char *what, uint8_t itemId) {
if(!this->fmt.overflowed()) return false;
ESP_LOGE(TAG, "%s %u doesn't fit in %u bytes, leaving it out", what, itemId, (unsigned)this->unitSize);
this->unitLen = 0;
return true;
}
// Starts a list item, after a comma unless it is the first.
size_t SnapshotWriter::itemStart() {
size_t pos = 0;
if(!this->firstItem) this->unit[pos++] = ',';
this->resetFmt(pos);
return pos;
}
// One piece of a group: its ref, then each linked shade, then the close.
// inList is whether it is an item of a list rather than the whole document.
void SnapshotWriter::groupUnit(const group_snapshot_t &group, bool inList) {
auto done = [this, inList]() {
this->groupOpen = false;
if(inList) this->idx++;
else this->nextStep();
};
if(!this->groupOpen) {
const size_t pos = inList ? this->itemStart() : 0;
if(!inList) this->resetFmt(0);
this->fmt.beginObject();
writeGroupRef(this->fmt, group);
this->fmt.beginArray("linkedShades");
if(this->overflowed("group", group.groupId)) { done(); return; }
this->unitLen = pos + this->fmt.length();
this->groupOpen = true;
this->lsIdx = 0;
this->firstLinked = true;
if(inList) this->firstItem = false;
return;
}
const shade_snapshot_t *shade = nullptr;
while(this->lsIdx < group.linkedShadeCount && !(shade = this->snap.shade(group.linkedShades[this->lsIdx]))) this->lsIdx++;
if(!shade) {
strcpy(this->unit, "]}");
this->unitLen = 2;
done();
return;
}
size_t pos = 0;
if(!this->firstLinked) this->unit[pos++] = ',';
this->resetFmt(pos);
this->fmt.beginObject();
writeShadeRef(this->fmt, *shade);
this->fmt.endObject();
this->lsIdx++;
if(this->overflowed("linked shade", shade->shadeId)) return;
this->unitLen = pos + this->fmt.length();
this->firstLinked = false;
}
void SnapshotWriter::produceUnit() {
this->unitLen = 0;
switch(this->steps[this->step]) {
case ST_HEADER:
this->resetFmt(0);
this->fmt.beginObject();
if(this->header) this->header(this->fmt, this->snap);
this->fmt.beginArray("rooms");
// Only fixed size values so this can't happen short of a bug.
if(this->fmt.overflowed()) ESP_LOGE(TAG, "Header doesn't fit in %u bytes", (unsigned)this->unitSize);
this->unitLen = this->fmt.length();
this->nextStep();
return;
case ST_ROOMS: {
if(this->idx >= this->snap.roomCount) { this->nextStep(); return; }
const room_snapshot_t &room = this->snap.rooms[this->idx++];
const size_t pos = this->itemStart();
this->fmt.beginObject();
writeRoom(this->fmt, room);
this->fmt.endObject();
if(this->overflowed("room", room.roomId)) return;
this->unitLen = pos + this->fmt.length();
this->firstItem = false;
return;
}
case ST_SHADES: {
if(this->idx >= this->snap.shadeCount) { this->nextStep(); return; }
const shade_snapshot_t &shade = this->snap.shades[this->idx++];
const size_t pos = this->itemStart();
this->fmt.beginObject();
writeShade(this->fmt, shade);
this->fmt.endObject();
if(this->overflowed("shade", shade.shadeId)) return;
this->unitLen = pos + this->fmt.length();
this->firstItem = false;
return;
}
case ST_GROUPS:
if(this->idx >= this->snap.groupCount) { this->nextStep(); return; }
this->groupUnit(this->snap.groups[this->idx], true);
return;
case ST_REPEATERS: {
if(this->idx >= this->snap.repeaterCount) { this->nextStep(); return; }
size_t pos = 0;
if(!this->firstItem) this->unit[pos++] = ',';
pos += snprintf(this->unit + pos, this->unitSize - pos, "%lu", (unsigned long)this->snap.repeaters[this->idx++]);
this->unitLen = pos;
this->firstItem = false;
return;
}
case ST_ROOM: {
const room_snapshot_t *room = this->snap.room(this->id);
if(!room) { this->literal("{\"status\":\"ERROR\",\"desc\":\"Room Id not found.\"}"); return; }
this->resetFmt(0);
this->fmt.beginObject();
writeRoom(this->fmt, *room);
this->fmt.endObject();
if(!this->overflowed("room", room->roomId)) this->unitLen = this->fmt.length();
this->nextStep();
return;
}
case ST_SHADE: {
const shade_snapshot_t *shade = this->snap.shade(this->id);
if(!shade) { this->literal("{\"status\":\"ERROR\",\"desc\":\"Shade Id not found.\"}"); return; }
this->resetFmt(0);
this->fmt.beginObject();
writeShade(this->fmt, *shade);
this->fmt.endObject();
if(!this->overflowed("shade", shade->shadeId)) this->unitLen = this->fmt.length();
this->nextStep();
return;
}
case ST_GROUP: {
const group_snapshot_t *group = this->snap.group(this->id);
if(!group) { this->literal("{\"status\":\"ERROR\",\"desc\":\"Group Id not found.\"}"); return; }
this->groupUnit(*group, false);
return;
}
case ST_OPEN_ARRAY: this->literal("["); return;
case ST_CLOSE_ARRAY: this->literal("]"); return;
case ST_TO_SHADES: this->literal("],\"shades\":["); return;
case ST_TO_GROUPS: this->literal("],\"groups\":["); return;
case ST_TO_REPEATERS: this->literal("],\"repeaters\":["); return;
case ST_CLOSE_DOC: this->literal("]}"); return;
default:
return;
}
}
/*********************************************************************
* Slots
********************************************************************/
int8_t SnapshotSlots::acquire() {
for(uint8_t i = 0; i < SNAPSHOT_SLOTS; i++) {
uint32_t expected = FREE;
if(!this->state[i].compare_exchange_strong(expected, WAITING)) continue;
// Only the holder of a WAITING slot touches its pointer until it is queued.
this->snaps[i] = new (std::nothrow) model_snapshot_t;
if(!this->snaps[i]) {
this->state[i].store(FREE);
return -1;
}
return static_cast<int8_t>(i);
}
return -1;
}
// The snapshot goes before the slot is marked free, so acquire() never sees a
// free slot that still holds one.
void SnapshotSlots::freeSlot(uint8_t slot) {
delete this->snaps[slot];
this->snaps[slot] = nullptr;
this->state[slot].store(FREE);
}
void SnapshotSlots::fill(uint8_t slot, void (*fill)(model_snapshot_t &snap)) {
uint32_t expected = WAITING;
if(!this->state[slot].compare_exchange_strong(expected, FILLING)) {
// The read went away before it was filled; nobody else will free it.
if(expected == ABANDONED) this->freeSlot(slot);
return;
}
model_snapshot_t &snap = *this->snaps[slot];
snap.startingAddress = 0;
snap.roomCount = snap.shadeCount = snap.groupCount = snap.repeaterCount = 0;
fill(snap);
expected = FILLING;
// Released while being filled: the read is gone, so free it here.
if(!this->state[slot].compare_exchange_strong(expected, FILLED)) this->freeSlot(slot);
}
bool SnapshotSlots::filled(uint8_t slot) const { return this->state[slot].load() == FILLED; }
void SnapshotSlots::cancel(uint8_t slot) { this->freeSlot(slot); }
void SnapshotSlots::release(uint8_t slot) {
// Still waiting for or being filled by loop(): it frees the slot when it sees
// ABANDONED. Filled: nothing else touches it, so free it now.
if(this->state[slot].exchange(ABANDONED) == FILLED) this->freeSlot(slot);
}

187
src/ModelSnapshot.h Normal file
View file

@ -0,0 +1,187 @@
#ifndef MODEL_SNAPSHOT_H
#define MODEL_SNAPSHOT_H
#include <atomic>
#include <stddef.h>
#include <stdint.h>
#include "ShadeTypes.h"
#include "JsonFormatter.h"
// A copy of the shade model taken on loop() for one read, so the reply
// describes a single moment (docs/adr/0002-loop-task-is-the-only-writer-of-the-shade-model.md),
// and the JSON the read routes write from it. Kept free of the shade model and
// the web server so it can be unit tested on the PC (pio test -e native).
// Positions are as clients see them: whole percentages, flipped for a shade
// with flipPosition, -1 when unset.
// No default member values: with them the SnapshotSlots would be stored in
// flash as initialized data (11 KB). The snapshot functions set every field;
// anything else starts from {}.
struct room_snapshot_t {
uint8_t roomId;
char name[21];
int8_t sortOrder;
};
struct linked_remote_snapshot_t {
uint32_t remoteAddress;
uint16_t lastRollingCode;
};
struct shade_snapshot_t {
uint8_t shadeId;
uint8_t roomId;
char name[21];
uint32_t remoteAddress;
uint32_t upTime;
uint32_t downTime;
uint32_t tiltTime;
uint16_t stepSize;
bool paired;
uint16_t lastRollingCode;
int8_t position;
int8_t target;
int8_t myPos;
int8_t direction;
uint8_t tiltType;
int8_t tiltPosition;
int8_t tiltTarget;
int8_t myTiltPos;
int8_t tiltDirection;
uint8_t shadeType;
uint8_t bitLength;
uint8_t proto;
uint8_t flags;
bool flipCommands;
bool flipPosition;
bool inGroup;
bool sunSensor;
bool light;
bool simMy;
uint8_t repeats;
int8_t sortOrder;
uint8_t gpioUp;
uint8_t gpioDown;
uint8_t gpioMy;
bool gpioLLTrigger;
uint8_t linkedRemoteCount;
linked_remote_snapshot_t linkedRemotes[SOMFY_MAX_LINKED_REMOTES];
};
struct group_snapshot_t {
uint8_t groupId;
uint8_t roomId;
char name[21];
uint32_t remoteAddress;
uint16_t lastRollingCode;
uint8_t bitLength;
uint8_t proto;
bool sunSensor;
bool flipCommands;
uint8_t flags; // its linked shades' flags together
uint8_t repeats;
int8_t sortOrder;
uint8_t linkedShadeCount;
uint8_t linkedShades[SOMFY_MAX_GROUPED_SHADES];
};
// Only the rooms, shades, groups and repeaters that exist, in model order.
struct model_snapshot_t {
uint32_t startingAddress;
uint8_t roomCount;
uint8_t shadeCount;
uint8_t groupCount;
uint8_t repeaterCount;
room_snapshot_t rooms[SOMFY_MAX_ROOMS];
shade_snapshot_t shades[SOMFY_MAX_SHADES];
group_snapshot_t groups[SOMFY_MAX_GROUPS];
uint32_t repeaters[SOMFY_MAX_REPEATERS];
const room_snapshot_t *room(uint8_t roomId) const;
const shade_snapshot_t *shade(uint8_t shadeId) const;
const group_snapshot_t *group(uint8_t groupId) const;
};
// The fields of one item, as every route has always written them.
void writeRoom(JsonFormatter &json, const room_snapshot_t &room);
void writeShadeRef(JsonFormatter &json, const shade_snapshot_t &shade);
void writeShade(JsonFormatter &json, const shade_snapshot_t &shade);
void writeGroupRef(JsonFormatter &json, const group_snapshot_t &group);
// The shade a group links to, or nullptr when it has gone.
typedef const shade_snapshot_t *(*shade_lookup_t)(uint8_t shadeId, void *ctx);
// writeGroupRef plus linkedShades, the refs of the shades it links to.
void writeGroup(JsonFormatter &json, const group_snapshot_t &group, shade_lookup_t lookup, void *ctx);
// The documents the read routes send.
enum class snapshot_doc_t : uint8_t {
controller, // /controller
discovery, // /discovery
rooms, // /rooms
shades, // /shades
groups, // /groups
room, // /room?roomId=
shade, // /shade?shadeId=
group // /group?groupId=
};
// Writes the fields of /controller or /discovery that aren't in the snapshot
// (device, radio and firmware details) into the open top-level object.
typedef void (*snapshot_header_t)(JsonFormatter &json, const model_snapshot_t &snap);
// Writes a document from a snapshot a piece at a time for a chunked response.
// Each room, shade, group and linked shade is formatted into unit on its own;
// one that doesn't fit is logged and left out so the JSON stays valid.
class SnapshotWriter {
public:
// id picks the item for the single-item documents. header is only called
// for controller and discovery. unit must outlive the writer.
SnapshotWriter(const model_snapshot_t &snap, snapshot_doc_t doc, uint8_t id, snapshot_header_t header, char *unit, size_t unitSize);
// Copies up to maxLen more bytes of the document into out. 0 once it is all
// out.
size_t generate(uint8_t *out, size_t maxLen);
private:
const model_snapshot_t &snap;
const uint8_t *steps;
uint8_t step = 0;
uint8_t id;
snapshot_header_t header;
BufferedJsonFormatter fmt;
char *unit;
size_t unitSize;
size_t unitLen = 0;
size_t consumed = 0;
uint8_t idx = 0;
uint8_t lsIdx = 0; // next linked shade of the group being written
bool groupOpen = false; // the group's ref is out, its linked shades are next
bool firstItem = true;
bool firstLinked = true;
void produceUnit();
void nextStep();
void literal(const char *text);
void resetFmt(size_t offset);
bool overflowed(const char *what, uint8_t itemId);
size_t itemStart();
void groupUnit(const group_snapshot_t &group, bool comma);
};
// The snapshots reads are served from. A read takes a slot on the web server
// task and queues a fill; loop() fills it; the read streams it and lets it go.
// The read may go away before loop() gets to it, so the slot's state decides
// who frees it: whichever side is last. A slot's snapshot (about 5.6 KB) is
// only allocated while a read holds it.
#define SNAPSHOT_SLOTS 2
class SnapshotSlots {
public:
// A free slot with its snapshot allocated, or -1 when every slot is in use
// or there is no memory. Any task.
int8_t acquire();
// Fills the slot with fill, unless its read has gone. loop() only.
void fill(uint8_t slot, void (*fill)(model_snapshot_t &snap));
// Whether loop() has filled the slot. The read's task.
bool filled(uint8_t slot) const;
// Valid from acquire() until the read releases the slot.
model_snapshot_t &snapshot(uint8_t slot) { return *this->snaps[slot]; }
// The read is done with the slot, or gone. The read's task.
void release(uint8_t slot);
// Frees a slot whose fill was never queued. The read's task.
void cancel(uint8_t slot);
private:
enum : uint32_t { FREE, WAITING, FILLING, FILLED, ABANDONED };
model_snapshot_t *snaps[SNAPSHOT_SLOTS] = {};
std::atomic<uint32_t> state[SNAPSHOT_SLOTS] = {};
void freeSlot(uint8_t slot);
};
#endif

View file

@ -2,8 +2,16 @@
#define SHADE_TYPES_H #define SHADE_TYPES_H
#include <Arduino.h> #include <Arduino.h>
// The kinds of shade, tilt and group. Kept free of the shade model so the // The kinds of shade, tilt and group, and how many of each the shade model
// shade maths can use them in the unit tests on the PC (pio test -e native). // holds. Kept free of the shade model so the shade maths can use them in the
// unit tests on the PC (pio test -e native).
#define SOMFY_MAX_SHADES 32
#define SOMFY_MAX_GROUPS 16
#define SOMFY_MAX_LINKED_REMOTES 7
#define SOMFY_MAX_GROUPED_SHADES 32
#define SOMFY_MAX_ROOMS 16
#define SOMFY_MAX_REPEATERS 7
enum class group_types : byte { enum class group_types : byte {
channel = 0x00 channel = 0x00

View file

@ -4,14 +4,6 @@
#include "ConfigSettings.h" #include "ConfigSettings.h"
#include "WResp.h" #include "WResp.h"
#define SOMFY_MAX_SHADES 32
#define SOMFY_MAX_GROUPS 16
#define SOMFY_MAX_LINKED_REMOTES 7
#define SOMFY_MAX_GROUPED_SHADES 32
#define SOMFY_MAX_ROOMS 16
#define SOMFY_MAX_REPEATERS 7
#include "SomfyFrame.h" #include "SomfyFrame.h"
#include "TxBuffer.h" #include "TxBuffer.h"
#include "RadioScheduler.h" #include "RadioScheduler.h"

View file

@ -12,6 +12,7 @@
#include "LoopQueue.h" #include "LoopQueue.h"
#include "ShadeRequests.h" #include "ShadeRequests.h"
#include "SomfyShadeOps.h" #include "SomfyShadeOps.h"
#include "ModelSnapshot.h"
#include "GitOTA.h" #include "GitOTA.h"
#include "esp_core_dump.h" #include "esp_core_dump.h"
#include "esp_system.h" #include "esp_system.h"
@ -39,6 +40,7 @@ GitUpdater git;
SomfyShadeOps shadeOps; SomfyShadeOps shadeOps;
ShadeRequests shadeRequests(shadeOps); ShadeRequests shadeRequests(shadeOps);
LoopQueue loopQueue; LoopQueue loopQueue;
SnapshotSlots snapshotSlots;
uint32_t oldheap = 0; uint32_t oldheap = 0;

102
src/SomfySnapshot.cpp Normal file
View file

@ -0,0 +1,102 @@
#include <string.h>
#include "SomfySnapshot.h"
extern SomfyShadeController somfy;
static void copyName(char *out, const char *name) {
strncpy(out, name, 20);
out[20] = '\0';
}
void snapshotRoom(SomfyRoom &room, room_snapshot_t &out) {
out.roomId = room.roomId;
copyName(out.name, room.name);
out.sortOrder = room.sortOrder;
}
void snapshotShade(SomfyShade &shade, shade_snapshot_t &out) {
out.shadeId = shade.getShadeId();
out.roomId = shade.roomId;
copyName(out.name, shade.name);
out.remoteAddress = shade.getRemoteAddress();
out.upTime = shade.upTime;
out.downTime = shade.downTime;
out.tiltTime = shade.tiltTime;
out.stepSize = shade.stepSize;
out.paired = shade.paired;
out.lastRollingCode = shade.lastRollingCode;
out.position = shade.transformPosition(shade.currentPos);
out.target = shade.transformPosition(shade.target);
out.myPos = shade.transformPosition(shade.myPos);
out.direction = shade.direction;
out.tiltType = static_cast<uint8_t>(shade.tiltType);
out.tiltPosition = shade.transformPosition(shade.currentTiltPos);
out.tiltTarget = shade.transformPosition(shade.tiltTarget);
out.myTiltPos = shade.transformPosition(shade.myTiltPos);
out.tiltDirection = shade.tiltDirection;
out.shadeType = static_cast<uint8_t>(shade.shadeType);
out.bitLength = shade.bitLength;
out.proto = static_cast<uint8_t>(shade.proto);
out.flags = shade.flags;
out.flipCommands = shade.flipCommands;
out.flipPosition = shade.flipPosition;
out.inGroup = shade.isInGroup();
out.sunSensor = shade.hasSunSensor();
out.light = shade.hasLight();
out.simMy = shade.simMy();
out.repeats = shade.repeats;
out.sortOrder = shade.sortOrder;
out.gpioUp = shade.gpioUp;
out.gpioDown = shade.gpioDown;
out.gpioMy = shade.gpioMy;
out.gpioLLTrigger = (shade.gpioFlags & static_cast<uint8_t>(gpio_flags_t::LowLevelTrigger)) != 0;
out.linkedRemoteCount = 0;
for(uint8_t i = 0; i < SOMFY_MAX_LINKED_REMOTES; i++) {
SomfyLinkedRemote &remote = shade.linkedRemotes[i];
if(remote.getRemoteAddress() == 0) continue;
linked_remote_snapshot_t &l = out.linkedRemotes[out.linkedRemoteCount++];
l.remoteAddress = remote.getRemoteAddress();
l.lastRollingCode = remote.lastRollingCode;
}
}
void snapshotGroup(SomfyGroup &group, group_snapshot_t &out) {
out.groupId = group.getGroupId();
out.roomId = group.roomId;
copyName(out.name, group.name);
out.remoteAddress = group.getRemoteAddress();
out.lastRollingCode = group.lastRollingCode;
out.bitLength = group.bitLength;
out.proto = static_cast<uint8_t>(group.proto);
out.sunSensor = group.hasSunSensor();
out.flipCommands = group.flipCommands;
out.repeats = group.repeats;
out.sortOrder = group.sortOrder;
// The same sum as SomfyGroup::updateFlags, without its event.
out.flags = 0;
for(uint8_t i = 0; i < SOMFY_MAX_GROUPED_SHADES && group.linkedShades[i] != 0; i++) {
SomfyShade *shade = somfy.getShadeById(group.linkedShades[i]);
if(shade) out.flags |= shade->flags;
}
out.linkedShadeCount = 0;
for(uint8_t i = 0; i < SOMFY_MAX_GROUPED_SHADES; i++) {
const uint8_t shadeId = group.linkedShades[i];
if(shadeId > 0 && shadeId < 255) out.linkedShades[out.linkedShadeCount++] = shadeId;
}
}
void fillSnapshot(model_snapshot_t &snap) {
snap.startingAddress = somfy.startingAddress;
for(uint8_t i = 0; i < SOMFY_MAX_ROOMS; i++)
if(somfy.rooms[i].roomId != 0) snapshotRoom(somfy.rooms[i], snap.rooms[snap.roomCount++]);
for(uint8_t i = 0; i < SOMFY_MAX_SHADES; i++)
if(somfy.shades[i].getShadeId() != 255) snapshotShade(somfy.shades[i], snap.shades[snap.shadeCount++]);
for(uint8_t i = 0; i < SOMFY_MAX_GROUPS; i++)
if(somfy.groups[i].getGroupId() != 255) snapshotGroup(somfy.groups[i], snap.groups[snap.groupCount++]);
for(uint8_t i = 0; i < SOMFY_MAX_REPEATERS; i++)
if(somfy.repeaters[i] != 0) snap.repeaters[snap.repeaterCount++] = somfy.repeaters[i];
}
const shade_snapshot_t *liveShade(uint8_t shadeId, void *ctx) {
SomfyShade *shade = somfy.getShadeById(shadeId);
if(!shade) return nullptr;
shade_snapshot_t *out = static_cast<shade_snapshot_t *>(ctx);
snapshotShade(*shade, *out);
return out;
}

17
src/SomfySnapshot.h Normal file
View file

@ -0,0 +1,17 @@
#ifndef SOMFY_SNAPSHOT_H
#define SOMFY_SNAPSHOT_H
#include "ModelSnapshot.h"
#include "Somfy.h"
// Copies the shade model into the ModelSnapshot types.
void snapshotRoom(SomfyRoom &room, room_snapshot_t &out);
void snapshotShade(SomfyShade &shade, shade_snapshot_t &out);
// Works the group's flags out from its shades without changing the group.
void snapshotGroup(SomfyGroup &group, group_snapshot_t &out);
// The whole shade model. loop() only.
void fillSnapshot(model_snapshot_t &snap);
// A shade_lookup_t over the live shade model for writeGroup. ctx is a
// shade_snapshot_t to copy into.
const shade_snapshot_t *liveShade(uint8_t shadeId, void *ctx);
#endif

View file

@ -11,6 +11,8 @@
#include "Somfy.h" #include "Somfy.h"
#include "WResp.h" #include "WResp.h"
#include "LoopQueue.h" #include "LoopQueue.h"
#include "ModelSnapshot.h"
#include "SomfySnapshot.h"
#include "Web.h" #include "Web.h"
#include "MQTT.h" #include "MQTT.h"
#include "GitOTA.h" #include "GitOTA.h"
@ -30,12 +32,11 @@ extern MQTTClass mqtt;
extern GitUpdater git; extern GitUpdater git;
extern ESPNetwork net; extern ESPNetwork net;
extern LoopQueue loopQueue; extern LoopQueue loopQueue;
extern SnapshotSlots snapshotSlots;
//#define WEB_MAX_RESPONSE 34768 //#define WEB_MAX_RESPONSE 34768
#define WEB_MAX_RESPONSE 4096 #define WEB_MAX_RESPONSE 4096
static char g_async_content[WEB_MAX_RESPONSE]; static char g_async_content[WEB_MAX_RESPONSE];
// /discovery logs where its time went when it takes at least this long.
#define DISCOVERY_SLOW_MS 50
// General responses // General responses
@ -202,140 +203,42 @@ static bool asyncHasParam(AsyncWebServerRequest *request, const char *name) {
return request->hasParam(name); return request->hasParam(name);
} }
// -- Serialization helpers (accept JsonFormatter& so both sync and async can use them) -- // -- Serialization helpers --
// Replies that still read the live shade model on this task (command and edit
// replies) copy one item into the ModelSnapshot types and write it the way the
// read routes do. The read routes themselves write from a ModelSnapshot that
// loop() fills (sendSnapshot).
static void serializeRoom(SomfyRoom *room, JsonFormatter &json) { static void serializeRoom(SomfyRoom *room, JsonFormatter &json) {
json.addElem("roomId", room->roomId); room_snapshot_t s;
json.addElem("name", room->name); snapshotRoom(*room, s);
json.addElem("sortOrder", room->sortOrder); writeRoom(json, s);
} }
static void serializeShadeRef(SomfyShade *shade, JsonFormatter &json) { static void serializeShadeRef(SomfyShade *shade, JsonFormatter &json) {
json.addElem("shadeId", shade->getShadeId()); shade_snapshot_t s;
json.addElem("roomId", shade->roomId); snapshotShade(*shade, s);
json.addElem("name", shade->name); writeShadeRef(json, s);
json.addElem("remoteAddress", (uint32_t)shade->getRemoteAddress());
json.addElem("paired", shade->paired);
json.addElem("shadeType", static_cast<uint8_t>(shade->shadeType));
json.addElem("flipCommands", shade->flipCommands);
json.addElem("flipPosition", shade->flipPosition);
json.addElem("bitLength", shade->bitLength);
json.addElem("proto", static_cast<uint8_t>(shade->proto));
json.addElem("flags", shade->flags);
json.addElem("sunSensor", shade->hasSunSensor());
json.addElem("hasLight", shade->hasLight());
json.addElem("repeats", shade->repeats);
} }
// paired and lastRollingCode are passed in for a response that reports the // paired and lastRollingCode are passed in for a response that reports the
// change the main loop is about to make. An unpair is held until it goes on // change the main loop is about to make. An unpair is held until it goes on
// air (holdLoopItem), so the answer comes true. // air (holdLoopItem), so the answer comes true.
static void serializeShade(SomfyShade *shade, JsonFormatter &json, bool paired, uint16_t lastRollingCode) { static void serializeShade(SomfyShade *shade, JsonFormatter &json, bool paired, uint16_t lastRollingCode) {
json.addElem("shadeId", shade->getShadeId()); shade_snapshot_t s;
json.addElem("roomId", shade->roomId); snapshotShade(*shade, s);
json.addElem("name", shade->name); s.paired = paired;
json.addElem("remoteAddress", (uint32_t)shade->getRemoteAddress()); s.lastRollingCode = lastRollingCode;
json.addElem("upTime", (uint32_t)shade->upTime); writeShade(json, s);
json.addElem("downTime", (uint32_t)shade->downTime);
json.addElem("paired", paired);
json.addElem("lastRollingCode", (uint32_t)lastRollingCode);
json.addElem("position", shade->transformPosition(shade->currentPos));
json.addElem("tiltType", static_cast<uint8_t>(shade->tiltType));
json.addElem("tiltPosition", shade->transformPosition(shade->currentTiltPos));
json.addElem("tiltDirection", shade->tiltDirection);
json.addElem("tiltTime", (uint32_t)shade->tiltTime);
json.addElem("stepSize", (uint32_t)shade->stepSize);
json.addElem("tiltTarget", shade->transformPosition(shade->tiltTarget));
json.addElem("target", shade->transformPosition(shade->target));
json.addElem("myPos", shade->transformPosition(shade->myPos));
json.addElem("myTiltPos", shade->transformPosition(shade->myTiltPos));
json.addElem("direction", shade->direction);
json.addElem("shadeType", static_cast<uint8_t>(shade->shadeType));
json.addElem("bitLength", shade->bitLength);
json.addElem("proto", static_cast<uint8_t>(shade->proto));
json.addElem("flags", shade->flags);
json.addElem("flipCommands", shade->flipCommands);
json.addElem("flipPosition", shade->flipPosition);
json.addElem("inGroup", shade->isInGroup());
json.addElem("sunSensor", shade->hasSunSensor());
json.addElem("light", shade->hasLight());
json.addElem("repeats", shade->repeats);
json.addElem("sortOrder", shade->sortOrder);
json.addElem("gpioUp", shade->gpioUp);
json.addElem("gpioDown", shade->gpioDown);
json.addElem("gpioMy", shade->gpioMy);
json.addElem("gpioLLTrigger", ((shade->gpioFlags & (uint8_t)gpio_flags_t::LowLevelTrigger) == 0) ? false : true);
json.addElem("simMy", shade->simMy());
json.beginArray("linkedRemotes");
for(uint8_t i = 0; i < SOMFY_MAX_LINKED_REMOTES; i++) {
SomfyLinkedRemote &lremote = shade->linkedRemotes[i];
if(lremote.getRemoteAddress() != 0) {
json.beginObject();
json.addElem("remoteAddress", (uint32_t)lremote.getRemoteAddress());
json.addElem("lastRollingCode", (uint32_t)lremote.lastRollingCode);
json.endObject();
}
}
json.endArray();
} }
static void serializeShade(SomfyShade *shade, JsonFormatter &json) { serializeShade(shade, json, shade->paired, shade->lastRollingCode); } static void serializeShade(SomfyShade *shade, JsonFormatter &json) { serializeShade(shade, json, shade->paired, shade->lastRollingCode); }
static void serializeGroupRef(SomfyGroup *group, JsonFormatter &json) { static void serializeGroupRef(SomfyGroup *group, JsonFormatter &json) {
group->updateFlags(); group_snapshot_t s;
json.addElem("groupId", group->getGroupId()); snapshotGroup(*group, s);
json.addElem("roomId", group->roomId); writeGroupRef(json, s);
json.addElem("name", group->name);
json.addElem("remoteAddress", (uint32_t)group->getRemoteAddress());
json.addElem("lastRollingCode", (uint32_t)group->lastRollingCode);
json.addElem("bitLength", group->bitLength);
json.addElem("proto", static_cast<uint8_t>(group->proto));
json.addElem("sunSensor", group->hasSunSensor());
json.addElem("flipCommands", group->flipCommands);
json.addElem("flags", group->flags);
json.addElem("repeats", group->repeats);
json.addElem("sortOrder", group->sortOrder);
} }
static void serializeGroup(SomfyGroup *group, JsonFormatter &json) { static void serializeGroup(SomfyGroup *group, JsonFormatter &json) {
serializeGroupRef(group, json); group_snapshot_t s;
json.beginArray("linkedShades"); snapshotGroup(*group, s);
for(uint8_t i = 0; i < SOMFY_MAX_GROUPED_SHADES; i++) { shade_snapshot_t linked;
uint8_t shadeId = group->linkedShades[i]; writeGroup(json, s, liveShade, &linked);
if(shadeId > 0 && shadeId < 255) {
SomfyShade *shade = somfy.getShadeById(shadeId);
if(shade) {
json.beginObject();
serializeShadeRef(shade, json);
json.endObject();
}
}
}
json.endArray();
}
static void serializeRooms(JsonFormatter &json) {
for(uint8_t i = 0; i < SOMFY_MAX_ROOMS; i++) {
SomfyRoom *room = &somfy.rooms[i];
if(room->roomId != 0) {
json.beginObject();
serializeRoom(room, json);
json.endObject();
}
}
}
static void serializeShades(JsonFormatter &json) {
for(uint8_t i = 0; i < SOMFY_MAX_SHADES; i++) {
SomfyShade &shade = somfy.shades[i];
if(shade.getShadeId() != 255) {
json.beginObject();
serializeShade(&shade, json);
json.endObject();
}
}
}
static void serializeGroups(JsonFormatter &json) {
for(uint8_t i = 0; i < SOMFY_MAX_GROUPS; i++) {
SomfyGroup &group = somfy.groups[i];
if(group.getGroupId() != 255) {
json.beginObject();
serializeGroup(&group, json);
json.endObject();
}
}
} }
static void serializeRepeaters(JsonFormatter &json) { static void serializeRepeaters(JsonFormatter &json) {
for(uint8_t i = 0; i < SOMFY_MAX_REPEATERS; i++) { for(uint8_t i = 0; i < SOMFY_MAX_REPEATERS; i++) {
@ -401,318 +304,117 @@ static void serializeGitRelease(GitRelease *rel, JsonFormatter &json) {
json.endObject(); json.endObject();
} }
// Memory-bounded streaming builder for /controller. Produces the JSON payload // The parts of /controller that aren't in the shade model.
// one small unit at a time into a fixed 3KB buffer, so a chunked HTTP response static void controllerHeader(JsonFormatter &json, const model_snapshot_t &snap) {
// can drain it as the TCP send window allows — no growing cbuf in the async json.addElem("maxRooms", (uint8_t)SOMFY_MAX_ROOMS);
// response stream. json.addElem("maxShades", (uint8_t)SOMFY_MAX_SHADES);
class ControllerChunker { json.addElem("maxGroups", (uint8_t)SOMFY_MAX_GROUPS);
public: json.addElem("maxGroupedShades", (uint8_t)SOMFY_MAX_GROUPED_SHADES);
enum : uint8_t { S_HEADER, S_ROOMS, S_SHADES, S_GROUPS, S_REPEATERS, S_DONE }; json.addElem("maxLinkedRemotes", (uint8_t)SOMFY_MAX_LINKED_REMOTES);
static constexpr uint16_t LS_OPEN = 0xFFFF; json.addElem("startingAddress", snap.startingAddress);
json.beginObject("transceiver");
json.beginObject("config");
serializeTransceiverConfig(json);
json.endObject();
json.endObject();
json.beginObject("version");
serializeGitVersion(json);
json.endObject();
}
// The parts of /discovery that aren't in the shade model.
static void discoveryHeader(JsonFormatter &json, const model_snapshot_t &snap) {
(void)snap;
const char *connType = "Unknown";
if(net.connType == conn_types_t::ethernet) connType = "Ethernet";
else if(net.connType == conn_types_t::wifi) connType = "Wifi";
json.addElem("serverId", settings.serverId);
json.addElem("version", settings.fwVersion.name);
json.addElem("latest", git.latest.name);
json.addElem("model", "ESPSomfyRTS");
json.addElem("hostname", settings.hostname);
json.addElem("authType", static_cast<uint8_t>(settings.Security.type));
json.addElem("permissions", settings.Security.permissions);
json.addElem("chipModel", settings.chipModel);
json.addElem("connType", connType);
json.addElem("checkForUpdate", settings.checkForUpdate);
json.beginObject("memory");
json.addElem("max", ESP.getMaxAllocHeap());
json.addElem("free", ESP.getFreeHeap());
json.addElem("min", ESP.getMinFreeHeap());
json.addElem("total", ESP.getHeapSize());
json.addElem("uptime", (uint64_t)millis());
json.endObject();
}
BufferedJsonFormatter fmt; // A read served from a ModelSnapshot that loop() fills, so the document
// describes one moment (docs/adr/0002-loop-task-is-the-only-writer-of-the-shade-model.md).
// Each item is formatted into a fixed 3 KB unit and streamed as the TCP send
// window allows, so there is no growing response buffer either.
struct SnapshotRead {
uint8_t slot;
char unit[3072]; char unit[3072];
size_t unitLen = 0; SnapshotWriter writer;
size_t consumed = 0; SnapshotRead(uint8_t slot, snapshot_doc_t doc, uint8_t id, snapshot_header_t header)
uint8_t section = S_HEADER; : slot(slot), writer(snapshotSlots.snapshot(slot), doc, id, header, unit, sizeof(unit)) {}
uint16_t idx = 0; // The response is gone, whether it finished or the client went away.
uint16_t lsIdx = LS_OPEN; ~SnapshotRead() { snapshotSlots.release(this->slot); }
bool firstInArray = true;
bool firstInLSArray = true;
size_t generate(uint8_t *out, size_t maxLen) {
size_t written = 0;
while(written < maxLen) {
if(consumed < unitLen) {
size_t n = unitLen - consumed;
if(n > maxLen - written) n = maxLen - written;
memcpy(out + written, unit + consumed, n);
consumed += n;
written += n;
} else {
produceNext();
if(unitLen == 0) return written;
}
}
return written;
}
private:
void resetFmt(size_t offset = 0) {
unit[offset] = 0;
fmt.setBuffer(unit + offset, sizeof(unit) - offset);
}
// An item too big for the unit buffer is left out so the JSON stays valid.
// Skipping one produces nothing, so go on to the next until something is
// produced or the document is done.
void produceNext() {
consumed = 0;
do produceUnit(); while(unitLen == 0 && section != S_DONE);
}
// True when the unit just formatted didn't fit. It is logged and skipped.
bool overflowed(const char *what, uint8_t id) {
if(!fmt.overflowed()) return false;
ESP_LOGE(TAG, "/controller: %s %u doesn't fit in %u bytes, leaving it out", what, id, (unsigned)sizeof(unit));
unitLen = 0;
return true;
}
void produceUnit() {
unitLen = 0;
switch(section) {
case S_HEADER: {
resetFmt();
fmt.beginObject();
fmt.addElem("maxRooms", (uint8_t)SOMFY_MAX_ROOMS);
fmt.addElem("maxShades", (uint8_t)SOMFY_MAX_SHADES);
fmt.addElem("maxGroups", (uint8_t)SOMFY_MAX_GROUPS);
fmt.addElem("maxGroupedShades", (uint8_t)SOMFY_MAX_GROUPED_SHADES);
fmt.addElem("maxLinkedRemotes", (uint8_t)SOMFY_MAX_LINKED_REMOTES);
fmt.addElem("startingAddress", (uint32_t)somfy.startingAddress);
fmt.beginObject("transceiver");
fmt.beginObject("config");
serializeTransceiverConfig(fmt);
fmt.endObject();
fmt.endObject();
fmt.beginObject("version");
serializeGitVersion(fmt);
fmt.endObject();
fmt.beginArray("rooms");
// Only fixed size values so this can't happen short of a bug.
if(fmt.overflowed()) ESP_LOGE(TAG, "/controller: header doesn't fit in %u bytes", (unsigned)sizeof(unit));
unitLen = fmt.length();
section = S_ROOMS;
idx = 0;
firstInArray = true;
return;
}
case S_ROOMS: {
while(idx < SOMFY_MAX_ROOMS && somfy.rooms[idx].roomId == 0) idx++;
if(idx >= SOMFY_MAX_ROOMS) {
strcpy(unit, "],\"shades\":[");
unitLen = strlen(unit);
section = S_SHADES;
idx = 0;
firstInArray = true;
return;
}
size_t pos = 0;
if(!firstInArray) unit[pos++] = ',';
resetFmt(pos);
fmt.beginObject();
serializeRoom(&somfy.rooms[idx], fmt);
fmt.endObject();
if(overflowed("room", somfy.rooms[idx].roomId)) { idx++; return; }
unitLen = pos + fmt.length();
idx++;
firstInArray = false;
return;
}
case S_SHADES: {
while(idx < SOMFY_MAX_SHADES && somfy.shades[idx].getShadeId() == 255) idx++;
if(idx >= SOMFY_MAX_SHADES) {
strcpy(unit, "],\"groups\":[");
unitLen = strlen(unit);
section = S_GROUPS;
idx = 0;
lsIdx = LS_OPEN;
firstInArray = true;
return;
}
size_t pos = 0;
if(!firstInArray) unit[pos++] = ',';
resetFmt(pos);
fmt.beginObject();
serializeShade(&somfy.shades[idx], fmt);
fmt.endObject();
if(overflowed("shade", somfy.shades[idx].getShadeId())) { idx++; return; }
unitLen = pos + fmt.length();
idx++;
firstInArray = false;
return;
}
case S_GROUPS: {
while(idx < SOMFY_MAX_GROUPS && somfy.groups[idx].getGroupId() == 255) idx++;
if(idx >= SOMFY_MAX_GROUPS) {
strcpy(unit, "],\"repeaters\":[");
unitLen = strlen(unit);
section = S_REPEATERS;
idx = 0;
firstInArray = true;
return;
}
SomfyGroup *g = &somfy.groups[idx];
if(lsIdx == LS_OPEN) {
size_t pos = 0;
if(!firstInArray) unit[pos++] = ',';
resetFmt(pos);
fmt.beginObject();
serializeGroupRef(g, fmt);
fmt.beginArray("linkedShades");
if(overflowed("group", g->getGroupId())) { idx++; return; }
unitLen = pos + fmt.length();
lsIdx = 0;
firstInArray = false;
firstInLSArray = true;
return;
}
SomfyShade *lshade = nullptr;
while(lsIdx < SOMFY_MAX_GROUPED_SHADES) {
uint8_t sid = g->linkedShades[lsIdx];
if(sid > 0 && sid < 255) {
lshade = somfy.getShadeById(sid);
if(lshade) break;
}
lsIdx++;
}
if(!lshade) {
strcpy(unit, "]}");
unitLen = 2;
idx++;
lsIdx = LS_OPEN;
return;
}
size_t pos = 0;
if(!firstInLSArray) unit[pos++] = ',';
resetFmt(pos);
fmt.beginObject();
serializeShadeRef(lshade, fmt);
fmt.endObject();
if(overflowed("linked shade", lshade->getShadeId())) { lsIdx++; return; }
unitLen = pos + fmt.length();
lsIdx++;
firstInLSArray = false;
return;
}
case S_REPEATERS: {
while(idx < SOMFY_MAX_REPEATERS && somfy.repeaters[idx] == 0) idx++;
if(idx >= SOMFY_MAX_REPEATERS) {
strcpy(unit, "]}");
unitLen = 2;
section = S_DONE;
return;
}
size_t pos = 0;
if(!firstInArray) unit[pos++] = ',';
pos += snprintf(unit + pos, sizeof(unit) - pos, "%lu", (unsigned long)somfy.repeaters[idx]);
unitLen = pos;
idx++;
firstInArray = false;
return;
}
case S_DONE:
default:
unitLen = 0;
return;
}
}
}; };
static void sendSnapshot(AsyncWebServerRequest *request, snapshot_doc_t doc, uint8_t id = 0, snapshot_header_t header = nullptr) {
const int8_t slot = snapshotSlots.acquire();
if(slot < 0) {
request->send(503, _encoding_json, F("{\"status\":\"ERROR\",\"desc\":\"Busy, try the read again\"}"));
return;
}
loop_item_t item;
item.type = loop_item_type_t::snapshot_fill;
item.slot = static_cast<uint8_t>(slot);
if(!loopQueue.submit(item)) {
snapshotSlots.cancel(slot);
sendQueueFull(request);
return;
}
auto read = std::make_shared<SnapshotRead>(static_cast<uint8_t>(slot), doc, id, header);
// The headers go now; the body waits until loop() has filled the snapshot.
// The library asks again on the next ACK or poll.
AsyncWebServerResponse *response = request->beginChunkedResponse(_encoding_json,
[read](uint8_t *buffer, size_t maxLen, size_t index) -> size_t {
(void)index;
if(!snapshotSlots.filled(read->slot)) return RESPONSE_TRY_AGAIN;
return read->writer.generate(buffer, maxLen);
});
request->send(response);
}
// -- Async handler implementations -- // -- Async handler implementations --
void Web::handleDiscovery(AsyncWebServerRequest *request) { void Web::handleDiscovery(AsyncWebServerRequest *request) {
if(request->method() == HTTP_POST || request->method() == HTTP_GET) { if(request->method() == HTTP_POST || request->method() == HTTP_GET) {
ESP_LOGD(TAG, "Async Discovery Requested"); ESP_LOGD(TAG, "Async Discovery Requested");
char connType[10] = "Unknown"; sendSnapshot(request, snapshot_doc_t::discovery, 0, discoveryHeader);
if(net.connType == conn_types_t::ethernet) strcpy(connType, "Ethernet");
else if(net.connType == conn_types_t::wifi) strcpy(connType, "Wifi");
// This handler runs on the async_tcp task, which also serves the sockets,
// so a slow one is logged with where its time went.
ElapsedTimer total, step;
uint32_t usHead, usMem, usRooms, usShades, usGroups, usSend;
AsyncJsonResp resp;
resp.beginResponse(request, g_async_content, sizeof(g_async_content));
resp.beginObject();
resp.addElem("serverId", settings.serverId);
resp.addElem("version", settings.fwVersion.name);
resp.addElem("latest", git.latest.name);
resp.addElem("model", "ESPSomfyRTS");
resp.addElem("hostname", settings.hostname);
resp.addElem("authType", static_cast<uint8_t>(settings.Security.type));
resp.addElem("permissions", settings.Security.permissions);
resp.addElem("chipModel", settings.chipModel);
resp.addElem("connType", connType);
resp.addElem("checkForUpdate", settings.checkForUpdate);
usHead = step.lap();
resp.beginObject("memory");
resp.addElem("max", ESP.getMaxAllocHeap());
resp.addElem("free", ESP.getFreeHeap());
resp.addElem("min", ESP.getMinFreeHeap());
resp.addElem("total", ESP.getHeapSize());
resp.addElem("uptime", (uint64_t)millis());
resp.endObject();
usMem = step.lap();
resp.beginArray("rooms");
serializeRooms(resp);
resp.endArray();
usRooms = step.lap();
resp.beginArray("shades");
serializeShades(resp);
resp.endArray();
usShades = step.lap();
resp.beginArray("groups");
serializeGroups(resp);
resp.endArray();
resp.endObject();
usGroups = step.lap();
resp.endResponse();
usSend = step.us();
net.needsBroadcast = true; net.needsBroadcast = true;
if(total.us() >= DISCOVERY_SLOW_MS * 1000) {
ESP_LOGW(TAG, "discovery: head %.1f mem %.1f rooms %.1f shades %.1f groups %.1f send %.1f ms",
usHead / 1000.0f, usMem / 1000.0f, usRooms / 1000.0f, usShades / 1000.0f, usGroups / 1000.0f, usSend / 1000.0f);
}
} }
else else
request->send(500, _encoding_text, "Invalid http method"); request->send(500, _encoding_text, "Invalid http method");
} }
void Web::handleGetRooms(AsyncWebServerRequest *request) { void Web::handleGetRooms(AsyncWebServerRequest *request) {
if(!this->isAuthenticated(request)) return; if(!this->isAuthenticated(request)) return;
if(request->method() == HTTP_POST || request->method() == HTTP_GET) { if(request->method() == HTTP_POST || request->method() == HTTP_GET) sendSnapshot(request, snapshot_doc_t::rooms);
AsyncJsonResp resp;
resp.beginResponse(request, g_async_content, sizeof(g_async_content));
resp.beginArray();
serializeRooms(resp);
resp.endArray();
resp.endResponse();
}
else request->send(404, _encoding_text, _response_404); else request->send(404, _encoding_text, _response_404);
} }
void Web::handleGetShades(AsyncWebServerRequest *request) { void Web::handleGetShades(AsyncWebServerRequest *request) {
if(!this->isAuthenticated(request)) return; if(!this->isAuthenticated(request)) return;
if(request->method() == HTTP_POST || request->method() == HTTP_GET) { if(request->method() == HTTP_POST || request->method() == HTTP_GET) sendSnapshot(request, snapshot_doc_t::shades);
AsyncJsonResp resp;
resp.beginResponse(request, g_async_content, sizeof(g_async_content));
resp.beginArray();
serializeShades(resp);
resp.endArray();
resp.endResponse();
}
else request->send(404, _encoding_text, _response_404); else request->send(404, _encoding_text, _response_404);
} }
void Web::handleGetGroups(AsyncWebServerRequest *request) { void Web::handleGetGroups(AsyncWebServerRequest *request) {
if(!this->isAuthenticated(request)) return; if(!this->isAuthenticated(request)) return;
if(request->method() == HTTP_POST || request->method() == HTTP_GET) { if(request->method() == HTTP_POST || request->method() == HTTP_GET) sendSnapshot(request, snapshot_doc_t::groups);
AsyncJsonResp resp;
resp.beginResponse(request, g_async_content, sizeof(g_async_content));
resp.beginArray();
serializeGroups(resp);
resp.endArray();
resp.endResponse();
}
else request->send(404, _encoding_text, _response_404); else request->send(404, _encoding_text, _response_404);
} }
void Web::handleController(AsyncWebServerRequest *request) { void Web::handleController(AsyncWebServerRequest *request) {
if(!this->isAuthenticated(request)) return; if(!this->isAuthenticated(request)) return;
if(request->method() == HTTP_POST || request->method() == HTTP_GET) { if(request->method() == HTTP_POST || request->method() == HTTP_GET) {
settings.printAvailHeap(); settings.printAvailHeap();
auto state = std::make_shared<ControllerChunker>(); sendSnapshot(request, snapshot_doc_t::controller, 0, controllerHeader);
AsyncWebServerResponse *response = request->beginChunkedResponse(_encoding_json,
[state](uint8_t *buffer, size_t maxLen, size_t index) -> size_t {
return state->generate(buffer, maxLen);
});
request->send(response);
} }
else request->send(404, _encoding_text, _response_404); else request->send(404, _encoding_text, _response_404);
} }
@ -923,15 +625,9 @@ void Web::handleRoom(AsyncWebServerRequest *request) {
if(request->method() == HTTP_GET) { if(request->method() == HTTP_GET) {
if(asyncHasParam(request, "roomId")) { if(asyncHasParam(request, "roomId")) {
int roomId = asyncParam(request, "roomId").toInt(); int roomId = asyncParam(request, "roomId").toInt();
SomfyRoom *room = somfy.getRoomById(roomId); // The status goes out before the snapshot is filled, so whether the room
if(room) { // exists is checked here, on the live model.
AsyncJsonResp resp; if(somfy.getRoomById(roomId)) sendSnapshot(request, snapshot_doc_t::room, roomId);
resp.beginResponse(request, g_async_content, sizeof(g_async_content));
resp.beginObject();
serializeRoom(room, resp);
resp.endObject();
resp.endResponse();
}
else request->send(500, _encoding_json, F("{\"status\":\"ERROR\",\"desc\":\"Room Id not found.\"}")); else request->send(500, _encoding_json, F("{\"status\":\"ERROR\",\"desc\":\"Room Id not found.\"}"));
} }
else request->send(500, _encoding_json, F("{\"status\":\"ERROR\",\"desc\":\"You must supply a valid room id.\"}")); else request->send(500, _encoding_json, F("{\"status\":\"ERROR\",\"desc\":\"You must supply a valid room id.\"}"));
@ -944,15 +640,8 @@ void Web::handleShade(AsyncWebServerRequest *request) {
if(request->method() == HTTP_GET) { if(request->method() == HTTP_GET) {
if(asyncHasParam(request, "shadeId")) { if(asyncHasParam(request, "shadeId")) {
int shadeId = asyncParam(request, "shadeId").toInt(); int shadeId = asyncParam(request, "shadeId").toInt();
SomfyShade *shade = somfy.getShadeById(shadeId); // As for /room, existence is checked on the live model.
if(shade) { if(somfy.getShadeById(shadeId)) sendSnapshot(request, snapshot_doc_t::shade, shadeId);
AsyncJsonResp resp;
resp.beginResponse(request, g_async_content, sizeof(g_async_content));
resp.beginObject();
serializeShade(shade, resp);
resp.endObject();
resp.endResponse();
}
else request->send(500, _encoding_json, F("{\"status\":\"ERROR\",\"desc\":\"Shade Id not found.\"}")); else request->send(500, _encoding_json, F("{\"status\":\"ERROR\",\"desc\":\"Shade Id not found.\"}"));
} }
else request->send(500, _encoding_json, F("{\"status\":\"ERROR\",\"desc\":\"You must supply a valid shade id.\"}")); else request->send(500, _encoding_json, F("{\"status\":\"ERROR\",\"desc\":\"You must supply a valid shade id.\"}"));
@ -965,15 +654,8 @@ void Web::handleGroup(AsyncWebServerRequest *request) {
if(request->method() == HTTP_GET) { if(request->method() == HTTP_GET) {
if(asyncHasParam(request, "groupId")) { if(asyncHasParam(request, "groupId")) {
int groupId = asyncParam(request, "groupId").toInt(); int groupId = asyncParam(request, "groupId").toInt();
SomfyGroup *group = somfy.getGroupById(groupId); // As for /room, existence is checked on the live model.
if(group) { if(somfy.getGroupById(groupId)) sendSnapshot(request, snapshot_doc_t::group, groupId);
AsyncJsonResp resp;
resp.beginResponse(request, g_async_content, sizeof(g_async_content));
resp.beginObject();
serializeGroup(group, resp);
resp.endObject();
resp.endResponse();
}
else request->send(500, _encoding_json, F("{\"status\":\"ERROR\",\"desc\":\"Group Id not found.\"}")); else request->send(500, _encoding_json, F("{\"status\":\"ERROR\",\"desc\":\"Group Id not found.\"}"));
} }
else request->send(500, _encoding_json, F("{\"status\":\"ERROR\",\"desc\":\"You must supply a valid group id.\"}")); else request->send(500, _encoding_json, F("{\"status\":\"ERROR\",\"desc\":\"You must supply a valid group id.\"}"));

View file

@ -0,0 +1,296 @@
#include <unity.h>
#include <string.h>
#include <string>
#include "ModelSnapshot.h"
static model_snapshot_t snap;
static char unitBuf[3072];
static room_snapshot_t room(uint8_t id, const char *name) {
room_snapshot_t r{};
r.roomId = id;
strcpy(r.name, name);
return r;
}
static shade_snapshot_t shade(uint8_t id, const char *name) {
shade_snapshot_t s{};
s.shadeId = id;
s.roomId = 2;
strcpy(s.name, name);
s.remoteAddress = 1000 + id;
s.upTime = 10000;
s.downTime = 11000;
s.tiltTime = 7000;
s.stepSize = 100;
s.paired = true;
s.lastRollingCode = 42;
s.position = 30;
s.target = 30;
s.myPos = -1;
s.tiltType = 2;
s.tiltPosition = 40;
s.tiltTarget = 40;
s.myTiltPos = -1;
s.shadeType = 1;
s.bitLength = 56;
s.inGroup = true;
s.repeats = 1;
return s;
}
static group_snapshot_t group(uint8_t id, const char *name) {
group_snapshot_t g{};
g.groupId = id;
strcpy(g.name, name);
g.remoteAddress = 2000 + id;
g.lastRollingCode = 5;
g.bitLength = 80;
g.repeats = 2;
return g;
}
static std::string json(void (*write)(JsonFormatter &)) {
static char buf[2048];
BufferedJsonFormatter fmt;
fmt.setBuffer(buf, sizeof(buf));
fmt.beginObject();
write(fmt);
fmt.endObject();
return std::string(buf, fmt.length());
}
static std::string shadeRefJson(uint8_t id) {
static uint8_t which;
which = id;
return json([](JsonFormatter &f) { writeShadeRef(f, *snap.shade(which)); });
}
static std::string shadeJson(uint8_t id) {
static uint8_t which;
which = id;
return json([](JsonFormatter &f) { writeShade(f, *snap.shade(which)); });
}
static std::string roomJson(uint8_t id) {
static uint8_t which;
which = id;
return json([](JsonFormatter &f) { writeRoom(f, *snap.room(which)); });
}
static std::string groupRefJson(uint8_t id) {
static uint8_t which;
which = id;
return json([](JsonFormatter &f) { writeGroupRef(f, *snap.group(which)); });
}
static void testHeader(JsonFormatter &json, const model_snapshot_t &s) {
json.addElem("startingAddress", s.startingAddress);
}
// The whole document, taken chunkSize bytes at a time like the web server does.
static std::string document(snapshot_doc_t doc, uint8_t id = 0, size_t chunkSize = 7, size_t unitSize = sizeof(unitBuf)) {
SnapshotWriter writer(snap, doc, id, testHeader, unitBuf, unitSize);
std::string out;
uint8_t chunk[64];
for(int guard = 0; guard < 100000; guard++) {
const size_t n = writer.generate(chunk, chunkSize);
if(n == 0) break;
out.append(reinterpret_cast<char *>(chunk), n);
}
return out;
}
void setUp() {
snap = model_snapshot_t{};
snap.startingAddress = 77;
snap.rooms[snap.roomCount++] = room(2, "Living");
snap.rooms[snap.roomCount++] = room(3, "Bed");
snap.shades[snap.shadeCount++] = shade(1, "Kitchen");
snap.shades[snap.shadeCount++] = shade(4, "Porch");
group_snapshot_t g = group(6, "All");
g.linkedShades[g.linkedShadeCount++] = 4;
g.linkedShades[g.linkedShadeCount++] = 9; // gone from the model
g.linkedShades[g.linkedShadeCount++] = 1;
snap.groups[snap.groupCount++] = g;
snap.repeaters[snap.repeaterCount++] = 12345;
snap.repeaters[snap.repeaterCount++] = 678;
}
void tearDown() {}
/*********************************************************************
* Items, as the routes have always written them
********************************************************************/
static void test_a_shade() {
snap.shades[0].linkedRemotes[snap.shades[0].linkedRemoteCount++] = { 5555, 7 };
TEST_ASSERT_EQUAL_STRING(
"{\"shadeId\":1,\"roomId\":2,\"name\":\"Kitchen\",\"remoteAddress\":1001,\"upTime\":10000,\"downTime\":11000,"
"\"paired\":true,\"lastRollingCode\":42,\"position\":30,\"tiltType\":2,\"tiltPosition\":40,\"tiltDirection\":0,"
"\"tiltTime\":7000,\"stepSize\":100,\"tiltTarget\":40,\"target\":30,\"myPos\":-1,\"myTiltPos\":-1,\"direction\":0,"
"\"shadeType\":1,\"bitLength\":56,\"proto\":0,\"flags\":0,\"flipCommands\":false,\"flipPosition\":false,"
"\"inGroup\":true,\"sunSensor\":false,\"light\":false,\"repeats\":1,\"sortOrder\":0,\"gpioUp\":0,\"gpioDown\":0,"
"\"gpioMy\":0,\"gpioLLTrigger\":false,\"simMy\":false,\"linkedRemotes\":[{\"remoteAddress\":5555,\"lastRollingCode\":7}]}",
shadeJson(1).c_str());
}
static void test_a_shade_ref() {
TEST_ASSERT_EQUAL_STRING(
"{\"shadeId\":1,\"roomId\":2,\"name\":\"Kitchen\",\"remoteAddress\":1001,\"paired\":true,\"shadeType\":1,"
"\"flipCommands\":false,\"flipPosition\":false,\"bitLength\":56,\"proto\":0,\"flags\":0,\"sunSensor\":false,"
"\"hasLight\":false,\"repeats\":1}",
shadeRefJson(1).c_str());
}
static void test_a_room() {
TEST_ASSERT_EQUAL_STRING("{\"roomId\":2,\"name\":\"Living\",\"sortOrder\":0}", roomJson(2).c_str());
}
static void test_a_group_ref() {
TEST_ASSERT_EQUAL_STRING(
"{\"groupId\":6,\"roomId\":0,\"name\":\"All\",\"remoteAddress\":2006,\"lastRollingCode\":5,\"bitLength\":80,"
"\"proto\":0,\"sunSensor\":false,\"flipCommands\":false,\"flags\":0,\"repeats\":2,\"sortOrder\":0}",
groupRefJson(6).c_str());
}
static void test_names_are_escaped() {
strcpy(snap.rooms[0].name, "A \"big\" room");
TEST_ASSERT_EQUAL_STRING("{\"roomId\":2,\"name\":\"A \\\"big\\\" room\",\"sortOrder\":0}", roomJson(2).c_str());
}
/*********************************************************************
* Documents
********************************************************************/
static std::string groupWithLinked() {
std::string g = groupRefJson(6);
g.pop_back(); // reopen it for linkedShades
return g + ",\"linkedShades\":[" + shadeRefJson(4) + "," + shadeRefJson(1) + "]}";
}
static void test_rooms_list() {
TEST_ASSERT_EQUAL_STRING(("[" + roomJson(2) + "," + roomJson(3) + "]").c_str(), document(snapshot_doc_t::rooms).c_str());
}
static void test_shades_list() {
TEST_ASSERT_EQUAL_STRING(("[" + shadeJson(1) + "," + shadeJson(4) + "]").c_str(), document(snapshot_doc_t::shades).c_str());
}
static void test_groups_list_skips_a_linked_shade_that_is_gone() {
TEST_ASSERT_EQUAL_STRING(("[" + groupWithLinked() + "]").c_str(), document(snapshot_doc_t::groups).c_str());
}
static void test_empty_lists() {
snap = model_snapshot_t{};
TEST_ASSERT_EQUAL_STRING("[]", document(snapshot_doc_t::rooms).c_str());
TEST_ASSERT_EQUAL_STRING("[]", document(snapshot_doc_t::shades).c_str());
TEST_ASSERT_EQUAL_STRING("[]", document(snapshot_doc_t::groups).c_str());
}
static void test_controller() {
const std::string expected = "{\"startingAddress\":77,\"rooms\":[" + roomJson(2) + "," + roomJson(3) +
"],\"shades\":[" + shadeJson(1) + "," + shadeJson(4) +
"],\"groups\":[" + groupWithLinked() +
"],\"repeaters\":[12345,678]}";
TEST_ASSERT_EQUAL_STRING(expected.c_str(), document(snapshot_doc_t::controller).c_str());
}
static void test_discovery_has_no_repeaters() {
const std::string expected = "{\"startingAddress\":77,\"rooms\":[" + roomJson(2) + "," + roomJson(3) +
"],\"shades\":[" + shadeJson(1) + "," + shadeJson(4) +
"],\"groups\":[" + groupWithLinked() + "]}";
TEST_ASSERT_EQUAL_STRING(expected.c_str(), document(snapshot_doc_t::discovery).c_str());
}
static void test_controller_of_an_empty_model() {
snap = model_snapshot_t{};
TEST_ASSERT_EQUAL_STRING("{\"startingAddress\":0,\"rooms\":[],\"shades\":[],\"groups\":[],\"repeaters\":[]}",
document(snapshot_doc_t::controller).c_str());
}
static void test_chunk_size_does_not_change_the_document() {
const std::string whole = document(snapshot_doc_t::controller, 0, 64);
for(size_t n = 1; n < 20; n++)
TEST_ASSERT_EQUAL_STRING(whole.c_str(), document(snapshot_doc_t::controller, 0, n).c_str());
}
static void test_single_items() {
TEST_ASSERT_EQUAL_STRING(roomJson(3).c_str(), document(snapshot_doc_t::room, 3).c_str());
TEST_ASSERT_EQUAL_STRING(shadeJson(4).c_str(), document(snapshot_doc_t::shade, 4).c_str());
TEST_ASSERT_EQUAL_STRING(groupWithLinked().c_str(), document(snapshot_doc_t::group, 6).c_str());
}
static void test_a_single_item_gone_by_the_snapshot_is_an_error() {
TEST_ASSERT_EQUAL_STRING("{\"status\":\"ERROR\",\"desc\":\"Shade Id not found.\"}", document(snapshot_doc_t::shade, 9).c_str());
TEST_ASSERT_EQUAL_STRING("{\"status\":\"ERROR\",\"desc\":\"Group Id not found.\"}", document(snapshot_doc_t::group, 9).c_str());
TEST_ASSERT_EQUAL_STRING("{\"status\":\"ERROR\",\"desc\":\"Room Id not found.\"}", document(snapshot_doc_t::room, 9).c_str());
}
static void test_an_item_too_big_for_the_unit_is_left_out() {
// Room JSON is about 40 bytes and a shade's well over 300, so a 200 byte
// unit takes the rooms and leaves the shades out, keeping the JSON valid.
TEST_ASSERT_EQUAL_STRING(("[" + roomJson(2) + "," + roomJson(3) + "]").c_str(),
document(snapshot_doc_t::rooms, 0, 7, 200).c_str());
TEST_ASSERT_EQUAL_STRING("[]", document(snapshot_doc_t::shades, 0, 7, 200).c_str());
}
static void test_a_left_out_item_does_not_leave_a_stray_comma() {
// Give the first shade every linked remote so it no longer fits a unit the
// second one does.
snap.shades[0].linkedRemoteCount = SOMFY_MAX_LINKED_REMOTES;
for(uint8_t i = 0; i < SOMFY_MAX_LINKED_REMOTES; i++) snap.shades[0].linkedRemotes[i] = { 4000000000u, 65535 };
const size_t fits = shadeJson(4).length() + 8;
TEST_ASSERT_TRUE(shadeJson(1).length() > fits);
TEST_ASSERT_EQUAL_STRING(("[" + shadeJson(4) + "]").c_str(), document(snapshot_doc_t::shades, 0, 7, fits).c_str());
}
/*********************************************************************
* Slots
********************************************************************/
static SnapshotSlots slots;
static int fills = 0;
static void countFill(model_snapshot_t &s) {
fills++;
s.shadeCount = 1;
}
static void test_two_reads_at_once_and_no_more() {
const int8_t a = slots.acquire();
const int8_t b = slots.acquire();
TEST_ASSERT_TRUE(a >= 0 && b >= 0 && a != b);
TEST_ASSERT_EQUAL_INT8(-1, slots.acquire());
slots.cancel(a);
slots.cancel(b);
}
static void test_a_read_is_filled_then_freed() {
fills = 0;
const int8_t s = slots.acquire();
TEST_ASSERT_FALSE(slots.filled(s));
slots.fill(s, countFill);
TEST_ASSERT_TRUE(slots.filled(s));
TEST_ASSERT_EQUAL_UINT8(1, slots.snapshot(s).shadeCount);
slots.release(s);
TEST_ASSERT_EQUAL_INT8(s, slots.acquire());
slots.cancel(s);
TEST_ASSERT_EQUAL_INT(1, fills);
}
static void test_a_read_gone_before_its_fill_is_not_filled() {
fills = 0;
const int8_t s = slots.acquire();
const int8_t other = slots.acquire();
slots.release(s);
// Not free yet: loop() still has the fill queued.
TEST_ASSERT_EQUAL_INT8(-1, slots.acquire());
slots.fill(s, countFill);
TEST_ASSERT_EQUAL_INT(0, fills);
TEST_ASSERT_EQUAL_INT8(s, slots.acquire());
slots.cancel(s);
slots.cancel(other);
}
static void test_a_cancelled_slot_is_free_again() {
const int8_t s = slots.acquire();
slots.cancel(s);
TEST_ASSERT_EQUAL_INT8(s, slots.acquire());
slots.cancel(s);
}
int main(int argc, char **argv) {
(void)argc;
(void)argv;
UNITY_BEGIN();
RUN_TEST(test_a_shade);
RUN_TEST(test_a_shade_ref);
RUN_TEST(test_a_room);
RUN_TEST(test_a_group_ref);
RUN_TEST(test_names_are_escaped);
RUN_TEST(test_rooms_list);
RUN_TEST(test_shades_list);
RUN_TEST(test_groups_list_skips_a_linked_shade_that_is_gone);
RUN_TEST(test_empty_lists);
RUN_TEST(test_controller);
RUN_TEST(test_discovery_has_no_repeaters);
RUN_TEST(test_controller_of_an_empty_model);
RUN_TEST(test_chunk_size_does_not_change_the_document);
RUN_TEST(test_single_items);
RUN_TEST(test_a_single_item_gone_by_the_snapshot_is_an_error);
RUN_TEST(test_an_item_too_big_for_the_unit_is_left_out);
RUN_TEST(test_a_left_out_item_does_not_leave_a_stray_comma);
RUN_TEST(test_two_reads_at_once_and_no_more);
RUN_TEST(test_a_read_is_filled_then_freed);
RUN_TEST(test_a_read_gone_before_its_fill_is_not_filled);
RUN_TEST(test_a_cancelled_slot_is_free_again);
return UNITY_END();
}