ci: coverage reports for the native and web UI tests

A coverage env (the native env with gcov counters) runs the native
tests in CI; gcovr writes the HTML report and a table on the run's
summary page. The web UI tests report coverage with Node's built-in
coverage, and genhtml turns its lcov into HTML. Both reports are run
artifacts, and pushes to main publish them to GitHub Pages.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
cjkas 2026-10-31 12:07:58 +01:00
parent db73190653
commit bd399db73d
5 changed files with 117 additions and 4 deletions

View file

@ -24,11 +24,27 @@ jobs:
# pioarduino's fork of PlatformIO Core, which the pioarduino platform needs. # pioarduino's fork of PlatformIO Core, which the pioarduino platform needs.
# Pinned: 6.2.0.post1 reports itself as 6.2.0-post.1, which the # Pinned: 6.2.0.post1 reports itself as 6.2.0-post.1, which the
# platform's ">=6.2.0" reads as a pre-release and refuses. # platform's ">=6.2.0" reads as a pre-release and refuses.
- name: Install PlatformIO - name: Install PlatformIO and gcovr
run: pip install pioarduino==6.2.0 run: pip install pioarduino==6.2.0 gcovr==8.8
# The coverage env is the native env with gcov counters: the same
# tests, measured.
- name: Run native unit tests - name: Run native unit tests
run: pio test -e native run: pio test -e coverage
# gcovr.cfg picks the files; the table goes to the run's summary page.
- name: Coverage report
run: |
mkdir -p coverage
gcovr --html-details coverage/index.html \
--markdown "$GITHUB_STEP_SUMMARY" --markdown-title "Native tests coverage"
- name: Upload coverage report
uses: actions/upload-artifact@v4
with:
name: coverage-cpp
path: coverage/
retention-days: 14
webui: webui:
name: Web UI tests name: Web UI tests
@ -44,8 +60,82 @@ jobs:
node-version: "22" node-version: "22"
# The protocol 2 client (data-src/js) against the recorded fixtures. # The protocol 2 client (data-src/js) against the recorded fixtures.
# Coverage: a table in the log, and lcov for the HTML report.
- name: Run web UI tests - name: Run web UI tests
run: node --test "test/js/*.test.js" run: |
mkdir -p coverage
node --test --experimental-test-coverage \
--test-coverage-include="data-src/js/**" \
--test-reporter=spec --test-reporter-destination=stdout \
--test-reporter=lcov --test-reporter-destination=coverage/lcov.info \
"test/js/*.test.js"
- name: Coverage report
run: |
sudo apt-get update -qq
sudo apt-get install -y -qq lcov
genhtml coverage/lcov.info --output-directory coverage/html \
--title "ESPSomfy-RTS web UI tests"
- name: Upload coverage report
uses: actions/upload-artifact@v4
with:
name: coverage-js
path: coverage/html/
retention-days: 14
# The coverage of main on the repo's GitHub Pages site
# (Settings > Pages > Source: GitHub Actions).
coverage-pages:
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
needs: [test, webui]
name: Publish coverage
runs-on: ubuntu-latest
permissions:
pages: write
id-token: write
environment:
name: github-pages
url: ${{ steps.deploy.outputs.page_url }}
concurrency:
group: pages
cancel-in-progress: false
steps:
- name: Download native tests coverage
uses: actions/download-artifact@v4
with:
name: coverage-cpp
path: site/cpp
- name: Download web UI tests coverage
uses: actions/download-artifact@v4
with:
name: coverage-js
path: site/js
- name: Index page
run: |
cat > site/index.html <<EOF
<!doctype html>
<meta charset="utf-8">
<title>ESPSomfy-RTS coverage</title>
<h1>ESPSomfy-RTS test coverage</h1>
<p>main at <a href="${{ github.server_url }}/${{ github.repository }}/commit/${{ github.sha }}">${GITHUB_SHA::7}</a>, $(date -u '+%Y-%m-%d %H:%M UTC')</p>
<ul>
<li><a href="cpp/">Native tests</a> (pio test -e native): only the sources the native env builds</li>
<li><a href="js/">Web UI tests</a> (data-src/js)</li>
</ul>
EOF
- name: Upload site
uses: actions/upload-pages-artifact@v3
with:
path: site
- name: Deploy to GitHub Pages
id: deploy
uses: actions/deploy-pages@v4
build: build:
# Don't build or publish firmware that fails its tests. # Don't build or publish firmware that fails its tests.

1
.gitignore vendored
View file

@ -12,6 +12,7 @@ esp32s3.svd
# Build, Logs and Output Folders # Build, Logs and Output Folders
build/ build/
coverage/
data/ data/
logs/ logs/
managed_components/ managed_components/

View file

@ -3,6 +3,8 @@
- Add or update tests for the code you change, even if nobody asked. - Add or update tests for the code you change, even if nobody asked.
- to test run `pio test -e native` - to test run `pio test -e native`
- the web UI's protocol 2 client (`data-src/js/`): `node --test "test/js/*.test.js"`. The minify step joins those files into `index.js`, so each one ends with a `module.exports` footer for Node and adds nothing global but its classes. - the web UI's protocol 2 client (`data-src/js/`): `node --test "test/js/*.test.js"`. The minify step joins those files into `index.js`, so each one ends with a `module.exports` footer for Node and adds nothing global but its classes.
- coverage of the native tests: `pio test -e coverage`, then `gcovr` (`pip install gcovr`), which reads `gcovr.cfg` and writes `coverage/coverage_details.html`. Files the native env doesn't build aren't in it. CI publishes the coverage of `main` to the repo's GitHub Pages.
- coverage of the web UI tests: add `--experimental-test-coverage --test-coverage-include="data-src/js/**"` to the `node --test` command.
## Building ## Building
- Firmware builds need pioarduino's Core (`pip install pioarduino`, or the pioarduino IDE extension). Stock PlatformIO Core refuses the pinned platform, see [ADR 0003](docs/adr/0003-pioarduino-idf55.md). - Firmware builds need pioarduino's Core (`pip install pioarduino`, or the pioarduino IDE extension). Stock PlatformIO Core refuses the pinned platform, see [ADR 0003](docs/adr/0003-pioarduino-idf55.md).

9
gcovr.cfg Normal file
View file

@ -0,0 +1,9 @@
# Coverage of the native tests: pio test -e coverage, then gcovr.
# Only the files the native env builds show up (build_src_filter);
# the rest of src/ needs the ESP32 and isn't measured.
filter = src/
object-directory = .pio/build/coverage
sort = uncovered-percent
print-summary = yes
html-title = ESPSomfy-RTS native tests
html-details = coverage/

View file

@ -161,3 +161,14 @@ build_flags =
-Isrc -Isrc
-Itest/stubs -Itest/stubs
-D_POSIX_THREAD_SAFE_FUNCTIONS -D_POSIX_THREAD_SAFE_FUNCTIONS
; The native tests with gcov counters, for the coverage report:
; pio test -e coverage && gcovr
; gcovr.cfg sets what the report covers and where it goes.
[env:coverage]
extends = env:native
build_flags =
${env:native.build_flags}
--coverage
-O0
-lgcov