Testing & Release Engineering · September 8, 2026
September 8, 2026 — Testing, Packaging & Release Discipline
A layered approach to unit tests, packaging, live validation, soak testing, and release discipline.
Different tests catch different failures
No single test proves an application is correct.
I now think in layers.
1. Focused unit/regression tests
Run the smallest relevant tests while developing a fix.
Purpose:
- fast feedback
- prove the bug exists
- prove the specific fix works
2. Full unit suite
python3 -m unittest discover -s tests -v
Purpose:
- catch regressions outside the changed module
3. Compilation/import validation
python3 -m compileall -q src tests
Purpose:
- syntax errors
- import problems
- code paths not reached by a focused test
4. Diff hygiene
git diff --check
Purpose:
- whitespace errors
- malformed patch details
Small command, useful habit.
5. Package build
python3 -m pip wheel . --no-deps --no-build-isolation -w /tmp/wheels
A source checkout can succeed while the wheel omits required assets.
6. Package audit
When an application ships data files, inspect the built package.
Questions:
- are all required assets present?
- are obsolete assets absent?
- does the packaged manifest match the source tree?
- are frame counts correct?
For exact asset identity, hashes or byte comparisons are stronger evidence than filenames.
7. Live smoke test
A GUI program needs a real launch test.
Unit tests cannot prove:
- the compositor accepts the window
- transparency looks right
- clicks actually reach the UI
- a popover releases an input grab
- animations feel correct
8. Soak test
A longer live run can reveal:
- timer accumulation
- memory/resource growth
- state that only gets stuck after repeated cycles
- intermittent warnings
Known-good checkpoints
A useful checkpoint records:
- commit SHA
- test count/results
- package build result
- live validation result
- important asset/runtime state
That makes later recovery much easier than saying “it worked earlier today.”
Release metadata must agree
Version strings in README, package metadata, tags, and releases should be reconciled before shipping.
Conflicting versions make debugging installations harder.
Main lesson
“It works on my checkout” is not a release standard. A trustworthy change survives focused tests, the full suite, packaging, and live use.
Regression testing grows with feature composition
As a project gains long-running features, the highest-risk failures move to boundaries between systems.
Instead of testing only:
Feed works
Focus works
Drag works
I also need combinations:
Focus → Drag → Focus resumes
Focus → Sleep → session pauses
Feed → level-up → recovery
Catalogue → bond update → refresh
Shutdown → pending reward → persistence
A regression watchlist is useful because these combinations are easy to forget even when individual unit suites are strong.
Release documentation is part of the release
I learned that code can be ready while the repository still tells an old story.
Before a release, audit:
- README
- changelog
- package version
- runtime version
- roadmap/current-status docs
- regression checklist
- known issues
- install/update instructions
- historical docs that might look current
If documentation says “do not add XP” after XP has intentionally shipped, that is a release defect even if the code works.
Environment-specific skips need accurate reporting
A test skipped because it requires a real session bus or compositor is not evidence that the behavior passed.
Release notes should distinguish:
- automated test passed
- test skipped because environment unavailable
- package/import validation passed
- live desktop QA still required
That keeps confidence grounded in what was actually verified.
Test the second launch, not only the first launch
A desktop application can pass a clean first launch and still fail when activated a second time.
Mochi updater QA exposed a case where a second open could leave the app transparent/frozen.
That makes these separate smoke tests:
cold launch
close/reopen
second activation
restart after update
restart after failure/recovery
GTK/Gio application identity, surviving process state, stale window references, compositor state, and restart sequencing can all make later activations behave differently from the first one.
“Launches successfully once” is therefore weaker evidence than it looks.
An updater is a release-and-recovery protocol
A built-in updater should be designed as a stateful transition, not a download button.
Useful stages are:
check version
→ reject unsafe downgrade
→ stage update
→ validate staged result
→ restart/switch
→ confirm healthy launch
→ rollback/recover if needed
Important properties include:
- do not destroy the known-good version before a replacement is ready
- keep enough information to recover
- guard against unintended downgrades
- make restart behavior testable
- test interruption/failure paths, not just the happy path
The updater belongs to release engineering because it controls how a known-good installation becomes a new one without trapping the user in a broken state.
Test the production composition, not only isolated parts
Mochi exposed a regression where individual behavior worked in isolation but failed when the real mixin hierarchy was assembled.
The failure came from method-resolution order: an earlier mixin consumed a hook before a later mixin could act on it.
That means composition itself is something I need to test.
For mixin-heavy, plugin-based, middleware, or dependency-injected systems, a useful regression ladder is:
isolated unit
→ subsystem composition
→ production object composition
→ live runtime
If the bug appeared only in the production object, a test that instantiates a simplified fake composition is weaker evidence.
Headless success is not visual-runtime success
The Mochi Lab appearance work also reinforced a confidence boundary.
A large non-display suite can verify:
- attachment metadata
- frame coverage
- fallback behavior
- package contents
- rendering logic that does not require a compositor
It still cannot prove:
- alignment feels correct during drag
- accessories visually crowd a pose
- transparency/compositing looks right
- GTK/Wayland behavior matches the desktop session
Automated coverage and live visual QA should be reported separately rather than blended into one “tests passed” claim.