-
1.4Release: 1.4 eleasee8668a21 · ·
## [1.4] — 2026-03-17 ### New Features #### `docker_exec` Step Standalone step for executing commands inside Docker containers — a compact shorthand for `docker_op: exec` with full `save`/`save_regex`/`save_map` support: ```yaml - docker_exec: "cat /opt/app/status.txt" services: ["clientd-1", "clientd-2"] expect: success save: status_output save_regex: "state=(\\w+)" ``` #### `save_regex` / `save_map` — Structured Output Extraction CLI, RPC, Bash, and docker_exec steps now support regex-based extraction: ```yaml - cli: "net srv -net {{network_name}} dump" node: node-1 save: raw_output save_regex: "address:\\s+(\\S+)" save_map: order_hash: "hash:\\s+(0x[a-f0-9]+)" price: "price:\\s+(\\d+\\.\\d+)" ``` Centralized via `RuntimeContext.save_output()` — supports first capture group or full match fallback, with warning on no-match. #### `wait: docker_exec` Polling Poll a command inside a Docker container until a condition is met: ```yaml - wait: docker_exec cmd: "test -S /opt/app/run/app.sock && echo ready" services: ["clientd-1"] contains: "ready" timeout: 20 interval: 1 ``` #### `build` / `devices` / `volumes` in `docker_compose_gen` Generated Docker Compose services now support custom image builds, device mappings, and volume mounts. The first generated service carries the full `build` definition; siblings reuse the image via `pull_policy: never`: ```yaml docker_compose_gen: - prefix: kelvpn-client count: "{{client_count}}" build: context: ../docker dockerfile: Dockerfile.client devices: ["/dev/net/tun:/dev/net/tun"] volumes: ["../configs:/etc/app:ro"] cap_add: [NET_ADMIN] environment: APP_MODE: client ``` `INSTANCE_ID` env var is automatically injected (1, 2, …N). #### `env_file` for Suites Suite descriptor supports `env_file` to inject environment variables into all provisioned nodes: ```yaml env_file: .env.stage ``` #### `docker_devices` / `environment` in Node Specs `NodeComposeSpec` and `NodeComposeGenSpec` now accept `docker_devices` and `environment` fields, and `cap_add` supports a `docker_capabilities` YAML alias: ```yaml node_compose: vpn-server-1: role: full docker_capabilities: [NET_ADMIN, SYS_PTRACE] docker_devices: ["/dev/net/tun:/dev/net/tun"] environment: VPN_MODE: server ``` #### `param_map` in `ConfigStep` Conditionally populate config keys from truthy scenario variables: ```yaml - config: "cellframe-node.cfg.d/debug-overrides.cfg" nodes: "{{_all_svc}}" param_map: DEBUG_REACTOR: "general.debug_reactor" DEBUG_HTTP: "general.debug_http" ``` Only variables with truthy values (`"true"`, `"1"`, `"yes"`) generate entries. If none match, the step is silently skipped. #### Generalized `cfg.d` Paths Config step supports arbitrary `.cfg.d/` paths, not just `cellframe-node.cfg.d/`: ```yaml - config: "network/vpntest.cfg.d/role.cfg" nodes: ["node-1"] set: role: "master" ``` #### `--param` / `-P` CLI Option for `run_tests` Pass ad-hoc KEY=VALUE parameters from the command line directly into scenario variables: ```bash stage_env.py run-tests tests/ -P vpn_client_count=5 -P debug=true ``` #### `--yes` / `-y` Flag for `start --clean` Skip the interactive confirmation prompt in CI pipelines: ```bash stage_env.py start --clean --yes ``` #### Per-Role / Per-Node Docker Customizations New `[role_docker_*]` and `[node_docker_*]` sections in `stage-env.cfg` for declarative Docker configuration: ```ini [role_docker_root] capabilities = NET_ADMIN,SYS_PTRACE devices = /dev/net/tun:/dev/net/tun environment = VPN_DEBUG=1 [node_docker_3] capabilities = NET_RAW extra_config = {"privileged": true} ``` #### CI Pipeline Improvements - SIGTERM/SIGHUP handler — `finally` blocks run on pipeline cancellation - PID file moved to `/tmp/` — system-wide lock, writable without root - Temp config support — avoid writing to read-only `/opt/` - Path redirection to CI work directory ### Bug Fixes - **Image build race** — build ALL missing images before `docker compose up`, not just the first service; correctly detect `build` directives - **Zombie prevention** — `init: true` added to all Docker containers - **Monotonic timeouts** — `wait: cli` and `wait: bash` polling loops use `time.monotonic()` with dynamic per-attempt timeouts instead of accumulated `elapsed += interval` - **Template-variable timeouts** — `timeout: "{{var}}"` resolved via `_resolve_timeout()` across all wait types - **curl fail-fast** — `-f` flag added to mirror_tester HTTP requests - **Absolute cache_dir** — normalized to relative path for Docker volumes - **Default base_http_port** — changed to 9079 to avoid collisions - **SnapshotManager init** — fixed constructor argument ordering bug - **Python 3.9 compatibility** — removed 3.10+ syntax constructs - **cfg.d routing** — any path containing `.cfg.d/` is handled as INI snippet (not just `cellframe-node.cfg.d/`) ### Stats - **24 files** changed - **+833 / −127** lines - **70 commits** --- -
1.3Release: [1.3] 2026-03-1030ef9c66 · ·
Changelog: stage-env 1.2 \u2192 1.3 52 \u043a\u043e\u043c\u043c\u0438\u0442\u0430, 80 \u0444\u0430\u0439\u043b\u043e\u0432 \u0438\u0437\u043c\u0435\u043d\u0435\u043d\u043e (+5266 / -1438 \u0441\u0442\u0440\u043e\u043a) \u041d\u043e\u0432\u044b\u0435 \u0432\u043e\u0437\u043c\u043e\u0436\u043d\u043e\u0441\u0442\u0438 (Features) Event-driven waiters \u0447\u0435\u0440\u0435\u0437 notify server \u0441 polling fallback \u2014 \u043c\u043e\u043d\u0438\u0442\u043e\u0440\u0438\u043d\u0433 \u0441\u043e\u0431\u044b\u0442\u0438\u0439 \u043d\u043e\u0434 \u0447\u0435\u0440\u0435\u0437 push-\u0443\u0432\u0435\u0434\u043e\u043c\u043b\u0435\u043d\u0438\u044f \u0432\u043c\u0435\u0441\u0442\u043e \u0447\u0438\u0441\u0442\u043e\u0433\u043e \u043f\u043e\u043b\u043b\u0438\u043d\u0433\u0430 (notify_client.py, waiters.py) Docker capabilities/devices \u2014 \u043f\u043e\u0434\u0434\u0435\u0440\u0436\u043a\u0430 docker_capabilities \u0438 devices \u0432 compose, \u0440\u0430\u0441\u0448\u0438\u0440\u0435\u043d\u0438\u0435 test_commands.py Config deploy \u0432 \u043a\u043e\u043d\u0442\u0435\u0439\u043d\u0435\u0440 \u2014 \u043a\u0430\u0441\u0442\u043e\u043c\u043d\u044b\u0435 \u0448\u0430\u0431\u043b\u043e\u043d\u044b, regenerate_cert, metrics-\u043e\u043f\u0435\u0440\u0430\u043d\u0434 (schema_metrics.py, step/metrics.py) \u0420\u0430\u0441\u0448\u0438\u0440\u0435\u043d\u0438\u0435 \u0434\u0432\u0438\u0436\u043a\u0430 \u0441\u0446\u0435\u043d\u0430\u0440\u0438\u0435\u0432 \u2014 custom operands, bash steps, cert generation \u0432 YAML-\u0441\u0446\u0435\u043d\u0430\u0440\u0438\u044f\u0445 \u041f\u043e\u0434\u0434\u0435\u0440\u0436\u043a\u0430 v5.8 \u2014 \u043e\u0442\u0434\u0435\u043b\u044c\u043d\u044b\u0435 genesis-\u0441\u0446\u0435\u043d\u0430\u0440\u0438\u0438 \u0438 \u043a\u043e\u043d\u0444\u0438\u0433\u0438 \u0434\u043b\u044f v5.8 (smart polling \u0432\u043c\u0435\u0441\u0442\u043e \u0441\u0442\u0430\u0442\u0438\u0447\u0435\u0441\u043a\u0438\u0445 wait) \u041c\u0443\u043b\u044c\u0442\u0438\u0432\u0435\u0440\u0441\u0438\u043e\u043d\u043d\u043e\u0435 \u043e\u043f\u0440\u0435\u0434\u0435\u043b\u0435\u043d\u0438\u0435 \u2014 \u0430\u0432\u0442\u043e\u0434\u0435\u0442\u0435\u043a\u0442 \u0432\u0435\u0440\u0441\u0438\u0438 genesis-\u0441\u0446\u0435\u043d\u0430\u0440\u0438\u0435\u0432 (v5.7/v5.8/v6.x) + stable-\u0441\u0438\u043c\u043b\u0438\u043d\u043a\u0438 node_compose + docker_compose \u2014 \u0434\u0438\u043d\u0430\u043c\u0438\u0447\u0435\u0441\u043a\u043e\u0435 \u0441\u043e\u0437\u0434\u0430\u043d\u0438\u0435 \u043d\u043e\u0434 \u0438\u0437 \u0442\u0435\u0441\u0442\u043e\u0432\u044b\u0445 \u0441\u044c\u044e\u0442\u043e\u0432 Self-contained cert-generator \u2014 \u0433\u0435\u043d\u0435\u0440\u0430\u0446\u0438\u044f \u0441\u0435\u0440\u0442\u0438\u0444\u0438\u043a\u0430\u0442\u043e\u0432, JSON CLI parsing, \u0440\u0430\u0441\u0448\u0438\u0440\u0435\u043d\u0438\u0435 eval namespace \u0418\u043d\u0442\u0435\u0440\u043d\u0430\u043b\u0438\u0437\u0430\u0446\u0438\u044f (i18n) \u2014 \u0440\u0443\u0441\u0441\u043a\u0430\u044f \u043b\u043e\u043a\u0430\u043b\u0438\u0437\u0430\u0446\u0438\u044f, i18n.py + translations_ru.py (336 \u0441\u0442\u0440\u043e\u043a) \u0414\u0435\u0442\u0430\u043b\u044c\u043d\u044b\u0439 \u0432\u044b\u0432\u043e\u0434 \u0442\u0435\u0441\u0442\u043e\u0432 \u2014 backtrace \u0434\u043b\u044f \u0443\u043f\u0430\u0432\u0448\u0438\u0445 waiters, \u043f\u043e\u0434\u0440\u043e\u0431\u043d\u044b\u0439 \u043e\u0442\u0447\u0451\u0442 \u0432 \u043a\u043e\u043d\u0446\u0435 \u0442\u0435\u0441\u0442\u0438\u0440\u043e\u0432\u0430\u043d\u0438\u044f \u0420\u0435\u0444\u0430\u043a\u0442\u043e\u0440\u0438\u043d\u0433 \u0413\u0440\u0430\u043d\u0443\u043b\u044f\u0440\u043d\u044b\u0435 Docker volume mounts \u2014 \u043d\u0430\u0441\u0442\u0440\u0430\u0438\u0432\u0430\u0435\u043c\u044b\u0435 \u0447\u0435\u0440\u0435\u0437 node_var_subdirs \u0423\u0434\u0430\u043b\u0435\u043d\u0438\u0435 \u0445\u0430\u0440\u0434\u043a\u043e\u0434\u0430 stagenet \u2014 \u0430\u0432\u0442\u043e\u0434\u0435\u0442\u0435\u043a\u0442 genesis version, regex \u0432 contains Snapshot executor hardening \u2014 \u0443\u0441\u0438\u043b\u0435\u043d\u0438\u0435 \u0438 code review \u0444\u0438\u043a\u0441\u044b \u0420\u0435\u043e\u0440\u0433\u0430\u043d\u0438\u0437\u0430\u0446\u0438\u044f \u0448\u0430\u0431\u043b\u043e\u043d\u043e\u0432 \u2014 v5.x \u2192 v5.7, \u0434\u043e\u0431\u0430\u0432\u043b\u0435\u043d v5.8, stable-\u0441\u0438\u043c\u043b\u0438\u043d\u043a\u0438 Boolean normalization \u0432 \u043a\u043e\u043d\u0444\u0438\u0433\u0430\u0445 \u0418\u0441\u043f\u0440\u0430\u0432\u043b\u0435\u043d\u0438\u044f (Fixes) \u0423\u0434\u0430\u043b\u0451\u043d \u043c\u0451\u0440\u0442\u0432\u044b\u0439 is_add_directly \u0438\u0437 ESBocs chain template \u0438 \u0434\u043e\u043a\u0443\u043c\u0435\u043d\u0442\u0430\u0446\u0438\u0438 \u0414\u0438\u043d\u0430\u043c\u0438\u0447\u0435\u0441\u043a\u043e\u0435 \u0438\u043c\u044f \u0441\u0435\u0442\u0438 \u2014 \u0437\u0430\u043c\u0435\u043d\u0430 hardcoded stagenet defaults ForStep list variable handling, suite vars \u0432 setup, node alias resolution Mempool_auto_types \u2014 \u0440\u0430\u0441\u0448\u0438\u0440\u0435\u043d\u0438\u0435 \u0434\u043b\u044f zerochain, \u043e\u0431\u0440\u0430\u0431\u043e\u0442\u043a\u0430 paused \u043a\u043e\u043d\u0442\u0435\u0439\u043d\u0435\u0440\u043e\u0432 MDBX: wipe global_db \u043d\u0430 \u043a\u043b\u043e\u043d\u0438\u0440\u043e\u0432\u0430\u043d\u043d\u044b\u0445 \u043d\u043e\u0434\u0430\u0445 (5.x), \u0443\u0434\u0430\u043b\u0435\u043d\u0438\u0435 stale mdbx.lck DAG-PoA: \u0443\u043c\u0435\u043d\u044c\u0448\u0435\u043d\u044b \u0442\u0430\u0439\u043c\u0430\u0443\u0442\u044b, fix chain sync logic Docker entrypoint \u0438 Dockerfile updates Python 3.9 \u0441\u043e\u0432\u043c\u0435\u0441\u0442\u0438\u043c\u043e\u0441\u0442\u044c \u0421\u043e\u0432\u043c\u0435\u0441\u0442\u0438\u043c\u043e\u0441\u0442\u044c \u0441 release-6.0 \u043d\u043e\u0434\u043e\u0439
-
1.2Release: 1.2 release22b1b1a6 · ·
## [1.2] — 2026-03-04 ### New Features #### Dynamic Node Provisioning from Suites (`node_compose`, `node_compose_gen`) Suites can add Cellframe nodes on top of the base topology without manually editing Docker Compose. `node_compose` — explicit per-node definitions: ```yaml node_compose: vpn-server-1: role: full source: genesis # genesis | node-<id> | fresh packages: "iperf3 tcpdump" setup: - cli: "srv_vpn enable" node: vpn-server-1 ``` `node_compose_gen` — parametric template-based generation: ```yaml node_compose_gen: - prefix: vpn-client count: "{{vpn_client_count}}" role: full source: genesis packages: "iperf3" ``` Expands to `vpn-client-1`, `vpn-client-2`, ...; IDs are assigned automatically after the last node in the base topology. #### Arbitrary Docker Services (`docker_compose`) Embed third-party containers (iperf, redis, grafana, etc.) directly in the suite descriptor: ```yaml docker_compose: iperf-server: image: networkstatic/iperf3 command: ["-s"] networks: vpntest: ipv4_address: 172.22.0.100 ``` #### Flexible Includes with Inline Variable Overrides `includes` now accepts both file paths and inline data blocks, allowing variable overrides between file inclusions: ```yaml includes: - ../common/vpn_defaults.yml # vpn_client_count: 3 - variables: # override vpn_client_count: 5 - ../common/vpn_nodes.yml # uses 5 ``` #### Self-Contained Cert-Generator Multi-stage `Dockerfile.cert-generator` clones SDK repos at build time. No host SDK mounts required at runtime — the binary is baked into the image. Configurable via `SDK_BRANCH`, `CELLFRAME_SDK_REPO`, `DAP_SDK_REPO`. `CMakeLists.txt` simplified: links against `cellframe_sdk` and `dap-sdk` shared libs instead of manually listing 15+ static libraries. #### Glob Patterns for Local Packages `local_path` in config now supports shell globs (`*.deb`, `build-*/*.deb`). Picks the latest match (sorted alphabetically) for deterministic results. #### JSON CLI Output Parsing (SDK 6.0+) `NodeStateManager` and `DockerComposeManager` parse JSON status format. Falls back to text format (5.x) on JSON parse failure — compatible with both SDK 5.x and 6.0+. #### Auto-Detection of Genesis Scenario Version The framework detects the Cellframe Node major version from the DEB package and automatically selects `v5.x/` or `v6.x/` genesis scenarios. Variables `cf_version_major`, `cf_version_minor` are available at runtime. #### Dynamic Network Name Removed hardcoded `stagenet`. `WaitForDatumStep.network` and `NetworkWaitStep.network` default to `"{{network_name}}"` — the name is taken from configuration. #### New `docker_op` Operations - `pause` / `unpause` — freeze/unfreeze a container to simulate connectivity loss - `logs` — retrieve logs with `tail:` (line count) and `save:` (into a variable) - `exec` — execute a command inside a container with `save:` ```yaml - docker_op: pause services: "node-1" - docker_op: logs services: "node-1" tail: 50 save: node_logs - docker_op: exec services: "node-1" command: "pgrep cellframe-node" save: pid_output ``` #### `snapshot` Extensions - `verify` operation — checks if a snapshot exists without raising an exception - `save:` field — stores the result (`True`/`False`) in a variable ```yaml - snapshot: verify name: cdb_ready save: has_cdb_snapshot ``` #### `wait` Extensions - `group:` / `key:` — wait for GlobalDB sync by group/key - `optional: true` — timeout does not raise an exception (for retry loops) #### `dpkg` Operand Install DEB packages on nodes directly from a scenario: ```yaml - dpkg: /path/to/package.deb nodes: ["node-1", "node-2"] ``` #### Custom Operands (`CustomOperandStep`) Extensible operand system — register new operands via `operand_registry` without modifying the framework core. #### `StepGroup`: `group:` / `name:` Alias Both keys are equivalent for naming a group: ```yaml - group: "Verify certs" # OK - name: "Verify certs" # also OK ``` #### Regex in `contains` CLI checks support regular expressions in `contains`: ```yaml - cli: "block list" contains: "have blocks: [1-9]" ``` ### Bug Fixes - **eval() Namespace Unification** — variables and builtins merged into a single dict for `eval()`. Expressions like `len(items) == 3` and `range(max_nodes)` now work correctly - **CLI Parser Always Initialized** — defaults applied even without `global_defaults.cli` - **CLI Defaults Applied Without Options Whitelist** — defaults added for commands not in the help cache - `expect_output:` (silently ignored by Pydantic) replaced with `contains:` across all genesis/e2e scenarios - `for: { var: x, in: [...] }` syntax fixed to correct `for: x` / `in: [...]` - `expect:` removed from `check:` sections (CLICheck does not support this field) - `ToolStep` — added variable substitution for `node` (`ctx.substitute`) - `Validator` — fixed handling of `SectionConfig`/`CheckSectionConfig`, added guard for `SuiteDescriptor` - CLI check executor — catches `asyncio.TimeoutError` instead of `subprocess.TimeoutExpired` - RPC step — `ctx.substitute()` called only for string parameters - Bash step — guard against `None` timeout (fallback to 300s) - Boolean value normalization in configs ### Configuration Templates Reorganized for multi-version support: ``` config/templates/ ├── v5.x/cellframe-node.cfg.j2 ├── v6.x/cellframe-node.cfg.j2 ├── network.cfg.j2 (formerly stagenet.cfg.j2) └── chains/ ├── main.cfg.j2 └── zerochain.cfg.j2 ``` -