Test, verification and defect
Test specification, run, result, evidence and defect as a nonconformity claim. The remediation work item has an independent lifecycle.
Bundle → Layer → Finding → Questions Filled
2 bundles · 3 layers · 4 findings · 9 questions
Specification and run This bundle records the expected behaviour and the context of a test run.
Test specification
This layer relates a test specification to its expected behaviour and execution context.
Expected behaviour
This finding identifies the expected behaviour against which the test evaluates the object.
- Which expected behaviour or requirement does this test specification verify?
- Is the expected behaviour for this test specification recorded separately from its test code?
Test run
This finding records the tested version, environment and any flakiness assessment.
- Which version of the object under test and which environment did this run use?
- Was this run's result assessed as flaky, and how was that assessment recorded?
Results and defects This bundle records verification scope and claims of nonconformity.
Verification evidence
This layer binds verification evidence to the version actually tested.
Result scope
This finding records evidence scope and identifies any proposed reuse without treating it as a run on another version.
- Which exact version does this verification evidence apply to?
- Which evidence reference, tested source version and proposed target version are identified in this evidence reuse proposal?
Defect
This layer records a nonconformity claim separately from its remediation work item.
Defect claim
This finding compares observed and expected behaviour and records defect attribution and re-verification.
- What observed and expected behaviour are recorded in this defect claim?
- Does this defect claim attribute the failure to the product or to the test itself?
- Has the production defect identified in this claim been re-verified on the fixed version?
Classifiers Filled
- Family
- Enterprise profiles
- Category
- Enterprise subject
- Entry kind
- subject
- Domain
- EnterpriseTEC
- Industry
- Cross-industry
- Tags
- EM-TEC-08W1subjectTestSpecificationTestRunTestResultDefectVerificationEvidence
What it is Filled
Test specification, run, result, evidence and defect as a nonconformity claim. The remediation work item has an independent lifecycle.
Why it exists Filled
Test specification, run, result, evidence and defect as a nonconformity claim. The remediation work item has an independent lifecycle.
Distinguishing features Filled
- A flaky result is recorded as flaky and is never counted as a pass.
- Every run pins the object under test and its environment.
- A defect is a nonconformity claim with observed and expected behaviour; its fix is a separate work item.
- Distinct from incidents, which are disruptions in operation rather than nonconformities found by verification.
What robots and AI may and may not do Filled
Must not
- Close a production defect automatically because a test was fixed.
- Count a flaky run as passed.
- Apply a verification result to a version it did not run on.
- Delete failing tests to make a build green.
- Edit recorded results or evidence after the run.
Only with a human decision
- Releasing with known open defects of high severity.
- Waiving a required verification.
May
- Aggregate results by version and environment.
- Flag flaky tests and keep their runs separate from passes.
- Link defects to tests that reproduce them and to fixed versions.
Moral aspects Filled
- Unverified releases can harm users, especially in safety-relevant software.
- False pass reports shift risk onto users who cannot see it.
Who is affected
- Users of the software
- Developers and testers
- Release approvers
Owners Filled
Steward
The CTO, the CIO or the service owner accountable for the system.
Master systems
- Software catalogue
- Source control
- CI/CD pipeline
- CMDB
- Observability platform
Links to other meta-models Filled
neighbor
- EM-PRD-03
references
- WM-SFT-015 - conceptual-candidate
What else AI and robots need to interact with it Filled
Identity and identifiers required Filled
- Identified by a test specification identifier; runs by run identifier with version and environment; defects by defect identifier.
Direct properties not applicable Not applicable
Not applicable
Enterprise record contour: physical properties belong to referenced world models.
Recognition optional Filled
- A record names a test, a version under test, an environment and a result, or an observed and expected behaviour.
- Often confused with the remediation ticket or with an incident.
Capabilities and actions required Filled
- Several tests can be linked to one defect with separate conclusions.
- Re-verification on another version stays a separate result.
Hazards and failure modes required Filled
- Hidden flakiness that masks real failures.
- Defects closed by changing tests rather than the product.
- Results applied to the wrong version.
Standards and interfaces required Filled
- ISO/IEC/IEEE 29119 series for software testing.
- ISO/IEC 25010 product quality model.
- IEEE 1044 classification for software anomalies.
Context of use required Filled
- Used by engineering, QA and release management teams.
- Regulated domains such as medical and automotive software require traceable verification.
Sources Filled
- AISMM/WM-SFT and ArchiMate: product, system and architecture
- CSDM: business application, service and runtime instance
- SPDX/OpenTelemetry/Google SRE: delivery, observation and reliability; choose by boundary
- OMG UML Testing Profile (UTP) - Checked live (HTTP 200) on 2026-10-07.
- IEEE/ISO/IEC International Standard for Software and systems engineering--Software testing--Part 3: Test documentation - Carried over from the published world model wm-sft-015-test-case-test-result; checked live (HTTP 200) on 2026-10-09.
- IEEE/ISO/IEC International Standard - Software and systems engineering--Software testing--Part 4: Test techniques - Carried over from the published world model wm-sft-015-test-case-test-result; checked live (HTTP 200) on 2026-10-09.
- ISO/IEC/IEEE International Standard Software and systems engineering -- Software testing -- Part 5: Keyword-Driven Testing - Carried over from the published world model wm-sft-015-test-case-test-result; checked live (HTTP 200) on 2026-10-09.
Open questions
- How to separate expected behavior from the test implementation?
- When is a test failure a product defect, and when a test defect?
- How to link a verification to an exact version?
- Установить границу и решение reuse/extend/new по действующим спецификациям.
- Подтвердить semantic crosswalk, права и source mastership.
- Выбрать immutable refs; провести проверки fixtures до заявления о публикационной готовности.
Machine files
Provenance
enterprise research programme · published-partial
Built from: enterprise/models/em-tec-08/brief.json, ver-cy/world-models/card-supplements/em-tec-08.json
Still in Russian: suggested owner, blocking decisions, vercy candidates. Translate in research/enterprise/i18n/units.en.json.