Test Strategy Preliminary Goals
Objective: Test provide meaning and are actionable
This means that all components that encompass at the level of the test { Unit, Integration, Application/Acceptance } provide detailed information within the system they are testing.
Primary Goals
- Should be repeatable in isolation
- Dockerization of common components possibility { Docker Compose Database Server }
- Coverage is on critical workflows
- Code and scripts of tests resides in same repository (otherwise code drift will cleave the components farther apart over time than the continents that drifted to form the Pacific Ocean)
- Automated and tied to build passing tests is required
- Input data is consistent in intent, but not content, randomization occurs on inputs when at all possible to hopefully identify risky inputs and situations
- Will run locally always as minimum
- Information and reports are readily available (at build level) as a metric for public view and distribution
- This could be reports in common formats such as CSV, TXT, PDF
- Images and recordings of tests are by default on and present
- Failed runs and outputs notify applicable committer and halt ability to incorporate affected code into protected branches (master+tags) until resolved
Extended Goals
- Decay of tests is automated so that tests knowledge and understanding is constantly updated: For example test checkin process will introduce limited degree of chaos, for example a git post commit hook (on non protected branches, excludes master+tags) or build process will delete code or move lines to create build and test failures every so random commits. Think of the above as introducing replication errors like in genetics which is what provides type checking, fitness, and every so often desired mutations on lifeforms