Improving Software: Automated Testing

Building testing after the code is written and deployed is hard and expensive, so why?

Feedback

This is the goal of all testing to get direct results from inputs and does it match the expected outputs

Reduce

Change risk and remove problems

Improve

Understanding of internal code structure

Team Effort

• One person writing the tests opinionates them too much and will miss important cases
• Tests communicate intent between code and people
• Maintaining a test suite takes substantial effort, distributing it makes it fair and values others input
• Tests must deliver value to those that write the code not users.  Users value the application not its test suite

Good tests

• Reliable
• Easy, simple, fast as possible to run (single command, click, etc)
• Consistent known inputs and expected outcomes
• Gives readable and actionable information about failures

Test Values Unit tests < Integration Test

Unit tests

• Require mocks, stubs
• Information is not holistic
• Add in is high effort on already working completed code i.e. live deployed now

Integration tests

• Harder to write, debug, and slow
• Require infrastructure: database, etc.
• Validates state and flow of complete business logic processes

Test useful repetitive tasks

• Login, signup, payments …
• Use 80% cases vs 20% edge cases (Edge case testing can be done by manual tester until it falls into 80% flow)
• Tests should be automated to run with code and under source control

Integration Test Helpers

• Docker images
• Public APIs
• Use shortcuts (Cypress over Selenium because it can directly access databases)
• Use real data if available and applicable over fake
• Pick a modern test framework xUnit, Fixie, nUnit not MSTest