Bytehub

Appearance

Testing ​

Used to check that a system behaves as expected:

  • gives correct results.
  • does not crash.

V-Model ​

Types of Tests ​

  • System test
    • complete system.
    • test functional behavior (use cases)
  • Unit test
    • class, method (in isolation)
  • Acceptance test
  • Performance test
  • Stress test
  • Platform test
  • Usability test
  • Integration test

Test Plan ​

Goal

  • Systematically test whether the system or unit
  • Behaves as expected if used in specified way Contents
  • List of execution scenarios → test cases
  • How the tests should be carried out
  • What the expected result is.

System Testing ​

  • Use cases to derive test cases
  • consider
    • Alternative flows in use cases
    • Different values used in use cases
    • Additional requirements
  • System treated as black box
    • Internal structure is unknown
    • Validation by observing behaviour

Unit Testing ​

  • One test plan for each class to be tested
    • Multiple test cases for each method
  • Consider method contracts (preconditions and postconditions)
  • Treat method as white box
    • Internal structure is known
    • Validation by inspecting method result and/or instance/class variables

Testing for unexpected situations ​

  • Primarily test for correct behavior
    • Input within specified bounds
    • Actions occur in the expected order
  • Sometimes we should also test robustness
    • Nothing bad happens when input is wrong

Testing and development process ​

  • Iterative process
    • Develop tests before implementation. (test-driven development) based on the specifications.
    • Perform unit tests after functionality is complete.
    • Extending the implementation requires extending the test plan.
    • Test results lead to further coding.
    • Test coverage results lead to extended test plan.
  • Regression
    • Bugs may be introduced in previously correct code.
    • rerun tests on a regular basis (regression testing)
  • Philosophies:
    • code then test
    • write tests then write code (test-driven development TDD)

Test Quality ​

  • Tests passing does not mean that the code is bug free, because realistically you cannot test all possibilities.
  • How to ensure tests have appropriate quality
    • Systematically develop tests based on class specifications
    • Apply best practices:
      • many test classes
      • small test cases
      • meaningful test case names
    • carry out test reviews.

Code Coverage ​

Code coverage can be measured in several ways:

  • Statement coverage: each statement is executed at least once.

  • Branch coverage: each branch is traversed (and every entry point is taken) at least onvce

  • Path coverage: all program paths are traversed at least once.

  • 100% coverage is difficult to achieve especially for path coverage: too many test cases would be needed.

  • Examples of hard to cover code.

    • Synthetic code generated by compiler
    • code that handles exceptional situations
    • conditions that combine cases
    • test code

Hints:

  • try to achieve as high statement coverage as possible (e.g > 80%)
  • keep testing in mind while designing code: TDD
  • you may have to refactor code to make it easier to test.