assert, and tests as a table

What you will be able to do

After this lesson you can

• explain why a program that seems to work may still be wrong,
• use assert to check input and to write a quick test,
• write a table of tests and a loop that runs it, and
• choose the cases that find the most bugs.

Seems to work

You wrote a function. You tried it with an input or two, and the answers looked right. Is it correct? Not necessarily. The bugs hide in the inputs that you did not try: the empty list, the edge of a range, the number that is exactly 75. A test is how you find out. It runs the code on an input, and compares the answer with one that you worked out in advance, from the rules, never from the code.

Why not from the code? If you copy the answer that the code gives, then a bug is copied into the test, and the test says "all is well". The expected answer must come from your understanding of what the function should do.

assert

Python has a statement made for checks: assert. It says: "this must be True". If it is not, the program stops with an AssertionError. You can add a message after a comma.

Predict first. Which of the four lines produce a message? Then run it.

• assert average([2, 4, 6]) == 4 is a test. It passes, so nothing is shown. A passing assert is silent.
• The assert len(numbers) > 0, "average of an empty list" inside the function is a check on the input. If somebody calls average([]), the function stops with a clear message. (For input from a user, use a ValueError as you learned. An assert is for mistakes of the programmer, not of the user.)
• The last test fails on purpose: average([1, 2]) is 1.5, not 2. We caught the AssertionError to show the message. In your tests, you would let it stop the program, so that you see it.

What happens when an assert fails and nobody catches it?

One caution: Python can be started with a flag that switches asserts off, so they are for testing, never for the real work of a program, such as validating a password.

A table of tests

A pile of asserts stops at the first failure. You would like to see all the failures at once. For this, use a table of tests: a list of cases, and a loop.

Choose cases that would catch the likely bugs: zero, a negative number, the edges, and an ordinary case.

Which cases?

You cannot test every input. A good set is small and catches most bugs.

Eight to twelve cases usually catch most bugs.

Common mistakes

• Taking the expected answer from the code. The test cannot find a bug that it copied.
• Testing only the usual cases. The bugs are at the edges.
• Using assert for user input. Asserts can be switched off. Use ValueError.
• Stopping at the first failure when a table would show all of them.
• Not running the tests again after a change. A fix can break a case that worked.

Practice

Start with two quick questions. Each New numbers gives you a fresh one, so you can practise as long as you like.

Then the labs and the problems:

1. Warm-up. Open the lab Fill the table. You write the expected answers for a function, from its rules.
2. Build. Open the problem Write the asserts and the problem Which cases?
3. Stretch. Open the lab Catch the bugs and write a table of cases that finds three hidden bugs.

Remember these

• A test compares the answer of the code with an answer worked out from the rules.
• assert condition, "message" stops with an AssertionError when the condition is False. It is silent when True.
• A table of tests and a loop show all the failures at once.
• Test one in each branch, both sides of each edge, the ends, and the empty case.

Check yourself

Try each one on your own first. If you are stuck, open a hint. Then check with Show the answer.

1. What does a passing assert show?

2. Why must the expected answer not be copied from the code?

3. Which cases would find the bug of mark > 75 instead of mark >= 75?