Project: fix the library

What you will be able to do

After this lesson you can

• read a table of failing tests,
• find each bug with the method of the last lesson,
• fix one bug at a time and run all the tests again, and
• write tests of your own for a library.

The situation

A friend wrote a small library of four text functions, and says "it works". You do not believe it, and you have a table of thirteen tests. The functions:

1. word_count(text): the number of words.
2. average_word_length(text): the average length of a word. For no words, 0.0.
3. most_common_word(text): the most common word. When two words tie, the one that comes first in the alphabet. For no words, None.
4. is_palindrome(text): does the text read the same both ways, ignoring capitals, spaces and punctuation?

Step 1: run the tests

The tests are written from the rules above, not from the code.

Group the failures by function. Three come from word_count, and two from is_palindrome. It is likely that each function has one bug that shows in more than one test. This is a useful habit: read all the failures first, then group them, before you fix anything.

Step 2: find each bug

For each bug, use the loop: reproduce, locate, explain, fix one thing, test again.

Bug 1: word_count.

• Reproduce: word_count("the cat"), with two spaces, gives 3.
• Locate: text.split(" ") cuts at every single space. Two spaces make an empty word between them. An empty text gives [""], one empty word, so the count would be 1, not 0. A space at the start or the end adds an empty word too.
• Explain: "split(" ") makes empty words, so the count is too big."
• Fix: text.split() with no argument drops all the white space.

Bug 2: average_word_length.

• Reproduce: average_word_length("") raises a ZeroDivisionError.
• Locate: for no words, len(words) is 0, and the sum is divided by 0.
• Explain: "the empty text has no words, and the code divides by their number."
• Fix: check if not words: return 0.0 first. (This is the empty edge case.)

Bug 3: most_common_word.

• Reproduce: most_common_word("b a a b") gives b, and the rule says a.
• Locate: the loop goes through the words in alphabetical order, and keeps the best so far. The condition is counts[word] >= counts[best]: with greater or equal, a later word with the same count replaces the earlier one. So the last of the tie wins.
• Explain: ">= lets a later word replace an equal one, but the first in the alphabet should stay."
• Fix: use >, strictly more. (A wrong condition at an edge, from the table of the usual suspects.)

Bug 4: is_palindrome.

• Reproduce: is_palindrome("Level") gives False.
• Locate: the code compares the text with its reverse, exactly. The capital L is not equal to the small l, and the spaces and commas of a sentence are in the way.
• Explain: "the comparison does not ignore capitals, spaces or punctuation."
• Fix: clean the text first: keep only the letters, in small letters.

Step 3: the fixed library

Set the words to 0: the empty text gives one empty piece with split(" "), the very bug in word_count("").

Step 4: extend it

Pick one or two, and do them without looking at a solution.

1. More tests. Add three tests of your own that find bugs that these tests miss. Think of edge cases: punctuation inside words, "don't", capital letters in most_common_word.
2. A new function. longest_word(text), with a rule for a tie. Write the tests first, then the function.
3. A random test. Check is_palindrome against a trusted answer: for any text t, is_palindrome(t + reverse_of(t)) must be True.
4. Use unittest. Rewrite the table as a test class with assertEqual and assertRaises.
5. A property. word_count(text + " " + text) must be twice word_count(text). Is it?

Common mistakes

• Fixing a bug without understanding it. Explain it first, in a sentence.
• Fixing two bugs at once. You do not know which fix worked.
• Changing the tests to match the code. The tests come from the rules.
• Not running all the tests after a fix.
• Forgetting the empty text. Three of the four bugs touch an edge.

Practice

Start with two quick questions. Each New numbers gives you a fresh one, so you can practise as long as you like.

Then the labs and the problems:

1. Warm-up. Open the lab Project: fix the library. The checker runs your fixed functions on many cases, some of which the lesson tests do not cover.
2. Build. Do Step 4 for one extension.
3. Stretch. Open the lab Catch the bugs again, and write tests for the library so that all four bugs are found.

Remember these

• Write the tests from the rules, and run them before you trust the code.
• For each failing test: reproduce, locate, explain, fix one thing, test again.
• The tests do not change when you fix the code. The code changes to meet the tests.
• Three out of four bugs here were at an edge: an empty text, a tie, a capital letter.

Check yourself

Try each one on your own first. If you are stuck, open a hint. Then check with Show the answer.

1. Why did the old word_count("") not give 0?

2. What does > instead of >= change in most_common_word?

3. Why must the tests stay the same when you fix the code?