Chapter 15. Refactoring 15.1. Handling bugs Despite your best efforts to write comprehensive unit tests, bugs happen. What do I mean by “bug”? A bug is a test case you haven't written yet. Example 15.1. The bug >>> import roman5 >>> roman5.fromRoman("") 1 0 1 Remember in the previous section when you kept seeing that an empty string would match the regular expression you were using to check for valid Roman numerals? Well, it turns out that this is still true for the final version of the regular expression. And that's a bug; you want an empty string to raise an InvalidRomanNumeralError exception just like any other sequence of characters that don't represent a valid Roman numeral. After reproducing the bug, and before fixing it, you should write a test case that fails, thus illustrating the bug. Example 15.2. Testing for the bug (romantest61.py) class FromRomanBadInput(unittest.TestCase): # previous test cases omitted for clarity (they haven't changed) def testBlank(self): """fromRoman should fail with blank string""" self.assertRaises(roman.InvalidRomanNumeralError, roman.fromRoman, "") 1 1 Pretty simple stuff here. Call fromRoman with an empty string and make sure it raises an InvalidRomanNumeralError exception. The hard part was finding the bug; now that you know about it, testing for it is the easy part. Since your code has a bug, and you now have a test case that tests this bug, the test case will fail: Example 15.3. Output of romantest61.py against roman61.py fromRoman should only accept uppercase input . ok toRoman should always return uppercase . ok fromRoman should fail with blank string . FAIL fromRoman should fail with malformed antecedents . ok fromRoman should fail with repeated pairs of numerals . ok fromRoman should fail with too many repeated numerals . ok fromRoman should give known result with known input . ok toRoman should give known result with known input . ok fromRoman(toRoman(n))==n for all n . ok toRoman should fail with non-integer input . ok toRoman should fail with negative input . ok toRoman should fail with large input . ok toRoman should fail with 0 input . ok ====================================================== ================ FAIL: fromRoman should fail with blank string ---------------------------------------------------------------------- Traceback (most recent call last): File "C:\docbook\dip\py\roman\stage6\romantest61.py", line 137, in testBlank self.assertRaises(roman61.InvalidRomanNumeralError, roman61.fromRoman, "") File "c:\python21\lib\unittest.py", line 266, in failUnlessRaises raise self.failureException, excName AssertionError: InvalidRomanNumeralError ---------------------------------------------------------------------- Ran 13 tests in 2.864s FAILED (failures=1) Now you can fix the bug. Example 15.4. Fixing the bug (roman62.py) This file is available in py/roman/stage6/ in the examples directory. def fromRoman(s): """convert Roman numeral to integer""" if not s: 1 raise InvalidRomanNumeralError, 'Input can not be blank' if not re.search(romanNumeralPattern, s): raise InvalidRomanNumeralError, 'Invalid Roman numeral: %s' % s result = 0 index = 0 for numeral, integer in romanNumeralMap: while s[index:index+len(numeral)] == numeral: result += integer index += len(numeral) return result 1 Only two lines of code are required: an explicit check for an empty string, and a raise statement. Example 15.5. Output of romantest62.py against roman62.py fromRoman should only accept uppercase input . ok toRoman should always return uppercase . ok fromRoman should fail with blank string . ok 1 fromRoman should fail with malformed antecedents . ok fromRoman should fail with repeated pairs of numerals . ok fromRoman should fail with too many repeated numerals . ok fromRoman should give known result with known input . ok toRoman should give known result with known input . ok fromRoman(toRoman(n))==n for all n . ok toRoman should fail with non-integer input . ok toRoman should fail with negative input . ok toRoman should fail with large input . ok toRoman should fail with 0 input . ok ---------------------------------------------------------------------- Ran 13 tests in 2.834s OK 2 1 The blank string test case now passes, so the bug is fixed. 2 All the other test cases still pass, which means that this bug fix didn't break anything else. Stop coding. Coding this way does not make fixing bugs any easier. Simple bugs (like this one) require simple test cases; complex bugs will require complex test cases. In a testing-centric environment, it may seem like it takes longer to fix a bug, since you need to articulate in code exactly what the bug is (to write the test case), then fix the bug itself. Then if the test case doesn't pass right away, you need to figure out whether the fix was wrong, or whether the test case itself has a bug in it. However, in the long run, this back-and-forth between test code and code tested pays for itself, because it makes it more likely that bugs are fixed correctly the first time. Also, since you can easily re-run all the test cases along with your new one, you are much less likely to break old code when fixing new code. Today's unit test is tomorrow's regression test. 15.2. Handling changing requirements Despite your best efforts to pin your customers to the ground and extract exact requirements from them on pain of horrible nasty things involving scissors and hot wax, requirements will change. Most customers don't know what they want until they see it, and even if they do, they aren't that good at articulating what they want precisely enough to be useful. And even if they do, they'll want more in the next release anyway. So be prepared to update your test cases as requirements change. Suppose, for instance, that you wanted to expand the range of the Roman numeral conversion functions. Remember the rule that said that no character could be repeated more than three times? Well, the Romans were willing to make an exception to that rule by having 4 M characters in a row to represent 4000. If you make this change, you'll be able to expand the range of convertible numbers from 1 3999 to 1 4999. But first, you need to make some changes to the test cases. Example 15.6. Modifying test cases for new requirements (romantest71.py) This file is available in py/roman/stage7/ in the examples directory. If you have not already done so, you can download this and other examples used in this book. import roman71 import unittest class KnownValues(unittest.TestCase): knownValues = ( (1, 'I'), (2, 'II'), (3, 'III'), (4, 'IV'), (5, 'V'), (6, 'VI'), (7, 'VII'), (8, 'VIII'), (9, 'IX'), (10, 'X'), (50, 'L'), (100, 'C'), (500, 'D'), (1000, 'M'), (31, 'XXXI'), (148, 'CXLVIII'), (294, 'CCXCIV'), (312, 'CCCXII'), (421, 'CDXXI'), (528, 'DXXVIII'), (621, 'DCXXI'), (782, 'DCCLXXXII'), (870, 'DCCCLXX'), (941, 'CMXLI'), [...]... programmer who says “Trust me.” 15.3 Refactoring The best thing about comprehensive unit testing is not the feeling you get when all your test cases finally pass, or even the feeling you get when someone else blames you for breaking their code and you can actually prove that you didn't The best thing about unit testing is that it gives you the freedom to refactor mercilessly Refactoring is the process of... unit tests is that the code being tested is never “ahead” of the test cases While it's behind, you still have some work to do, and as soon as it catches up to the test cases, you stop coding.) Example 15.8 Coding the new requirements (roman72.py) This file is available in py/roman/stage7/ in the examples directory """Convert to and from Roman numerals""" import re #Define exceptions class RomanError(Exception):... you explicitly stopped it with the regular expression pattern matching You may be skeptical that these two small changes are all that you need Hey, don't take my word for it; see for yourself: Example 15.9 Output of romantest72.py against roman72.py fromRoman should only accept uppercase input ok toRoman should always return uppercase ok fromRoman should fail with blank string ok fromRoman should... these for loops need to be updated as well to go up to 4999 Now your test cases are up to date with the new requirements, but your code is not, so you expect several of the test cases to fail Example 15.7 Output of romantest71.py against roman71.py fromRoman should only accept uppercase input ERROR toRoman should always return uppercase ERROR 1 fromRoman should fail with blank string ok fromRoman... Usually, “better” means “faster”, although it can also mean “using less memory”, or “using less disk space”, or simply “more elegantly” Whatever it means to you, to your project, in your environment, refactoring is important to the long-term health of any program Here, “better” means “faster” Specifically, the fromRoman function is slower than it needs to be, because of that big nasty regular expression... worth trying to do away with the regular expression altogether (it would be difficult, and it might not end up any faster), but you can speed up the function by precompiling the regular expression Example 15.1 0 Compiling regular expressions >>> import re >>> pattern = '^M?M?M?$' >>> re.search(pattern, 'M') 1 >>> compiledPattern = re.compile(pattern) 2 >>> compiledPattern . Chapter 15. Refactoring 15. 1. Handling bugs Despite your best efforts to write comprehensive. I mean by “bug”? A bug is a test case you haven't written yet. Example 15. 1. The bug >>> import roman5 >>> roman5.fromRoman("")