The second assignment in CSC302 focused on implementing change requests (CRs) for the JFreeChart project. Given a list of 10 CRs from the JFreeChart bug tracking repo, our team had to select two CRs that we would have to fix. As with many projects, implementing the fix to a given CR is only half of the solution. Once the fix for a defect has been implemented, you need to verify that the proposed fix is valid.
So here is where testing comes in. A series of carefully constructed test cases should be able kill two birds with one stone (at least in theory). First, the tests should be able to verify the validity of a proposed fix. Second, the tests need to ensure that fixes didn’t break anything in the project. Because the defects are of a visual nature, I proposed to make everyone’s life a lot easier by using automation-assisted testing to help us solve this problem.
Our team chose to work on CR#1886044 (Add fancy log tick labels) and CR#1410121 (Strange label truncation). I won’t summarize the defects here, but essentially both CRs are visual defects that deal with the way JFreeChart is drawing the x-axis labels.
The first thing that we should do is to make sure that the defect itself still exists and that it is consistently reproducible. If you’re lucky, the person who submitted the CR has a set of steps to reproduce that you can follow. If you’re really lucky, the submitter will also attach some nice screen shots, or even code snippets to help you reproduce the problem. However, if you have no such luck your best bet is to go back to the submitter and ask for more details and even some steps to reproduce. They do want to get their defect fixed right??
The strange label truncation will be easy to reproduce; all we have to do is copy and paste the provided code snippet and run it. The log tick marks, on the other hand, will require a bit more effort. We’ll have to figure out how to draw the exact scale that the submitter was using for this defect. Fortunately for us, some investigation into the library revealed that the defect had to do with the LogFormat class and the way log axes are drawn in the x-axis. Using the previous test app as a template, we can re-draw a new graph that satisfy these requirements.
In the process of reproducing the defect, we’ve also formulated a pretty good idea about what our test parameters will need to be. For the log tick marks, we will need to define various x intervals to ensure that values that fall in that range do not show up in scientific notation. We will also need to adjust the dimensions of the JFrame so that the numbers maintain their “normal” format even after resizing. For the strange label truncation defect, we know that the truncation changes according to window dimensions. We also know that the amount of truncation depends on the length of each token in the label, and on the rotation of the label in the graph. Now that’s a lot of parameters to test, so are you sure you want to modify every parameter by hand for each and every test case?
Now before we all go insane and offload the work to India, let’s take a few steps back and think about this as the lazy computer scientists that we all are. Lucky for us, we’re not doing blackbox testing — we’re doing greybox testing; we have complete access to the code, but the tests should be done with a focus on customer/user acceptance. So let’s make life a lot easier for ourselves and lets throw in some automation to help us do our tests.
The first thing that we should do is make sure that our tests can always be executed with one command; so we built an ant file that will automatically compile and run our test code. Testing is boring enough, so let’s make it as painless as possible.
Second, we need to add hooks into our test apps so that we can manually adjust our parameters while the program is running. Using the strange label truncation as an example, we should be able to dynamically specify the window width, window height, and text label rotation while the test app is running. We should not have to delve into the code, modify the parameters, recompile and run every time we need to verify a particular test case. To address this problem, we introduce a new “configuration window” into the test app. The configuration window will let the tester manually adjust the desired test parameters on the test window:
Okay, so we now have a way of manually adjusting parameters that are “interesting” from a testing perspective. But even with this fancy new window, you’re going to end up doing manual entry for over a hundred test values 
So let’s take this one step further and automate the process of specifying test parameters. We’ll modify the test app so that it will automatically set up the JChart with our test case parameters. All the human tester will have to do is look at the damn thing and specify whether or not the JFreeChart objects look okay. If it doesn’t look okay, print out a log message of the test parameters that were used so that the tester can go back and reproduce that test case using the configuration window.
After this upgrade, the test app can now accept a plain test file as an argument. Each line of the text file should contain three numbers separated by commas: width,height,rotation. Each line of the text file will act as a test case. The test app will then go through the configuration file line by line and set up the JChart with the given parameters, prompting the user for validation at each step.
Using this setup, executing and verifying over 50 test cases became a quick 5 minute task. Just run the test app and click either Yes or No depending on whether or not the JChart looks okay. If a few test cases failed, you can look back at the console output to see a log of which test cases failed and what parameters were used. Using that information, you could either investigate into the defect further of pass it off to the developers to help them in their bug hunting.
See, testing doesn’t have to be completely painful and tedious. Just be lazy and think like a computer scientist!