Saturday, January 28, 2012

Understanding the power of isolated object testing

I spent some time last night watching J.B. Rainsberger's excellent InfoQ presentation entitled "Integration Tests are a Scam". I've been recently contemplating why I favor unit testing (or as J.B. calls it, isolated object testing) over integration testing and I found his presentation particularly relevant. I highly recommend watching it. He also has a series of blog entries that support the presentation (Parts 1, 2, and 3). I also recommend reading those. They're truly gems.

 

Testing to prove basic correctness

I find many developers using integrated tests as a way to prove the basic correctness of the class or system under test. J.B. writes that "While integration tests offer value in other contexts, too many programmers use them to show basic correctness, and when they do that they waste a tremendous amount of time and effort." Integrated testing can be used within a project (I'm personally fond of acceptance testing), but integrated testing should not be used to prove basic correctness of your code. Focused, isolated object tests (aka unit tests using test doubles) should be used for this endeavor. If you discover behavior that a collaborator demonstrates and you have not accounted for in your isolated object tests, you should mimic this behavior in your test doubled collaborator contracts. You want to cover as much of your code with isolated object tests. We'll talk more about contract tests later.

J.B. mentions using an integrated test to learn about how a collaborator might support its contract, but that this integrated test is not included in the basic correctness test suite. I'm wondering if there isn't some other test suite here that we could use to keep integrated tests that support our learning the runtime and external dependencies. This test suite would be run periodically, but is not part of the whole continuous integration process of building a software system. Need to noodle on this more.

 

Isolated object testing

J.B. states in his aforementioned presentation that he does not use the term "unit testing" and instead favors a more focused term of "isolated object testing". He makes a point to call out the isolated word; these tests isolate the class under test by using test doubles to stub or mock the collaborators of the class under test. These tests focus on a single object and a singular behavior. Any collaborations are realized using test doubles.

I tend to agree that the phrase "unit testing" is a weak phrase describing the type of testing that we use to drive design of a single class and a single method. These tests focus on the responsibility of a single method of a single class. These tests also help you focus on understanding the contracts of collaboration between this class under test and its collaborators. This is really where the design part of the equation comes in. Classes that are hard to test are screaming out that the design is wrong. The tests are saying you have too much responsibility in the class under test or there are too many collaborators. Using your fast isolated object tests, you can engage in a reactive design activity, moving responsibilities out of the class under test, create new collaborators, and other design changes to make .

 

The collaboration contract

J.B. rightfully makes a big deal about contracts (aka interfaces) to explicitly specify collaboration protocols between objects. He mentions that in Domain-Driven Design, popularized by the Eric Evans book of the same name, three concepts can be used to express a domain-driven model: Values, Entities, and Services. All Services should have contracts and those contracts manifest themselves as interfaces. By specifying interfaces, we explicitly declare the protocol supported by each interface implementation. Constraining ourselves to interface types when specifying collaborations results in looser coupled systems, which is considered a "Good Thing". When engaging in isolated object testing, J.B. details the concept of collaboration tests and the closely associated contract tests. This concept of collaboration tests and contract tests is something new to me and is a big reason the presentation was so valuable in my eyes.

 

Collaboration tests and contract tests

After watching the presentation, I seem to be doing a pretty good job of using collaboration tests, but I'm not making the association to contract tests. Collaboration tests prove that the client interacts with its collaborators correctly; the client sends the correct messages and message arguments to the collaborator and appropriately handles all outputs from the collaborator. This is traditionally what I have used mock objects for and that seems to be what J.B. is arguing one should do for collaboration testing.

Contract testing, on the other hand, deals with testing that an interface implementation accurately respects the interface it is implementing. Does the implementor support the contract it declares to support? I haven't typically written these types of tests, but I'm going to start. Interestingly, I don't see a lot of this sort of testing in the wild. The part I really dig about contract tests as Rainsberger explains them is that they can be reused across interface implementations. He uses the List interface and two of its implementations, ArrayList and LinkedList, and details how to use implementation inheritance in the tests to DRY up your contract tests.

It really gets interesting when he declares that for every collaboration test that implies that a test double behavior, there better be a corresponding contract test that demonstrates that the interface implementation(s) actually does support that behavior. Same goes for values returned: if a test double returns a value in a collaboration test, there should be a contract test that demonstrates that the real implementation(s) does actually return that value. This is where I think selective, integrated learning tests can help you discover how your classes on the edges of a system may act when integrated to real external dependencies. But again, those integrated tests are not providing basic code correctness semantics. They're in your project to help you learn, but are not part of the isolated object tests suite. Don't lump them with your isolated tests and don't run them as part of your code/update repos/run tests/commit cadence.  This learning test suite should be run periodically during the day, but not part of the CI build process.

The death spiral of integrated testing for proving basic code correctness


Both Rainsberger's tutorial presentation and his blog postings go into great detail to the fallacy of using integrated testing to prove code correctness. I won't rehash what he has to say about it. My conclusions that I draw from his material are:

  • Focus on isolated object tests and isolation techniques and learn how to do collaboration and contract testing.
  • If you need to learn, by all means, write an integrated test. If you need to reproduce a defect, initially write it as an integrated test. However, in all cases, take what you have learned and replicate that in your isolated object test suite.  Replicate the behavior you discovered in the integration test(s) with test doubles.  This will ensure that your isolated object tests stay true to the behavior of the integrated system.
  • Don't get sucked into the downward death spiral of using integrated testing to guide your test-driven design efforts. It will only cause you pain.


J.B.'s series on Integrated Tests are a Scam can be found here.



 

7 comments:

  1. I absolutely love "death spiral". I wish I'd called it that.

    Thank you for your support of the idea. I'm particularly interested in knowing how it helps you in your work.

    ReplyDelete
  2. Solid advice. Thanks Chris. Using "isolated" to mean "single object and singular behavior" works better for my feeble mind than the word "unit". As I think about it, what we're calling unit tests on my current project might in some cases not be proving basic code correctness. I probably should take one of your TDD classes at DevJam. ;-)

    ReplyDelete
  3. Hmmm... I've seen so-called isolated object tests make so many assumptions about the collaborators it uses that they become a maintenance nightmare, false-redbaring constantly. And I've seen integration tests become so easy to write and so able to survive refactoring that you will take them out of my cold, dead hands to prove the correctness of my code. So I have trouble with this one.

    ReplyDelete
  4. This comment has been removed by the author.

    ReplyDelete
  5. Colonel Nikolai, I respect your opinion. I'm not saying that I don't value integrated tests, I just tend not to use them for code correctness. I've been on more than one project that has allowed integrated testing to get out of hand and turn the entire testing feedback loop on its head. Anything over 2 minute for a full run of the integration test suite kills the feedback loop.

    I love your comment about false-redbarring. Yep, isolation techniques, if not properly controlled, can be a huge mess. You definitely have to understand what the contract is when using test doubles. Thanks for the read and the comments.

    ReplyDelete
  6. Do you know how to keep a good mood. Just remember to pay attention to the fashion trend of Paris Hermes such as Hermes Constance Handbag or the other styles like Kelly Hermes Bag. I promise you will relax yourself. So come on, know more fashion style arround you and make yourself more confidence. You will like it.

    ReplyDelete
  7. Otr214428
    INTERNATIONAL CONCEPT OF WORK FROM HOME
    Work from home theory is fast gaining popularity because of the freedom and flexibility that comes with it. Since one is not bound by fixed working hours, they can schedule their work at the time when they feel most productive and convenient to them. Women & Men benefit a lot from this concept of work since they can balance their home and work perfectly. People mostly find that in this situation, their productivity is higher and stress levels lower. Those who like isolation and a tranquil work environment also tend to prefer this way of working. Today, with the kind of communication networks available, millions of people worldwide are considering this option.

    Women & Men who want to be independent but cannot afford to leave their responsibilities at home aside will benefit a lot from this concept of work. It makes it easier to maintain a healthy balance between home and work. The family doesn't get neglected and you can get your work done too. You can thus effectively juggle home responsibilities with your career. Working from home is definitely a viable option but it also needs a lot of hard work and discipline. You have to make a time schedule for yourself and stick to it. There will be a time frame of course for any job you take up and you have to fulfill that project within that time frame.

    There are many things that can be done working from home. A few of them is listed below that will give you a general idea about the benefits of this concept.

    Baby-sitting
    This is the most common and highly preferred job that Women & Men like doing. Since in today's competitive world both the parents have to work they need a secure place to leave behind their children who will take care of them and parents can also relax without being worried all the time. In this job you don't require any degree or qualifications. You only have to know how to take care of children. Parents are happy to pay handsome salary and you can also earn a lot without putting too much of an effort.

    Nursery
    For those who have a garden or an open space at your disposal and are also interested in gardening can go for this method of earning money. If given proper time and efforts nursery business can flourish very well and you will earn handsomely. But just as all jobs establishing it will be a bit difficult but the end results are outstanding.

    Freelance
    Freelance can be in different wings. Either you can be a freelance reporter or a freelance photographer. You can also do designing or be in the advertising field doing project on your own. Being independent and working independently will depend on your field of work and the availability of its worth in the market. If you like doing jewellery designing you can do that at home totally independently. You can also work on freelancing as a marketing executive working from home. Wanna know more, email us on workfromhome.otr214428@gmail.com and we will send you information on how you can actually work as a marketing freelancer.


    Internet related work
    This is a very vast field and here sky is the limit. All you need is a computer and Internet facility. Whatever field you are into work at home is perfect match in the software field. You can match your time according to your convenience and complete whatever projects you get. To learn more about how to work from home, contact us today on workfromhome.otr214428@gmail.com and our team will get you started on some excellent work from home projects.


    Diet food
    Since now a days Women & Men are more conscious of the food that they eat hence they prefer to have homemade low cal food and if you can start supplying low cal food to various offices then it will be a very good source of income and not too much of efforts. You can hire a few ladies who will help you out and this can be a good business.

    Thus think over this concept and go ahead.

    ReplyDelete