Showing posts with label testing. Show all posts

December 12, 2011

Yet Another Agile Reading List

(Updated October 2018)

I'm often asked for an agile reading list and each time I wish that I had already blogged it. What's kept me from doing it so far is that I tend to write a somewhat new list each time depending on the requester's level of experience, unique situation, and context. Such a list is also likely to change over time, but that shouldn't stop me from posting it -- this post is my thoughts at this point in time. My thoughts are likely to change.

That said, and without further ado or proof reading, here is an agile reading list.

Getting Started
Just go read the Scrum Guide. If you've never read it, you really should. It's short. But you might need something beyond that to gain a better understanding of how and why agile works, so don't stop there...

I could make this easy for me and just say read Cohn's signature series plus the Beck signature series. That'd keep you busy for a while and cover the topic pretty well.

But I can put a little more effort into this. Don't let this long list scare you. Start with this 1st block, then supplement with some of the others. I'm writing this from the point of view of helping a total agile novice get started.

Most importantly, you need to understand the agile values: http://agilemanifesto.org/ and http://pmdoi.org/ and either http://www.extremeprogramming.org/values.html or http://www.softwarereality.com/lifecycle/xp/four_values.jsp or http://xp123.com/xplor/xp0209b/index.shtml.

Here are some links about agile in general and two specific agile processes, XP and Scrum, and one lean/agile approach called Kanban. Just read as little or as much of these as you desire to get the gist of what it's all about:
http://en.wikipedia.org/wiki/Agile_development
http://en.wikipedia.org/wiki/Extreme_Programming
http://en.wikipedia.org/wiki/Scrum_%28development%29
http://en.wikipedia.org/wiki/Kanban_%28development%29

You need a good understanding of the Scrum process since it's the market share leader right now. I really really like Kenny Rubin's way with words and his Essential Scrum book is excellent. This is good: Succeeding with Agile: Software Development Using Scrum by Mike Cohn. Subscribe to Mike's blog.

Here's a good short, quick, easy to read intro to Scrum: "The Elements of Scrum" by Sims and Johnson. It includes some extra stuff that you really should do if you are using Scrum. I liked that. Well written.

I think I'll throw in http://scrumpocketguide.com/# from my friend Peter Saddington. I haven't read it, but it sounds short. :-)

Alternatives or Slightly More Advanced Items
Subscribe to my blog and LeadingAgile's blog :-).

These next few are a bit old, but you need to have read something by "Uncle Bob" and Alistair. You could substitute some other book by them if you wish:
Agile Software Development, Principles, Patterns, and Practices by Robert C. Martin.
Agile Software Development by Alistair Cockburn.

These eXtreme Programming books are great and short and solidly apply to Scrum:
Planning XP by Beck.
XP Installed by Jeffries.
XP Explored by Wake.
Don't get the original white XP Explained book by Beck. It's talks about Load Factors and Ideal Engineering Days which I don't recommend.

If you want to broaden beyond Scrum and XP, read about Crystal Clear and the other Crystal methods by Alistair Cockburn.

Product Owner
Now we move beyond the basics into some specifics. First, let's take a look at the Product Owner related literature:

http://www.agilemodeling.com/artifacts/userStory.htm

Agile Estimating and Planning by Mike Cohn.

User Stories Applied: For Agile Software Development by Mike Cohn.

Here are some great posts from my colleague on The Product Owner:
http://www.leadingagile.com/2011/03/why-a-product-owner-team/
http://www.leadingagile.com/2011/03/product-owner-team-who-needs-to-play/
http://www.leadingagile.com/2011/03/one-real-life-product-owner-team/
http://www.leadingagile.com/2011/03/product-owner-team-design-considerations/

Scrum Product Ownership by Robert Galen. It's not an intro to agile. It's a good, thorough book on product ownership. It's full of good advice and tips and warnings against certain dysfunctional behaviors. The book contains some typos and passive voice that will annoy some people.

The Product Owner role in agile is a big job. And very important. Many say it can't be filled by just one person because no one has all the necessary skills. They say you often need a team of people with diverse skills and responsibilities to do the job well. I tend to agree. Here is some food for thought:
http://blog.xebia.com/2008/05/22/scrum-the-mythical-product-owner-role/

Scrum Master
Next, someone on your team needs to understand retrospectives in depth. If you don't get what you need to know out of the above books, this one is wonderful: Agile Retrospectives: Making Good Teams Great by Esther Derby. Subscribe to Esther's blog.

I highly recommend Scrum Mastery by Geoff Watts. This is an excellent book filled with very practical and though provoking advice.

Lyssa's book, Coaching Agile Teams, is very useful, but the novice should read the others first.

Technical
Here are some more assorted topics:

Agile Testing: A Practical Guide for Testers and Agile Teams by Lisa Crispin. And check out her sequel also: More Agile Testing.

Read Beck's TDD book, Fowler's Refactoring book, Evans' Domain Driven Design book, and Williams' Pair Programming book.

Specification By Example by Gojko Adzic should be in this list as should The Cucumber Book by Wynne and Hellesoy.

Lean
At some point, you might need to know what people mean by Lean as it applies to software. Poppendieck wrote the books on it:

Implementing Lean Software Development: From Concept to Cash by Mary Poppendieck

Leading Lean Software Development: Results Are not the Point by Mary Poppendieck.

To really understand lean, really and truly, read Toyota Kata by Mike Rother. This is about getting everyone in the org to make process improvement part of their daily job, to be very intentional about understanding the current and target condition, and to be very intentional about mentoring everyone about everything I just mentioned. This text will help you have a proper understanding about everything else you read about lean, all the techniques, all the metrics, all the approaches.

Then read The Goal: A Process of Ongoing Improvement by Eliyahu M. Goldratt, a delightful book. We also have his Theory of Constraints (ToC) text. Goldratt is becoming a very important person to have read, at least for lean/agile consultants.

However, if you read Goldratt, you must also read The Principles of Product Development Flow by Reinertsen. Reinertsen takes exception with the over simplified or at least naïve implementations of ToC (et. al.).

ToC brings us to the kanban process which is gaining popularity. Kanban by David J Anderson is a wonderful book,but lacks specifics on how to do the metrics when the rubber meets the road. Subscribe to the kanbandev yahoo group. Anderson's Agile Management I would say is more advanced and should be read by serious agile managers or coaches. It's a good cross section of lean, pre-kanban, ToC, etc.

Rother's book Learning To See is about value stream mapping from a manufacturing perspective. I enjoyed working through it, but i'm weird that way. Most people in software development organizations won't need this.

Scale
There are books on scaling agile: Practices for Scaling Lean & Agile Development by Larman and Vodde. And Scaling Software Agility, Leffingwell.

Manage
Management 3.0 by Jurgen Appelo is the most important book for anyone leading or managing IT knowledge workers.

Dan Pink's Drive: understand autonomy, mastery and purpose.

Not sure whether to put DeMarco's Slack, Getting Past Burnout, Busywork and the Myth of Total Efficiency here or up under Lean. I have a blog post on slack.

Fin
Sadly, I cannot recommend this book, even thought I have a chapter in it: XP 2003 Proceedings.

Finally, find a local agile/lean/scrum user group to attend and make connections. I can tell you all about the ones in Atlanta and can connect you to people who know all about the ones in Ohio and Chicago and RTP, NC.

There are probably a thousand other folks with their own agile reading list. I wouldn't mind it at all if you were to post your list here or link from here to your site.

June 2, 2011

Engineering Practices Necessary for Scrum

Atlanta has a wonderful tech community. User groups abound. We even have three different agile user groups that meet each month (or more than that if you count the craftsmanship meetup as an agile group and SPIN as partly so). Earlier this year Semeer Bendre asked on twitter why we have these different communities, copying Mike Cottmeyer, Dennis Stevens and I. I tweeted back that #atlantascrum tends to focus more on #scrum whereas #agileatlanta is more free to cover other stuff, such as dev practices. Mike Cottmeyer quickly corrected me, saying they cover fundamentally the same material and XP isn't oob at #scrumatlanta. Then to prove his point, to show just how wrong I was, Mike signed me up to do a talk at #atlantascrum on "Exploring the Engineering Practices Necessary for Scrum." (Let that be a warning to you to tweet wisely.) This blog post came out of that talk. It's pretty much what I said, edited some for this format.

If I have any bias concerning this topic it is that I'm a long-time strong believer in eXtreme Programming. Heck, I was agile when agile wasn't cool. In fact, it wasn't even called agile yet back then. We founded XP Atlanta before the manifesto meeting in Snowbird. And back in 2000 my team gave me a red cape with a big XP on the back. It wasn't until later that I realized that they thought of me more as a super-villain than a super-hero. They wanted me to wear the cape so they could see me coming. And hear me when I made the swooshing noises. But I digress.

I'm otherwise fairly balanced with half my career using a plan driven approach and the later half using an agile approach; and most of my career as a programmer but more than several years in management and product management.

ENGINEERING PRACTICES IN XP

So, first, before we look at Scrum, let's look at the engineering practices in XP. This is the briefest of summaries of these practices.

As an aside, the non-programmers on our teams need to understand and appreciate what this stuff is. Be kind and explain it to them in a way they can understand. Offer them a non-technical, or at least very lightly technical, lunch and learn. This will help us all work together in harmony.

Pair Programming is far more than continuous code review. The pair is thinking on multiple levels: one about the low-level details, names, indentation and syntax while the other is thinking about the big picture, whether this test is sufficient, whether it's the right test, what's the next step, about the next refactoring, and may be helping look up something in the language or API docs. Pairing improves quality and speed of development, or it at least increases the longevity of the codebase by holding entropy at bay. Pairing supports simple design and helps enforce the use of these other practices (peer pressure).

If you are just watching the other guy code, you aren't pairing. Pairing is active. Who has the keyboard should change every minute or three. Try using the ping-pong technique whereby one programmer codes a failing test. The other programmer then makes that test pass and codes the next failing test. Then the first programmer takes over again, and so forth.

Test-Driven Development is where a test (that fails) is coded before the production code (that makes the test pass) is written. TDD supports Refactoring and Collective Code Ownership and Continuous Integration and other things like small releases and iterative development. Out of doing TDD you get automated unit tests for all code. (Or automated acceptance tests for all stories for ATDD.) A related technique is to write an automated test to reveal a defect before the defect is addressed.

Continuous Integration is where you integrate changes daily and preferably much more often. The opposite, of course, is leaving integration until later, which is often a tedious, lengthy, buggy and painful process. A student of lean will learn that leaving integration for later will lessen the likelihood of good luck. Or, delay causes waste.

Refactoring is improving the design without changing the functionality. Many don't do this right or well. It's best done "in the small" all the time, not in large steps and infrequent. The later may be design change, but I'm loathe to call it refactoring. Good examples are small things like rename class, extract method, move method, introduce variable, etc. Most of these common refactorings are supported by the IDEs we use today, such as Eclipse, Intellij, and VisualStudio with Resharper. Proper Refactoring supports simple design and testing.

Coding Standards supports Pairing and Collective Code Ownership. I don't recommend writing your own. There are many published coding standards. I suggest adopting one of those.

Collective Code Ownership is where everyone on the team is responsible for that team's code. All team members may change all the code. We're assuming we have a small team (or teams) of 7 +/- 3 coders. This practice reduces delay (waiting/blocks) and risk. For this, you need pairing and tests.

Simple design -- It's simple design, not simplistic design. Simple design makes code understandable and therefore more maintainable and modifiable.

System Metaphor is a story that everyone can tell about how the system works. This usually yields a system of names. It supports Collective Code Ownership and Simple Design.

So, that's just the XP engineering practices, not all of XP's practices. And that's about XP. That's not about Scrum.

BACK TO SCRUM

But the topic proposed for Atlanta Scrum group was "Exploring the Engineering Practices Necessary to Support Scrum".

So let's get around to that, but before we do, let me take one more detour. If I wanted to find a definition of agile on the internet, where might I look? I'd look on Wikipedia: http://en.wikipedia.org/wiki/Agile_development

Lo and behold, a definition of agile. And, there's a picture of Sutherland. I'm a little miffed that Scrum gets top billing, but at least if I page down, I see a picture of a couple pair-programmers practicing XP.

Now, what are agile engineering practices? I've shown you the XP ones above. What are the 'Scrum' or 'agile' engineering practices? Can't I find that on wikipedia too? Not as of today, but it was suggested to me during my talk that I should fix that.

My point here is that there is no commonly accepted definition of what the agile engineering practices should be, at least not well enough to make it to wikipedia. I say all that to point out that the stuff written below is more of my opinion than are the facts about XP I've stated above.

ENGINEERING PRACTICES NECESSARY FOR SCRUM

What are the engineering practices necessary to support scrum? Go ahead. Write them down. I'll wait.

That's a nice list you have there, but does Scrum make them necessary? Are they necessary because you are doing Scrum? Is it specific to Scrum or is it agile in general? Here's my list of engineering practices necessary to support Scrum:

none

I categorically say no to all the practices on your list. None. There are no engineering practices necessary to support Scrum.

The question is a non-starter for me. And I say that as an XP fan, a process that believes in the importance of a specific set of engineering practices. I take exception on two grounds. The first is the word necessary.

Necessary is a bit of a strong word. It's kind of like saying always or never. You never take out the trash. You always make me walk the dog. You never put your dirty clothes in the hamper. You always leave the car on empty. You never say you love me any more. Things like that.

The second is the assumption that if you are doing Scrum you must also be involved in software development somehow. You might not be engineering anything at all.

Lots have blogged on this and there was a good debate on twitter about it (involving Jeffries or Shalloway or Sutherland or Schwaber?). It was about Scrum being sufficient or whether it needs development practices. One side said Scrum by itself is insufficient. The other said Scrum is what it is, it's fine as defined, please do use engineering practices, but Scrum doesn't need to be changed.

My wife runs our household using scrum. She's not the housewife, homemaker, domestic engineer: She's the product owner. (My 13 year old daughter who is made to clean the bathrooms is the scum master. But that's another story.) The point is, there are no engineering practices needed for Scrum to be useful in a household.

A team I once coached was the payroll department. They weren't engineering anything. They were building nothing. But they had initiatives and duties that they managed using Scrum. No engineering practices needed.

Pillar's recruiting team uses an agile approach. Pillar's on-boarding process uses an agile approach. Pillar's management uses an agile approach to manage our internal process improvement initiatives. These don't need engineering practices.

However, I know what you mean. I understand the real question. Maybe this is the question to ask:

When you are doing software development and Scrum, are there engineering practices that help?

Now you can bring out your list again.

But does Scrum make them necessary? Or are they universally necessary when doing software development in the current set of popular high level programming languages, namely Java and C# and Ruby? For the most part, I'd say universally helpful. So, we should scratch Scrum from that question and state it thusly:

Are there engineering practices that help if you happen to be on a project involving computer programming?

Absolutely.

Given that, here's my ordered list of useful software engineering practices. I am representing these as tiers upon which the more basic and most important are the foundation upon which the others are built.

FIRST TIER

Starting at the bottom, we have Continuous Integration and Automated Unit Testing. You get a lot out of this. It can hook most people, being a gateway drug, I mean, can help most people and it's a great place to start. Everything else is harder without this

I didn't list source code control, because, everyone does that now, right? I guess there are some things I just assume, and that's one of them.

I could put pairing in the bottom and that would make the other things more likely, but the rest could happen without pairing, so that is later. Besides, the business reality is that it's hard to find enough pair-programmers, so I'm resigned to think that I'll often have a non-pairing team somewhere in my organization.

Similarly, TDD makes sure UTs, refactoring, and emergent-design happen and has other advantages. But you canunit test without TDD, so TDD is higher up in my list.

How about sustainable pace? Is that a technical practice? I'm thinking it's more of a (project) management practice than a strictly technical practice. Otherwise, I'd stick it here.

SECOND TIER

You need tests before refactoring. Some would argue that you need refactoring to do automated testing. Lots of truth to that, but strictly speaking, you could test without refactoring. But you need TDD and Refactoring to support Tier Three.

THIRD TIER

In this tier is Simple Design. Simple is hard. To make things simple, you have to be quite knowledgeable about numerous design concepts. And you also need to be just plain good. You have to be knowledgeable in areas that can make your designs anything but simple if misused, which is why I said you also have to be 'just plain good' in addition to knowledgeable.

Also in this tier is Knowledge of Design Patterns. I didn't say use of design patterns. Use them in your conversations as shorthand, for a concise, rich with meaning way of communicating. But use them in your code carefully.

You should know SOLID design principles if your system is object oriented.

Emergent Design and Emergent Architecture is about recognizing the fact that we aren't going to get it right the first time. Let the design emerge/evolve over time as we learn. Even very very smart folks, the top folks in this field say they don't get it right the 1st time. (Seems like I should try to give a reference to that. Eh, I'll just put an unsubstantiated claim that I bet Cunningham, Jeffries, Uncle Bob, Joshua Kerievsky, Kent Beck, Corey Hanes, and may others would agree.)

This tier is really all supported with 'Ongoing Education', continuous learning. I really think there's lots to learn there and the field continues to grow. This is so difficult to do really really well, that we can never stop learning.

FOURTH TIER

Collective Code Ownership and Pair Programming require a Coding Standard.

As I mentioned earlier, Pair Programming is really a foundational technique, but I can't say that it's for everyone. I'm a strong believer in the technique and have required most of my employees give it a try for at least a month, though it can take two or three to really get accustomed to the approach.

FIFTH TIER

This is the just good old good to know tier. In here are some things that may change or be replaced by something else over time, more so than the ones in the other layers. This tier also includes things to have in your tool box, though you may never pull them out to use them:

Code Coverage
Cyclomatic Complexity
Other Code Metrics
BDD
ATDD
Domain Driven Design

Some might argue with me for putting ATDD or maybe BDD so high up. There are numerous techniques for automated testing at a higher level than unit testing that teams can choose from. I didn't want to list them all and I didn't want to say that one was better than others in all cases.

Likewise, Domain Driven Design is a useful technique, but I can't say every team must use that technique over another.

BACK TO SCRUM

Is there anything in Scrum that should lead us to do these things? You need these things to be agile software development team and these things support time boxed iterations. But there is also the Definition of Done. Some Scrum teams define what done means and hopefully it includes explicitly or implicitly some of these things.

After my talk at Atlanta Scrum on this topic last week I gave a demonstration/explanation of TDD. Surprisingly, I got more positive feedback about that than I did the presentation. Not that either was stellar, but I was expecting a different crowd than what showed up.

Contact me for a TDD demonstration or agile lunch-n-learn at your office.

December 13, 2007

On Automated Testing

Test everything somewhere. Test nothing everywhere.
-- Andrew Fuqua, December 2007

We were talking about automated testing, particularly test coverage, in Agile Atlanta the other day when I coined that aphorism. At least I don't think I've heard it anywhere else before. The concept had been in my head forever and I've worded it similarly thousands of times, but perhaps not so succinctly.

When writing test code it's good to have almost everything tested in some fashion. But there is little reason to test anything multiple times. Just cover everything well, in one layer or another. Any concept well covered in unit tests doesn't need to be tested in higher layer functional tests. Any concept well covered in integration tests doesn't need to be tested in customer acceptance tests. It would be inefficient to cover the same area so many times.

Of course, it's okay to leave certain cases to be tested by hand, as long as they are relatively few in number. Leave what can't be automated easily for manual testing.

February 1, 2001

Continuous Integration

Authors: Matt Di Iorio, Andrew Fuqua, Charlie Hubbard

Have you ever been on a project that practiced big-bang integration – wait until all the features are done, then throw it all together and see what happens? It usually turns out just like the name implies – a totally chaotic, primordial soup of software. Just getting the code to compile is an amazing feat. Then come the bugs. How painful it is to harness the power of nature to integrate your code!

Or say you have a daily build. But for some reason your daily build is broken daily. So you spend half the day tracking down the slacker who checked in the bad code.

"Ah, but my daily build never breaks", you say. Great! But have you ever spent more than a day tracking down one bug? I thought so.

Our team has overcome these problems using simple techniques: We check-in frequently. And we build and test continuously.

"We measure success one build at a time"

Our automated build and test process runs continuously – not daily. Well, it actually runs every half-hour, but that's close enough to continuous for me.

By building and testing continuously, we find it trivial to figure out what broke the build. Given the short time span between builds, the system is usually built with only one developer's changes at a time. If it broke, we readily know why.

By checking in frequently, we avoid file contention. This means never having to merge two sets of changes.

By doing both of these, we never need more than a few minutes to integrate a piece of code into the system. The code in MKS is always buildable – it always works – every minute of every day. I can't stress this enough: it is never in a broken state. That certainly makes integration easy. And since we check-in frequently, we never have much to integrate.

XP's practices build on each other. That's what makes the process so strong. Like XP, continuous integration's practices are multiplicative. Continuous integration draws its strength from good unit tests and frequent check-ins: The tests are more valuable because they are run more often. The frequent builds are more valuable because by running the tests they do more work each time.

Our House

You will get some benefit just by increasing check-in and build/test frequency. We discovered that some additional things we do make continuous integration even better. You may have different needs and different practices. But here is what we do.

Farewell Make

We broke our love affair with Make. She was never a very accommodating partner – always wanting things to be perfect, never compromising. I wanted spaces; She wanted tabs. We just couldn't go on.

We started using a tool called Ant. It's an extensible, cross-platform build tool written in Java. When we discovered that it uses XML for the build file, we just had to use it. Ant pulls from MKS, compiles the code, builds and signs the jar files, runs the tests, and lets us know if anything failed. A Windows Scheduled Task kicks it off every 30 minutes.

One cool Ant feature is the ability to write "build listeners". We wrote a listener to yell at us when the build fails. To notify the developers we use a "net send" command. We wanted to make it intrusive as possible so the developers couldn't ignore failures.

Developers need to build and test on their workstations before checking in their changes. Our IDE allows us to build and test. But we also wanted each developer be able to build with the same process as on the "official" build machine. This required us to modify our architecture but it helps keep the build running smoothly.

"Just the facts, ma'am"

Once you start running an automated build, you've got to see the results. Our results are on a web page. Prominently shown at the top is the Success / Failure message. Then come the details. If the build was not successful, you can see which files didn't compile or which tests failed.

When the build does fail, it only takes us a couple minutes to find and fix the problem. (Remember all the benefits I mentioned above?) In keeping with the spirit of "continuous", we don't wait 20 minutes for the next regularly scheduled build. We can kick off an unscheduled build at any time from our build page.

Our build page even has a cool plot of the number of unit tests run each day. That helps us focus on writing more tests.

Keep it up

It's always good to be on the lookout for things that you can do to make your life easier (or safer). For example, Mike Slifcak suggested we build the database from scratch before running all the tests. This ensures our production code and our install code don't get out of sync. Now we have continuous integration of the install and production code. See how much you can automate.

Our main goal with the build process was to get the most value by doing the least amount of work. There were lots things we could have added to our build process, like automatically logging all the changes between builds. But that wasn't necessary and would only give us more to maintain. Stay light!

Links

http://www.martinfowler.com/articles/continuousIntegration.html
Martin Fowler's article on continuous integration that got us started.

http://jakarta.apache.org/ant/
Here's the great great Ant.

Automated Unit Testing

Authors: Greg Houston, Andrew Fuqua, Tom Gagnier

Did you hear the one about the programmer who wouldn't rewrite some bad code because he was afraid of breaking something else? Since the product couldn't be easily enhanced, innovation was stilted and the company's market value tanked. Our protagonist spent the next two years maintaining that same ugly code. It's not a funny story, but it happens all the time. It doesn't have to. We can rewrite code without fear. We don't have to be the butt of that sad joke. All we need are fully automated unit tests to validate the behavior of our classes and methods.

"Okay, fine," you may be thinking, "but where do these tests come from? They don't exactly fall from the sky." No, they don't. Programmers write tests before they write the production code. Start right now – Write tests for each new method. It's easy and it has done wonders for innumerable projects.

But Wait, There's More!

A.K.A., The Benefits

You can not safely repair or rewrite poor code unless you have tests to prove that things still work. If you do have tests, you can restructure, replace, rewrite until your heart is content. So unit testing enables Refactoring , which keeps working code from degrading as changes are made. When you have unit tests, you have confidence to change any code, even code written by someone else. If the tests pass after making your change, you know you didn't break anything. If the tests worked before, but don't work now, you know you broke it! It's black and white. This enables shared ownership of the code.

Wow! Now we have clean, refactored, tested code and confident programmers. But isn't refactoring and extra testing going to slow us down? NO! Developers actually program faster with fewer defects when using Unit Tests: First, clean, refactored, tested code is easier (read: quicker) to maintain. Second, the tests either pass or fail. The programmers get immediate feedback. They aren't left wondering whether they've tested everything. Third, when tests fail, the code is still fresh in mind – the bugs are easier to find and fix . Fourth, the tests are run every time the code is changed. A programmer is not going to repeat by hand all manual tests ever performed on a class. It's not possible and if it were it would be too time consuming. Happily, automated tests can be run numerous times every day . Finally, you know when you are done because all the tests run!

But wait! That's not all. Automated Unit Testing actually identifies design mistakes . When a design involves too much complexity or coupling, unit tests are difficult to write. By restructuring the design to make it testable we accomplish goals of good design: Low Coupling, and High Cohesion.

All this for one LOW PRICE!

Benefits Outweigh Costs

Write Unit Tests test for everything that could possibly break, using automated tests that must run perfectly all the time. Writing the test code adds to the initial overhead of writing the production code. And, you MUST maintain the tests just as you maintain production code. Thankfully, once written, tests rarely need to change. As explained earlier, automated tests pay for themselves in programmer speed and code quality. With a little practice, you'll be able to write tests quickly.

TESTIFY!

Did you actually do this or is this just theory?

Yes, we actually did and do this. Here's a little testimony from three of ISS's internal projects (code names are used).

Stingray and Diablo

The SAFEsuite Events team started writing automated unit tests in the Stingray project (Events Controlled Release). On that effort, we only wrote tests for the Java Applet portion of the system. At first, our team had difficulty. In a short time, we developed habits and techniques to easily incorporate unit testing into development. When shipped, the Applet had over 100 tests. The experience was so positive that our current project (Diablo) uses unit tests throughout the entire system. The Engineers now rely on unit tests . The benefits have been demonstrated. We only want to work with code that has unit tests. Unit tests have enabled the team to go faster, simplify design, measure progress, and share ownership .

Magellan

The Magellan team started writing automated tests in early to mid 2000, after the project was well underway. They, at first, struggled to internalize test writing. But even under extreme deadline pressure, they learned to test and saw benefits. Most importantly, they stuck with it even after the original unit-test evangelist had left the team. Now, they have over 70 tests. They're hooked – They'll never write code without writing tests.

But First, the Requirements…

You can't expect these benefits with just any old wad of test code. There are a few qualities the tests must possess:

Easy to run – Programmers are lazy. They need to be able to press a single button to run the tests (all or a subset). Make it easy and developers will run them frequently. Make it hard, and no one will run them at all. Easy and Automated means the tests should not require any user input.

Self-checking – Programmers are lazy. They don't want to (and won't) examine a bunch of output to determine if anything failed. The test should check its own results and simply report "Pass" or "Fail".

Fast – Programmers are impatient. They'll be running many tests several times per hour. The tests must be quick. Tests that take too long will have to be optimized. We often use stubs: sometimes to fake I/O or provide a quicker access method; sometimes to short-circuit certain initializations.

Getting Started

The easiest way to get started is to grab someone who has done this and ask them to show you.

We use an off-the-shelf framework to manage the execution and reporting: JUnit for Java, CppUnit for C++. These excellent free frameworks are available online at xprogramming.com/software.htm. These frameworks make it easy to write and automate the Unit Tests.

Create unit tests when writing new code, fixing bugs, and refactoring . Don't worry about adding tests to existing code. Start out writing tests for new code and for any changes you make in existing code. You'll gradually add tests for the old logic as you make enhancements and fix defects. When making changes, first write tests for the parts of the system the changes will affect. The objective is to test everything that could possibly break. Use judgment; there is no need to go nuts. Use risk to drive which tests to implement. You should concentrate on where the risk is. [Refactoring; Martin Fowler; pp 101] There is a point of diminishing returns with testing – when you think you've done enough, stop!

IF and only IF your team is not writing enough tests, visibly focus on testing by tracking the total number of unit tests each day. We have code you can use that counts the tests and makes a cool plot (Figure 1).

Unit Testing the GUI

Don't. Don't test the GUI by firing UI events, scripting user input, scraping screens etc. Instead, use design patterns that separate the visuals from the logic behind the GUI. The rule is "Do No Processing In Your GUI Code." Some patterns that are popular are Model-View-Controller (MVC) [See getting_started2, search java.sun.com for MVC or read Krasner, Pope, A cookbook for using the model-view-controller user interface paradigm in Smalltalk-80, Journal of Object-Oriented Programming , 1(3):26-49, August/September 1988] and Application Façade [appfacades.pdf].

Write Unit Tests that exercise the logic of the system. With MVC, the tests would exercise the Model. For an Application Façade, the tests would exercise the Façade or the classes within the Façade.

Testing the visual portion of the interface is better left for humans. This is one area where we need the expertise that can only be provided by a skilled Quality Assurance team.

Debugging with Unit Tests

Defects indicate a missing unit test. Before coding a fix, develop a test that exposes the defect. Once it's fixed, your job is done – all the tests pass so Quit, Enough Done! (You will not have to run lots of tests by hand.) After fixing a defect, it is important to take a moment to think about the possibility of other missing unit tests. Write all those tests and make them pass. You'll quickly get good at this.

Unit Tests and Functional Test

Don't confuse "white-box" Unit Testing with "black-box" Functional Testing. Functional tests verify the entire end-to-end operation of each feature. Typically functional tests relate directly to system requirements (MRD). We still need automated functional tests (customer acceptance tests). And we still need a QA teams for their unique perspective and abilities. Unit Tests are very small tests on individual classes and methods of the system. They don't require the entire system to execute. A test may cover several classes or methods but the smaller and more specific the better. Since unit tests are written by development / for development and since all tests must pass before any changes are checked in, QA will never see a failed unit test.