Unit testing Cocoa user interfaces: Cocoa Bindings
[info]chanson
About a year ago, I wrote about unit testing target-action connections for Cocoa user interfaces. That covers the traditional mechanism by which user interfaces have typically been constructed in Cocoa since the NeXTstep days. However, with the release of Mac OS X 10.3 Panther we've had a newer interface technology available — Cocoa bindings — which has presented some interesting application design and testing challenges.

Among other hurdles, to properly use Cocoa bindings in your own applications, you need to ensure that the code you write properly supports key-value coding and key-value observing. However, since the release of Mac OS X 10.4 Tiger, the necessary APIs have been available to easily do test-driven development of your application's use of Cocoa bindings, following a trust, but verify approach. (It's also been quite easy from the start to test your support for key-value coding and key-value observing, to ensure that your code meets the necessary prerequisites for supporting bindings. I can write more on this topic in another post if anyone is interested.)

The key to writing unit tests for Cocoa bindings is the -infoForBinding: method in AppKit's NSKeyValueBindingCreation informal protocol. Using this simple method, you can interrogate any object that has a binding for all of the information about that binding! It simply returns a dictionary with three keys:
  1. NSObservedObjectKey, which is the object that the binding is bound to;
  2. NSObservedKeyPathKey, which is the key path that is bound — in Interface Builder terms, this is the controller key path combined with the model key path, with a dot in between them; and
  3. NSOptionsKey, which is a dictionary of additional binding options unique to the binding. These are all of those additional checkboxes and pop-ups in the Interface Builder bindings inspector for setting things like a value transformer.
By specifying what this dictionary should contain for a particular binding, you can describe the binding itself and thus start doing test-driven development of your Cocoa bindings-based user interface. Note that all of the system-supported binding names — as well as the binding option names — are specified in <AppKit/NSKeyValueBinding.h> and are documented, too!

Let's take a simple example, like the one in last year's target-action example, of a window controller whose window has a static text field in it. The field should have its value bound to the name of a person through an object controller for that person. Assume that I've already created the test case and set up some internal methods on my window controller to refer to the contents of the window via outlets, and to load the window (without displaying it) in -setUp just like in the target-action example.

First, to see that my text field has a value binding, I might write something like this:
- (void)testPersonNameFieldHasValueBinding {
    NSTextField *personNameField = [_windowController personNameField];

    NSDictionary *valueBindingInfo = [personNameField infoForBinding:NSValueBinding];
    STAssertNotNil(valueBindingInfo,
        @"The person name field's value should be bound.");
}
Of course, this tells us nothing about how the binding should be configured, so it needs some fleshing out...

Let's check the object and key path for the binding.
- (void)testPersonNameFieldHasValueBinding {
    NSTextField *personNameField = [_windowController personNameField];

    NSDictionary *valueBindingInfo = [personNameField infoForBinding:NSValueBinding];
    STAssertNotNil(valueBindingInfo,
        @"The person name field's value should be bound.");

    NSObjectController *personController = [_windowController personController];
    STAssertEquals([valueBindingInfo objectForKey:NSObservedObjectKey], personController,
        @"The person name field should be bound to the person controller.");

    STAssertEqualObjects([valueBindingInfo objectForKey:NSObservedKeyPathKey], @"name",
        @"The person name field's value should be bound to the 'name' key.");
}
Not very exciting, and a little verbose, but it'll easily lead us through what needs to be set up in Interface Builder for this binding to work. If you want to cut down the verbosity, you can of course extract a method to do the basic checking...
- (BOOL)object:(id)object shouldHaveBinding:(NSString *)binding
            to:(id)boundObject throughKeyPath:(NSString *)keyPath
{
    NSDictionary *info = [object infoForBinding:binding];

    return ([info objectForKey:NSObservedObjectKey] == boundObject)
            && [[info objectForKey:NSObservedKeyPathKey] isEqualToString:keyPath];
}

- (void)testPersonNameFieldHasValueBinding {
    NSTextField *personNameField = [_windowController personNameField];
    NSObjectController *personController = [_windowController personController];
    
    STAssertTrue([self object:personNameField shouldHaveBinding:NSValue
                           to:personController throughKeyPath:@"name"],
    @"Bind person name field's value to the person controller's 'name' key path.");
}
If you're writing code that needs, say, a value transformer, it's a simple matter to extend this model to also check that the correct value transformer class name is specified for the NSValueTransformerNameBindingOption key in the binding options dictionary returned for NSOptionsKey.

You can even extract these kinds of checks into your own subclass of SenTestCase that you use as the basis for all of your application test cases. This will let you write very concise specifications for how your user interface should be wired to the rest of the code, that you can use to just walk through Interface Builder and connect things together — as well as use to ensure that you don't break it accidentally by making changes to other items in Interface Builder.

This is the real power of test-driven development when combined with Cocoa: Because you can trust that the framework will do the right thing as long as it's set up right, you simply need to write tests that specify how your application's interface should be set up. You don't need to figure out how to create events manually, push them through the run loop or through the window's -sendEvent: method, how to deal with showing or not showing the window during tests, or anything like that. Just ensure that your user interface is wired up correctly and Cocoa will take care of the rest.

Separation of Concerns
[info]chanson
Jeremy Miller, The Shade Tree Developer, On Writing Maintainable Code:
Separation of Concerns is the Alpha and Omega of design principles. No other design principle that I'm going to discuss -- be it loose coupling, high cohesion, encapsulation, minimal duplication, or orthogonality -- is possible without Separation of Concerns. Simply put, strive to do one thing at a time in your code. Layering. Divide and conquer. A lot of the other principles are about enabling a system to change with minimal effort and risk. Before you can even think about that, you need to be able to build the system, and then understand that code.
Jeremy hits the nail on the head here: The single most important factor in ease of software maintenance is how well-factored the software is. If its code commonly does more than one thing at once or otherwise mixes concerns, apply the Extract Method, Extract Class, Extract Interface, and Replace Conditional with Polymorphism refactorings liberally until the software is broken up in such a way that each logical unit of code only does one thing.

Also, be sure to check out Working Effectively with Legacy Code by Michael Feathers. (Here's a good review by James Robertson.) It contains a lot of techniques for taking legacy code — which Feathers defines as any code without unit tests — and breaking it up in well-defined ways into code for which it's possible to write unit tests so further, safer refactoring can be done more quickly. It's based on this paper, Working Effectively with Legacy Code, available from the ObjectMentor web site.

Programming as Typing
[info]chanson
Bruce Eckel, Programming as Typing:
I think the problem is that while many programmers understand that programming happens in the mind, and the code itself is just an artifact of the process, outside the field, the code looks like it's what you're doing. (An understandable perception, since the code is the core deliverable). So if it's about the code and not the mental process behind the code, it makes sense that you would do whatever you can to produce the code as fast and as cheaply as possible, and to discard anything that appears to hinder the creation of code. From this follows the logical ideas that coding is typing, so you want to see people typing all the time, and that 10 people can type faster than one person so if you can hire ten Indians cheaper than one US programmer, you're going to get a lot more typing done, so you'll get big economic leverage.
This describes exactly the behavior I saw in the contract software development world after the dot-com crash in early 2000. Managers and businesspeople in many organizations thought programmers were just typists and sought a "secretarial pool" model of software development whether within their companies, outsourced, or outsourced offshore.

Of course, anyone who has actually developed software knows that isn't how it works. But in a lot of organizations the people who are making decisions about software development are, unfortunately, not just aren't software developers themselves but have never been software developers. I'm very thankful that I haven't had to deal with anything like that since moving to California.

Keith Ray also has some good comments on Bruce's piece.

Unit testing Cocoa user interfaces: Target-Action
[info]chanson
It's really great to see that a lot of people are adopting unit testing for their projects and dramatically improving their quality. Test-driven development and agile development methodologies built around it are really taking off. However, a lot of people still feel that their user interface is difficult to test through code, and either requires a capture-playback tool or requires a different design approach based heavily on interfaces/protocols to get right.

In last year's post Trust, but verify. I tried to dispel some of the mystery of testing your application's user interface when using the Cocoa frameworks. However, I've still had a lot of (entirely well-justified!) requests for examples of how to put it into practice. So here's a simple example of what I'd do to write a unit test for a button in a window that's supposed to perform some action.

First, when implementing my window, I'd follow the standard Cocoa pattern of having a custom NSWindowController subclass to manage my window. This window controller will have an outlet connected to each of the views in the window, and will also wind up with a private accessor method — used only within the class and any subclasses, and in testing — for getting the value of each of its outlets. This design flows naturally from the test which I would write to specify that the window should contain a button. First, here's the skeleton into which I'd put tests:
// TestMyWindow.h

#import <SenTestingKit/SenTestingKit.h>

@class MyWindowController;

@interface TestMyWindow : SenTestCase {
    MyWindowController *_windowController;
    NSWindow *_window;
}
@end

// TestMyWindow.m

#import "TestMyWindow.h"
#import "MyWindowController_Private.h"

@implementation TestMyWindow

- (void)setUp {
    // MyWindowController knows its nib name and
    // invokes -initWithWindowNibName: in -init
    _windowController = [[MyWindowController alloc] init];

    // Load the window, but don't show it.
    _window = [_windowController window];
}

- (void)tearDown {
    [_windowController release];
    _window = nil; // owned by _windowController
}

@end
That's the infrastructure into which I'd put my other test methods for this window. For example, I'll want to specify the nib name for the window controller and ensure that it actually knows its window:
- (void)testNibName {
    STAssertEqualObjects([_windowController windowNibName], @"MyWindow",
      @"The nib for this window should be MyWindow.nib");
}

- (void)testWindowLoading {
    STAssertNotNil(_window,
      @"The window should be connected to the window controller.");
}
Now let's check that I have a "Do Something" button in the window, and that it sends an action directly to the window controller.
- (void)testDoSomethingButton {
    // _doSomethingButton is a private method that returns the button
    // conected to the doSomethingButton outlet
    NSButton *doSomethingButton = [_windowController _doSomethingButton];
    
    STAssertNotNil(doSomethingButton,
      @"The window should have a 'Do something' button.");
    
    STAssertEqualObjects([doSomethingButton title], @"Do Something",
      @"The button should be titled accordingly.");

    STAssertEquals([doSomethingButton action], @selector(doSomething:),
      @"The button should send -doSomething: to its target.");

    STAssertEquals([doSomethingButton target], _windowController,
      @"The button should send its action to the window controller.");
}
You'll notice something I'm not doing in the above: I'm not simulating interaction with the interface. This is the core of the trust, but verify approach to unit testing of your user interface.

I can trust that as long as I verify everything is hooked up properly that Cocoa will cause the button to send its action message to its target — whether it's a specific object or, if the target is nil, the responder chain — whenever the button is clicked while it's enabled and not hidden. I don't need to simulate a user event, and I don't even need to display the interface while running the unit tests. All I need to do is inspect, through code, that everything is wired up correctly.

Note that I can do way more than the above in testing my interface design, too. For example, I can ensure that the control layout is correct according to what my interface designer has specified, by checking bounding rectangles for example. But testing only the functionality of my interface has significant advantages, too. For example, it doesn't matter if I wind up using a custom kind of button to achieve exactly the kind of look and feel or behavior I need. It doesn't matter if I wind up changing the layout in response to feedback. No matter what I do, I'll know that functionality won't accidentally break while I'm messing around in Interface Builder — even if I completely rip out my interface and replace it with a new one!

This approach can also be used for testing Cocoa bindings using the -infoForBinding: method that was introduced in Mac OS X 10.4 Tiger. I hope to write up a post soon on how to approach Cocoa bindings using these same techniques, but it should be fairly straightforward given the above and the above documentation.

Update: I've struck through the check of the button's title above, because you may or may not want to do that. For example, if you're primarily running your unit tests against your development localization, you may want to put it in. But if you want to run your unit tests against a localized build of your application, you'll probably want to avoid checking a localized title against an English string. A "have your cake and eat it too" strategy might be to keep a variable somewhere in your application that can be used to selectively disable checks of only localized strings.

Update July 7, 2007: I've finally written a post, Unit testing Cocoa user interfaces: Cocoa bindings, on how to write tests for Cocoa bindings. Now there's no excuse for not doing test-driven development of your Cocoa user interfaces!

"Uncle Bob" gets a MacBook Pro
[info]chanson
Robert C. "Uncle Bob" Martin of ObjectMentor bought a new MacBook Pro and appears to be very pleased with it. Uncle Bob is quite well-known in the object-oriented design and agile development & Extreme Programming communities, and it's great to see him join the Macintosh world.

I saw him speak at a C-SPIN meeting a few years back — Testing is about Specification, not Verification (March 2003) — and it was painful watching him fight his Windows laptop while setting up. I suspect between his new MacBook Pro and Keynote he'll be much, much happier. I suspect he'll also find Xcode, Cocoa, and Objective-C very interesting too, especially coming from Java and Microsoft .NET.

"Enterprise" thought leadership?
[info]chanson
David Heinemeier Hansson, creator of Rails at 37signals, takes James McGovern — some Java/J2EE author — to task for his über-lame rant against Ruby in the Enterprise in a great post titled Boy, is James McGovern enterprise or what!

So by Enterprise, Architect, and Enterprise Architect standards, this gent must be the top of the pop. Thus, allow me to make this perfectly clear: I would be as happy as a clam never to write a single line of software that guys like James McGovern found worthy of The Enterprise.

If Ruby, Rails, and the rest of the dynamic gang we're lumped together to represent, is not now, nor ever, McGovern Enterprise Ready™, I say hallelujah! Heck, I'll repeat that in slow motion just to underscore my excitement: HAL-LE-LU-JAH!

With that out of the way, we're faced with a more serious problem. How do we fork the word enterprise? The capitalized version has obviously been hijacked by McGovern and his like-minded to mean something that is synonymous with hurt and pain and torment.

Indeed, McGovern's rant reads more like a parody of a rant than the real thing:
13. Lets say there is a sixteen week project and the productivity stuff was true and Ruby could save me an entire three weeks which would be significant. Since Ruby is a new vendor and not represented by existing vendors I already do business with, do you think that I will spend more than three weeks in just negotiating the contract?
Yes, because there is some vendor out there named "Ruby that you need to sign a contract with before you can begin a project.

Despite his claims to be agile, McGovern obviously doesn't know the first thing about agile development. People come first, sure, but agile development doesn't say that tools aren't important. Not using good tools makes it harder for good people to do good work.

That's why I love developing software for Mac OS X and why I love helping people develop software on Mac OS X: We have great tools like Cocoa, Core Data, Interface Builder, OCUnit, WebObjects, and Xcode, and these can be used by great developers to do great things.

Making Better Games with Test-Driven Development
[info]chanson
Noel Llopis (Games from Within) and Sean Houghton, Backwards Is Forward: Making Better Games with Test-Driven Development:

One of the questions we had when we jumped into TDD is whether it was going to hold for high-level code. We had seen in practice from previous projects that we can certainly do TDD to create low-level and intermediate-level libraries (math, collision, messaging, etc). But would it really work for high-level code that would build on low-level code?

The answer is an unconditional yes. We have developed a full codebase doing TDD from the start, and we had no difficulty writing high-level code with TDD. Things like character state machines, game flow, or specific game entities were done through TDD without any problems, and greatly benefited from the TDD approach.

Noel's blog is great, and this paper is being presented at the 2006 Game Developers Conference.

Noel and High Moon Studios have been a lot of pioneering work using Extreme Programming in the game development space. Game development can definitely make good use of Extreme Programming to manage the development process, dramatically improve the quality of code, and do wonders for scheduling accuracy and schedule predictability.

Extreme Programming in game development also presents special challenges due to the exploratory nature of a lot of the work, as well as the highly-interactive nature of the software itself. As Noel points out, you wind up writing much more finely-factored code when doing TDD, which will be alien to a lot of game developers but will help greatly with maintenance, debugging, and the ever-more-important portability. (Not just between Windows and Mac OS X, but also between Xbox and PlayStation 2 and Nintendo GameCube and Xbox 360 and PlayStation 3 and Nintendo Revolution...)

My Top Three Agile Practices
[info]chanson
On the Extreme Programming mailing list, Rob Evans recently mentioned that he had been asked to define a software development methodology for his company's entire IT organization. To help him define this methodology, he asked list participants to name their top three agile practices to use when moving an organization to agile methods.

Here are my top three agile methods, all taken from Extreme Programming:
  1. Test-Driven Development
  2. Small Releases
  3. The Planning Game
All of these practices work well together, they can be done in virtually any development environment on virtually any platform, and they don't require much adjustment in the actual developer workday (though they may require significant adjustments in the project planning and scheduling process).

Test-Driven Development gives you a huge amount of ROI when it comes to keeping code quality and maintainability high. This is particularly true if you refactor every time you make a test pass; if you get the tests passing but don't remove duplication and keep your design clean, you can wind up with very well tested spaghetti.

Small Releases — working in terms of short iterations and having a complete, production-quality deliverable at the end of every iteration — keep progress on a project extremely visible. You can tell exactly when and what progress is being made, allowing progressively more accurate decisions to be made about your project and making its return-on-investment extremely visible.

The Planning Game works with small releases to really put the business drivers of the project behind the wheel. They get to decide what is most important to work on in an iteration — and adjust this from iteration to iteration as their requirements change — and the developers performing the work get to decide exactly how much effort it will take to complete. How much work to actually schedule in an iteration is not dictated by either group, but rather by successively-more-accurate measurement of what can be accomplished in an iteration. This means that business can front-load a project's ROI, development isn't having all of schedule, features, and resources dictated to it, and the project's schedule actually gets more rather than less accurate over the life of the project.

With varying degrees of efficiency, all of the above can be done for a team of 1 or a team of 100. They can be done with a team all in one room or with every member on a different continent. And, in my experience, even if these are the only methods you adopt, they form a solid process with which to deliver quality software with accurate schedules and budgets.

Plug: Xcode Unit Testing can be used very effectively to do Test-Driven Development of applications and frameworks using a variety of Mac OS X technologies. You can even do Test-Driven Development of your software's Core Data data model and human interface!

I've also found OmniOutliner to be very effective for managing user stories on small projects.

Kent Beck on Developer Testing
[info]chanson
Since I've mentioned testing...

Go on over to IT Conversations and listen to Kent Beck on Developer Testing at the Developer Testing Forum in Palo Alto last November.

Kent's the author of eXtreme Programming Explained and Test Driven Development By Example. If you're still writing code without writing tests for it, you really need to hear what Kent has to say.

Bursting? What bursting?
[info]chanson
Christopher Koch, Bursting the CMM Hype, CIO Magazine:
The depth and wisdom of the CMM itself is unquestioned by experts on software development. If companies truly adopt it and move up the ladder of levels, they will get better at serving their customers over time, according to anecdotal evidence. But a high CMM level is not a guarantee of quality or performance—only process. It means that the company has created processes for monitoring and managing software development that companies lower on the CMM scale do not have. But it does not necessarily mean those companies are using the processes well.
I thought this article was going to be about "bursting the CMM hype," instead it's mostly about bursing CMM claims. And not claims by or about the CMM, but the claims of some firms to particular CMM levels.

Here are the plain facts: CMM level is not a very good predictor of likely project success and choosing or declining vendors on CMM level is pointless.

There's a little lip service paid to this in the article in the next paragraph:
"Having a higher maturity level significantly reduces the risk over hiring a [company with a lower level], but it does not guarantee anything," says Jay Douglass, director of business development at the SEI. "You can be a Level 5 organization that produces software that might be garbage."
It doesn't address why though.

The CMM claims to to indicate the repeatability and quality of the software process used by a development organization. However, it makes a similar assumption to the IEEE's fatally-flawed Software Engineering Body of Knowledge (SWEBOK) effort: That a waterfall process is The Right Way to develop software, and if we just all follow a waterfall process more strictly and do more work up front and dot more Is and cross more Ts that we'll attain process Nirvana. At that point everyone will have a prescribed role following prescribed tasks, all entirely documented, and software development will truly be a form of manufacturing.

This is a 1950s industrial-management fantasy. It doesn't represent the reality of the successful lean manufacturing world today, and has to my knowledge never been truly representative of successful software development efforts. Today, anyone starting a manufacturing effort needs to study lean manufacturing carefully so they don't make all the same mistakes that were made between 1800 and 1980. And anyone starting a software development effort needs to study agile development so they don't mistake software development for manufacturing, and so they don't repeat the mistakes of 1950-2000.

The CMM isn't all bad. For example, it's been said that a shop that does Extreme Programming rigorously would only need to do a little extra work to qualify for a CMM Level 3 assessment. And XP is actually fairly easy to follow rigorously in some domains when doing new development — sure, it requires discipline, but not tedium such as filling out traceability matrices. But why are there additions necessary to attain Level 3, "Defined Process"? Because CMM doesn't just care that a process is defined but what that process is and how it was arrived at.

For instance, in my company, the process is imposed by fiat: I decide what my company's process is. Period. CMM Level 3 would have me conduct some form of study on process requirements, and then ensure that the process selected meets those requirements, before deciding that I really do have a defined process! In short, CMM requires bureaucracy even though none may be necessary.

The CMM can be useful as a guide in certain circumstances. But if you're going to evaluate vendors against it you should take the time to select what parts of it you care about, understand why you care about them in your particular situation, and then evaluate your potential vendors against those parts that you care about. Theoretically a Level 4 or Level 5 organization will meet your requirements by default, but there may be plenty of shops that have been assessed at lower levels — or haven't been assessed at all — that also meet all of your requirements. As the article says,
There is still no substitute for deep due diligence.

A good user story format
[info]chanson
On the Extreme Programming list, Dan North of ThoughtWorks posted an interesting format for user stories on an XP project.

This very succinctly documents what feature an individual user story represents, what benefit it delivers and to whom. It also covers the customer acceptance test for the user story very simply, in a way that lends itself to automation.

On the front of the card, you have the title of the story, followed by a statement of the form "As a X I want Y so that Z." X is the beneficiary of the user story. Y is what the system will actually do when the user story is implemented. Z is the benefit the user story will deliver.

The back of the card is divided down the middle. The first half is labeled I do this... and describes an action. The second half is labeled This happens... and describes the result of that action. Once every action in the first column actually gives the result in the second column, implementation of the user story is finished.

Using OmniOutliner for User Stories
[info]chanson
Mike Zorkek saw on the HighBay development weblog that I use OmniOutliner to manage User Stories. He asked for an example, I gave him one, and he helpfully suggested I write a weblog entry. Sometimes the light bulb needs a little prodding before it goes off...

Here's what I do: I create three columns in an outline: Story, Points, and Time. User stories are written in the Story column. Point estimates get put in the Points column. Actual time taken gets put in the Time column.

I make the Points and Time columns calculated. That way, I can see at a glance (by using hierarchy) just how many points a particular collection of stories totals to, or a how long a particular set of finished stories took.

I typically have one group for the story backlog, another group for the current iteration, and a bunch of closed groups for previous iterations.

Here's an idealized text representation:
StoryPointsTime
[ ] Iteration 3610.0
   [X] Story "foo"210.0
   [ ] Story "bar"3
   [ ] Story "baz"1
[X] Iteration 2632.0
[X] Iteration 1631.75
[ ] Backlog16
   [ ] Story "quux"1
   [ ] Story "quuux"3
   [ ] Story "quuuux"2
   [ ] Story "quuuuux"3
   [ ] Story "quuuuuux"4
   [ ] Story "quuuuuuux"3
Obviously a user story is going to have more text than just 'Story "title"' but OmniOutliner scales pretty well to story-sized amounts of text.

Something I did on one project was to create an additional "Points Done" calculated column. After checking off a story as complete I'd also put its point total in there. That way I could look at the iteration's line in the outline and see not just how many points were in the iteration overall, but what portion was already done.

Time-and-Materials Versus Fixed-Bid
[info]chanson
There's a great discussion taking place right now on the Cocoa-Dev mailing list about custom software development. (The original poster called it "freelance programming.") One of the posters said they always try to work on a fixed bid, where their bid is based on the value to their client. This is what I posted in response, very lightly edited for formatting.

I prefer not to work on fixed-bid projects. I do time-and-materials, because of the methodology I use. I've run into too many situations where attempts to fully-specify systems up front just aren't realistic. So I work according to an agile development methodology inspired by Extreme Programming.

I work in very short iterations and incrementally deliver functionality to my clients, using the Planning Game to determine their needs in the form of user stories. They're only paying for what they get, and they're getting the most important stuff first. That's first according to their most current business needs, rather than what they thought their business needs would be three or six months from when they first contacted me about the project.

http://c2.com/cgi-bin/wiki?ExtremeProgramming
http://c2.com/cgi-bin/wiki?PlanningGame
http://c2.com/cgi-bin/wiki?TestDrivenDevelopment

One of Cocoa's hidden strengths is that it plays very nicely with Extreme Programming. There are several great unit testing frameworks that you can use with Project Builder to do Test Driven Development. The rapid turnaround of Cocoa makes it feasible to do even significant features in one- to three-week iterations. And the dynamism of Objective-C almost perfectly matches that of the Smalltalk environment Extreme Programming was developed in.

I'm still surprised that the Extreme Programming crowd hasn't latched on to Cocoa and Objective-C.

More on software project failure and iterative development
[info]chanson
Not to keep harping on the same subject, but...

Dr. Randall W. Jensen, Lessons Learned From Another Failed Software Contract, STSC CrossTalk.

Unfortunately this example is very common. Industry software delivery statistics are quite dismal. Fifty percent of commercial software products are delivered over schedule, 33 percent are cancelled, and 75 percent are operational failures. Government software delivery statistics are similar.

The following lessons learned discussion is based upon a post-mortem analysis of this avionics software development. The intriguing analysis results show this project was neither unique nor abnormal. The problems that surfaced during the project's life were common in the mid- 1980s environment and are still common today.

The article goes on to describe the impact of cost and schedule of various issues on an avionics modernization project that failed, and how they contributed to the project's failure.

Here's one choice quote: The absence of a valid estimate is a primary cause of cost and schedule overruns, programs that spiral out of control, and failed programs.

One way around this problem is to not fall into the "whole-project estimate" trap to begin with. If instead of trying to come up with an estimate for the cost of an entire project, certain bounds are set at the start and you develop the project in an iterative fashion, you're more likely to get the most needed functionality on-budget and on-time with fewer defects than if you try to pre-plan every little nuance of development.


Another of the important lessons learned from the failed project: Lesson 4: Instant experience is a myth.

Remember this the next time you talk to an outsourcer about doing a software project. If they can't point to real projects where they've used a technology, or give concrete numbers for how long they've been using the technology, be very wary. For example, there are a lot of companies in the Mac OS X market that are claiming Macintosh experience that the people doing the work don't actually have, as well as more specialized experience (such as Cocoa and Objective-C experience) that they don't actually have.

That doesn't mean everybody involved needs to be a guru. It's great for organizational development to have some gurus and some people new to a technology on the same project — it helps bring those people up to speed much faster and helps institutionalize good habits besides. But make sure that the team taking on a software project has at least some guru-level developers involved.

Everybody involved in software development should read STSC CrossTalk. It's a good magazine, brought to us by the Air Force. Sure, it has some focus on defense software projects and waterfall-style methodologies, but much of what's in it is rather broadly applicable.

Defect Management in an Agile Development Environment
[info]chanson
Don Opperthauser, Defect Management in an Agile Development Environment, STSC CrossTalk
Let me begin with a case in point. More than a decade ago, I and the other future AgileTek co-founders received a functional specification from a large ($13 billion today) consumer products company. It was not an overly complex system, but the functional specification ran to more than 400 pages. The painstaking detail of the document was impressive; every detail of the user interface, validation rules, and exactly how everything was to work was all spelled out. We got the job.

While the software was intended for use by the field sales force, our customer was the information technology (IT) organization. We suggested that perhaps it would be wise for our development team to sit down with some of the intended users and review the specifications. We were told that the IT folks had already done that and, moreover, the effort had taken up more of the users' time than they wanted to give; there was no need for any further review. All we needed to do was to build and test the software to spec - what we call spec conversion in our business.

What seemed like a straightforward task of turning the specifications into bits and bytes got complicated when we discovered that what it said on page 83 contradicted what it said on page 183 and so forth. Could we have possibly analyzed, absorbed, and understood those 400 pages well enough to catch such problems before we began? Of course not. More importantly, do you think that anyone in the sales force really analyzed, absorbed, and understood those 400 pages even though they approved the specification? Most assuredly not! Their eyes probably glazed over around page 20, and they had no choice but to approve a specification they neither had the time nor skill to understand.

This is, in a nutshell, why I've switched bDistributed.com away from doing RFP (Request for Proposal) work to working on a time-and-materials basis using an iterative agile methodology. I've had people tell me that "the industry has been able to do spec work for 30 years" but the fact of the matter is that the industry has been trying and failing to do it for 30 years.

The only way to really succeed at spec work is to significantly pad both your time and cost estimates to build yourself enough of a buffer to handle all of the unforseen circumstances. In today's economy this is a losing proposition; too many competitors are too hungry to be honestly realistic with their estimates and wind up lowballing their bids to get work.

I think it's far better for everybody involved — the client, the service provider, and the eventual end users — to use an iterative, agile approach to software development. More details on the approach I've chosen are in an article I wrote late last year for my company's web site, Story-Driven Planning: Delivering Business Value Fast.

In a nutshell, the client determines what features they want in the form of "user stories" that each will take less than a single one-to-three-week iteration to complete. (Iteration length is fixed and chosen up-front; two-week iterations are the norm though one-week iterations work on smaller projects.) The developers estimate the user stories in terms of abstract units of effort (points), and estimate how many points they can do in an iteration. The client prioritizes the user stories and the developers work on them to completion. At the end of an iteration, the number of completed stories (in terms of points) is used to determine how many points' worth can be chosen for the next iteration. And at the beginning of the next iteration, the client gets to add new stories, remove existing stories, and re-prioritize existing stories.

This means the client is always in charge of what's being worked on, and gets to have the features that are most important to them up front. And since user stories are always worked through to completion they're ready to ship at the end of an iteration — not "implemented but needing QA."

More writin'
[info]chanson
I've posted another article in the Articles section of my company's web site. This one's on the potential Return on Agile Development.

I'm going to try to write an average of one to two articles like this every week for my company web site. I'll also probaby mention them here, and if you want to keep up to date you can always use a feed-monitoring tool like NetNewsWire to subscribe to the RSS feed.

I'm doing this because I've found that the more good, hyperlinked, textual content there is on my company's site, the more hits I get from search engines. (Especially Google.) And with my site layout, I've found that people tend to browse around once they're there. Now I just need to figure out how to turn up the sales pressure a notch without being crass about it.

Influential Software Projects
[info]chanson
I was thinking today about which large software projects have had huge impacts on the way we create software. The ones I can think of off the top of my head are:
IBM OS/360
The project that gave rise to Fred Brooks' The Mythical Man-Month and the concept of software engineering.

Space Shuttle Onboard Software
The project which inspired the Carnegie Mellon University Software Engineering Institute Capability Maturity Model.

Chrysler C3 Payroll System
The project that led to eXtreme Programming and other agile methodologies.
Any other suggestions? I'd like to have an up-to-date list at some point.