Previous Entry Add to Memories Share Next Entry
Xcode: Unit Testing Cocoa Applications
latest
chanson
Yesterday, I talked about how to add unit tests to Cocoa frameworks using Xcode. There's only a little more set-up required to add tests to Objective-C Cocoa applications.

First, turn off ZeroLink for the application target you want to test. Just as with a framework, your unit tests will be built as a test bundle containing only the tests, completely separate from the code being tested. The test bundle will access the code in your application by linking against it, and you can't link against something built with ZeroLink. Note: You only need to do this for Xcode 2.1 through Xcode 2.5. Xcode 3.0 removed support for ZeroLink, since the linker is now sufficiently fast as to obviate it.

Next, add a new Cocoa Unit Test Bundle target to your application project. This is the target that will actually contain your tests. When you add the test bundle target, Xcode shows its Info window. In the General tab of this window, press the + button and choose your application target from the sheet that appears. This will make your test bundle target depend on your application target, so you can just build your test bundle all the time and have your application automatically rebuild first.

Now you actually need to make your test bundle link against your application. You do this using the Bundle Loader property under Linker Options in your test bundle target's configurations. You need to set the value of this option to the path of your application executable and not just its wrapper: $(BUILT_PRODUCTS_DIR)/MyApplication.app/Contents/MacOS/MyApplication. The $(BUILT_PRODUCTS_DIR) variable is expanded at build time to be the path to the build products for the currently-selected configuration, which lets you avoid using an absolute path.

The final step is to tell the unit testing infrastructure how to run your tests. For a framework, the dynamic loader does the heavy lifting; when the test rig loads your test bundle, the loader will automatically load the frameworks it depends on including the one you're testing. However, for an application to be tested the application must be launched and the test bundle injected into it. You can specify that this should happen using the Test Host property under Unit Testing in your test bundle target's configurations. Just as with the Bundle Loader property, you need to set the value to the full path of your application executable. Since you've already done that once, you can just re-use the value of the Bundle Loader setting by specifying $(BUNDLE_LOADER) — this is the underlying variable that the property is associated with.

Now just like with frameworks you can add test cases to your test bundle just by adding new files based on the Cocoa Objective-C test case class template. Your application code can remain in your application target and your unit testing code can remain in your test bundle target. Whenever you build your test bundle target, once everything is built your application will launch, its tests will be run, and it will quit — and the test results will be reported via the Build Results window just like compiler and linker errors.

This series of posts on Xcode 2.1 has been a little goldmine :-)

Thanks!

Debugging

(Anonymous)

2005-08-28 03:55 pm (UTC)

Chris, Any idea how to use the debugger with the unit tests? That would be tremendously valuable!

-
Dave Hein

See Xcode 2.1: Debugging Cocoa application unit tests for an overview of how to debug into your tests. The gist is that you just have to configure your executable to run the tests, and then you can bring it up under the debugger and hit breakpoints in your tests just like you can in your app.

Injection Rocks

(Anonymous)

2005-09-20 06:22 am (UTC)

I've said it before, but it bears repeating that I love how the test injection works. That's an integration feature that is just really nice.

James Duncan Davidson

nice :)
;))

Xcode 3.0

(Anonymous)

2007-11-17 03:29 am (UTC)

Are these still correct instructions for Xcode 3.0?


They should still be correct for Xcode 3.0; the only thing that Xcode 3.0 changes is how you debug application tests, because you have to specify DYLD_FALLBACK_FRAMEWORK_PATH in your executable's environment. That doesn't affect running unit tests as part of your build (the RunUnitTests script does it for you).

I just had to delete a comment that asked about the iPhone SDK. Please remember that you have to agree to a Non-Disclosure Agreement (NDA) to get access to the iPhone SDK, and that means you can't discuss it in public.

Re: Xcode 3.0

(Anonymous)

2010-10-28 02:25 am (UTC)

That is absurd. I ask myself: why am I sitting here trying to figure out OCUnit?

Using Instruments with unit tests

(Anonymous)

2008-07-26 05:44 pm (UTC)

I have a cocoa application with unit tests that I created by following your instructions. They are working great. I have them set up so that tests are injected into the executable and run during builds. I would like to check for memory leaks using instruments. Can I use Instruments when the unit tests run and see if the unit tests cause leaks?

Thanks

I haven't been able to make this work

(Anonymous)

2008-09-03 02:12 am (UTC)

Hi Chris,

I'm learning Cocoa after years of working with other systems. I'm pretty familiar with the concepts, but I'm new enough to Cocoa that the syntax details are still tripping me up.

I'm writing a tutorial document on how to use XCode 3, SenTestingKit, OCMock, and git. For solving a couple of problems I had, your instructions were the best I found, but I still was not able to make it work.

I've put my tutorial and my code at http://xorandor.com/FirstCocoaApp

If you have some time, I'd appreciate some help on:
1) Running the unit tests as part of the build process in Xcode 3.1, and
2) Debugging the unit tests.

Once I get past those problem, I think I'll be able to work through my issues with OCMock and finish the tutorial.

Thanks,

Pat

Re: I haven't been able to make this work

chanson

2008-09-03 02:49 pm (UTC)

Please post a focused question on Apple's xcode-users mailing list. I'm not the only person who can help with any issues people run into trying to use Xcode unit testing; there are lots of people using it successfully, and many are on the mailing list.

Thanks.

I'm using XCode 3.1.1 and was able to add a "Unit Test Bundle" target but could not find where to set the Bundle Loader property. It's probably right in front of my face, but I could use a little help locating where to make this setting.

Re: Where to set the Bundle Loader property?

chanson

2008-11-24 05:32 am (UTC)

It's one of the build settings in the configurations of your Unit Test Bundle target. Double-click your target to get a Target Info window, click the Build tab, and then set the Configuration pop-up to All Configurations and the Settings pop-up to All Settings. Then you can use the search field to search for, say, Loader, and it'll show the Bundle Loader setting. Or you can just scroll the list to find the Bundle Loader setting under the Linking section.

Thank you!
The problem for me was that I had not set the Target info window to Show "All Settings"

Unexported symbols...

(Anonymous)

2009-01-17 05:54 pm (UTC)

One hiccup I ran into was that the application "library" doesn't necessarily export all the symbols that a test would reasonably want to use. I worked around this by unchecking the "Symbols Hidden By Default" checkbox in the application's build parameters. But it would be nice if there was a way to get "the magic" to link up without changing the export rules on my app.

Any suggestions for a better way to set this up? I wonder if there's some way to transform the application binary after it's built, to make its symbols exported for the Unit Test bundle's benefit.


Re: Unexported symbols...

(Anonymous)

2009-01-17 05:55 pm (UTC)

Oh and by the way, this is Daniel Jalkut. Hi :)


Automated procedure

(Anonymous)

2009-01-27 12:19 am (UTC)

Thanks so much Chris, your posts were life-savers. Here's an applescript to set up an xcode project for unit testing and debugging those tests, following your procedure. I had to remove the comments for length restrictions. The full script can be found at http://clipperadams.com/programming/xcode_unit_test_applescript.html.

(*
To debug the unit tests, in the "General" tab of the executable info window, the "Working Directory" must be set to "Build Products directory"
Modify the testFrameworkPath for your system
*)
set testFrameworkPath to "/Developer/Library/Frameworks/SenTestingKit.framework"

tell application "Xcode"
tell project of active project document
set projectName to name
set executablePath to "$(BUILT_PRODUCTS_DIR)/" & projectName & ".app/Contents/MacOS/" & projectName
tell executable name
make new launch argument with properties {name:"-SenTest All", active:true}
make new environment variable with properties {name:"DYLD_FALLBACK_FRAMEWORK_PATH", value:"$(DEVELOPER_LIBRARY_DIR)/Frameworks", active:true}
make new environment variable with properties {name:"XCInjectBundleInto", value:executablePath, active:true}
make new environment variable with properties {name:"XCInjectBundle", value:"Unit Tests.octest", active:true}
make new environment variable with properties {name:"DYLD_INSERT_LIBRARIES", value:"$(DEVELOPER_LIBRARY_DIR)/PrivateFrameworks/DevToolsBundleInjection.framework/DevToolsBundleInjection", active:true}
end tell
set unitTestTemplate to target template "Cocoa/Unit Test Bundle"
make new target at end of targets with data unitTestTemplate with properties {name:"Unit Tests"}
tell root group
make new group with properties {name:"Test Cases"} at beginning of groups
end tell
tell group "Linked Frameworks"
set testFrameworkFile to make new file reference with properties {full path:testFrameworkPath, name:"SenTestingKit.framework"}
end tell
add testFrameworkFile to (get link binary with libraries phase of target "Unit Tests")
set value of build setting "BUNDLE_LOADER" of build configuration "Debug" of target "Unit Tests" to executablePath
set value of build setting "TEST_HOST" of build configuration "Debug" of target "Unit Tests" to "$(BUNDLE_LOADER)"
set active target to target "Unit Tests"
end tell
end tell
activate application "Xcode"
tell application "System Events"
tell application process "Xcode"
tell outline 1 of scroll area 1 of splitter group 1 of group 1 of window projectName
tell (first row whose value of text field 1 of group 1 = "Targets")
set targetGroupDisclosureTriangle to UI element 1 of group 1
if value of targetGroupDisclosureTriangle = 0 then
perform action "AXPress" of targetGroupDisclosureTriangle
end if
end tell
select (first row whose value of text field 1 of group 1 = "Unit Tests")
end tell
perform action "AXPress" of menu item "Get Info" of menu 1 of menu bar item "File" of menu bar 1
tell window "Target “Unit Tests” Info"
click radio button "General" of tab group 1
perform action "AXPress" of button 1 of splitter group 1 of tab group 1
tell sheet 1
select (first row of outline 1 of scroll area 1 whose value of text field 1 = projectName)
click button "Add Target"
end tell
click button 1
end tell
end tell
end tell

Re: Automated procedure

zenspider

2009-10-26 11:14 pm (UTC)

this doesn't quite work anymore but is fairly easy to fix (tho I haven't figured out how to make it work via the script menu in xcode!?!?).

switch from: "tell application process" to: "tell process"

switch from: "window projectName" to: "window 1"

The former may not be necessary... I was trying to figure out the problem with running via the script menu and it may be muddled in. The latter is necessary as xcode 3.2 has a much more complex window name (or is a result of using the all-in-one project layout?).

Testing C functions ...

(Anonymous)

2009-02-13 05:57 pm (UTC)

Do I need to do anything special to test normal C functions?

My references to cocoa symbols resolve ok, but when I try and reference C functions defined in my application code, I get "symbol not found" errors during linking, even though nm seems to indicate that they are present in the symbol table.


exited abnormally with code 139

(Anonymous)

2009-04-01 09:36 am (UTC)

Hi Chris,

Thank you very much for the guides on unit testing cocoa apps! I've had some "weird" problems with the unittests though. in some cases the rig have filed with an error message similar to the above when I do unit test for an iPhone application. I just want to share that if I set the active SDK to Simulator - iPhone OS 2.2 all is well. I honestly have no idea why this is, and I don't have the skills to find out :)

Test Host "exited abnormally with code 127"

(Anonymous)

2009-04-08 03:23 pm (UTC)

Chris -
Do these instructions apply to Xcode 3.1.2?

I have a project, the executable runs fine (It's just the default one that gets created, opens a minimally functional window.)

I've added some classes and test cases and get the formentioned error.

I did NOT set the "Target Membership" property as described in the framework version of this tutorial. I did set the Active target to the Test bundle... it had no effect.

Any ideas which way to go on this one? I did really like to develop using unit tests... in fact... i see no other way.

Thanks for these instructions and for your response in advance.

Bobby

PS. This question has been asked on the xcode-users group. No response, yet.

http://lists.apple.com/archives/xcode-users/2009/Feb/msg00341.html

Hi Chris, thanks for the great tutorial! I have still a question: is it possible to use unit testing when building a cocoa application from command line with xcodebuild install? I've got undefined symbols for all application methods called in my test class :( Everything goes fine with xcodebuid build or built from Xcode. It can be reproduced even with very basic projects. Is it possible at all? Thanks, Severin

(Deleted comment)

not working on xcode 3.2

(Anonymous)

2009-11-25 05:18 pm (UTC)

I'm using xcode 3.2, but its not possible for me to debug my test cases, allthough I did what you wrote.

???