Unit testing framework suggestions?

  • Greetings,

    Sorry for all the newbie questions.  But I'm afraid you're going to have to put up these for the next 6 months.  ;)

    I've got enough code that I want to start doing unit testing.  So far I've only discovered this one:

        <http://unitkit.org/>

    Any comments about UnitKit (good experiences, likes, dislikes, ...)?  Are there any alternatives that people would like to suggest.

    Thanks,

    James

    --
    James Bucanek <mailto:<privatereply...>
  • On Sep 22, 2004, at 5:22 PM, James Bucanek wrote:

    I've got enough code that I want to start doing unit testing. So far
    I've only discovered this one:

    <http://unitkit.org/>

    Any comments about UnitKit (good experiences, likes, dislikes, ...)?
    Are there any alternatives that people would like to suggest.
    See Also:

    http://www.sente.ch/software/ocunit/

    -jcr

    On Sep 22, 2004, at 5:22 PM, James Bucanek wrote:

    I've got enough code that I want to start doing unit testing. So far
    I've only discovered this one:

    <http://unitkit.org/>

    Any comments about UnitKit (good experiences, likes, dislikes, ...)?
    Are there any alternatives that people would like to suggest.
    See Also:

    http://www.sente.ch/software/ocunit/

    -jcr

    John C. Randolph <jcr...> (408) 974-8819
    Sr. Cocoa Software Engineer,
    Apple Worldwide Developer Relations
    http://developer.apple.com/cocoa/index.html
  • On Wed, 22 Sep 2004 17:22:29 -0700 James Bucanek <subscriber...>
    wrote:
    >
    > I've got enough code that I want to start doing unit testing.  So far I've
    only discovered this one:
    >
    > <http://unitkit.org/>
    >
    > Any comments about UnitKit (good experiences, likes, dislikes, ...)?  Are
    there any alternatives that people would like to suggest.

    See also

    http://testkit.sourceforge.net

    Tim
  • At 20:56 Uhr -0500 22.09.2004, Tim Hart wrote:

    BTW -- does anybody know some good tutorials about Unit testing? I
    don't mean about how to use the frameworks, but rather on how to do
    it for the most benefit, and why it's better than other methods, and
    how this relates to Cocoa and Objective C.

    E.g. UnitKit keeps separate "test objects", which I find kinda odd.
    I'm wondering why one shouldn't put the tests in the object in need
    of the testing instead... things like that.
    --
    Cheers,
    M. Uli Kusterer
    ------------------------------------------------------------
            "The Witnesses of TeachText are everywhere..."
                        http://www.zathras.de
  • On Thursday, September 23, 2004, at 02:54PM, M. Uli Kusterer <Witness.of.TeachText...> wrote:

    > At 20:56 Uhr -0500 22.09.2004, Tim Hart wrote:
    >> See also
    >>
    >> http://testkit.sourceforge.net
    >
    > BTW -- does anybody know some good tutorials about Unit testing? I
    > don't mean about how to use the frameworks, but rather on how to do
    > it for the most benefit, and why it's better than other methods, and
    > how this relates to Cocoa and Objective C.

    A good resource for unit testing in general:
    http://c2.com/cgi/wiki?UnitTestTutorial

    Why is it better than other methods? I'm a unit testing fan, but 'better' is a very subjective question.

    The biggest benefit I gain by keeping a suite of unit tests is this:
    I have a suite that I can run on-demand, that takes a fraction of the time it would take to manually test everything, and is a reasonable measure of the overall health of the system. A breaking test is a guarantee that there is either something wrong with the code - or the test. A unit test written to 'uncover' a bug guarantees that the same bug will not reappear (although the same symptoms due to *another* bug might). Unit tests aren't really useful unless they are run on a regular, frequent basis.

    Note that a passing suite of unit tests does *not* guarantee flawless code. FIT tests and user acceptance testing is still A Very Good Idea(tm). Unit tests do aid debugging by giving you a direct and immediate entry point into the method/function that isn't behaving appropriately.

    There are several styles to unit testing. Martin Fowler compares 2 types of unit testing here

    http://www.martinfowler.com/articles/mocksArentStubs.html

    TestKit has a beginning API for mock objects, but the API is currently absent code to count/verify how many times a method is called. That will follow soon.

    How does this relate to Cocoa/Objective-C? I've not seen an article relating to this. Can you be more specific with your questions?

    > E.g. UnitKit keeps separate "test objects", which I find kinda odd.
    > I'm wondering why one shouldn't put the tests in the object in need
    > of the testing instead... things like that.

    One of the main purposes of a unit testing suite is to enable the developer to concentrate on the tests, and not on the setup/identification and execution of those tests. A very convenient way to do this in Objective-C is to look for all subclasses of a particular class. In the case of TestKit, it's TWTestCase. That requirement does limit the developer from making tests part of the class being tested.

    I don't know that I would want to couple my tests to the class in question so tightly, though. As a separate library, I have the choice of delivering my test suite along with the code itself to the client, or not. The client has a choice of installing the test suite on the same machine - or not. It's a bit more flexible.

    NUnit, which is a unit testing kit for C# (http://www.nunit.org), would actually allow you to bind your test cases to the class being tested if your chose. The reason this is possible is due to C#'s attributes - class metadata. NUnit test cases do not have to inherit from any particular class.

    > --
    > Cheers,
    > M. Uli Kusterer
    > ------------------------------------------------------------
    > "The Witnesses of TeachText are everywhere..."
    > http://www.zathras.de
    >
    >
  • On Sep 22, 2004, at 5:22 PM, James Bucanek wrote:
    > I've got enough code that I want to start doing unit testing.  So far
    > I've only discovered this one:
    >
    > <http://unitkit.org/>
    >
    > Any comments about UnitKit (good experiences, likes, dislikes, ...)?
    > Are there any alternatives that people would like to suggest.

    I have used:

    a) ObjCUnit:  http://oops.se/objcunit
    This is a unit testing framework based on JUnit. I was pleased with it,
    although have not used it recently. It is at version 1.2, circa 2002,
    so I guess it hasn't changed recently either. It has some level of mock
    object support as well, which I have not used.

    b) OCUnit:  http://www.sente.ch/software/ocunit
    This is still what I use at work. It is also much like JUnit, although
    I think the docs state that it is actually modeled after the SmallTalk
    SUnit project (which would make OCUnit a sibling of JUnit, rather than
    an adaptation of it). I have been very satisfied with it and it works
    much as you'd expect, with basic Xcode integration. OCUnit is at
    version 38 from June of 2004.

    For mock object support, Sen:te recommends OCMock, from Mulle
    Kybernetik. I have not used OCMock much yet, but it looks very
    interesting:
        http://www.mulle-kybernetik.com/software/OCMock

    Subsequent to using the above, I have also read about, but not used:

    c) TestKit:  http://testkit.sourceforge.net
    This is another adaptation of JUnit, which has already been mentioned
    on this thread. It looks nice, and the documentation is very good. It
    features mock object support and also a GUI test runner. TestKit is at
    version 0.9.4 (June 2004).

    d) UnitKit:  http://unitkit.org
    I hadn't even heard of this one until this thread. It is a bit
    different than the others in that it seems to to be specific to
    Objective-C and intended mainly for Xcode. It does not clone the JUnit
    interface although the concepts are similar. It is at version 1.0.1
    (August 2004).

          ----------

    I would be very interested to hear people's real-world experiences with
    any of these. It is pretty hard to realistically try them all in a
    production environment. At Five Speed we use OCUnit because the
    interface is familiar, it is authored by Sen:te and so carries their
    reputation, and because it does now have Xcode integration (although
    that used to be lacking). A couple years ago we used ObjCUnit, but the
    main reason we switched (I think) was that OCUnit seemed more "alive"
    in terms of active development.

    I personally am interested to hear about real-world experiences with
    UnitKit, since its approach is the most different.

    Cheers,
    --
    Mason Mark
    Five Speed Software, Inc.
    http://www.fivespeed.com
  • > http://www.martinfowler.com/articles/mocksArentStubs.html
    >
    > TestKit has a beginning API for mock objects, but the API is currently
    > absent code to count/verify how many times a method is called. That
    > will follow soon.
    >

    For an implementation of mock objects see OCMock, at
    http://www.mulle-kybernetik.com/software/OCMock/

    Marco Scheurer
    Sen:te, Lausanne, Switzerland  http://www.sente.ch
  • On Sep 23, 2004, at 22:51, M. Uli Kusterer wrote:

    > E.g. UnitKit keeps separate "test objects", which I find kinda odd.
    > I'm wondering why one shouldn't put the tests in the object in need of
    > the testing instead... things like that.

    Most implementations of testing kits implements tests in a subclass of
    TestCase.

    One reason is that tests can have their own state and ivars, and it
    wouldn't be a good idea to mix this with the class you want to test.

    Another is that tests are written from the perspective of a user of the
    class rather than as "internal" tests. A good practice is to write the
    tests before you write the tested code. If you do so, your tests can be
    seen as a formal specification of the code to be written.

    Another is that a lot of stuff is usually done in the TestCase class,
    so that writing a test is as simple as writing a single method in a
    subclass of TestCase.

    All the xUnit packages are more or less derived from the original
    Smalltalk "SUnit". The seminal article is by Kent Beck, and can be
    found at http://www.armaties.com/testfram.htm . So there is also an
    historical reason for doing it in subclasses of Testcase.

    However, this can be done differently. For instance, Marcel Weiher
    advocates putting the tests in a category of the tested class.

    Marco Scheurer
    Sen:te, Lausanne, Switzerland  http://www.sente.ch
  • On Sep 23, 2004, at 1:51 PM, M. Uli Kusterer wrote:
    > BTW -- does anybody know some good tutorials about Unit testing? I
    > don't mean about how to use the frameworks, but rather on how to do it
    > for the most benefit, and why it's better than other methods, and how
    > this relates to Cocoa and Objective C.

    Kent Beck, "Test Driven Development" is good.  As is Martin Fowler,
    "Refactoring" and Beck's original book on Extreme Programming.

    Some of the references on the OCUnit site, to Beck et al's papers on
    the original Smalltalk unit testing framework are also pretty good.

      -- Chris
  • At 4:56 Uhr +0200 24.09.2004, Marco Scheurer wrote:
    > One reason is that tests can have their own state and ivars, and it
    > wouldn't be a good idea to mix this with the class you want to test.

      A-ha! Thanks, that makes a lot of sense. I knew there had to be some
    obvious reason.

    > (...)
    >
    > All the xUnit packages are more or less derived from the original
    > Smalltalk "SUnit". The seminal article is by Kent Beck, and can be
    > found at http://www.armaties.com/testfram.htm  . So there is also an
    > historical reason for doing it in subclasses of Testcase.
    >
    > However, this can be done differently. For instance, Marcel Weiher
    > advocates putting the tests in a category of the tested class.

      Yes, that was what I thought. It wouldn't be too hard to use e.g.
    the preprocessor to remove these tests from the class for the final
    production build. Since I know that I prefer things that are related
    (like code and the tests that go with it, or code and its
    documentation) together in one file, because otherwise it's too easy
    to have them go out of sync.

      But of course, if my test wants to keep state, it'd obviously be a
    little bothersome to have that in the same class. It would even
    violate encapsulation, come to think of it. Thanks, I think that
    aspect is now a little clearer to me.

      Just a thought: Does any of the kits have some macros for "default
    tests"? E.g.  some way to just say: Arguments to this function (or
    its result, or whatever) range from X to Y, and then it would
    generate code to try the edges of that range (both inside and out)
    automatically?
    --
    Cheers,
    M. Uli Kusterer
    ------------------------------------------------------------
            "The Witnesses of TeachText are everywhere..."
                        http://www.zathras.de
  • On Sep 24, 2004, at 2:00 PM, M. Uli Kusterer wrote:

    > At 4:56 Uhr +0200 24.09.2004, Marco Scheurer wrote:
    >> One reason is that tests can have their own state and ivars, and it
    >> wouldn't be a good idea to mix this with the class you want to test.
    >
    > A-ha! Thanks, that makes a lot of sense. I knew there had to be some
    > obvious reason.
    >
    >> (...)
    >>
    >> All the xUnit packages are more or less derived from the original
    >> Smalltalk "SUnit". The seminal article is by Kent Beck, and can be
    >> found at http://www.armaties.com/testfram.htm . So there is also an
    >> historical reason for doing it in subclasses of Testcase.
    >>
    >> However, this can be done differently. For instance, Marcel Weiher
    >> advocates putting the tests in a category of the tested class.
    >
    > Yes, that was what I thought. It wouldn't be too hard to use e.g. the
    > preprocessor to remove these tests from the class for the final
    > production build. Since I know that I prefer things that are related
    > (like code and the tests that go with it, or code and its
    > documentation) together in one file, because otherwise it's too easy
    > to have them go out of sync.

    Not really if you run your tests after each change, or frequently (at
    least daily). A lot of our users actually prefer to keep the tests
    separated from the code, in a different target or a different project.
    One reason is to avoid deploying the test code and the testing
    framework (although I think that in some cases it can be nice to be
    able to run the tests at the customer's site).

    > But of course, if my test wants to keep state, it'd obviously be a
    > little bothersome to have that in the same class. It would even
    > violate encapsulation, come to think of it. Thanks, I think that
    > aspect is now a little clearer to me.
    >
    > Just a thought: Does any of the kits have some macros for "default
    > tests"? E.g.  some way to just say: Arguments to this function (or its
    > result, or whatever) range from X to Y, and then it would generate
    > code to try the edges of that range (both inside and out)
    > automatically?

    The answer is certainly not like this, AFAIK none of the kits
    automatically generate code. To do this kind of tests with OCUnit you
    would use one of a bunch of macros to test that exceptions are thrown
    or not, for instance:

    - (void) testSetRatioRange
    {
    STAssertNoThrow ([x setRatio: minimum], nil);
    STAssertNoThrow ([x setRatio: maximum], nil);
    STAssertThrowsSpecificNamed ([x setRatio:minimum - epsilon],
    NSException, NSInvalidArgumentException, nil);
    STAssertThrowsSpecificNamed ([x setRatio:maximum + epsilon],
    NSException, NSInvalidArgumentException, nil);
    }

    I'm not sure that automatically generating such code is worth the
    effort.

    Marco Scheurer
    Sen:te, Lausanne, Switzerland  http://www.sente.ch
  • M. Uli Kusterer wrote on Friday, September 24, 2004:

    > At 4:56 Uhr +0200 24.09.2004, Marco Scheurer wrote:
    >> One reason is that tests can have their own state and ivars, and it
    >> wouldn't be a good idea to mix this with the class you want to test.
    >
    > A-ha! Thanks, that makes a lot of sense. I knew there had to be some
    > obvious reason.

    The documentation for UnitKit points out another reason why test classes are kept separate.  Namely, the performance of an application directly impacted by the size of the code and data structures.  Leaving your test code in your production code just makes for a bloated app.

    And if you're thinking that you'll remove all of the test code via the pre-processor, that's a hazardous road to go down.  You want to test the code that you're going to ship, not the code that you *think* will still work after you flip some compiler switches.  Been there, done that, got burned.

    --
    James Bucanek <mailto:<privatereply...>
  • At 14:51 Uhr +0200 24.09.2004, Marco Scheurer wrote:
    > The answer is certainly not like this, AFAIK none of the kits
    > automatically generate code. To do this kind of tests with OCUnit
    > you would use one of a bunch of macros to test that exceptions are
    > thrown or not, for instance:
    >
    > - (void) testSetRatioRange
    > {
    > STAssertNoThrow ([x setRatio: minimum], nil);
    > STAssertNoThrow ([x setRatio: maximum], nil);
    > STAssertThrowsSpecificNamed ([x setRatio:minimum - epsilon],
    > NSException, NSInvalidArgumentException, nil);
    > STAssertThrowsSpecificNamed ([x setRatio:maximum + epsilon],
    > NSException, NSInvalidArgumentException, nil);
    > }
    >
    > I'm not sure that automatically generating such code is worth the effort.

      Well, I guess one could just have a macro that does a typical
    sequence of such tests, right? After all, testing a minimum and
    maximum and that it doesn't or does throw on these should be a pretty
    common request.
    --
    Cheers,
    M. Uli Kusterer
    ------------------------------------------------------------
            "The Witnesses of TeachText are everywhere..."
                        http://www.zathras.de
  • On Sep 24, 2004, at 5:00 AM, M. Uli Kusterer wrote:
    > Just a thought: Does any of the kits have some macros for "default
    > tests"? E.g.  some way to just say: Arguments to this function (or its
    > result, or whatever) range from X to Y, and then it would generate
    > code to try the edges of that range (both inside and out)
    > automatically?

    Blatant plug alert:

    I work for a company called Parasoft (http://www.parasoft.com) that
    does this for lots of languages (C++, Java, C#), but not for
    Objective-C. The technology is pretty impressive; the tools can inspect
    the code and intelligently determine what tests are useful (it pulls
    out constants, follows Design By Contract, keeps results for regression
    tests, etc). If you can actually say that you'd be willing to buy a
    theoretical ObjCTest, give the company a call and tell them so; I'd
    love to work on the product!

    Seth A. Roby            The Amazing Llama < mail or AIM me at tallama at mac dot
    com>
    "Life is like an exploded clown. It's really funny until you figure out
    what just happened."
  • I found the book interesting, but not too useful for specifically what
    Uli was asking about.  I remember wishing for more examples of actual
    test cases.  It's more about the big project management picture, and
    selling the benefits of extreme programming, and very little about
    actually doing unit testing.

    Daniel

    On Sep 23, 2004, at 11:37 PM, Chris Hanson wrote:
    > Kent Beck, "Test Driven Development" is good.  As is Martin Fowler,
    > "Refactoring" and Beck's original book on Extreme Programming.
  • I said:

    > I found the book interesting, but not too useful for specifically what
    > Uli was asking about.  I remember wishing for more examples of actual
    > test cases.  It's more about the big project management picture, and
    > selling the benefits of extreme programming, and very little about
    > actually doing unit testing.

    Whoops!  I neglected to indicate the book I'm talking about: the Beck
    book "Extreme Programming Explained."

    On Sep 23, 2004, at 11:37 PM, Chris Hanson wrote:
    > Kent Beck, "Test Driven Development" is good.  As is Martin Fowler,
    > "Refactoring" and Beck's original book on Extreme Programming.
  • Uli, I am so glad you started this discussion. I wanted to do it this
    for the last two years or so, but never really found time...

    On Sep 23, 2004, at 10:51 PM, M. Uli Kusterer wrote:

    > E.g. UnitKit keeps separate "test objects", which I find kinda odd.
    > I'm wondering why one shouldn't put the tests in the object in need of
    > the testing instead... things like that.

    I use testing frameworks for many years - something like 15 - long
    before xUnit, XP, Agile etc were born. And I used OCUnit almost from
    the day it was born, until ...

    Until one of my students (a bright chap) came to me and told me that he
    wasted a day before he could write his first test method. His exact
    words were "... testing  with OCUnit is not the simplest thing that
    could possibly work" [OCUnit was at the time probably the only
    framework for ObjC - but my student was criticizing the method, not the
    framework]

    Well, this gave me a lot of food for thinking. At that time I was
    already reading a very early draft of Kent's TDD book, and happy I was
    not - not because of the technique, but because of the technology... So
    after thinking about this for almost two months, finally I came to the
    conclusion that the problem is exactly the one you discovered -
    separation of TestCase from the class being tested [well, in reality
    this is not always true -- but please read my replays to Marco I intend
    to write later today for more].

    At the time Drew and I was starting to put our thoughts about
    multi-volume text book on Cocoa programming (still work in [slow]
    progress), so we had a beer together and write down (on a A6 index
    card) the requirements for a testing framework to be publish in the
    book. Here what we scribbled:
    - the integration to any cocoa project (existing or new) should not
    take longer then a minute
    - one should learn the basics in max 5 mins
    - test could be placed anywhere (in TestCase or directly in the class).

    Later, we added three additional requirements:
    - there should be no technical distinction between unit and acceptance
    tests,
    - the framework should be able to test the UI, and
    - test runs should have memory - ignore for now, it is a subject of
    completely different thread ...

    We ended up writing something that at first looked like a simplified
    version of xUnit. But is was not! Very soon I discovered that actually
    it is completely new beast. The difference is more on how it changed my
    way of testing - the flow. Now actually I can not call it testing any
    more ... it feels more like compiling, or debugging - a real integrated
    part of the development process. And it is part of the program you
    write (as it should be - I discovered this much later) - not part of
    the IDE as the original xUnit was supposed. I am sure all this has some
    far reaching philosophical consequences, and although I discussed it
    privately with few prominent XPers and Pragmatic Programmers, I never
    found time to go deeper into this.

    I had the chance to teach Cocoa to a (very) large Apple third party
    software provider, and to test most of our ideas. I observed something
    very surprising - developers who were exposed to testing framework for
    the first time were able to write their first test within 10 min, and
    after a day they were semi-experts. But there we two developers with
    prior JUnit experience. They struggled  really badly! Actually they
    needed about a week before feel comfortable with our framework. It
    might sound strange, but I went through the same initial struggle. I am
    not sure, but this could be comparable to the transition of X11
    developers to Cocoa - the first thing they ask for is the "while(1) {
    //get next event }" loop - it takes them some time before they stop
    feeling the need for such a beast...

    Drew and I want to make the sources and some texts freely available,
    and we have a permission already to do this, but realistically this
    will not happen before the end of october. So I hope you stay tuned :-)

    Georg Tuparev
    Tuparev Technologies
    Klipper 13
    1186 VR Amstelveen
    The Netherlands
    Mobile: +31-6-55798196
  • It happens that I disagree with Marco once each decade (so do not
    expect me to start discussing with him next 10 years) ... but I feel
    this is one of these occasions...

    Also read this in the context of my previous replay to Uli.

    On Sep 24, 2004, at 4:56 AM, Marco Scheurer wrote:

    > On Sep 23, 2004, at 22:51, M. Uli Kusterer wrote:
    >
    >> E.g. UnitKit keeps separate "test objects", which I find kinda odd.
    >> I'm wondering why one shouldn't put the tests in the object in need
    >> of the testing instead... things like that.
    >
    > Most implementations of testing kits implements tests in a subclass of
    > TestCase.

    True. As far as I know only NUnit - and I believe because of technical
    reasons (I might be wrong - never spent too much time on it).

    >
    > One reason is that tests can have their own state and ivars, and it
    > wouldn't be a good idea to mix this with the class you want to test.

    You might be right is theory, but at least I and all the people I
    enjoyed working with almost never kept state between two tests.
    Actually, if I think very hard, I cannot find a single case where my
    need of using state was not because of stinky code - or bad design, or
    bad refactoring. In our testing framework, we create an instance of the
    tested class for every single test method. In cases where we need
    singleton we need to override the setUp and tearDown methods. For class
    like App Delegate (instantiated in a nib file) we write:

    + (id)setUp { return [NSApp delegate]; }
    + (void)tearDown:(id)testInstance  { }

    This feels strange at the beginning - normally one needs to implement
    setUp and tearDown for each TestCase class, but  with our framework we
    need from time to time to knock them off - most often though one never
    touches them. Also note the "+" in front of these methods. This is not
    a typo :-)

    Here a typical test method:

    - (void)testVisibleWindow {
        SEConfirmIsTrue(tableWindow != nil, @"Window should not be nil");
        SEConfirmIsTrue([tableWindow isVisible],
                        @"Window should be visible after the application is
    launched");
    }

    UI testing made easy...

    > Another is that tests are written from the perspective of a user of
    > the class rather than as "internal" tests. A good practice is to write
    > the tests before you write the tested code. If you do so, your tests
    > can be seen as a formal specification of the code to be written.

    Developer writes tests first - to help him or her to write the real
    code. So you write a test not as a user, but as an author. Tests are
    also the best known to me way of documenting - and you want to keep the
    doc close to the code - otherwise they will get out of sync - therefore
    test are so good - they must be in sync if they have to pass! [I am
    sure this might sound like Knuth's Literate Programming - in retrospect
    I have to acknowledge that after Drew and I wrote the first version of
    our testing framework, I though that this is the closest practical way
    of doing literate programming for Cocoa]

    > Another is that a lot of stuff is usually done in the TestCase class,
    > so that writing a test is as simple as writing a single method in a
    > subclass of TestCase.

    Luckily we have categories in ObjC. And no one stops you putting this
    "lot of stuff" to a category of NSObject :-) But more importantly,
    there is no need to have a lot of stuff in TestCase class.

    [...]

    Georg Tuparev
    Tuparev Technologies
    Klipper 13
    1186 VR Amstelveen
    The Netherlands
    Mobile: +31-6-55798196
  • On Sep 24, 2004, at 2:51 PM, Marco Scheurer wrote:

    >> Yes, that was what I thought. It wouldn't be too hard to use e.g.
    >> the preprocessor to remove these tests from the class for the final
    >> production build. Since I know that I prefer things that are related
    >> (like code and the tests that go with it, or code and its
    >> documentation) together in one file, because otherwise it's too easy
    >> to have them go out of sync.
    >
    > Not really if you run your tests after each change, or frequently (at
    > least daily). A lot of our users actually prefer to keep the tests
    > separated from the code, in a different target or a different project.

    Why?

    > One reason is to avoid deploying the test code and the testing
    > framework (although I think that in some cases it can be nice to be
    > able to run the tests at the customer's site).

    Some cases? Always! This saved us many debugging hours. And users are
    in general impressed. When they see that your program has 3000 tests,
    they are really impressed! And because we test UI too, they think it is
    fun too - too see all these windows hopping around. BTW, we had one
    client who told us to keep the tests always on. This "helped him to
    understand better developers intentions"...

    Georg Tuparev
    Tuparev Technologies
    Klipper 13
    1186 VR Amstelveen
    The Netherlands
    Mobile: +31-6-55798196
  • On Sep 24, 2004, at 5:27 PM, James Bucanek wrote:

    >> A-ha! Thanks, that makes a lot of sense. I knew there had to be some
    >> obvious reason.
    >
    > The documentation for UnitKit points out another reason why test
    > classes are kept separate.  Namely, the performance of an application
    > directly impacted by the size of the code and data structures.
    > Leaving your test code in your production code just makes for a
    > bloated app.

    Well, we did made measurement for our last two projects - depending on
    the application and on the computer, the performance deteriorates
    between 0% and 3% (on average 1%). Not worth discussing - specially
    because tests helped us often to have very good performance to start
    with.

    Georg Tuparev
    Tuparev Technologies
    Klipper 13
    1186 VR Amstelveen
    The Netherlands
    Mobile: +31-6-55798196
  • On Sep 24, 2004, at 5:57 PM, M. Uli Kusterer wrote:

    >> - (void) testSetRatioRange
    >> {
    >> STAssertNoThrow ([x setRatio: minimum], nil);
    >> STAssertNoThrow ([x setRatio: maximum], nil);
    >> STAssertThrowsSpecificNamed ([x setRatio:minimum - epsilon],
    >> NSException, NSInvalidArgumentException, nil);
    >> STAssertThrowsSpecificNamed ([x setRatio:maximum + epsilon],
    >> NSException, NSInvalidArgumentException, nil);
    >> }
    >>
    >> I'm not sure that automatically generating such code is worth the
    >> effort.
    >
    > Well, I guess one could just have a macro that does a typical
    > sequence of such tests, right? After all, testing a minimum and
    > maximum and that it doesn't or does throw on these should be a pretty
    > common request.

    That's not the point! There are many test-coverage frameworks that are
    as useful as Rational designing tool...

    Good unit tests help the developer to:
    - design the application,
    - document the sources,
    - gain confidence (courage) needed to make major refactorings,
    - fulfill the promise of delivering business value to the clients (e.g.
    by demonstrating that acceptance tests run).

    None of these tasks is to be autogenerated by macro!

    Georg Tuparev
    Tuparev Technologies
    Klipper 13
    1186 VR Amstelveen
    The Netherlands
    Mobile: +31-6-55798196
  • On Sep 26, 2004, at 01:33, Georg Tuparev wrote:

    > It happens that I disagree with Marco once each decade (so do not
    > expect me to start discussing with him next 10 years) ... but I feel
    > this is one of these occasions...

    Oh! I'l have to show some disagreement too then!

    >> One reason is that tests can have their own state and ivars, and it
    >> wouldn't be a good idea to mix this with the class you want to test.
    >
    > You might be right is theory, but at least I and all the people I
    > enjoyed working with almost never kept state between two tests.
    > Actually, if I think very hard, I cannot find a single case where my
    > need of using state was not because of stinky code - or bad design, or
    > bad refactoring.

    I admit that it is not frequent, and one could certainly go away
    without that possibility.

    > In our testing framework, we create an instance of the tested class
    > for every single test method.

    If I understand this correctly, this is exactly what is done in OCUnit
    and SmalltalkUnit.

    > In cases where we need singleton we need to override the setUp and
    > tearDown methods. For class like App Delegate (instantiated in a nib
    > file) we write:
    >
    > + (id)setUp { return [NSApp delegate]; }
    > + (void)tearDown:(id)testInstance  { }
    >
    > This feels strange at the beginning - normally one needs to implement
    > setUp and tearDown for each TestCase class,

    No. You can but don't have to implement setUp and tearDown. You do it
    only when you want to initialize all your TestCase tests to a common
    state.

    > but  with our framework we need from time to time to knock them off -
    > most often though one never touches them. Also note the "+" in front
    > of these methods. This is not a typo :-)

    OCUnit also has +setUp...

    > Here a typical test method:
    >
    > - (void)testVisibleWindow {
    > SEConfirmIsTrue(tableWindow != nil, @"Window should not be nil");
    > SEConfirmIsTrue([tableWindow isVisible],
    > @"Window should be visible after the application
    > is launched");
    > }
    >
    > UI testing made easy...

    I fail to see any differences with how this would be done using OCUnit,
    except the name of the macros. Until I see more of this kit, I'm still
    unconvinced: it looks to me like one of the many "I can make it
    simpler" attempt, which evolves in mostly the same thing as OCUnit, a
    good example of the NIH syndrome.

    With an installer package, target and file templates and a README, I
    doubt that it would take anyone more than 5 minutes to write a first
    test using OCUnit, unless of course one is new to the whole idea.

    >> Another is that tests are written from the perspective of a user of
    >> the class rather than as "internal" tests. A good practice is to
    >> write the tests before you write the tested code. If you do so, your
    >> tests can be seen as a formal specification of the code to be
    >> written.
    >
    > Developer writes tests first - to help him or her to write the real
    > code. So you write a test not as a user, but as an author. Tests are
    > also the best known to me way of documenting - and you want to keep
    > the doc close to the code - otherwise they will get out of sync -
    > therefore test are so good - they must be in sync if they have to
    > pass!

    If you test all the time (say after each change), it doesn'y matter
    where your tests are, they will not get out of sync. If you don't, it
    doesn't matter where they are either, they will get out of sync. So
    this to "keep your tests in sync" is not a reason for having them in
    one class or another.

    > [I am sure this might sound like Knuth's Literate Programming - in
    > retrospect I have to acknowledge that after Drew and I wrote the first
    > version of our testing framework, I though that this is the closest
    > practical way of doing literate programming for Cocoa]

    I said that much when talking about a "formal specification".

    In truth my point about writing tests first also has little to do with
    where they are implemented. You can of course start writing your class
    by writing the test methods. But it seems more natural to me to do so
    as an "outsider", the customer who specifies what the methods are
    supposed to do.

    >> Another is that a lot of stuff is usually done in the TestCase class,
    >> so that writing a test is as simple as writing a single method in a
    >> subclass of TestCase.
    >
    > Luckily we have categories in ObjC. And no one stops you putting this
    > "lot of stuff" to a category of NSObject :-) But more importantly,
    > there is no need to have a lot of stuff in TestCase class.

    Yes, as I said that's a possibility. And then in order to keep your
    files shorter you may want to put the testing categories in separate
    files, and if you do so you gain nothing by having the tests in a
    category instead of a subclass of TestCase. However you have lost some
    flexibility, like the ability to store a state when you need it.

    marco

    Marco Scheurer
    Sen:te, Lausanne, Switzerland  http://www.sente.ch
  • On Sep 26, 2004, at 01:41, Georg Tuparev wrote:

    > On Sep 24, 2004, at 2:51 PM, Marco Scheurer wrote:
    >>
    >> Not really if you run your tests after each change, or frequently (at
    >> least daily). A lot of our users actually prefer to keep the tests
    >> separated from the code, in a different target or a different
    >> project.
    >
    > Why?
    >
    >> One reason is to avoid deploying the test code and the testing
    >> framework (although I think that in some cases it can be nice to be
    >> able to run the tests at the customer's site).
    >
    > Some cases? Always! This saved us many debugging hours.

    Yes, especially for custom software, having the possibility to ask your
    customer to launch the application in test mode in his environment can
    be helpful.

    However, most of our users told us they wanted to be able to ship
    without the tests and the testing infrastructure. This also make sense,
    especially for "shrink-wrap": they are some issues with installing and
    depending on a 3rd party framework.

    So we leave that choice to our users.

    > And users are in general impressed. When they see that your program
    > has 3000 tests, they are really impressed! And because we test UI too,
    > they think it is fun too - too see all these windows hopping around.
    > BTW, we had one client who told us to keep the tests always on. This
    > "helped him to understand better developers intentions"...

    Funny, some say that you write tests to better understand the customer
    intentions... ;-)

    marco

    Marco Scheurer
    Sen:te, Lausanne, Switzerland  http://www.sente.ch
  • On Sep 26, 2004, at 13:01, Marco Scheurer wrote:
    >> Here a typical test method:
    >>
    >> - (void)testVisibleWindow {
    >> SEConfirmIsTrue(tableWindow != nil, @"Window should not be nil");
    >> SEConfirmIsTrue([tableWindow isVisible],
    >> @"Window should be visible after the application
    >> is launched");
    >> }
    >>
    >> UI testing made easy...
    >
    > I fail to see any differences with how this would be done using
    > OCUnit, except the name of the macros.

    Sorry to followup on my own post, but I hit reply to soon! Another
    difference is of course that by writing the tests in a category, you
    have access to the internal state of the class (like the ivar
    tableWindow), and you do not need access methods...

    Marco Scheurer
    Sen:te, Lausanne, Switzerland  http://www.sente.ch
  • On 25 Sep 2004, at 16:32, Georg Tuparev wrote:

    >
    > Well, this gave me a lot of food for thinking. At that time I was
    > already reading a very early draft of Kent's TDD book, and happy I was
    > not - not because of the technique, but because of the technology...
    > So after thinking about this for almost two months, finally I came to
    > the conclusion that the problem is exactly the one you discovered -
    > separation of TestCase from the class being tested [well, in reality
    > this is not always true -- but please read my replays to Marco I
    > intend to write later today for more].

    Yes, this was my impression as well, at first simply a conjecture and a
    code smell:  parallel class hierarchies.  Now that I've worked with
    JUnit for well over a year, I am absolutely convinced that my decision
    to allow the class to test itself for MPWTest was precisely right.

    > At the time Drew and I was starting to put our thoughts about
    > multi-volume text book on Cocoa programming (still work in [slow]
    > progress), so we had a beer together and write down (on a A6 index
    > card) the requirements for a testing framework to be publish in the
    > book. Here what we scribbled:
    > - the integration to any cocoa project (existing or new) should not
    > take longer then a minute
    > - one should learn the basics in max 5 mins
    > - test could be placed anywhere (in TestCase or directly in the class).

    Hmm...I guess I should (have) advertise(d) MPWTestKit a little more
    heavily, because it has these attributes.  There are hooks for more
    flexibility if you need it, but the basics are absolutely trivial.

    +testSelectors
    {
    return [NSArray
    arrayWithObjects:@"myWonderfulTest1",@"myWonderfulTest2",nil];
    }

    +myWonderfulTest1
    {
    }
    +myWonderfulTest2
    {
    }

    [Automatic test-case discovery could be added, but I actually find the
    explicit listing of my tests useful]

    [snip more requirements]

    > We ended up writing something that at first looked like a simplified
    > version of xUnit. But is was not! Very soon I discovered that actually
    > it is completely new beast. The difference is more on how it changed
    > my way of testing - the flow. Now actually I can not call it testing
    > any more ... it feels more like compiling, or debugging - a real
    > integrated part of the development process. And it is part of the
    > program you write (as it should be - I discovered this much later) -
    > not part of the IDE as the original xUnit was supposed.

    YES!  That is exactly my experience with MPWTestKit.  The tests are an
    integral part of each class that is written.  They are always there.
    They do NOT produce a dependency on the unit test framework, which only
    adds infrastructure to run the tests in a controlled/grouped fashion.

    In addition to the tests being automatically associated by a class,
    they are automatically grouped by framework.
  • On Sep 26, 2004, at 1:01 PM, Marco Scheurer wrote:

    >> [I am sure this might sound like Knuth's Literate Programming - in
    >> retrospect I have to acknowledge that after Drew and I wrote the
    >> first version of our testing framework, I though that this is the
    >> closest practical way of doing literate programming for Cocoa]
    >
    > I said that much when talking about a "formal specification".
    >
    > In truth my point about writing tests first also has little to do with
    > where they are implemented. You can of course start writing your class
    > by writing the test methods. But it seems more natural to me to do so
    > as an "outsider", the customer who specifies what the methods are
    > supposed to do.

    I hate split universes! It feels schizophrenic! If you by a book, you
    have the text and all illustrations together. Imagine how it will feel
    if they are separated in two books (three in the case of programming:
    sources, tests, documentation)... But when I separate all these three
    components, soon (after a month or so) I start feeling outsider.

    Also I do not want to be an outsider to my own sources! Someone needs
    to be an insider after all. I also feel strong about (collective) code
    ownership, and I and my colleagues like to be proud about our craft. We
    cannot be proud for something we feel like outsiders. And emotions
    correlate strongly with productivity.

    About your comment of "yet another do it simpler framework". You are
    right to suggest that every framework evolves and gets more complex
    with the time. The same will happen (already did) with our framework.
    But, it evolves in different direction. Our test analysis gets more
    sophisticated, and our UI nicer. But tests are just tests - very
    simple. They will not get more clever. There is no TestCase, or
    TestSuite , there is just no place where one can add bells and
    whistles.

    Georg Tuparev
    Tuparev Technologies
    Klipper 13
    1186 VR Amstelveen
    The Netherlands
    Mobile: +31-6-55798196
  • On Sep 26, 2004, at 5:37, Marco Scheurer wrote:
    > Another difference is of course that by writing the tests in a
    > category, you have access to the internal state of the class (like the
    > ivar tableWindow), and you do not need access methods...

    A good suite of unit tests enables refactoring. However, if the tests
    access internals of the production code, the tests are getting in the
    way of refactoring (because they would always need to be changed
    together with the production code). That's why I think ideally unit
    tests should only test the public/official api, for which categories
    are not necessary.

    Besides, if all unit tests were in the form of categories, how would
    one test a production code category?

    Christian
  • On Sep 26, 2004, at 5:59, Marcel Weiher wrote:
    > On 25 Sep 2004, at 16:32, Georg Tuparev wrote:
    >> finally I came to the conclusion that the problem is exactly the one
    >> you discovered - separation of TestCase from the class being tested
    >> [well, in reality this is not always true -- but please read my
    >> replays to Marco I intend to write later today for more].
    >
    > Yes, this was my impression as well, at first simply a conjecture and
    > a code smell:  parallel class hierarchies.  Now that I've worked with
    > JUnit for well over a year, I am absolutely convinced that my decision
    > to allow the class to test itself for MPWTest was precisely right.

    Parallel class hierarchies are a code smell if both hierarchies are
    production code. We're dealing with one hierarchy of production code
    and another one of unit testing code. To me, that doesn't smell.

    In my experience, for anything but trivial systems, these two class
    hierarchies are not an exact mirror of each other. Often times I have
    more than one test class per production class. These test classes test
    different aspects of the production class and usually have different
    setups. Sometimes I also have a testclass to test the interaction
    between certain production classes. And other timed I don't have a test
    class for a production class at all because I only test what could
    possibly fail and what requires me to do TDD.

    Christian
  • [sorry for the smime-less repost]

    On Sep 26, 2004, at 5:37, Marco Scheurer wrote:
    > Another difference is of course that by writing the tests in a
    > category, you have access to the internal state of the class (like the
    > ivar tableWindow), and you do not need access methods...

    A good suite of unit tests enables refactoring. However, if the tests
    access internals of the production code, the tests are getting in the
    way of refactoring (because they would always need to be changed
    together with the production code). That's why I think ideally unit
    tests should only test the public/official api, for which categories
    are not necessary.

    Besides, if all unit tests were in the form of categories, how would
    one test a production code category?

    Christian
  • [sorry for the smime-less repost]

    On Sep 26, 2004, at 5:59, Marcel Weiher wrote:
    > On 25 Sep 2004, at 16:32, Georg Tuparev wrote:
    >> finally I came to the conclusion that the problem is exactly the one
    >> you discovered - separation of TestCase from the class being tested
    >> [well, in reality this is not always true -- but please read my
    >> replays to Marco I intend to write later today for more].
    >
    > Yes, this was my impression as well, at first simply a conjecture and
    > a code smell:  parallel class hierarchies.  Now that I've worked with
    > JUnit for well over a year, I am absolutely convinced that my decision
    > to allow the class to test itself for MPWTest was precisely right.

    Parallel class hierarchies are a code smell if both hierarchies are
    production code. We're dealing with one hierarchy of production code
    and another one of unit testing code. To me, that doesn't smell.

    In my experience, for anything but trivial systems, these two class
    hierarchies are not an exact mirror of each other. Often times I have
    more than one test class per production class. These test classes test
    different aspects of the production class and usually have different
    setups. Sometimes I also have a testclass to test the interaction
    between certain production classes. And other timed I don't have a test
    class for a production class at all because I only test what could
    possibly fail and what requires me to do TDD.

    Christian
  • On Sep 26, 2004, at 19:25, Christian Pekeler wrote:

    > On Sep 26, 2004, at 5:37, Marco Scheurer wrote:
    >> Another difference is of course that by writing the tests in a
    >> category, you have access to the internal state of the class (like
    >> the ivar tableWindow), and you do not need access methods...
    >
    > A good suite of unit tests enables refactoring. However, if the tests
    > access internals of the production code, the tests are getting in the
    > way of refactoring (because they would always need to be changed
    > together with the production code). That's why I think ideally unit
    > tests should only test the public/official api, for which categories
    > are not necessary.

    Oh yes, I agree totally. I did not see this as an advantage, just a
    difference.

    Marco Scheurer
    Sen:te, Lausanne, Switzerland  http://www.sente.ch
  • On Sep 26, 2004, at 13:59, Marcel Weiher wrote:

    > Hmm...I guess I should (have) advertise(d) MPWTestKit a little more
    > heavily, because it has these attributes.  There are hooks for more
    > flexibility if you need it, but the basics are absolutely trivial.
    >
    > +testSelectors
    > {
    > return [NSArray
    > arrayWithObjects:@"myWonderfulTest1",@"myWonderfulTest2",nil];
    > }
    >
    > +myWonderfulTest1
    > {
    > }
    > +myWonderfulTest2
    > {
    > }

    Hey, now I have to disagree with not only Georg, but also Marcel!

    Don't you think that the fact that these tests need to be in class
    methods smelly? Of course you keep them "close" to the tested class,
    but you still have an instance of one class testing an instance of
    another class.

    If the idea is to have both testing and tested code in the same file,
    nothing forbid having two classes in the same file either.

    And I don't buy the argument that writing tests in a category rather
    than a class change the whole perspective on unit testing and how it
    makes it so much more integrated in the development activity.

    Of course OCUnit has TestCase and TesSuite, but that's an
    implementation detail with which nobody needs to care if they don't
    want to. They are not getting in the way of writing tests. For
    instances TestSuite are automatically created to group the test cases
    of different compilation units.

    I'm a fan of categories, but in that case, I don't see the point.
    Writing a test using OCUnit and, I suppose, many others is just as
    trivial:

    @implementation MyTestCase:TestCase
    - (void) testWonder1
    {
    }
    @end

    One way or another the only required activities are to write test
    methods and to run them.

    Marco Scheurer
    Sen:te, Lausanne, Switzerland  http://www.sente.ch
  • On Sep 26, 2004, at 8:23 PM, Marco Scheurer wrote:

    >
    > On Sep 26, 2004, at 19:25, Christian Pekeler wrote:
    >
    >> On Sep 26, 2004, at 5:37, Marco Scheurer wrote:
    >>> Another difference is of course that by writing the tests in a
    >>> category, you have access to the internal state of the class (like
    >>> the ivar tableWindow), and you do not need access methods...
    >>
    >> A good suite of unit tests enables refactoring. However, if the tests
    >> access internals of the production code, the tests are getting in the
    >> way of refactoring (because they would always need to be changed
    >> together with the production code). That's why I think ideally unit
    >> tests should only test the public/official api, for which categories
    >> are not necessary.
    >
    > Oh yes, I agree totally. I did not see this as an advantage, just a
    > difference.
    >
    > Marco Scheurer
    > Sen:te, Lausanne, Switzerland  http://www.sente.ch

    Wait a second folks. I thought the discussion was about Unit Tests and
    not about Acceptance Tests. Besides, if I cannot tests internals of a
    class, who will give me the courage to refactor these internals? Often
    these internals make more then 90% of the class. For example  a
    Telescope class might have a public method isTracking that simply
    returns a BOOL ivar set by an internal method called 10x each second
    that that corrects the geometry of the flex mirror, reads a frame from
    the helper CCD, does some nasty image convolution, calls eigen vectors
    based RMS routine and passes the result to a weighting function that
    finally decides if the telescope is still tracking or the object was
    lost. For a user of the class Telescopes these might be unimportant
    details, but for me - the author of the class - these are the most
    important methods, and I would like to be able to test them no matter
    if someone have academic or other objections...

    BTW, a good suite of tests enables refactoring by first braking few
    existing tests! If I make even a simple refactoring, say "rename
    method", and my test bar is still green I know I am in trouble and I do
    not have enough tests! When the tests are broken, I start fixing them.
    When the bar is green again, I commit. If the bar stays red for longer
    then an hour, I revert.

    Georg Tuparev
    Tuparev Technologies
    Klipper 13
    1186 VR Amstelveen
    The Netherlands
    Mobile: +31-6-55798196
  • On Sep 26, 2004, at 7:35 PM, Christian Pekeler wrote:

    > In my experience, for anything but trivial systems, these two class
    > hierarchies are not an exact mirror of each other. Often times I have
    > more than one test class per production class. These test classes test
    > different aspects of the production class and usually have different
    > setups. Sometimes I also have a testclass to test the interaction
    > between certain production classes. And other timed I don't have a
    > test class for a production class at all because I only test what
    > could possibly fail and what requires me to do TDD.

    If you read carefully my initial email, you would be surprised to
    discover that I am not saying that all test code should stay in the
    class being tested. But it is my experience, that this feels most
    naturally for about 70% of all cases. For these 70% of all tests, I do
    not want to be forced to use TestCase subclasses, because of reasons I
    explained in an earlier posting. For these 70% of the tests, it is also
    TSTTCPW to keep them close to the code they test, and this is a good
    thing!

    If I have to chose between both extremes (xUnit - all tests re in
    TestCase subclasses, and Marcel's way - non of them is in a TestCase
    class) I would confidently choose MPWTestKit...

    Georg Tuparev
    Tuparev Technologies
    Klipper 13
    1186 VR Amstelveen
    The Netherlands
    Mobile: +31-6-55798196
  • On 26 Sep 2004, at 12:12, Marco Scheurer wrote:

    > On Sep 26, 2004, at 01:41, Georg Tuparev wrote:
    >>
    >> Some cases? Always! This saved us many debugging hours.
    >
    > Yes, especially for custom software, having the possibility to ask
    > your customer to launch the application in test mode in his
    > environment can be helpful.
    >
    > However, most of our users told us they wanted to be able to ship
    > without the tests and the testing infrastructure. This also make
    > sense, especially for "shrink-wrap": they are some issues with
    > installing and depending on a 3rd party framework.

    Yes, that is precisely why MPWTest does not create a dependency on the
    test-framework.  You can leave your tests in the code without having to
    ship the testing framework.

    > So we leave that choice to our users.

    There is always choice:  you can put the categories in a separate file,
    lib or framework and ship without it.  You can also create a subclass
    for your tests, if you want to.  But you don't have to, and that's very
    important, IMHO.

    Marcel
  • On 26 Sep 2004, at 12:01, Marco Scheurer wrote:

    > On Sep 26, 2004, at 01:33, Georg Tuparev wrote:

    >>> One reason is that tests can have their own state and ivars, and it
    >>> wouldn't be a good idea to mix this with the class you want to test.
    >>
    >> You might be right is theory, but at least I and all the people I
    >> enjoyed working with almost never kept state between two tests.
    >> Actually, if I think very hard, I cannot find a single case where my
    >> need of using state was not because of stinky code - or bad design,
    >> or bad refactoring.
    >
    > I admit that it is not frequent, and one could certainly go away
    > without that possibility.

    Precisely.  So you should create a design that allows for the
    infrequent case, but caters to the frequent case.

    >
    >>> Another is that a lot of stuff is usually done in the TestCase
    >>> class, so that writing a test is as simple as writing a single
    >>> method in a subclass of TestCase.
    >>
    >> Luckily we have categories in ObjC. And no one stops you putting this
    >> "lot of stuff" to a category of NSObject :-) But more importantly,
    >> there is no need to have a lot of stuff in TestCase class.
    >
    > Yes, as I said that's a possibility. And then in order to keep your
    > files shorter you may want to put the testing categories in separate
    > files, and if you do so you gain nothing by having the tests in a
    > category instead of a subclass of TestCase. However you have lost some
    > flexibility, like the ability to store a state when you need it.

    The opposite is true.  You do not lose any flexibility, because you
    always *can* create a new class if you want to, with all the state you
    require.  You just don't *have to* create a new class, for all those
    times where you don't need it.
  • On 26 Sep 2004, at 00:33, Georg Tuparev wrote:

    > In our testing framework, we create an instance of the tested class
    > for every single test method.

    You do?  Automatically by the test framework?  I tend to leave this up
    to the test method in question (which will be a class method of the
    class under test unless you configure it differently).
  • On Sep 27, 2004, at 1:20 AM, Marcel Weiher wrote:

    >> In our testing framework, we create an instance of the tested class
    >> for every single test method.
    >
    > You do?  Automatically by the test framework?  I tend to leave this up
    > to the test method in question (which will be a class method of the
    > class under test unless you configure it differently).

    TSTTCPW as usual. Most of the time our default behavior is all we need.
    For the exceptions we override the setUp method...

    Georg Tuparev
    Tuparev Technologies
    Klipper 13
    1186 VR Amstelveen
    The Netherlands
    Mobile: +31-6-55798196
  • On 26 Sep 2004, at 21:57, Georg Tuparev wrote:
    > On Sep 26, 2004, at 7:35 PM, Christian Pekeler wrote:
    >
    >> In my experience, for anything but trivial systems, these two class
    >> hierarchies are not an exact mirror of each other. Often times I have
    >> more than one test class per production class. These test classes
    >> test different aspects of the production class and usually have
    >> different setups. Sometimes I also have a testclass to test the
    >> interaction between certain production classes. And other timed I
    >> don't have a test class for a production class at all because I only
    >> test what could possibly fail and what requires me to do TDD.
    >
    > If you read carefully my initial email, you would be surprised to
    > discover that I am not saying that all test code should stay in the
    > class being tested. But it is my experience, that this feels most
    > naturally for about 70% of all cases. For these 70% of all tests, I do
    > not want to be forced to use TestCase subclasses, because of reasons I
    > explained in an earlier posting. For these 70% of the tests, it is
    > also TSTTCPW to keep them close to the code they test, and this is a
    > good thing!
    >
    > If I have to chose between both extremes (xUnit - all tests re in
    > TestCase subclasses, and Marcel's way - non of them is in a TestCase
    > class) I would confidently choose MPWTestKit...

    Too much honor heaped on me:  MPWTest isn't nearly that extreme.  It
    does allow, and I do use, completely separate test-classes.  They are
    just rather rare, the majority of tests do not requie, and thus do not
    get, a separate class.

    Cheers,

    Marcel

    --
    Marcel Weiher                Metaobject Software Technologies
    <marcel...>        www.metaobject.com
    Metaprogramming for the Graphic Arts.  HOM, IDEAs, MetaAd etc.
      1d480c25f397c4786386135f8e8938e4
  • On 26 Sep 2004, at 19:57, Marco Scheurer wrote:

    > On Sep 26, 2004, at 13:59, Marcel Weiher wrote:
    >
    >> Hmm...I guess I should (have) advertise(d) MPWTestKit a little more
    >> heavily, because it has these attributes.  There are hooks for more
    >> flexibility if you need it, but the basics are absolutely trivial.
    >>
    >> +testSelectors
    >> {
    >> return [NSArray
    >> arrayWithObjects:@"myWonderfulTest1",@"myWonderfulTest2",nil];
    >> }
    >>
    >> +myWonderfulTest1
    >> {
    >> }
    >> +myWonderfulTest2
    >> {
    >> }
    >
    > Hey, now I have to disagree with not only Georg, but also Marcel!
    >
    > Don't you think that the fact that these tests need to be in class
    > methods smelly?

    No.  Also, they don't *need* to be in class methods. You can, if you
    want to, override the +testFixture method to return any object you
    please, and that will then be the test-fixture for that class.
    However, the default is the class itself and I hardly every found
    myself sufficiently motivated to change it.

    > Of course you keep them "close" to the tested class, but you still
    > have an instance of one class testing an instance of another class.

    Huh?  I have the class testing instance of itself.  This seems
    perfectly reasonable and has caused me absolutely no problems.  Why do
    you say an instance of one class testing an instance of another class?
    Do you mean an instance of the metaclass??

    > If the idea is to have both testing and tested code in the same file,
    > nothing forbid having two classes in the same file either.

    Sure.  And if there is some reason why I *do* need a subclass to test,
    then I do precisely that, for example when testing a category of a
    class that already has tests.  I could also do it if I actually needed
    extra state.

    So it's YAGNI and DTSTTCPW:  yes, the things you state *can* happen,
    but they are trivial to deal with when they do and for the 90% of the
    time that they are not needed there is a simpler solution.

    > And I don't buy the argument that writing tests in a category rather
    > than a class change the whole perspective on unit testing and how it
    > makes it so much more integrated in the development activity.

    I wouldn't buy it from a purely theoretical point of view either, but
    after a little more than 1 year of JUnit (after a number of years with
    MPWTest), I can say that it has definitely been the case.  YMMV.

    > Of course OCUnit has TestCase and TesSuite, but that's an
    > implementation detail with which nobody needs to care if they don't
    > want to. They are not getting in the way of writing tests. For
    > instances TestSuite are automatically created to group the test cases
    > of different compilation units.

    Yes, MPWTest creates both TestSuites and TestCases automatically
    (though you could interfere and get your own grouping).

    > I'm a fan of categories, but in that case, I don't see the point.
    > Writing a test using OCUnit and, I suppose, many others is just as
    > trivial:
    >
    > @implementation MyTestCase:TestCase
    > - (void) testWonder1
    > {
    > }
    > @end

    Now I have to wonder:  you complained above about an instance of one
    class testing an instance of a different class (which wasn't the case),
    and this is precisely the case here.  How is MyTestCase related to the
    class being tested?

    > One way or another the only required activities are to write test
    > methods and to run them.

    Not quite.  At the very least, you need to link in the testing
    framework.

    Marcel

    --
    Marcel Weiher                Metaobject Software Technologies
    <marcel...>        www.metaobject.com
    Metaprogramming for the Graphic Arts.  HOM, IDEAs, MetaAd etc.
      1d480c25f397c4786386135f8e8938e4
  • On 27 Sep 2004, at 00:33, Georg Tuparev wrote:

    >
    > On Sep 27, 2004, at 1:20 AM, Marcel Weiher wrote:
    >
    >>> In our testing framework, we create an instance of the tested class
    >>> for every single test method.
    >>
    >> You do?  Automatically by the test framework?  I tend to leave this
    >> up to the test method in question (which will be a class method of
    >> the class under test unless you configure it differently).
    >
    > TSTTCPW as usual. Most of the time our default behavior is all we
    > need. For the exceptions we override the setUp method...

    Hmm...I have to say I don't see how this is simple.  It is very
    automatic and convenient, but not exactly simple.  How does the unit
    test framework know what message to send to get an instance of a class?
      +new?

    I am guessing you have a convention, and am intrigued to hear that you
    say it is working well.

    Cheers,

    Marcel

    --
    Marcel Weiher                Metaobject Software Technologies
    <marcel...>        www.metaobject.com
    Metaprogramming for the Graphic Arts.  HOM, IDEAs, MetaAd etc.
      1d480c25f397c4786386135f8e8938e4
  • On 26 Sep 2004, at 18:43, Christian Pekeler wrote:

    > [sorry for the smime-less repost]
    >
    > On Sep 26, 2004, at 5:59, Marcel Weiher wrote:
    >> On 25 Sep 2004, at 16:32, Georg Tuparev wrote:
    >>> finally I came to the conclusion that the problem is exactly the one
    >>> you discovered - separation of TestCase from the class being tested
    >>> [well, in reality this is not always true -- but please read my
    >>> replays to Marco I intend to write later today for more].
    >>
    >> Yes, this was my impression as well, at first simply a conjecture and
    >> a code smell:  parallel class hierarchies.  Now that I've worked with
    >> JUnit for well over a year, I am absolutely convinced that my
    >> decision to allow the class to test itself for MPWTest was precisely
    >> right.
    >
    > Parallel class hierarchies are a code smell if both hierarchies are
    > production code.

    Says who??

    > We're dealing with one hierarchy of production code and another one of
    > unit testing code. To me, that doesn't smell.

    To me it does.  Because I still have to build/maintain both hierarchies.

    > In my experience, for anything but trivial systems, these two class
    > hierarchies are not an exact mirror of each other.

    Well, that might be because it is difficult to keep them in sync.  When
    I use JUnit, I also have this experience.  With MPWTest, I do not,
    because keeping the tests with the code under test is trivially easy.
    I find the latter much more natural and easy.

    > Often times I have more than one test class per production class.
    > These test classes test different aspects of the production class and
    > usually have different setups.

    Hmm..when I have different setups, I don't have to create separate
    classes.  I just factor out a different setup method in my test code.
    It seems pretty obvious that certain rigidities in your unit testing
    setup are forcing you to introduce entities that aren't strictly
    necessary.

    > Sometimes I also have a testclass to test the interaction between
    > certain production classes.

    I then tend to have an object that handles the interaction between
    those production classes.  However, nothing at all prevents me from
    adding a test-class if I feel the need for it.  It is just that I don't
    have to do it if there is no need, and I rarely feel the need for it.

    That I *can* put tests in categories of the class under test does *not*
    imply that I *must* do so.

    > And other timed I don't have a test class for a production class at
    > all because I only test what could possibly fail and what requires me
    > to do TDD.

    Then how did the class get written, under TDD?  When you look at that
    particular class, how do you find the tests for it?

    To me, almost all this looks like symptoms of *having* to maintain a
    parallel class hierarchies, and not being able to keep them in sync.
    Why you would think of them as *good* things puzzles me.

    Cheers,

    Marcel
  • On Sep 27, 2004, at 1:47 AM, Marcel Weiher wrote:
    >
    >> Of course you keep them "close" to the tested class, but you still
    >> have an instance of one class testing an instance of another class.
    >
    > Huh?  I have the class testing instance of itself.  This seems
    > perfectly reasonable and has caused me absolutely no problems.  Why do
    > you say an instance of one class testing an instance of another class?
    > Do you mean an instance of the metaclass??

    Yes, you've got method of the class object testing methods of the
    instance. These tests are no more internal than when implemented in
    another class. In both cases we probably start by instantiating an
    instance of the tested class. The testing code that needs to be written
    is the same.

    >> And I don't buy the argument that writing tests in a category rather
    >> than a class change the whole perspective on unit testing and how it
    >> makes it so much more integrated in the development activity.
    >
    > I wouldn't buy it from a purely theoretical point of view either, but
    > after a little more than 1 year of JUnit (after a number of years with
    > MPWTest), I can say that it has definitely been the case.  YMMV.

    Not fair. Comparing testing with Junit and testing with OCUnit is a bit
    like comparing programming in Java and programming in Objective C. I
    don't like JUnit either.

    >> I'm a fan of categories, but in that case, I don't see the point.
    >> Writing a test using OCUnit and, I suppose, many others is just as
    >> trivial:
    >>
    >> @implementation MyTestCase:TestCase
    >> - (void) testWonder1
    >> {
    >> }
    >> @end
    >
    > Now I have to wonder:  you complained above about an instance of one
    > class testing an instance of a different class (which wasn't the
    > case), and this is precisely the case here.  How is MyTestCase related
    > to the class being tested?

    I was not complaining, but pointing out that having tests as class
    methods of the tested class or as instance methods of another class is
    not fundamentally different.

    MyTestCase is a client of the tested class. It includes its interface.
    Again there is no difference, if you decide to move your category in
    other files, you would also need to include relevant headers.

    >> One way or another the only required activities are to write test
    >> methods and to run them.
    >
    > Not quite.  At the very least, you need to link in the testing
    > framework.

    Well sure, and your tests run when you don't link to your framework?

    Please note that I've said since the beginning that not subclassing
    TestCase was also possibility. And I'm not saying that it is
    necessarily worse, but that the claims that doing so makes the testing
    activity so much better, simpler, easier and enjoyable are wrong in my
    opinion.

    marco
  • >>> Of course you keep them "close" to the tested class, but you still
    >>> have an instance of one class testing an instance of another class.
    >>
    >> Huh?  I have the class testing instance of itself.  This seems
    >> perfectly reasonable and has caused me absolutely no problems.  Why
    >> do you say an instance of one class testing an instance of another
    >> class?  Do you mean an instance of the metaclass??
    >
    > Yes, you've got method of the class object testing methods of the
    > instance.

    Yes, that is the default case, with which I have no problems so far.
    If one would want to have the test methods as instance, all one would
    have to do is override +testFixture to return an instance.  So far, I
    haven't found the need for this.

    > These tests are no more internal than when implemented in another
    > class.

    I didn't claim they were, except for organizational purposes.

    > In both cases we probably start by instantiating an instance of the
    > tested class. The testing code that needs to be written is the same.

    It is very similar, but not the same.  Again, I didn't claim there were
    any significant differences.  You just don't need an extra class, one
    that is not related to the class under test.

    >
    >>> And I don't buy the argument that writing tests in a category rather
    >>> than a class change the whole perspective on unit testing and how it
    >>> makes it so much more integrated in the development activity.
    >>
    >> I wouldn't buy it from a purely theoretical point of view either, but
    >> after a little more than 1 year of JUnit (after a number of years
    >> with MPWTest), I can say that it has definitely been the case.  YMMV.
    >
    > Not fair. Comparing testing with Junit and testing with OCUnit is a
    > bit like comparing programming in Java and programming in Objective C.
    > I don't like JUnit either.

    I was referring to one specific aspect where OCUnit matches JUnit.
    What do you not like about JUnit?
    Save that that feature, I find JUnit fine (apart from having to work in
    Java in the first place, but that is a completely separate issue).

    So I don't think your objection is warranted.

    >
    > I was not complaining, but pointing out that having tests as class
    > methods of the tested class or as instance methods of another class is
    > not fundamentally different.

    I have found that it is significantly different, because you are
    leaving something out that is unnecessary and gets in the way.

    > MyTestCase is a client of the tested class. It includes its interface.
    > Again there is no difference, if you decide to move your category in
    > other files, you would also need to include relevant headers.

    This is not true.  First, there still is a difference, because a
    category is still part of the class.  Second, I usually don't decide to
    do that.

    >>> One way or another the only required activities are to write test
    >>> methods and to run them.
    >>
    >> Not quite.  At the very least, you need to link in the testing
    >> framework.
    >
    > Well sure, and your tests run when you don't link to your framework?

    Yes.  It is the test-tool that links to the test framework.  It then
    just loads the framework under test and everything is fine.

    > Please note that I've said since the beginning that not subclassing
    > TestCase was also possibility. And I'm not saying that it is
    > necessarily worse, but that the claims that doing so makes the testing
    > activity so much better, simpler, easier and enjoyable are wrong in my
    > opinion.

    Is your opinion backed by experience with both styles?

    Marcel

    --
    Marcel Weiher                Metaobject Software Technologies
    <marcel...>        www.metaobject.com
    Metaprogramming for the Graphic Arts.  HOM, IDEAs, MetaAd etc.
      1d480c25f397c4786386135f8e8938e4
  • On 26 sep 2004, at 22.44, Georg Tuparev wrote:
    > BTW, a good suite of tests enables refactoring by first braking few
    > existing tests!

    Your interpretation of the notion "refactoring" must be very different
    than my interpretation. To me, refactoring is about changing the
    internal structure of some code without changing its external behavior.
    That is, a good suite of tests enables refactoring by adequately
    covering the external behavior that one wants to preserve.

    In my experience, testing the internals of a class (in the sense of
    testing internal state by examining instance variables, etc) turns the
    whole process of unit testing into a slow-flowing blob of syrup.
    Personally, I'm moving farther and farther away from this kind if
    testing.

    To me, unit testing is a design technique that enables me to work
    faster. That the internal state of one object is represented by an
    intricate mixture of booleans isn't important (it might smell though,
    which is important :-). What is important however, is that my test
    suite--should need rise--allows me to switch to another implementation
    without being afraid of breaking anything.

    Regards,
    Malte

    --
    Malte Tancred
    Computer Programmer
    Oops AB, http://oops.se/
  • On Sep 27, 2004, at 3:57, Marcel Weiher wrote:
    > On 26 Sep 2004, at 18:43, Christian Pekeler wrote:
    >> On Sep 26, 2004, at 5:59, Marcel Weiher wrote:
    >>> On 25 Sep 2004, at 16:32, Georg Tuparev wrote:
    >>>> finally I came to the conclusion that the problem is exactly the
    >>>> one you discovered - separation of TestCase from the class being
    >>>> tested [well, in reality this is not always true -- but please read
    >>>> my replays to Marco I intend to write later today for more].
    >>>
    >>> Yes, this was my impression as well, at first simply a conjecture
    >>> and a code smell:  parallel class hierarchies.  Now that I've worked
    >>> with JUnit for well over a year, I am absolutely convinced that my
    >>> decision to allow the class to test itself for MPWTest was precisely
    >>> right.
    >>
    >> Parallel class hierarchies are a code smell if both hierarchies are
    >> production code.
    >
    > Says who??

    What? You mean if I say something, it doesn't carry enough authority
    for you? ;) OK, I restate as: In my personal experience, I have never
    perceived parallel class hierarchies as a code smell if one hierarchy
    is for production code and the other for unit tests.

    Parallel class hierarchies indicate too tight coupling, duplication, a
    lack of abstraction and usually result in hard to change code. Since
    tight coupling is the very purpose of unit testing (as we want to test
    every public method of a specific class), and since we usually want to
    test concrete implementations instead of abstractions, I'm of the
    opinion that parallel class hierarchies are not a problem in the
    context of unit testing.

    >> We're dealing with one hierarchy of production code and another one
    >> of unit testing code. To me, that doesn't smell.
    >
    > To me it does.  Because I still have to build/maintain both
    > hierarchies.

    If you have to maintain a test category instead of a test class per
    production class, I fail to see a significant difference for practical
    purposes.

    >> In my experience, for anything but trivial systems, these two class
    >> hierarchies are not an exact mirror of each other.
    >
    > Well, that might be because it is difficult to keep them in sync.

    I'm usually not even trying to keep them in sync. They just happen to
    start out looking parallel in the beginning. Nothing good or bad about
    this, IMHO.

    >> Often times I have more than one test class per production class.
    >> These test classes test different aspects of the production class and
    >> usually have different setups.
    >
    > Hmm..when I have different setups, I don't have to create separate
    > classes.  I just factor out a different setup method in my test code.

    Sure, that's another good way of doing this. I've recently met the
    author of the upcoming XUnit Patterns book. He told me that he doesn't
    like the one setUp/tearDown per test class approach because they're
    invoked implicitly and therefore make the individual test methods
    harder to read. I don't mind the implicit invocation as it makes my
    test code shorter, but every now and then have explicit setUps as well,
    especially when the test methods in a test class all have different
    requirements.

    >> And other timed I don't have a test class for a production class at
    >> all because I only test what could possibly fail and what requires me
    >> to do TDD.
    >
    > Then how did the class get written, under TDD?  When you look at that
    > particular class, how do you find the tests for it?

    It can happen as a result of refactoring. For example I create a common
    super class for two existing classes that have some form of code
    duplication, then I usually don't create a test for the new superclass
    until I have to. Or if instances of class A create instance of class B,
    and B happens to be a simple container, then I usually just have a
    simple test for B in the testclass for A. When working with Apple's
    frameworks, I find TDD is not always practical. For example, I don't
    usually test my controllers (but try to keep them as small and simple
    as possible).

    > To me, almost all this looks like symptoms of *having* to maintain a
    > parallel class hierarchies, and not being able to keep them in sync.
    > Why you would think of them as *good* things puzzles me.

    I didn't mean to imply that I find them good, they just don't bother me.

    I think we're splitting hairs. None of the different point of views in
    this thread (not just in this mail) justify a big argument in my
    opinion. I think it's great that we have more than one unit testing
    solutions and I'm very interested in seeing how they're different. I'm
    a bit disappointed that only the usual suspects are writing in this
    thread, indicating that unit testing is still not a mainstream activity
    in this community. I'm looking forward to the day that Apple releases
    all their frameworks and libraries with the sources of the
    corresponding unit tests and a community which wouldn't accept anything
    less.

    Christian
  • > Parallel class hierarchies indicate too tight coupling, duplication, a
    > lack of abstraction and usually result in hard to change code.

    Being somewhat of a simpleton, I see parallel class hierarchies as the
    same thing in two places, and thus a trivial violation of
    OnceAndOnlyOnce.  A lot of the other problems come as a result of this.

    > Since tight coupling is the very purpose of unit testing (as we want
    > to test every public method of a specific class),

    So why not attach the test-code directly to the code to be tested?
    Especially if it is less work to do so (the coupling is fully
    automatic) and we have reduced our entities ( Occam's Razor, DTSTTCPW )

    > and since we usually want to test concrete implementations instead of
    > abstractions, I'm of the opinion that parallel class hierarchies are
    > not a problem in the context of unit testing.

    Two objections:  first, your argument seems inconsistent.  On the one
    hand, you state that you want tight coupling, on the other hand you
    don't perceive the fact that the hierarchies do get out of sync as a
    problem.  Second, my practical experience with both styles leads me to
    conclude that it is a problem, at least relative to the solution
    without quasi-parallel hierarchies.

    >>> We're dealing with one hierarchy of production code and another one
    >>> of unit testing code. To me, that doesn't smell.
    >>
    >> To me it does.  Because I still have to build/maintain both
    >> hierarchies.
    >
    > If you have to maintain a test category instead of a test class per
    > production class, I fail to see a significant difference for practical
    > purposes.

    You don't *have* to maintain a category, it is just useful for
    documentation purposes.  The practical difference is that the tests are
    directly, unambiguously and simply attached to the class to be tested.
    So you have fewer entities in your program/framework.  Less to think
    about.  More chunking, more abstraction...good.  One very simple
    benefit:  if you view unit tests as a form of documentation, it is
    obvious where to find the unit tests for a particular class.  With
    xUnit style, this is far from obvious.

    Also, a class is a somewhat more heavyweight entity than a mere
    category:  it occupies the global namespace, you have to give it a
    name, etc. It also has to be a subclass of TestCase and therefore
    implies a link-time + runtime dependency on the unit testing framework.

    But really, I think this boils down to you being both quite accustomed
    to xUnit style, and not having practical experience with the other
    style.  Try to step back for a moment, and maybe consider what Georg
    said about how different programmers reacted to their unit testing
    framework (which I do not know).  It was the ones with xUnit experience
    that had the most difficulty.

    >>> In my experience, for anything but trivial systems, these two class
    >>> hierarchies are not an exact mirror of each other.
    >>
    >> Well, that might be because it is difficult to keep them in sync.
    >
    > I'm usually not even trying to keep them in sync.

    I thought you want them to be highly coupled?  How do you find the
    tests that belong to a class?

    > They just happen to start out looking parallel in the beginning.
    > Nothing good or bad about this, IMHO.

    I beg to differ.

    >
    >>> Often times I have more than one test class per production class.
    >>> These test classes test different aspects of the production class
    >>> and usually have different setups.
    >>
    >> Hmm..when I have different setups, I don't have to create separate
    >> classes.  I just factor out a different setup method in my test code.
    >
    > Sure, that's another good way of doing this. I've recently met the
    > author of the upcoming XUnit Patterns book. He told me that he doesn't
    > like the one setUp/tearDown per test class approach because they're
    > invoked implicitly and therefore make the individual test methods
    > harder to read.

    Yep, that's also what I found.  I want tests to be as obvious as
    possible.

    >> To me, almost all this looks like symptoms of *having* to maintain a
    >> parallel class hierarchies, and not being able to keep them in sync.
    >> Why you would think of them as *good* things puzzles me.
    >
    > I didn't mean to imply that I find them good, they just don't bother
    > me.

    Compared to not having unit testing:  full agreement.  However, coming
    from MPWTest I found the separate class hierarchies of tests required
    by JUnit to be bothersome.

    > I think we're splitting hairs.

    Hmm...I agree that you are splitting hairs, trying to pooh pooh
    something that you haven't actually used that (a) is simpler and (b)
    easier to use than existing unit testing.  The differences may appear
    small to you in theory, but I find them to be actually significant in
    practice, having actually used both styles and having found xUnit style
    lacking (though still absolutely superior to not unit testing).

    > None of the different point of views in this thread (not just in this
    > mail) justify a big argument in my opinion.

    So why are you arguing?

    > I think it's great that we have more than one unit testing solutions
    > and I'm very interested in seeing how they're different. I'm a bit
    > disappointed that only the usual suspects are writing in this thread,
    > indicating that unit testing is still not a mainstream activity in
    > this community.

    If you go back to Georg's post, and actually also to Uli's, you will
    see that the arguments presented are directly connected with trying to
    make unit testing even simpler and more straightforward, which can only
    help the cause.

    When I came upon xUnit, I found that I didn't get it.  There seemed all
    this extra complexity/baggage that just baffled me, especially coming
    out of YAGNI-land :-)  It seemed to me that it could be a lot simpler,
    especially in Objective-C.  And it turns out that it *could* be a lot
    simpler and more comfortable to boot.

    And it seems I am not the only one baffled:

    On 23 Sep 2004, at 21:51, M. Uli Kusterer wrote:
    >
    > E.g. UnitKit keeps separate "test objects", which I find kinda odd.
    > I'm wondering why one shouldn't put the tests in the object in need of
    > the testing instead... things like that.

    Just because you are sufficiently clever that things like that don't
    baffle you, doesn't mean that the rest of us are as well.  So, if you
    want more unit testing, help make it easier!

    Unit testing does not *have* to be in xUnit style.  In fact, the
    TDD-gurus wholeheartedly recommend writing your own unit testing
    framework.

    > I'm looking forward to the day that Apple releases all their
    > frameworks and libraries with the sources of the corresponding unit
    > tests and a community which wouldn't accept anything less.

    That would be great, but I am not entirely convinced that Apple
    frameworks have been or are developed in TDD style...

    Marcel

    --
    Marcel Weiher                Metaobject Software Technologies
    <marcel...>        www.metaobject.com
    Metaprogramming for the Graphic Arts.  HOM, IDEAs, MetaAd etc.
      1d480c25f397c4786386135f8e8938e4
  • > Wait a second folks. I thought the discussion was about Unit Tests and
    > not about Acceptance Tests. Besides, if I cannot tests internals of a
    > class, who will give me the courage to refactor these internals? Often
    > these internals make more then 90% of the class. For example  a
    > Telescope class might have a public method isTracking that simply
    > returns a BOOL ivar set by an internal method called 10x each second
    > that that corrects the geometry of the flex mirror, reads a frame from
    > the helper CCD, does some nasty image convolution, calls eigen vectors
    > based RMS routine and passes the result to a weighting function that
    > finally decides if the telescope is still tracking or the object was
    > lost. For a user of the class Telescopes these might be unimportant
    > details, but for me - the author of the class - these are the most
    > important methods, and I would like to be able to test them no matter
    > if someone have academic or other objections...

    Now I'm not a heavywight in UnitTesting as some of you are, but I
    always took such occasions as a sign that one class is doing too much
    and that I should add a small class that encapsulates exactly these
    internals. So the Telescope merely becomes some sort of Facade to the
    real functionality.

    Well, now that I think about it I'm a big fan of very little classes.
    Though I'm still lacking lot's of experience.

    After thinking some more: It seems to me  that if you have test-classes
    and source separated, it kind of enforces the use of many very small
    classes. Integrating the Tests in the Classes on the other hand enables
    them to do more - as it's still convenient to test.

    Now I'm not sure which of the goals are more desirable to me or if
    there is a middle ground to go with. But this seems to make the
    engineering tradeoff more clear. Do you agree or am I overlooking
    something?

    > BTW, a good suite of tests enables refactoring by first braking few
    > existing tests! If I make even a simple refactoring, say "rename
    > method", and my test bar is still green I know I am in trouble and I
    > do not have enough tests! When the tests are broken, I start fixing
    > them. When the bar is green again, I commit. If the bar stays red for
    > longer then an hour, I revert.

    <g> When I rename a Method and a Test breaks I rather start swearing at
    my stupid tools that weren't good enough  to rename the method call in
    the test too. </g>

    Is there some sign on the horizon that we finally will be getting
    SmallTalk or Eclipse Level Refactoring support? Right now the
    Objective-C Tools are _very_ week. :(

    cu Martin
previous month september 2004 next month
MTWTFSS
    1 2 3 4 5
6 7 8 9 10 11 12
13 14 15 16 17 18 19
20 21 22 23 24 25 26
27 28 29 30      
Go to today