A coding pattern that does not work under Garbage Collection

  • I've been testing some of our older libraries under GC, and have found
    one coding pattern (that may well have been bad practice anyway) that
    really does not work well under GC.

    I used to use convenience constructors for NSMutableData objects as a
    kind of lazy replacement for malloc.

    That is, an assignment like this:

      float *myPointer = [[NSMutableData
    dataWithLength:size*sizeof(float)] mutableBytes]; // Memory freed at
    end of event loop

    could be used in place of

      float *myPointer = (float*)malloc(size*sizeof(float)); // Needs a
    free() later

    The advantage of the first, under traditional memory management, was
    that the memory would presumably be cleaned up at the end of the
    current event loop (whereas the second would require a malloc
    downstream in the code).

    It seems like this (not surprisingly) doesn't work reliably under GC,
    as the collector obviously has no way of knowing that you are later
    looking at the memory originally addressed by myPointer.  It keeps
    track of objects and not pointers.

    One possible solution is to replace the first assignment with

      NSMutableData *myData = [NSMutableData
    dataWithLength:size*sizeof(float)];

    and use

      ((float*)[myData mutableBytes])  in place of myPointer.

    The other option is to just use malloc/free pairs - which I don't mind
    to a certain extent but the autoreleased convenience constructor was
    nice when there were a lot of little chunks of data to allocate which
    would not be needed after the end of the event loop.  I guess the
    concept of a convenience constructor no longer has the same
    significance it did pre-GC.

    As there are probably better ways of approaching this kind of
    situation, I'd be interested in any suggestions or comments.  The way
    I was doing it before was probably just bad...  still this explained a
    few spooky bugs in one of our apps under GC.

    Cheers,

    Rick
  • On Nov 9, 2007, at 10:39 AM, Rick Hoge wrote:

    > I've been testing some of our older libraries under GC, and have
    > found one coding pattern (that may well have been bad practice
    > anyway) that really does not work well under GC.
    >
    > I used to use convenience constructors for NSMutableData objects as
    > a kind of lazy replacement for malloc.
    >
    > That is, an assignment like this:
    >
    > float *myPointer = [[NSMutableData dataWithLength:size*sizeof
    > (float)] mutableBytes]; // Memory freed at end of event loop

    [...]

    > It seems like this (not surprisingly) doesn't work reliably under
    > GC, as the collector obviously has no way of knowing that you are
    > later looking at the memory originally addressed by myPointer.  It
    > keeps track of objects and not pointers.

    You kept a (default, no strong or weak modifier) pointer to data
    inside the object, but no one kept a pointer to the object, so it was
    collected.

    > As there are probably better ways of approaching this kind of
    > situation, I'd be interested in any suggestions or comments.  The
    > way I was doing it before was probably just bad...

    You can allocate GC'd memory with NSAllocateCollectable.

    Jim
  • [...]

    >> You kept a (default, no strong or weak modifier) pointer to data
    >> inside the object, but no one kept a pointer to the object, so it
    >> was collected.

    So if I had added the __strong modifier to the pointer assignment, the
    collector would know not that the memory was still in use?

    >> As there are probably better ways of approaching this kind of
    >> situation, I'd be interested in any suggestions or comments.  The
    >> way I was doing it before was probably just bad...
    >
    > You can allocate GC'd memory with NSAllocateCollectable.

    Thanks - this is very helpful.  I am trying to find code examples but
    the usage is fairly clear.  Playing with this in Instruments it looks
    like it does what it's supposed to.

    Rick
  • On Nov 9, 2007, at 11:47 AM, Rick Hoge wrote:

    >>> You kept a (default, no strong or weak modifier) pointer to data
    >>> inside the object, but no one kept a pointer to the object, so it
    >>> was collected.
    >
    > So if I had added the __strong modifier to the pointer assignment,
    > the collector would know not that the memory was still in use?

    (Not a GC expert...)

    No, I think you're still screwed. Since you can't know how NSData
    manages its bucket of bytes, you need to make sure the NSData object
    stays around for as long as you want to work with the bytes.

    The salient point was that no one kept a reference to the object
    which owned the bytes you were pointing at, and when the object got
    collected as a result, it's internal storage also went away.

    Really not so different from the autorelease case, except there you
    were guaranteed the lifetime of the parent autorelease pool. In a GC
    world the collector will collect unreferenced objects when it gets
    around to it, so you better keep a strong reference to the object for
    as long as you need it.

    Jim
  • On Nov 9, 2007, at 8:57 AM, Jim Correia wrote:
    > (Not a GC expert...)
    >
    > No, I think you're still screwed. Since you can't know how NSData
    > manages its bucket of bytes, you need to make sure the NSData object
    > stays around for as long as you want to work with the bytes.
    >
    > The salient point was that no one kept a reference to the object
    > which owned the bytes you were pointing at, and when the object got
    > collected as a result, it's internal storage also went away.
    >
    > Really not so different from the autorelease case, except there you
    > were guaranteed the lifetime of the parent autorelease pool. In a GC
    > world the collector will collect unreferenced objects when it gets
    > around to it, so you better keep a strong reference to the object
    > for as long as you need it.

    Jim is correct and this is an unfortunate edge case that there wasn't
    time to address in Leopard.  Fortunately, the solution is straight
    forward;  keep the NSData object around for as long as you need its
    memory to stick around.

    b.bum
  • If you aren't averse to ObjC++, there's always the traditional C++
    solution for a block of memory that has a controlled lifetime:

    vector<float> myData(size);
    float *myPointer = &myData[0]; // or just use myData directly, it
    will work the same as a C array in the majority of cases

    Then myData should last until it falls out of scope. This is easier
    than malloc because you don't have to worry about edge cases where
    you fail to free it properly (e.g. calling return in the middle of a
    function).

    Only thing I'm not sure about—if you raise an ObjC exception, I don't
    know if myData would be leaked. Not sure how well ObjC++ exceptions
    handle C++ cleanup.

    On Nov 9, 2007, at 7:39 AM, Rick Hoge wrote:

    >
    > I've been testing some of our older libraries under GC, and have
    > found one coding pattern (that may well have been bad practice
    > anyway) that really does not work well under GC.
    >
    > I used to use convenience constructors for NSMutableData objects as
    > a kind of lazy replacement for malloc.
    >
    > That is, an assignment like this:
    >
    > float *myPointer = [[NSMutableData dataWithLength:size*sizeof
    > (float)] mutableBytes]; // Memory freed at end of event loop
    >
    > could be used in place of
    >
    > float *myPointer = (float*)malloc(size*sizeof(float)); // Needs a
    > free() later
    >
    > The advantage of the first, under traditional memory management,
    > was that the memory would presumably be cleaned up at the end of
    > the current event loop (whereas the second would require a malloc
    > downstream in the code).
    >
    > It seems like this (not surprisingly) doesn't work reliably under
    > GC, as the collector obviously has no way of knowing that you are
    > later looking at the memory originally addressed by myPointer.  It
    > keeps track of objects and not pointers.
    >
    > One possible solution is to replace the first assignment with
    >
    > NSMutableData *myData = [NSMutableData dataWithLength:size*sizeof
    > (float)];
    >
    > and use
    >
    > ((float*)[myData mutableBytes])  in place of myPointer.
    >
    > The other option is to just use malloc/free pairs - which I don't
    > mind to a certain extent but the autoreleased convenience
    > constructor was nice when there were a lot of little chunks of data
    > to allocate which would not be needed after the end of the event
    > loop.  I guess the concept of a convenience constructor no longer
    > has the same significance it did pre-GC.
    >
    > As there are probably better ways of approaching this kind of
    > situation, I'd be interested in any suggestions or comments.  The
    > way I was doing it before was probably just bad...  still this
    > explained a few spooky bugs in one of our apps under GC.
    >
    > Cheers,
    >
    > Rick
  • > Hi,
    >
    > The GUI can say "kernel extension, allow all traffic". But the kernel
    > extension can't say "GUI, we have a new log entry"? So how does the
    > GUI get the log entries?
    >
    > Bye,
    > Nils

    Use IODataQueue

    http://developer.apple.com/documentation/Darwin/Reference/
    KernelIOKitFramework/IODataQueue/Classes/IODataQueue_/
  • On Nov 9, 2007 9:54 AM, John Stiles <JStiles...> wrote:
    > If you aren't averse to ObjC++, there's always the traditional C++
    > solution for a block of memory that has a controlled lifetime:
    >
    > vector<float> myData(size);
    > float *myPointer = &myData[0]; // or just use myData directly, it
    > will work the same as a C array in the majority of cases
    >
    > Then myData should last until it falls out of scope. This is easier
    > than malloc because you don't have to worry about edge cases where
    > you fail to free it properly (e.g. calling return in the middle of a
    > function).
    >
    > Only thing I'm not sure about—if you raise an ObjC exception, I don't
    > know if myData would be leaked. Not sure how well ObjC++ exceptions
    > handle C++ cleanup.

    In 32-bit, it isn't handled at all (you'll have to catch the Obj-C
    exceptions, destroy the vector yourself, and rethrow the exception).
    In 64-bit, Obj-C exceptions *are* C++ exceptions (and vice-versa), so
    all is well (all the normal C++ stack unwinding/destructor calling
    goodness happens)

    --
    Clark S. Cox III
    <clarkcox3...>
  • On Nov 9, 2007, at 10:33 AM, Clark Cox wrote:

    > On Nov 9, 2007 9:54 AM, John Stiles <JStiles...> wrote:
    >> If you aren't averse to ObjC++, there's always the traditional C++
    >> solution for a block of memory that has a controlled lifetime:
    >>
    >> vector<float> myData(size);
    >> float *myPointer = &myData[0]; // or just use myData
    >> directly, it
    >> will work the same as a C array in the majority of cases
    >>
    >> Then myData should last until it falls out of scope. This is easier
    >> than malloc because you don't have to worry about edge cases where
    >> you fail to free it properly (e.g. calling return in the middle of a
    >> function).
    >>
    >> Only thing I'm not sure about—if you raise an ObjC exception, I don't
    >> know if myData would be leaked. Not sure how well ObjC++ exceptions
    >> handle C++ cleanup.
    >
    > In 32-bit, it isn't handled at all (you'll have to catch the Obj-C
    > exceptions, destroy the vector yourself, and rethrow the exception).
    > In 64-bit, Obj-C exceptions *are* C++ exceptions (and vice-versa), so
    > all is well (all the normal C++ stack unwinding/destructor calling
    > goodness happens)

    It's a bit of a bummer how many of the ObjC runtime improvements
    aren't making it back to 32-bit, isn't it?

    I know, binary compatibility and all, but if there was a way to opt-
    in, that would sure be nice. Our apps aren't going 64-bit any time
    soon, but I'd appreciate improved exception handling (and I guess I
    could give up posing ;)  ).
  • >> (Not a GC expert...)

    But you read the manual better than I did :)

    >> No, I think you're still screwed. Since you can't know how NSData
    >> manages its bucket of bytes, you need to make sure the NSData
    >> object stays around for as long as you want to work with the bytes.
    >>
    >> The salient point was that no one kept a reference to the object
    >> which owned the bytes you were pointing at, and when the object got
    >> collected as a result, it's internal storage also went away.
    >>
    >> Really not so different from the autorelease case, except there you
    >> were guaranteed the lifetime of the parent autorelease pool. In a
    >> GC world the collector will collect unreferenced objects when it
    >> gets around to it, so you better keep a strong reference to the
    >> object for as long as you need it.
    >
    > Jim is correct and this is an unfortunate edge case that there
    > wasn't time to address in Leopard.  Fortunately, the solution is
    > straight forward;  keep the NSData object around for as long as you
    > need its memory to stick around.

    This is not a problem - just good to be aware of.

    Regarding NSAllocateCollectible, just to make sure I am not
    misunderstanding - I can allocate memory like the example below?

      __strong int* tempInt = (__strong
    int*)NSAllocateCollectable(sizeof(int), NSScannedOption);

    and rely on the garbage collector to figure out when I've reached a
    point where tempInt could no longer possibly be used to access that
    memory?  so no leaks even without a free()?

    Thanks for this and the other very informative replies to this post.

    Rick
  • On Nov 9, 2007, at 10:46 AM, John Stiles wrote:
    > It's a bit of a bummer how many of the ObjC runtime improvements
    > aren't making it back to 32-bit, isn't it?
    >
    > I know, binary compatibility and all, but if there was a way to opt-
    > in, that would sure be nice. Our apps aren't going 64-bit any time
    > soon, but I'd appreciate improved exception handling (and I guess I
    > could give up posing ;)

    Like 64 bit, it would require having another entire copy of all of
    dynamic libraries compiled for the "modern" Objective-C runtime...

    b.bum
  • Hmm. Well, that is a bit thorny.

    On Nov 9, 2007, at 3:16 PM, Bill Bumgarner wrote:

    > On Nov 9, 2007, at 10:46 AM, John Stiles wrote:
    >> It's a bit of a bummer how many of the ObjC runtime improvements
    >> aren't making it back to 32-bit, isn't it?
    >>
    >> I know, binary compatibility and all, but if there was a way to
    >> opt-in, that would sure be nice. Our apps aren't going 64-bit any
    >> time soon, but I'd appreciate improved exception handling (and I
    >> guess I could give up posing ;)
    >
    > Like 64 bit, it would require having another entire copy of all of
    > dynamic libraries compiled for the "modern" Objective-C runtime...
    >
    > b.bum
    >
  • On Nov 9, 2007, at 3:18 PM, John Stiles wrote:
    > Hmm. Well, that is a bit thorny.

    Seriously.  Users and their pesky expectations that their apps will
    keep working when a new release of the OS ships.  How selfish.

    At the least, the days of 32 bits are numbered.  Might be a 4 digit
    number but, with the upgrade to the Mac Mini, Apple's entire lineup of
    computers is 64 bit capable.

    b.bum
  • On Nov 9, 2007, at 3:28 PM, Bill Bumgarner wrote:

    > On Nov 9, 2007, at 3:18 PM, John Stiles wrote:
    >> Hmm. Well, that is a bit thorny.
    >
    > Seriously.  Users and their pesky expectations that their apps will
    > keep working when a new release of the OS ships.  How selfish.
    >
    > At the least, the days of 32 bits are numbered.  Might be a 4 digit
    > number but, with the upgrade to the Mac Mini, Apple's entire lineup
    > of computers is 64 bit capable.

    That's only one small part of the equation. Certainly there are
    hurdles other than installed base to consider.

    The machines we are targeting for our next product are all 64-bit
    capable, but we will be a 32-bit app. It is not an easy transition
    when you are inheriting a large preexisting codebase that needs to
    remain 32-bit compatible, and while there are benefits to 64-bit,
    there are also costs to consider.
  • On Nov 9, 2007, at 11:35 AM, Rick Hoge wrote:
    > Regarding NSAllocateCollectible, just to make sure I am not
    > misunderstanding - I can allocate memory like the example below?
    >
    > __strong int* tempInt = (__strong
    > int*)NSAllocateCollectable(sizeof(int), NSScannedOption);
    >
    > and rely on the garbage collector to figure out when I've reached a
    > point where tempInt could no longer possibly be used to access that
    > memory?  so no leaks even without a free()?
    >
    > Thanks for this and the other very informative replies to this post.

    That is correct.

    b.bum
previous month november 2007 next month
MTWTFSS
      1 2 3 4
5 6 7 8 9 10 11
12 13 14 15 16 17 18
19 20 21 22 23 24 25
26 27 28 29 30    
Go to today