NSOperationQueue Memory Leak

  • I have an audio application that processes numerous audio streams into
    ten-second clips, compresses the clips, and saves them to disk.
    Naturally, I wanted to move from single-threaded processing to
    multithreaded processing for the clip compression/writing. That was
    easily done using the NSOperationQueue and the NSInvocation classes,
    and it worked nicely, apart from a bad memory leak that I can't seem
    to eradicate. So I finally wrote a small test application to
    encapsulate what I was doing in the larger program, and it too has a
    leak -- a big one!

    The test application simply spawns 10000 threads whose sole job is to
    display "Hi, I am thread xx" on the console. It then sleeps and spawns
    10000 more. At the end of the run some 38 MB of  real memory has been
    used, and it never goes down.  (I'm not using garbage collection, but
    I think I'm doing things correctly with the autorelease pool.)

    Can someone tell me what I'm doing wrong?

    Thanks!

    Mike

    ---------------------------------------------------------------------------------------------------
    #import <Foundation/Foundation.h>

    @interface OperationQueueTester : NSObject
    {
    }

    - (void) run;
    - (void) sayHi:(id)blah;
    - (void) queueOperations:(int) count;

    @end

    @implementation OperationQueueTester

    - (id) init
    {
    return self;
    }

    - (void) run
    {
    // Allow a bit of time to have a look at the initial memory usage.
    NSLog(@"Starting in 5 seconds...");
    sleep(5);

    // Throw a bunch of simple operations into the operation queue.
    [self queueOperations:10000];
    sleep(10);
    [self queueOperations:10000];

    // Allow a bit of time to look at the final memory usage.
    NSLog(@"All threads have finished processing.");
    sleep(3600);
    }

    - (void) queueOperations:(int) count
    {
    NSAutoreleasePool *pool = [[NSAutoreleasePool alloc] init];
    NSInvocationOperation *processor;
    NSOperationQueue *operationQueue = [[[NSOperationQueue alloc] init]
    autorelease];
    NSLog(@"Preparing to queue operations");

    // Add multiple "process:" operations to the queue with no argument.
    for (int i = 0; i < count; i++)
    {
      processor = [[NSInvocationOperation alloc] initWithTarget:self
    selector:@selector(sayHi:) object:nil];
      [operationQueue addOperation:processor];
    }
    NSLog(@"All threads queued.");
    [operationQueue waitUntilAllOperationsAreFinished];
    [pool release];
    }

    - (void) sayHi:(id)blah
    {
    static int counter = 0;
    NSLog(@"Hi. I am thread %d", counter++);
    }

    @end

    /**
    Main routine, in which an object is instantiated to queue a large
    number of operations into an operation queue.
    **/
    int main (int argc, const char * argv[])
    {
        NSAutoreleasePool * pool = [[NSAutoreleasePool alloc] init];

    OperationQueueTester *queueTester = [[OperationQueueTester alloc]
    init];
    [queueTester run];
        [pool drain];
        return 0;
    }
  • On Aug 6, 2008, at 7:38 AM, Mike Simmons wrote:

    > processor = [[NSInvocationOperation alloc] initWithTarget:self
    > selector:@selector(sayHi:) object:nil];

    Why aren't you releasing this object?

    Nick Zitzmann
    <http://www.chronosnet.com/>
  • Inside your for loop you are allocating 10,000 NSInvocationOperation
    objects without adding them to your autorelease pool.  Adding
    [processor autorelease]; as the last line of the for loop killed
    memory leakage for me.  Without it my real memory would jump 10 MB
    each time queueOperations: ran, with it I had a high-water mark of
    12.02 MB, and I extended it to run through 10 iterations.

    Brock
  • Hi Mike

    > processor = [[NSInvocationOperation alloc] initWithTarget:self
    > selector:@selector(sayHi:) object:nil];
    You are not releasing the operation objects you create!.  You should
    have released them soon
    after they were added to the queue.
    --
    [operationQueue addOperation:processor];
    [processor release];  //<<<----
    --

    Also you are not releasing the OperationQueue that you create.  Since
    you are not using garbage collection,
    you will have to manage the memory on your own.  Read more on it:

    http://developer.apple.com/documentation/Cocoa/Conceptual/MemoryMgmt/Memory
    Mgmt.html


    Regards
    Shripada

    On 07-Aug-08, at 6:33 AM, <cocoa-dev-request...> wrote:

    > Message: 7
    > Date: Wed, 06 Aug 2008 08:38:09 -0500
    > From: Mike Simmons <mikesimmons...>
    > Subject: NSOperationQueue Memory Leak
    > To: <Cocoa-dev...>
    > Message-ID: <E7A0013D-FB65-48AE-9321-05A93F6EEAC9...>
    > Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes
    >
    > I have an audio application that processes numerous audio streams into
    > ten-second clips, compresses the clips, and saves them to disk.
    > Naturally, I wanted to move from single-threaded processing to
    > multithreaded processing for the clip compression/writing. That was
    > easily done using the NSOperationQueue and the NSInvocation classes,
    > and it worked nicely, apart from a bad memory leak that I can't seem
    > to eradicate. So I finally wrote a small test application to
    > encapsulate what I was doing in the larger program, and it too has a
    > leak -- a big one!
    >
    > The test application simply spawns 10000 threads whose sole job is to
    > display "Hi, I am thread xx" on the console. It then sleeps and spawns
    > 10000 more. At the end of the run some 38 MB of  real memory has been
    > used, and it never goes down.  (I'm not using garbage collection, but
    > I think I'm doing things correctly with the autorelease pool.)
    >
    > Can someone tell me what I'm doing wrong?
    >
    > Thanks!
    >
    > Mike
    >
    > ---------------------------------------------------------------------------------------------------
    > #import <Foundation/Foundation.h>
    >
    >
    > @interface OperationQueueTester : NSObject
    > {
    > }
    >
    > - (void) run;
    > - (void) sayHi:(id)blah;
    > - (void) queueOperations:(int) count;
    >
    > @end
    >
    > @implementation OperationQueueTester
    >
    > - (id) init
    > {
    > return self;
    > }
    >
    > - (void) run
    > {
    > // Allow a bit of time to have a look at the initial memory usage.
    > NSLog(@"Starting in 5 seconds...");
    > sleep(5);
    >
    > // Throw a bunch of simple operations into the operation queue.
    > [self queueOperations:10000];
    > sleep(10);
    > [self queueOperations:10000];
    >
    > // Allow a bit of time to look at the final memory usage.
    > NSLog(@"All threads have finished processing.");
    > sleep(3600);
    > }
    >
    > - (void) queueOperations:(int) count
    > {
    > NSAutoreleasePool *pool = [[NSAutoreleasePool alloc] init];
    > NSInvocationOperation *processor;
    > NSOperationQueue *operationQueue = [[[NSOperationQueue alloc] init]
    > autorelease];
    > NSLog(@"Preparing to queue operations");
    >
    > // Add multiple "process:" operations to the queue with no argument.
    > for (int i = 0; i < count; i++)
    > {
    > processor = [[NSInvocationOperation alloc] initWithTarget:self
    > selector:@selector(sayHi:) object:nil];
    > [operationQueue addOperation:processor];
    > }
    > NSLog(@"All threads queued.");
    > [operationQueue waitUntilAllOperationsAreFinished];
    > [pool release];
    > }
    >
    > - (void) sayHi:(id)blah
    > {
    > static int counter = 0;
    > NSLog(@"Hi. I am thread %d", counter++);
    > }
    >
    > @end
    >
    > /**
    > Main routine, in which an object is instantiated to queue a large
    > number of operations into an operation queue.
    > **/
    > int main (int argc, const char * argv[])
    > {
    > NSAutoreleasePool * pool = [[NSAutoreleasePool alloc] init];
    >
    > OperationQueueTester *queueTester = [[OperationQueueTester alloc]
    > init];
    > [queueTester run];
    > [pool drain];
    > return 0;
    > }
    >
    >
    >
    > ------------------------------

    -----------------------------------------------
    Robosoft Technologies - Come home to Technology

    Disclaimer: This email may contain confidential material. If you were not an intended recipient, please notify the sender and delete all copies. Emails to and from our network may be logged and monitored. This email and its attachments are scanned for virus by our scanners and are believed to be safe. However, no warranty is given that this email is free of malicious content or virus.
  • On Wed, Aug 6, 2008 at 9:38 AM, Mike Simmons <mikesimmons...> wrote:
    > I have an audio application that processes numerous audio streams into
    > ten-second clips, compresses the clips, and saves them to disk. Naturally, I
    > wanted to move from single-threaded processing to multithreaded processing
    > for the clip compression/writing. That was easily done using the
    > NSOperationQueue and the NSInvocation classes, and it worked nicely, apart
    > from a bad memory leak that I can't seem to eradicate. So I finally wrote a
    > small test application to encapsulate what I was doing in the larger
    > program, and it too has a leak -- a big one!
    >
    > The test application simply spawns 10000 threads whose sole job is to
    > display "Hi, I am thread xx" on the console. It then sleeps and spawns 10000
    > more. At the end of the run some 38 MB of  real memory has been used, and it
    > never goes down.  (I'm not using garbage collection, but I think I'm doing
    > things correctly with the autorelease pool.)
    >
    > Can someone tell me what I'm doing wrong?

    Two things:

    1) As others have already pointed out, you're managing your memory
    incorrectly. This is the main problem, and they've covered it.

    2) Expecting your "real memory" usage to decrease. I assume you're
    using Activity Monitor or top. These tools display how much memory
    your process has requested from the OS. However, requesting memory
    from the OS is expensive. Often the allocator, when freeing memory,
    will not return it to the OS, but instead just keep it so that it can
    be reused for the next allocation. Having your memory usage in these
    tools not go down can be a good warning sign (and having it increase
    without limit is a definite problem) but you should use a dedicated
    tool such as MallocDebug or ObjectAlloc to determine if you're leaking
    and why.

    Mike
previous month august 2008 next month
MTWTFSS
        1 2 3
4 5 6 7 8 9 10
11 12 13 14 15 16 17
18 19 20 21 22 23 24
25 26 27 28 29 30 31
Go to today