NSURLConnection leaking on multi-threaded apps?
-
So, as a follow-up to my previous threads discussion, which I do want to keep on-topic, it appears that NSURLConnection leaks whenever it's called inside a method that's been detached from the main thread. Namely, I tested my application with threading and without, and the thread-less version has no leaks. Just to reiterate, I never actually call the NSURLConnection methods directly, I DO have autorelease pools set up properly, and there is NOTHING else that is leaking (apart from the NSUC-associated classes), even though I do create (and manage to properly discard) a lot of other data, including XML documents, strings, etc. I'm guessing that the reason NSURLConnection gets called at all is because I call NSXMLDocument initWithContentsOfURL:options:error: to get data, and that method uses NSURLConnection internally? (if not, I'm really lost because, again, I never actually call NSURLConnection myself).
I found this page online which seems to illustrate this problem more concisely (though it calls NSUC directly), if I'm reading the code correctly: http://paste.lisp.org/display/21630 . There, a loop performs 100000 NSURLConnections, each on its own thread. When the program is run through ObjectAlloc, each instance of Connection, Request, and all other associated data is never released. Since the data requested is not that great, the leak isn't as noticeable, but if you're retrieving a significant amount of data each time, the numbers can add up quickly.
Can someone confirm for me that this is actually a bug and not something I can solve on my own, or am I really just missing something?
Paul
P.S. The unreleased NSURLConnections are also accompanied (among other things) by unreleased NSLocks and immutable CFStrings. The NSLocks, if I'm understanding things correctly, are coming from synchronous requests sent by NSURLConnection, which lock the current thread. -
On 5 mar 2007, at 00.51, Paul Borokhov wrote:
> So, as a follow-up to my previous threads discussion, which I do
> want to keep on-topic, it appears that NSURLConnection leaks
> whenever it's called inside a method that's been detached from the
> main thread. Namely, I tested my application with threading and
> without, and the thread-less version has no leaks. Just to
> reiterate, I never actually call the NSURLConnection methods
> directly, I DO have autorelease pools set up properly, and there is
> NOTHING else that is leaking (apart from the NSUC-associated
> classes), even though I do create (and manage to properly discard)
> a lot of other data, including XML documents, strings, etc. I'm
> guessing that the reason NSURLConnection gets called at all is
> because I call NSXMLDocument initWithContentsOfURL:options:error:
> to get data, and that method uses NSURLConnection internally? (if
> not, I'm really lost because, again, I never actually call
> NSURLConnection myself).
I don't know what the official word for thread safety is for these
classes. This page doesn't mention them as being either thread safe,
or unsafe:
<http://developer.apple.com/documentation/Cocoa/Conceptual/
Multithreading/articles/CocoaSafety.html>
When there is no documentation, you should assume that it's only OK
to use the class from the main thread.
I've seen crashes when trying to use NSURLConnection from background
threads. Each NSURLConnection instance was only ever used from one
particular thread, but it's my guess that they share some common
resource - a cache or something like that - that triggered my
problem. That said, I'm not 100% sure that it was because of the
threading, and not some stupid mistake of mine...
I suggest you file a bug report / enhancement request to ask Apple to
update the documentation regarding thread safety for these classes.
j o a r -
If you can't find a solution for the leak, I would suggest just using
NSURLConnection directly yourself then. From your main thread use
its asynchronous methods to get the data, and then init an XML
document with that.
Mike.
On 4 Mar 2007, at 23:51, Paul Borokhov wrote:
> So, as a follow-up to my previous threads discussion, which I do
> want to keep on-topic, it appears that NSURLConnection leaks
> whenever it's called inside a method that's been detached from the
> main thread. Namely, I tested my application with threading and
> without, and the thread-less version has no leaks. Just to
> reiterate, I never actually call the NSURLConnection methods
> directly, I DO have autorelease pools set up properly, and there is
> NOTHING else that is leaking (apart from the NSUC-associated
> classes), even though I do create (and manage to properly discard)
> a lot of other data, including XML documents, strings, etc. I'm
> guessing that the reason NSURLConnection gets called at all is
> because I call NSXMLDocument initWithContentsOfURL:options:error:
> to get data, and that method uses NSURLConnection internally? (if
> not, I'm really lost because, again, I never actually call
> NSURLConnection myself).
> I found this page online which seems to illustrate this problem
> more concisely (though it calls NSUC directly), if I'm reading the
> code correctly: http://paste.lisp.org/display/21630 . There, a loop
> performs 100000 NSURLConnections, each on its own thread. When the
> program is run through ObjectAlloc, each instance of Connection,
> Request, and all other associated data is never released. Since the
> data requested is not that great, the leak isn't as noticeable, but
> if you're retrieving a significant amount of data each time, the
> numbers can add up quickly.
> Can someone confirm for me that this is actually a bug and not
> something I can solve on my own, or am I really just missing
> something?
> Paul
>
> P.S. The unreleased NSURLConnections are also accompanied (among
> other things) by unreleased NSLocks and immutable CFStrings. The
> NSLocks, if I'm understanding things correctly, are coming from
> synchronous requests sent by NSURLConnection, which lock the
> current thread.
-
On Sun, 4 Mar 2007 15:51:06 -0800, Paul Borokhov <lensovetp...> said:
> So, as a follow-up to my previous threads discussion, which I do want to keep
> on-topic, it appears that NSURLConnection leaks whenever it's called inside a
> method that's been detached from the main thread. Namely, I tested my
> application with threading and without, and the thread-less version has no
> leaks. Just to reiterate, I never actually call the NSURLConnection methods
> directly, I DO have autorelease pools set up properly, and there is NOTHING
> else that is leaking (apart from the NSUC-associated classes), even though I
> do create (and manage to properly discard) a lot of other data, including XML
> documents, strings, etc. I'm guessing that the reason NSURLConnection gets
> called at all is because I call NSXMLDocument
> initWithContentsOfURL:options:error: to get data, and that method uses
> NSURLConnection internally?
This sounds like a variant of the NSURLConnection bug that causes a sporadic
crash when loading Core Data data from an XML store. The
NSPersistentStoreCoordinator calls NSXMLDocument
initWithContentsOfURL:options:error: which calls NSURLConnection and things
rapidly go south because of threading (and it's all Apple's code, so doing
something about it is quite tricky).
So, to sum up, yes, it's known that there are memory-management problems
with NSURLConnection and threads, and Apple's own code falls prey to these
problems. m.
--
matt neuburg, phd = <matt...>, <http://www.tidbits.com/matt/>
A fool + a tool + an autorelease pool = cool!
AppleScript: the Definitive Guide - Second Edition!
<http://www.amazon.com/gp/product/0596102119> -
I don't know if this will help you, but our app was leaking memory in
NSURLConnection as well. I didn't realize it could be related to
creating the NSURLConnection on a secondary thread -- that may very
well be the case.
In any event, we found that the leak went away if we set the capacity
of the URL cache to zero.
[[NSURLCache sharedURLCache] setMemoryCapacity:0];
[[NSURLCache sharedURLCache] setDiskCapacity:0];
In our case, we don't want any of the data cached anyway, so this
made sense.
-john
On Mar 11, 2007, at 9:12 AM, Matt Neuburg wrote:
> On Sun, 4 Mar 2007 15:51:06 -0800, Paul Borokhov
> <lensovetp...> said:
>> So, as a follow-up to my previous threads discussion, which I do
>> want to keep
>> on-topic, it appears that NSURLConnection leaks whenever it's
>> called inside a
>> method that's been detached from the main thread. Namely, I tested my
>> application with threading and without, and the thread-less
>> version has no
>> leaks. Just to reiterate, I never actually call the
>> NSURLConnection methods
>> directly, I DO have autorelease pools set up properly, and there
>> is NOTHING
>> else that is leaking (apart from the NSUC-associated classes),
>> even though I
>> do create (and manage to properly discard) a lot of other data,
>> including XML
>> documents, strings, etc. I'm guessing that the reason
>> NSURLConnection gets
>> called at all is because I call NSXMLDocument
>> initWithContentsOfURL:options:error: to get data, and that method
>> uses
>> NSURLConnection internally?
>
> This sounds like a variant of the NSURLConnection bug that causes a
> sporadic
> crash when loading Core Data data from an XML store. The
> NSPersistentStoreCoordinator calls NSXMLDocument
> initWithContentsOfURL:options:error: which calls NSURLConnection
> and things
> rapidly go south because of threading (and it's all Apple's code,
> so doing
> something about it is quite tricky).
>
> So, to sum up, yes, it's known that there are memory-management
> problems
> with NSURLConnection and threads, and Apple's own code falls prey
> to these
> problems. m.
>
> --
> matt neuburg, phd = <matt...>, <http://www.tidbits.com/matt/>
> A fool + a tool + an autorelease pool = cool!
> AppleScript: the Definitive Guide - Second Edition!
> <http://www.amazon.com/gp/product/0596102119
-
Thanks for the tip, but unfortunately it didn't help in this case. By the way, I am now using NSURLConnections directly, rather than NSXMLDoc initWithContents... here's the code:
-(NSData *)fetchXMLData:(NSURL *)theURL {
NSURLRequest *urlRequest = [NSURLRequest requestWithURL:theURL cachePolicy:NSURLRequestReloadIgnoringCacheData timeoutInterval:[self connectionTimeoutInterval]];
NSHTTPURLResponse *xmlResponse;
NSError *fetchError;
NSData *xmlData = [NSURLConnection sendSynchronousRequest:urlRequest returningResponse:&xmlResponse error:&fetchError];
return xmlData;
}
Adding the NSURLCache calls didn't help, though I would expect that since the NSURLRequest already has a cachePolicy of NSURLRequestReloadIgnoringCacheData.
Anyone have any more info on this? Christ..
Paul
----------Original Message----------
On 3/13/07 14:07 , John Mikros (<jmikros...>) wrote:
> I don't know if this will help you, but our app was leaking memory in
> NSURLConnection as well. I didn't realize it could be related to
> creating the NSURLConnection on a secondary thread -- that may very
> well be the case.
>
> In any event, we found that the leak went away if we set the capacity
> of the URL cache to zero.
>
> [[NSURLCache sharedURLCache] setMemoryCapacity:0];
> [[NSURLCache sharedURLCache] setDiskCapacity:0];
>
> In our case, we don't want any of the data cached anyway, so this
> made sense.
>
> -john
>
> On Mar 11, 2007, at 9:12 AM, Matt Neuburg wrote:
>
>> On Sun, 4 Mar 2007 15:51:06 -0800, Paul Borokhov <lensovetp...> said:
>>> So, as a follow-up to my previous threads discussion, which I do
>>> want to keep
>>> on-topic, it appears that NSURLConnection leaks whenever it's
>>> called inside a
>>> method that's been detached from the main thread. Namely, I tested my
>>> application with threading and without, and the thread-less version has no
>>> leaks. Just to reiterate, I never actually call the NSURLConnection
>>> methods
>>> directly, I DO have autorelease pools set up properly, and there is
>>> NOTHING
>>> else that is leaking (apart from the NSUC-associated classes), even
>>> though I
>>> do create (and manage to properly discard) a lot of other data,
>>> including XML
>>> documents, strings, etc. I'm guessing that the reason NSURLConnection gets
>>> called at all is because I call NSXMLDocument
>>> initWithContentsOfURL:options:error: to get data, and that method uses
>>> NSURLConnection internally?
>>
>> This sounds like a variant of the NSURLConnection bug that causes a
>> sporadic
>> crash when loading Core Data data from an XML store. The
>> NSPersistentStoreCoordinator calls NSXMLDocument
>> initWithContentsOfURL:options:error: which calls NSURLConnection and things
>> rapidly go south because of threading (and it's all Apple's code, so doing
>> something about it is quite tricky).
>>
>> So, to sum up, yes, it's known that there are memory-management problems
>> with NSURLConnection and threads, and Apple's own code falls prey to these
>> problems. m.
