CoreData, reset a persistent store

  • Hello,

    I am wondering how to reset a persistent store used by a single data
    model.
    -A simple solution would be to fetch all objects and delete them;
    this consumes much memory.
    -I am not sure that deleting the file under the persistent store is a
    good idea.
    -the "reset" method of the managed object model is not designed for
    that.

    Did I miss something ?

    Regards,

    Pierre Chatelier
  • Op 14-nov-2006, om 14:43 heeft Pierre Chatelier het volgende geschreven:

    > Hello,
    >
    > I am wondering how to reset a persistent store used by a single
    > data model.
    > -A simple solution would be to fetch all objects and delete them;
    > this consumes much memory.
    > -I am not sure that deleting the file under the persistent store is
    > a good idea.
    > -the "reset" method of the managed object model is not designed for
    > that.
    >
    > Did I miss something ?

    If I understand you correctly, yes. In Xcode, simply change the url
    in your appDelegate's (or controllers) .m file:
    url = [NSURL fileURLWithPath: [applicationSupportFolder
    stringByAppendingPathComponent: @"someProject.xml"]];
    ( see http://www.cocoadevcentral.com/articles/000085.php on how to
    set the correct type).
    The build and run will take a bit longer the first time, but that's
    all. The store itself you'll find under the theUser/Library/
    Application Support/SomeProject/
    The NSPersistentStoreCoordinator is designed not to care which store
    you use, as long as it confirms to the expecations of the
    ManagedObjectContext which in turn looked to the data model for
    guidance.

    Elise van Looij
  • >> I am wondering how to reset a persistent store used by a single
    >> data model.
    > If I understand you correctly, yes. In Xcode, simply change the url
    > in your appDelegate's (or controllers) .m file:
    > url = [NSURL fileURLWithPath: [applicationSupportFolder
    > stringByAppendingPathComponent: @"someProject.xml"]];
    > ( see http://www.cocoadevcentral.com/articles/000085.php on how to
    > set the correct type).
    > The build and run will take a bit longer the first time, but that's
    > all. The store itself you'll find under the theUser/Library/
    > Application Support/SomeProject/
    > The NSPersistentStoreCoordinator is designed not to care which
    > store you use, as long as it confirms to the expecations of the
    > ManagedObjectContext which in turn looked to the data model for
    > guidance.

    I do not understand... Do you mean that I should replace the
    persistent-file, at run-time, by an empty one that could be embedded
    in the application resources ?
    My problem is that my persistent store contains cached data, and that
    sometimes i would like to empty it to avoid infinite growing. And
    that not method of NSPersistentCoordinator proposed to empty the files.

    Regards,

    Pierre Chatelier
  • Op 14-nov-2006, om 15:37 heeft Pierre Chatelier het volgende geschreven:

    >>> I am wondering how to reset a persistent store used by a single
    >>> data model.
    >> If I understand you correctly, yes. In Xcode, simply change the
    >> url in your appDelegate's (or controllers) .m file:
    >> url = [NSURL fileURLWithPath: [applicationSupportFolder
    >> stringByAppendingPathComponent: @"someProject.xml"]];
    >> ( see http://www.cocoadevcentral.com/articles/000085.php on how to
    >> set the correct type).
    >> The build and run will take a bit longer the first time, but
    >> that's all. The store itself you'll find under the theUser/Library/
    >> Application Support/SomeProject/
    >> The NSPersistentStoreCoordinator is designed not to care which
    >> store you use, as long as it confirms to the expecations of the
    >> ManagedObjectContext which in turn looked to the data model for
    >> guidance.
    >
    > I do not understand... Do you mean that I should replace the
    > persistent-file, at run-time, by an empty one that could be
    > embedded in the application resources ?
    > My problem is that my persistent store contains cached data, and
    > that sometimes i would like to empty it to avoid infinite growing.
    > And that not method of NSPersistentCoordinator proposed to empty
    > the files.

    No, at runtime I don't think that would be a good idea. You might
    want to take a look at Apple's NSPersistentStoreCoordinator Class
    Reference. But as far as the cache growing to gigantic proportions, I
    don't think you need to worry about that. If I understand the section
    on Uniquing and Faulting in the Core Data Programming Guide [http://
    developer.apple.com/documentation/Cocoa/Conceptual/CoreData/
    index.html?http://developer.apple.com/documentation/Cocoa/Conceptual/
    CoreData/Articles/cdAccessorMethods.html#//apple_ref/doc/uid/
    TP40002154] correctly, Core Data takes care of all that. Don't fix it
    if it ain't broke.

    Elise van Looij
  • > No, at runtime I don't think that would be a good idea. You might
    > want to take a look at Apple's NSPersistentStoreCoordinator Class
    > Reference.
    That's what I did, but there is no method to clear a persistent store.

    > But as far as the cache growing to gigantic proportions, I don't
    > think you need to worry about that.
    I do worry. I don't want my cache file to take dozen of GB, even if
    only a part of it is in memory (thanks to faulting).

    So far, what I do is:
    -lock the managed object context
    -reset the managed object context to clear all pending changes
    -call removePersistentStore:error on the PersistentStoreCoordinator
    -remove the underlying file with NSFileManager
    -call addPersistentStoreWithType:configuration:URL:options:error on
    the PersistentStoreCoordinator to re-create the file
    -unlock the managed object context

    What I was expecting was a method like
    @interface NSManagedObjectContext
    -(void) clear;
    @end

    that would remove, from each associated persistent stores, all object
    of the managed object *fetched or not*. And there is apparently no
    method like this.
    This could be a feature request, but I first want to be sure that I
    did not miss anything.

    Regards,

    Pierre Chatelier
  • On Nov 14, 2006, at 11:09 AM, Elise van Looij wrote:
    > No, at runtime I don't think that would be a good idea. You might
    > want to take a look at Apple's NSPersistentStoreCoordinator Class
    > Reference. But as far as the cache growing to gigantic proportions,
    > I don't think you need to worry about that. If I understand the
    > section on Uniquing and Faulting in the Core Data Programming Guide

      Actually, that depends on the store type. If you're using, say,
    NSSQLiteStoreType, you probably don't really need to worry about
    performance that much.

      Pierre, have you actually tried just fetching and removing the
    objects, testing the performance? Remember what they say about
    premature optimization ...

    --
    I.S.
  • > Actually, that depends on the store type. If you're using, say,
    > NSSQLiteStoreType, you probably don't really need to worry about
    > performance that much.
    I am not worrying about performance, I just want to reset my cache
    from time to time, because it has no reason to spoil too much space.

    > Pierre, have you actually tried just fetching and removing the
    > objects, testing the performance?
    This consumes too much memory. I do not have any figures but this is
    not logical to load data before getting rid of it !

    Regards,

    Pierre Chatelier
  • > I am not worrying about performance, I just want to reset my cache
    > from time to time, because it has no reason to spoil too much space.

      Fair enough, but I consider memory consumption / paging a pretty
    significant part of 'performance'.

    > This consumes too much memory. I do not have any figures but this
    > is not logical to load data before getting rid of it !

      I have no idea just how much data you're talking about, but in
    most cases once you delete the objects you've fetched, then save the
    store, the data's no longer your problem (assuming proper memory
    management). You haven't provided any information that indicates a
    problem with memory consumption actually exists in your case. Try it,
    measure it with Shark, and you'll know.

      Core Data with a SQL store and a well-designed model performs
    amazingly well. Then again, the documentation does say, "... although
    of course reading 1GB of data into memory is an expensive operation
    no matter how efficient the repository ..."

      If you *do* find a problem (and you're not loading 1 GB of data
    into memory), it can likely be solved by adjusting your data model.
    Test your assumptions, you may be surprised. In any case, more
    specific information is needed to give you more specific answers. :-)

      You haven't specified the store type you're using, whether your
    app is document-based, or the amount of data you're working with and
    how it's structured. If you post this information, you might get a
    clearer answer or a better suggestion.

      I hope this is at least somewhat helpful.

    --
    I.S.
  • > You haven't specified the store type you're using, whether your
    > app is document-based, or the amount of data you're working with
    > and how it's structured. If you post this information, you might
    > get a clearer answer or a better suggestion.
    You are right. I thought that it was only a little question, with an
    answer like "you're right" or "you're wrong, just do that <snip>",
    but it seems debatable :-)

    So, here are more informations :

    -I have an application that computes some data. Each computed data is
    put inside an object that has some attributes : the data, the options
    used for computation... The data makes it a large structure, but it
    is a simple one : there is no relationship properties in the class,
    only attributes.

    -Now, I want to cache the data that is computed, for later use in the
    same application. ("Later" means some other day, the life time is
    greater that the application run-time). CoreData with SQLLiteStore is
    of great help, since it allows to store much data, but to keep in
    memory only what is necessary.

    -So, I have a very simple data model that holds only one entity : an
    NSManagedObject with the attributes described above (data,
    options...), and I can request the NSManageObjectContext to load some
    cached object instead of computing the data again.

    -Now, I want to let the user free the cache directly from a menu of
    the application. The user may want to do that to save some space on
    its disk, or to force triggering computations again instead of using
    cached values (let's say for benchmarking or debbugging). Depending
    on the user and the way he's using the application, the cache file
    may be from some mega-bytes to some giga-bytes.

    -So, to delete the cache, I was expecting a way to tell the
    NSManagedObjectModel : "ok, forget all". But it does not empty the
    file under the NSPersistentStoreCoordinator ! So I have to do that
    myself. A simple way is to fetch all objects, remove them and save,
    but this is a pain to load all that data in memory, for nothing !
    So far, what I do is:
    -lock the managed object context
    -reset the managed object context to clear all pending changes
    -call removePersistentStore:error on the PersistentStoreCoordinator
    -remove the underlying file with NSFileManager
    -call addPersistentStoreWithType:configuration:URL:options:error on
    the PersistentStoreCoordinator to re-create the file
    -unlock the managed object context
    this is simple, it works, but I personnally find it strange that the
    API cannot do that for me. The API could do even better, it could
    delete all objects of my NSManagedObjectModel (even if they are
    faults), in every persistent store attached to that model.

    > Fair enough, but I consider memory consumption / paging a pretty
    > significant part of 'performance'.
    Ok

    > Then again, the documentation does say, "... although of course
    > reading 1GB of data into memory is an expensive operation no matter
    > how efficient the repository ..."
    Yes, so my problem is to avoid loading, since it is totally useless
    (I just want to delete the data). You can call it optimization, but
    in this case I think it is just common sense.

    > I hope this is at least somewhat helpful.
    Certainly !

    Regards,

    Pierre Chatelier
  • Op 14-nov-2006, om 17:20 heeft Pierre Chatelier het volgende geschreven:

    >> No, at runtime I don't think that would be a good idea. You might
    >> want to take a look at Apple's NSPersistentStoreCoordinator Class
    >> Reference.
    > That's what I did, but there is no method to clear a persistent store.
    >
    >> But as far as the cache growing to gigantic proportions, I don't
    >> think you need to worry about that.
    > I do worry. I don't want my cache file to take dozen of GB, even if
    > only a part of it is in memory (thanks to faulting).
    >
    > So far, what I do is:
    > -lock the managed object context
    > -reset the managed object context to clear all pending changes
    > -call removePersistentStore:error on the PersistentStoreCoordinator
    > -remove the underlying file with NSFileManager
    > -call addPersistentStoreWithType:configuration:URL:options:error on
    > the PersistentStoreCoordinator to re-create the file
    > -unlock the managed object context

    Sorry, I don't know enough about memory management to help you with
    that. What I don't quite understand from your solution, though, is
    why you want to discard all the pending changes. If you don't want to
    have any pending changes, don't allow your users to make any changes.
    That way you won't have to worry about throwing them away.

    Elise van Looij
  • Have you tested doing a full retrieval and deletion?  When core data
    retrieves the objects, they are in a faulted state.  As long as you
    do not touch any of the fields inside of the objects they will remain
    faulted and take up a minimal amount of memory.

    As others have suggested, you might be prematurely optimizing this
    issue.  I would suggest trying a retrieval/delete and test it with
    shark to see if there is even a memory issue to solve.

    Marcus S. Zarra
    Zarra Studios LLC
    Simply Elegant Software for OS X
    www.zarrastudios.com

    On Nov 15, 2006, at 9:47 AM, Elise van Looij wrote:

    > Op 14-nov-2006, om 17:20 heeft Pierre Chatelier het volgende
    > geschreven:
    >
    >>> No, at runtime I don't think that would be a good idea. You might
    >>> want to take a look at Apple's NSPersistentStoreCoordinator Class
    >>> Reference.
    >> That's what I did, but there is no method to clear a persistent
    >> store.
    >>
    >>> But as far as the cache growing to gigantic proportions, I don't
    >>> think you need to worry about that.
    >> I do worry. I don't want my cache file to take dozen of GB, even
    >> if only a part of it is in memory (thanks to faulting).
    >>
    >> So far, what I do is:
    >> -lock the managed object context
    >> -reset the managed object context to clear all pending changes
    >> -call removePersistentStore:error on the PersistentStoreCoordinator
    >> -remove the underlying file with NSFileManager
    >> -call addPersistentStoreWithType:configuration:URL:options:error
    >> on the PersistentStoreCoordinator to re-create the file
    >> -unlock the managed object context
    >
    > Sorry, I don't know enough about memory management to help you with
    > that. What I don't quite understand from your solution, though, is
    > why you want to discard all the pending changes. If you don't want
    > to have any pending changes, don't allow your users to make any
    > changes. That way you won't have to worry about throwing them away.
    >
    > Elise van Looij
    >
    >
    > _______________________________________________
    > Do not post admin requests to the list. They will be ignored.
    > Cocoa-dev mailing list      (<Cocoa-dev...>)
    > Help/Unsubscribe/Update your Subscription:
    > http://lists.apple.com/mailman/options/cocoa-dev/<mzarra...>
    >
    > This email sent to <mzarra...>
  • > Have you tested doing a full retrieval and deletion?
    What are you calling a retrival ? A fetch ?

    > When core data retrieves the objects, they are in a faulted
    > state.  As long as you do not touch any of the fields inside of the
    > objects they will remain faulted and take up a minimal amount of
    > memory.
    Woah, you mean that an object remains a fault even when fetched, as
    long as I do not access the attributes ? I did not understand that, I
    will have a look.

    Regards,

    Pierre Chatelier
  • This is why I've stepped back from the thread a bit. Until there's
    actually a problem to solve, it's a waste of time and energy to
    speculate.

      Pierre, this is two people so far telling you to test / profile in
    Shark. Of course if you're happy physically removing the store and
    setting everything back up, it's definitely *a* solution. IMHO, it's
    easier to just delete the objects in the store rather than removing
    it entirely. Who knows - your user may have attached metadata to the
    store file that he/she doesn't want removed. I really don't think
    you'll run into a performance issue doing so.

    --
    I.S.

    On Nov 15, 2006, at 12:24 PM, Marcus S. Zarra wrote:

    > Have you tested doing a full retrieval and deletion?  When core
    > data retrieves the objects, they are in a faulted state.  As long
    > as you do not touch any of the fields inside of the objects they
    > will remain faulted and take up a minimal amount of memory.
    >
    > As others have suggested, you might be prematurely optimizing this
    > issue.  I would suggest trying a retrieval/delete and test it with
    > shark to see if there is even a memory issue to solve.
    >
    > Marcus S. Zarra
    > Zarra Studios LLC
    > Simply Elegant Software for OS X
    > www.zarrastudios.com
    >
    > On Nov 15, 2006, at 9:47 AM, Elise van Looij wrote:
    >
    >> Op 14-nov-2006, om 17:20 heeft Pierre Chatelier het volgende
    >> geschreven:
    >>
    >>>> No, at runtime I don't think that would be a good idea. You
    >>>> might want to take a look at Apple's
    >>>> NSPersistentStoreCoordinator Class Reference.
    >>> That's what I did, but there is no method to clear a persistent
    >>> store.
    >>>
    >>>> But as far as the cache growing to gigantic proportions, I don't
    >>>> think you need to worry about that.
    >>> I do worry. I don't want my cache file to take dozen of GB, even
    >>> if only a part of it is in memory (thanks to faulting).
    >>>
    >>> So far, what I do is:
    >>> -lock the managed object context
    >>> -reset the managed object context to clear all pending changes
    >>> -call removePersistentStore:error on the PersistentStoreCoordinator
    >>> -remove the underlying file with NSFileManager
    >>> -call addPersistentStoreWithType:configuration:URL:options:error
    >>> on the PersistentStoreCoordinator to re-create the file
    >>> -unlock the managed object context
    >>
    >> Sorry, I don't know enough about memory management to help you
    >> with that. What I don't quite understand from your solution,
    >> though, is why you want to discard all the pending changes. If you
    >> don't want to have any pending changes, don't allow your users to
    >> make any changes. That way you won't have to worry about throwing
    >> them away.
    >>
    >> Elise van Looij
    >>
    >>
    >> _______________________________________________
    >> Do not post admin requests to the list. They will be ignored.
    >> Cocoa-dev mailing list      (<Cocoa-dev...>)
    >> Help/Unsubscribe/Update your Subscription:
    >> http://lists.apple.com/mailman/options/cocoa-dev/<mzarra...>
    >>
    >> This email sent to <mzarra...>
    >
    > _______________________________________________
    > Do not post admin requests to the list. They will be ignored.
    > Cocoa-dev mailing list      (<Cocoa-dev...>)
    > Help/Unsubscribe/Update your Subscription:
    > http://lists.apple.com/mailman/options/cocoa-dev/idiotsavant2005%
    > 40gmail.com
    >
    > This email sent to <idiotsavant2005...>
  • > Pierre, this is two people so far telling you to test / profile
    > in Shark.
    Actually, I am not familiar with shark at all, I do not know how to
    get a graph of memory usage and of CPU usage, but I made a test with
    ActivityMonitor

    -deleting the persistent store file takes no time at all and no
    resources
    -with a 23MB cache (which is rather small in my case), fetching all
    objects, deleting them and saving the context takes approximatively
    10 seconds and consumes about 20MB of real memory

    The difference is significative.

    Regards,

    Pierre Chatelier
  • I would be interested in seeing the code as that seems excessive.

    Any chance you can share just that little snippet that handles the
    delete routine?

    Marcus S. Zarra
    Zarra Studios LLC
    Simply Elegant Software for OS X
    www.zarrastudios.com

    On Nov 15, 2006, at 12:12 PM, Pierre Chatelier wrote:

    >> Pierre, this is two people so far telling you to test / profile
    >> in Shark.
    > Actually, I am not familiar with shark at all, I do not know how to
    > get a graph of memory usage and of CPU usage, but I made a test
    > with ActivityMonitor
    >
    > -deleting the persistent store file takes no time at all and no
    > resources
    > -with a 23MB cache (which is rather small in my case), fetching all
    > objects, deleting them and saving the context takes approximatively
    > 10 seconds and consumes about 20MB of real memory
    >
    > The difference is significative.
    >
    > Regards,
    >
    > Pierre Chatelier
    >
  • > Any chance you can share just that little snippet that handles the
    > delete routine?
    Sure !
    Here it is :

    //first version, fetch all, deletes, save
        [managedObjectContext lock];
        NSFetchRequest* fetchRequest = [[NSFetchRequest alloc] init];
        [fetchRequest setEntity:[NSEntityDescription
    entityForName:@"CachedFeatures"
    inManagedObjectContext:managedObjectContext]];
        [fetchRequest setPredicate:[NSPredicate
    predicateWithFormat:@"SELF != nil"]];
        NSArray* allObjects = [managedObjectContext
    executeFetchRequest:fetchRequest error:&error];
        NSLog(@"[allObjects count] = %d", [allObjects count]);
        NSEnumerator* enumerator = [allObjects objectEnumerator];
        id object = nil;
        while((object = [enumerator nextObject]))
          [managedObjectContext deleteObject:object];
        [managedObjectContext save:&error];
        [managedObjectContext unlock];

    //second version : delete file
        [managedObjectContext lock];
        [managedObjectContext reset];//to drop pending changes
        if ([persistentStoreCoordinator
    removePersistentStore:persistentStore error:&error])
        {
          NSURL* storeURL = [NSURL fileURLWithPath:[self
    pathForPersistentStore]];
          [[NSFileManager defaultManager] removeFileAtPath:[storeURL
    path]  handler:nil];
          [self addPersistentStore];//recreates the persistent store
        }
        [managedObjectContext unlock];
  • On Nov 15, 2006, at 2:37 PM, Pierre Chatelier wrote:
    > [fetchRequest setPredicate:[NSPredicate
    > predicateWithFormat:@"SELF != nil"]];

      May I ask what the purpose of this line is?

    --
    I.S.
  • >> [fetchRequest setPredicate:[NSPredicate
    >> predicateWithFormat:@"SELF != nil"]];
    > May I ask what the purpose of this line is?
    This is the predicate I use to fetch all objects (in order to delete
    them afterwards).

    > while((object = [enumerator nextObject])) {
    > if (![object isFault]) {
    > NSLog(@"Object is not faulted");
    > }
    > [managedObjectContext deleteObject:object];
    > }
    No "Object is not faulted" is displayed in the log.

    > Do you have anything in the awakeFromFetch method by chance?
    No, I do not have rewritten awakeFromFetch. Actually my entity is of
    class NSManagedObject I did not subclass it.

    Pierre
  • On Nov 15, 2006, at 4:19 PM, Pierre Chatelier wrote:
    >>> [fetchRequest setPredicate:[NSPredicate
    >>> predicateWithFormat:@"SELF != nil"]];
    >> May I ask what the purpose of this line is?
    > This is the predicate I use to fetch all objects (in order to
    > delete them afterwards).

      It's utterly unnecessary and probably not right. Get rid of
    it. :-) All you need is the entity name (without setting a predicate)
    and all instances of that entity will be fetched.

    --
    I.S.
  • >>>> [fetchRequest setPredicate:[NSPredicate
    >>>> predicateWithFormat:@"SELF != nil"]];
    >>> May I ask what the purpose of this line is?
    >> This is the predicate I use to fetch all objects (in order to
    >> delete them afterwards).
    > It's utterly unnecessary and probably not right. Get rid of it. :-)
    You're right, this is not necessary. I used to believe that a
    predicate was required, but it's not the case.

    Thanks to that discussion, I now understand better the difference
    between faulting and internal-caching of CoreData, but it does not
    change the problem.
    My problem is still to clear my persistent store without fetching data.

    More precise times :

    //the code
        [managedObjectContext lock];
        NSFetchRequest* fetchRequest = [[NSFetchRequest alloc] init];
        [fetchRequest setEntity:[NSEntityDescription
    entityForName:@"CachedFeatures"
    inManagedObjectContext:managedObjectContext]];
        NSLog(@"fetch");
        NSEnumerator* allObjectsEnumerator = [[managedObjectContext
    executeFetchRequest:fetchRequest error:&error] objectEnumerator];
        NSLog(@"delete");
        id object = nil;
        while((object = [allObjectsEnumerator nextObject]))
          [managedObjectContext deleteObject:object];
        NSLog(@"save");
        [managedObjectContext save:&error];
        NSLog(@"done");
        [managedObjectContext unlock];

    //the log (see the durations : fetch is 4 seconds : that's a lot, and
    save is 7 seconds ! (that's for ~8000 objects))
    2006-11-15 23:17:16.842 MyApp[2799] fetch
    2006-11-15 23:17:20.576 MyApp[2799] delete
    2006-11-15 23:17:20.721 MyApp[2799] save
    2006-11-15 23:17:27.805 MyApp[2799] done

    Pierre
  • > That is the issue with the memory and the speed then.  Somehow
    > those objects are becoming fully realized which is causing the
    > memory spike and probably the delay as well.
    I disagree ! The objects *are* faults (remember, no "Object is not
    faulted" is displayed).
    However, since they were fetched, CoreData did cache them. This has a
    cost in speed and memory. My concern is to prevent that caching.

    Pierre
  • On 11/15/06, Pierre Chatelier <pierre.chatelier...> wrote:
    >> That is the issue with the memory and the speed then.  Somehow
    >> those objects are becoming fully realized which is causing the
    >> memory spike and probably the delay as well.
    > I disagree ! The objects *are* faults (remember, no "Object is not
    > faulted" is displayed).

    ...confused...

    Your code is the following...

    > if (![object isFault]) {
    > NSLog(@"Object is not faulted");
    > }

    Based on the if logic the log statement can be more clearly stated as
    "object data must be in memory". Review the documentation for
    -[NSManagedObject isFault].

    You are seeing the log message so the object has been loaded into memory.

    -Shawn
  • > If you look at the code, I negated the call (![object isFault])
    > which means they are fully realized hence your memory consumption.
    Yes, we agree on that

    > These objects either are already in memory or are some how firing
    > on the retrieval
    Since they were fetched, yes, they must be in memory; they can be
    faults (and they are), but the data is cached and firing the fault
    costs almost nothing.

    > Fetching takes minimal memory and time, it is the firing of the
    > fetch into a fully realized object that costs the time and memory.
    Here we disagree !

    My opinion comes from the following documentation : CoreData
    Programming Guide > Faulting and Uniquing > Realizing a managed object
    we can read :

    1)If you execute a fetch using executeFetchRequest:error:, this
    always results in a round trip to the persistent store to fetch the
    data. The objects returned in the results array are fully realized,
    and their data is stored in a cache (held by the persistent store
    coordinator).

    my comment : that's what I do : I fetch, and this results in faults-
    but-realized objects

    2)If you fire a fault, Core Data does not go back to the store if the
    data is available in the cache. With a cache hit, converting a fault
    into a realized managed object is very fast—it is basically the same
    as normal instantiation of a managed object. If the data is not
    available in the cache, Core Data automatically executes a fetch for
    the fault object; this results in a round trip to the persistent
    store to fetch the data, and again the data is cached in memory.

    my comment : "converting a fault into a realized managed object is
    very fast", is true in my case since data was fetched and is
    availabel in the cache.

    Regards,

    Pierre _______________________________________________
    Do not post admin requests to the list. They will be ignored.
    Cocoa-dev mailing list      (<Cocoa-dev...>)
    Help/Unsubscribe/Update your Subscription:
    http://lists.apple.com/mailman/options/cocoa-dev/<cocoa...>

    This email sent to <cocoa...>
  • On 11/15/06, Pierre Chatelier <pierre.chatelier...> wrote:
    >> If you look at the code, I negated the call (![object isFault])
    >> which means they are fully realized hence your memory consumption.
    > Yes, we agree on that
    >
    >> These objects either are already in memory or are some how firing
    >> on the retrieval
    > Since they were fetched, yes, they must be in memory; they can be
    > faults (and they are), but the data is cached and firing the fault
    > costs almost nothing.
    >
    >> Fetching takes minimal memory and time, it is the firing of the
    >> fetch into a fully realized object that costs the time and memory.
    > Here we disagree !
    >
    > My opinion comes from the following documentation : CoreData
    > Programming Guide > Faulting and Uniquing > Realizing a managed object
    > we can read :
    >
    > 1)If you execute a fetch using executeFetchRequest:error:, this
    > always results in a round trip to the persistent store to fetch the
    > data. The objects returned in the results array are fully realized,
    > and their data is stored in a cache (held by the persistent store
    > coordinator).

    This implies that you are fully loading all objects of entity kind
    "CachedFeatures" from the persistent store (loading from disk) to just
    go an delete them since you are using executeFetchRequest:error:. You
    are not utilizing any cache for this fetch (but you are getting the
    cache populated). This is likely unneeded overhead.

    Stepping back ... what are "CachedFeatures" objects? Do you even want
    the data those objects contain in the persistent store? If not the
    consider making its fields transient.

    -Shawn
  • > This implies that you are fully loading all objects of entity kind
    > "CachedFeatures" from the persistent store (loading from disk) to
    > just go an delete them since you are using
    > executeFetchRequest:error:. You are not utilizing any cache for
    > this fetch (but you are getting the cache populated). This is
    > likely unneeded overhead.
    Exactly ! And I want to avoid this overhead. I began this thread of
    discussion to know if there was a way to tell a persistent store (or
    a managed object context) to remove objects without fetching them.

    > Stepping back ... what are "CachedFeatures" objects ?
    It's some data related to my application

    > Do you even want the data those objects contain in the persistent
    > store?
    Absolutely ! But sometimes I want to reset the persistent store.

    Regards,

    Pierre Chatelier
  • On Nov 15, 2006, at 4:44 PM, Pierre Chatelier wrote:

    >> This implies that you are fully loading all objects of entity kind
    >> "CachedFeatures" from the persistent store (loading from disk) to
    >> just go an delete them since you are using
    >> executeFetchRequest:error:. You are not utilizing any cache for
    >> this fetch (but you are getting the cache populated). This is
    >> likely unneeded overhead.
    > Exactly ! And I want to avoid this overhead. I began this thread of
    > discussion to know if there was a way to tell a persistent store
    > (or a managed object context) to remove objects without fetching them.

    Not knowing how your data model... I wonder why you are executing a
    fetch?

    Do you have a top level managed object that has a one-to-many
    relationship with "CachedFeatures" entity?

    If you do then it could be as simple as iterating over the collection
    that you could get form such a top level object instead of fetching
    that collection from the persistent store. The former shouldn't
    trigger faulting (again not knowing what relationship and delete
    semantics CacheFeatures has).

    If you don't have such an object... why don't you? :)

    -Shawn
  • > Not knowing how your data model... I wonder why you are executing a
    > fetch?
    I execute a fetch because I found no other way to delete objects from
    a persistent store

    > Do you have a top level managed object that has a one-to-many
    > relationship with "CachedFeatures" entity?
    No

    > If you don't have such an object... why don't you? :)
    Because my persistent store is a big database. Sometimes I pick up
    things in it; but when I want to delete its content, there is no
    particular reason that *all* the content has been used in the
    program ! So I have no external collection refering all the objects
    of my database.

    Regards,

    Pierre
  • On Nov 14, 2006, at 8:43 AM, Pierre Chatelier wrote:

    > I am wondering how to reset a persistent store used by a single
    > data model.
    > -A simple solution would be to fetch all objects and delete them;
    > this consumes much memory.
    > -I am not sure that deleting the file under the persistent store is
    > a good idea.

    So at this point you really want to discard all the data in the store
    (and don't care about any metadata in the store)?

    Can you structure your application in such a way that at this point,
    you tear down your persistence stack (persistent store coordinator,
    managed object model, and any managed object contexts), remove the
    file on disk, then bring the persistence stack back up?

    Jim
  • On Nov 15, 2006, at 6:03 PM, Pierre Chatelier wrote:

    >> If you don't have such an object... why don't you? :)
    > Because my persistent store is a big database. Sometimes I pick up
    > things in it; but when I want to delete its content, there is no
    > particular reason that *all* the content has been used in the
    > program ! So I have no external collection refering all the objects
    > of my database.

    Maybe I'm misunderstanding either Shawn's question or your response,
    but it seems like an easy way to delete the contents of the store is
    to have all of the objects in the store point to a single object. A
    giant to-many relationship.

    Setup the delete rule so that when the "master" object is deleted, it
    cascades to all of the other objects. I'm not sure if this would be
    faster than fetching everything and deleting it, but it might be.
    Worth trying.

        - Scott
  • On Nov 14, 2006, at 5:43 AM, Pierre Chatelier wrote:

    > I am wondering how to reset a persistent store used by a single data
    > model.
    > -A simple solution would be to fetch all objects and delete them;
    > this consumes much memory.
    > -I am not sure that deleting the file under the persistent store is
    > a good idea.
    > -the "reset" method of the managed object model is not designed for
    > that.

    There's no such thing as "resetting" a persistent store.  A persistent
    store can have data inserted into it or deleted from it, but it can't
    be "emptied."

    Instead, you can just do the following:

    (1) Make sure you have no existing instances of any managed objects
    associated with that persistent store.

    (2) Ask any persistent store coordinator to which your persistent
    store has been added to remove it.

    (3) Delete the underlying file corresponding to the persistent store.

    Then you can add a new persistent store to any coordinator with the
    same URL and get a new, empty persistent store at that URL as a result.

      -- Chris
  • To give a more abstract perspective of Chris's answer ("There's no
    such thing as "resetting" a persistent store."):

    On Nov 15, 2006, at 4:44 PM, Pierre Chatelier wrote:

    >> Do you even want the data those objects contain in the persistent
    >> store?
    > Absolutely ! But sometimes I want to reset the persistent store.
    >
    You're thinking about this the wrong way round; managed objects do not
    represent what's in the store, the store is a persistent repository
    for managed objects...

    mmalc
  • > So at this point you really want to discard all the data in the
    > store (and don't care about any metadata in the store)?
    I want to delete in the persistent store all the objects that have
    been put inside by a known NSManagedObjectModel

    > Can you structure your application in such a way that at this
    > point, you tear down your persistence stack (persistent store
    > coordinator, managed object model, and any managed object
    > contexts), remove the file on disk, then bring the persistence
    > stack back up?

    > (1) Make sure you have no existing instances of any managed objects
    > associated with that persistent store.
    > (2) Ask any persistent store coordinator to which your persistent
    > store has been added to remove it.
    > (3) Delete the underlying file corresponding to the persistent store.
    > Then you can add a new persistent store to any coordinator with the
    > same URL and get a new, empty persistent store at that URL as a
    > result.

    If you look at my first posts, this is exactly the solution that I
    chose, because it was very efficient. I began this thread of
    discussion because I was wondering if it was the only way to make it
    efficient.

        [managedObjectContext lock];
        [managedObjectContext reset];
        if ([persistentStoreCoordinator
    removePersistentStore:persistentStore error:&error])
        {
          NSURL* storeURL = [NSURL fileURLWithPath:[self
    pathForPersistentStore]];
          [[NSFileManager defaultManager] removeFileAtPath:[storeURL
    path]  handler:nil];
          [self addPersistentStore];
        }
        [managedObjectContext unlock];

    > Setup the delete rule so that when the "master" object is deleted,
    > it cascades to all of the other objects
    Interesting idea ! Should be tried ! But I have no "master" object.
    It means that I should modify my data model to workaround a method I
    miss in the CoreData API. I was told to take care of "premature
    optimization", but at least, by deleting the file directly, I was not
    breaking my program as a workaround ;-)
    But I will try in the future.

    > You're thinking about this the wrong way round; managed objects do
    > not represent what's in the store, the store is a persistent
    > repository for managed objects...
    Ok, I understand the problem. There could be several
    ManagedObjectContext associated to the persistent store, so I just
    cannot tell "remove managed objects" because managed objects have
    sense only in a context, i.e. they have been fetched...
    But in that case, I would like to tell the persistent store "forget
    all object of a given entity". And there is no method like that.

    Personnally, I am satisfied with the discussion, and I think it can
    be closed. I would like to sum up the different points we've talked
    about :

    -Fetching an object will always result in time and memory consumption
    since even faulted, the fetched data is cached.
    -So far, a persistent store cannot be requested unless we use a
    managed object context (which implie a fetch). So, an "object" of the
    persistent store has no meaning until it is actually fetched by a
    context.
    -My request would be to Apple, to allow interaction with the
    persistent store, either with SQLLite requests, or with a minimal set
    of function like "removeAllEntities:(NSEntityDescription*) cascade:
    (BOOL)". This could be used to clear the persistent store without
    fetching in a managed object context.
    -So far, a solution to my innitial problem (clearing a persistent
    store as minimal cost), is to remove by hand the underlying file.

    Thanks to everybody

    Regards,

    Pierre Chatelier
  • >
    On Nov 16, 2006, at 12:07 AM, Pierre Chatelier wrote:

    >> You're thinking about this the wrong way round; managed objects do
    >> not represent what's in the store, the store is a persistent
    >> repository for managed objects...
    > Ok, I understand the problem. There could be several
    > ManagedObjectContext associated to the persistent store, so I just
    > cannot tell "remove managed objects" because managed objects have
    > sense only in a context, i.e. they have been fetched...
    > But in that case, I would like to tell the persistent store "forget
    > all object of a given entity". And there is no method like that.
    >
    Again that's the wrong way round to think about it.  Core Data is an
    *object graph management* and persistence framework.  The store is
    just somewhere objects go to sleep at night.  You could get away with
    doing what you ask in your situation because you have a single
    entity.  In the general case, there may be several entities with
    relationships between them.  If you want to remove all instances of an
    entity, then you must ensure that the resulting object graph is valid
    (there may be relationship constraints that need to be applied) and
    you can only do this by bringing into memory all the instances of the
    entity...

    mmalc
  • > Again that's the wrong way round to think about it.  (...) If you
    > want to remove all instances of an entity, then you must ensure
    > that the resulting object graph is valid (there may be relationship
    > constraints that need to be applied)
    Yes, but the NSPersistentStoreCoordinator is initialized with the
    model, so the persistent store should know how to cascade deletions
    by traversing relationships

    Regards,

    Pierre
  • On Nov 16, 2006, at 2:40 AM, Pierre Chatelier wrote:

    >> Again that's the wrong way round to think about it.  (...) If you
    >> want to remove all instances of an entity, then you must ensure
    >> that the resulting object graph is valid (there may be relationship
    >> constraints that need to be applied)
    > Yes, but the NSPersistentStoreCoordinator is initialized with the
    > model, so the persistent store should know how to cascade deletions
    > by traversing relationships
    >
    Cascading deletions per se is not the issue; the issue is whether the
    resulting object graph is in a valid state.  This can only, in the
    general case, be determined by having the objects in memory.

    (Consider what happens if you allow the object graph to be saved in an
    invalid state, for example if the destination of a required
    relationship is not present: The next time the user loads the store
    and edits a value -- perhaps completely unrelated to the "problem"
    object -- then tries to save, they will be unable to do so due to a
    validation error...)

    mmalc
  • >>> Again that's the wrong way round to think about it.  (...) If you
    >>> want to remove all instances of an entity, then you must ensure
    >>> that the resulting object graph is valid (there may be
    >>> relationship constraints that need to be applied)
    >> Yes, but the NSPersistentStoreCoordinator is initialized with the
    >> model, so the persistent store should know how to cascade
    >> deletions by traversing relationships
    > Cascading deletions per se is not the issue; the issue is whether
    > the resulting object graph is in a valid state.  This can only, in
    > the general case, be determined by having the objects in memory.
    I do not understand why you must have the objects in memory to know
    if this is a valid state... I was expecting the persistent store to
    be able to understand relationships, but perhaps it's not the case ?

    > (Consider what happens if you allow the object graph to be saved in
    > an invalid state, for example if the destination of a required
    > relationship is not present:
    Do you mean, for instance, the following :
    -Let's suppose I have a persistent store containing two types of
    entities A and B, with relation ships between them
    -Let's suppose the persistent store allows me to delete all entities
    of type A.
    -if I remove all entities A in the persistent store, only the
    managedObjectContext is able to determine cascading deletions. The
    persistent store does not know which B entities were connected and
    should be removed ?

    Thanks for your patience

    Regards,

    Pierre Chatelier
  • On Nov 16, 2006, at 4:33 AM, Pierre Chatelier wrote:

    >>>> Cascading deletions per se is not the issue; the issue is whether
    >>>> the resulting object graph is in a valid state.  This can only,
    >>>> in the general case, be determined by having the objects in memory.
    > I do not understand why you must have the objects in memory to know
    > if this is a valid state... I was expecting the persistent store to
    > be able to understand relationships, but perhaps it's not the case ?

    Relationships are not the only way to determine validity.  You can
    also have business logic that determines whether individual objects
    are valid and/or whether an entire sub-graph of objects is valid.

    >> (Consider what happens if you allow the object graph to be saved in
    >> an invalid state, for example if the destination of a required
    >> relationship is not present:
    > Do you mean, for instance, the following :
    > -Let's suppose I have a persistent store containing two types of
    > entities A and B, with relation ships between them
    > -Let's suppose the persistent store allows me to delete all entities
    > of type A.
    > -if I remove all entities A in the persistent store, only the
    > managedObjectContext is able to determine cascading deletions. The
    > persistent store does not know which B entities were connected and
    > should be removed ?

    When you attempt to save the managed object context, it will check not
    just the validity constraints that are specified in your managed
    object model, but also any validity constraints that are implemented
    in code using NSManagedObject's validation methods.

      -- Chris
  • > When you attempt to save the managed object context, it will check
    > not just the validity constraints that are specified in your
    > managed object model, but also any validity constraints that are
    > implemented in code using NSManagedObject's validation methods.
    Ok, I understand better.

    Thanks

    Pierre
previous month november 2006 next month
MTWTFSS
    1 2 3 4 5
6 7 8 9 10 11 12
13 14 15 16 17 18 19
20 21 22 23 24 25 26
27 28 29 30      
Go to today