Core Data merge and "statement is still active" error?

  • The goal is to do insertions/changes/deletes on a background thread
    and then after a save merge those changes into the main threads
    managed object context and thus update the UI.

    I have a situation where NSManagedObjectContextDidSaveNotification is
    forwarded from a background thread's managed object context to the
    main thread's managed object context (each thread has it's own managed
    object context). The main thread does some processing like this (in a
    subclass of NSPersistentDocument).

    - (void)_mergeChangesFromContextDidSaveNotification:
    (NSNotification*)notification
    {
    NSManagedObjectContext* __managedObjectContext = [self
    managedObjectContext];
    NSDate* __date = [[[NSFileManager defaultManager]
    fileAttributesAtPath:[[self fileURL] path] traverseLink:YES]
    objectForKey:NSFileModificationDate];

    [__managedObjectContext
    mergeChangesFromContextDidSaveNotification:notification];
    [__managedObjectContext processPendingChanges];
    [[__managedObjectContext undoManager] enableUndoRegistration];

    NSLog(@"%s *1* hasChanges=%d", __FUNCTION__, [__managedObjectContext
    hasChanges]);

    [self setFileModificationDate:__date];

    if ([__managedObjectContext hasChanges]) {
      [self saveDocument:self];
    }
    }

    - (void)mergeChangesFromContextDidSaveNotification:
    (NSNotification*)notification
    {
    NSManagedObjectContext* __managedObjectContext = [self
    managedObjectContext];

    NSLog(@"%s *0* hasChanges=%d", __FUNCTION__, [__managedObjectContext
    hasChanges]);

    if ([__managedObjectContext hasChanges]) {
      NSDate* __date = [[[NSFileManager defaultManager]
    fileAttributesAtPath:[[self fileURL] path] traverseLink:YES]
    objectForKey:NSFileModificationDate];

      [self setFileModificationDate:__date];
      [self saveDocument:self];
    }

    [[__managedObjectContext undoManager] disableUndoRegistration];

    for (NSManagedObject* __object in [[notification userInfo]
    objectForKey:NSUpdatedObjectsKey]) {
      [__managedObjectContext refreshObject:[__managedObjectContext
    objectWithID:[__object objectID]] mergeChanges:NO];
    }

    [self
    performSelector
    :@selector(_mergeChangesFromContextDidSaveNotification:)
    withObject:notification afterDelay:0.0];
    }

    If I forgo the refreshObject:mergeChanges: the error doesn't occur,
    but then another problem occurs where sometimes a relationship doesn't
    merge correctly between 2 objects (even when there are no changes in
    the main thread's managed object context the merge will sometimes
    delete part of one side of the relationship. This happens with many-to-
    many relationships including reflexive). By calling
    refreshObject:mergeChanges: I am forcing that object to be faulted and
    re-fetched with the correct data in the store.

    It seems that sometimes the call to
    mergeChangesFromContextDidSaveNotification: causes an error that shows
    up in the output as "statement is still active". It appears that
    object controllers that are bound to a managed object context receive
    a notification "_NSObjectsChangedInManagingContextPrivateNotification"
    and they do some fetching and the error occurs when they do a fetch
    request.

    I ran with the Core Data threading assertions in the _debug version of
    the library and nothing special is reported.

    Has anyone encountered this problem before and found a solution?

    --
    Michael
  • Michael,

    This error is almost always a multi-threading problem.  When you used
    the _debug version of Core Data, did you enable the threading
    assertions with the user default -com.apple.CoreData.ThreadingDebug
    3 ?  What version of OSX are you using ?  Is there a reason you're not
    using Core Data's -mergeChangesFromContextDidSaveNotification: ?

    If the object is modified (inserted, update, or deleted), you should
    pass in YES to -refreshObject:mergeChanges:

    > It seems that sometimes the call to
    > mergeChangesFromContextDidSaveNotification: causes an error that shows
    > up in the output as "statement is still active". It appears that
    > object controllers that are bound to a managed object context receive
    > a notification "_NSObjectsChangedInManagingContextPrivateNotification"
    > and they do some fetching and the error occurs when they do a fetch
    > request.

    This implies you're modifying a MOC on a background thread that's
    bound into Cocoa Bindings.  Cocoa Bindings and much of AppKit expect
    to effectively own their objects on the main thread.  Whichever MOC
    the object controllers are observing is probably being misused.

    - Ben
  • On Jun 3, 2008, at 5:52 PM, Ben Trumbull wrote:

    > Michael,
    >
    > This error is almost always a multi-threading problem.  When you
    > used the _debug version of Core Data, did you enable the threading
    > assertions with the user default -com.apple.CoreData.ThreadingDebug
    > 3 ?

    Yes, I also see this in the console:

    2008-06-03 18:26:08.824 Bug5912058[5770:10b] CoreData Multithreading
    assertions enabled.

    > What version of OSX are you using ?

    I'm using 10.5.3 (9D34), installed
    apple_debug_and_profile_libraries_for_mac_os_x_10.5.3_9d34.mpkg.zip
    yesterday.

    > Is there a reason you're not using Core Data's -
    > mergeChangesFromContextDidSaveNotification: ?
    >
    >
    > If the object is modified (inserted, update, or deleted), you should
    > pass in YES to -refreshObject:mergeChanges:

    Actually I am using mergeChangesFromContextDidSaveNotification:, I am
    calling refreshObject:mergeChanges: with NO on the NSUpdatedObjectsKey
    objects just before as this avoids the problem of Bug 5937572.

    Originally I was just using the managed object context's
    mergeChangesFromContextDidSaveNotification: but I noticed that the
    merges weren't reflecting what had been saved in regards to some
    objects relationships. For example:

    1. Perform some insertions and changes on a background thread. One of
    these changes is to add the inserted objects to one of the objects
    relationships (which is a many-to-many reflexive relationship). Let's
    say ObjectB which exists in the background thread's MOC now has 20
    friends (10 of the friends are newly inserted and 10 are existing
    objects).
    2. The background thread's MOC saves, and is registered to receive
    it's own MOC did save notification which it forwards to the main
    thread using performSelectorOnMainThread:
    3. The main thread's MOC has no changes and is fed the save
    notification by calling mergeChangesFromContextDidSaveNotification:
    and passing the notification.

    At this point we can now look at ObjectA (which is the same as ObjectB
    except it is owned by the main thread's MOC) which will show that it
    has 10 friends. It should have 20 but the merge has deleted the 10
    friends that were just inserted on the background thread, but just for
    ObjectA. Looking at those 10 objects that were deleted on the main
    thread you will find that they are still friends with ObjectA. This
    leaves the object graph in an inconsistent state.

    >
    >
    >> It seems that sometimes the call to
    >> mergeChangesFromContextDidSaveNotification: causes an error that
    >> shows
    >> up in the output as "statement is still active". It appears that
    >> object controllers that are bound to a managed object context receive
    >> a notification
    >> "_NSObjectsChangedInManagingContextPrivateNotification"
    >> and they do some fetching and the error occurs when they do a fetch
    >> request.
    >
    > This implies you're modifying a MOC on a background thread that's
    > bound into Cocoa Bindings.  Cocoa Bindings and much of AppKit expect
    > to effectively own their objects on the main thread.  Whichever MOC
    > the object controllers are observing is probably being misused.
    >

    The controllers are bound to the MOC on the main thread. The
    "statement is still active" exception occurs in the call to
    mergeChangesFromContextDidSaveNotification: on the main thread's MOC.

    I have an Xcode project that can show both bug 5937572, where the
    object graph is left in an inconsistent state after a merge and bug
    5982319, where the "statement is still active" exception occurs during
    a merge. The project is already attached to bug 5982319 (Bug-1.zip)
    and is setup to show the behavior regarding that issue. To see the
    other issue comment out lines 154-189 and uncomment lines 193-237 in
    MyDocument.m

    I also have the archive project here: http://www.dazys.com/Bug-1.zip

    --
    Michael
  • On Jun 3, 2008, at 8:15 PM, Michael Link wrote:

    >> Is there a reason you're not using Core Data's -
    >> mergeChangesFromContextDidSaveNotification: ?
    >>
    >> If the object is modified (inserted, update, or deleted), you
    >> should pass in YES to -refreshObject:mergeChanges:
    >
    > Actually I am using mergeChangesFromContextDidSaveNotification:, I
    > am calling refreshObject:mergeChanges: with NO on the
    > NSUpdatedObjectsKey objects just before as this avoids the problem
    > of Bug 5937572.

    You shouldn't call -refreshObject:mergeChanges:NO on an object with
    changes (passing YES is okay).  The reason for this is if the object
    is deleted or inserted, nuking that state is just plain weird.  The
    graph of objects will not behave they way you would wish.  A nuked
    inserted object is stuck in limbo, and a nuked deleted object can be a
    stale pointer for any of the objects related to it across inverse
    relationships.  Nuking an updated object also has a high probability
    of corrupting the relationship state for other objects with
    relationships to it.

    Since the objects on the main thread aren't supposed to have changes,
    skipping the changed ones shouldn't be a burden.

    > At this point we can now look at ObjectA (which is the same as
    > ObjectB except it is owned by the main thread's MOC) which will show
    > that it has 10 friends. It should have 20 but the merge has deleted
    > the 10 friends that were just inserted on the background thread, but
    > just for ObjectA. Looking at those 10 objects that were deleted on
    > the main thread you will find that they are still friends with
    > ObjectA. This leaves the object graph in an inconsistent state.

    The object on the main thread seems to think it has local changes to
    the "friends" relationship and during the merge this is trumping the
    changes you've just made in the other context.  I'm still
    investigating the issue with the new example.

    > I have an Xcode project that can show both bug 5937572, where the
    > object graph is left in an inconsistent state after a merge and bug
    > 5982319, where the "statement is still active" exception occurs
    > during a merge. The project is already attached to bug 5982319
    > (Bug-1.zip) and is setup to show the behavior regarding that issue.
    > To see the other issue comment out lines 154-189 and uncomment lines
    > 193-237 in MyDocument.m

    Ah.  I see.  The array controller is posting a KVO notification in the
    middle of the fetch operation.  You can work around the problem by
    disabling "Use Lazy Fetching" on the array controller.

    - Ben
  • On Jun 5, 2008, at 4:37 PM, Ben Trumbull wrote:

    >
    > On Jun 3, 2008, at 8:15 PM, Michael Link wrote:
    >
    >>> Is there a reason you're not using Core Data's -
    >>> mergeChangesFromContextDidSaveNotification: ?
    >>>
    >>> If the object is modified (inserted, update, or deleted), you
    >>> should pass in YES to -refreshObject:mergeChanges:
    >>
    >> Actually I am using mergeChangesFromContextDidSaveNotification:, I
    >> am calling refreshObject:mergeChanges: with NO on the
    >> NSUpdatedObjectsKey objects just before as this avoids the problem
    >> of Bug 5937572.
    >
    > You shouldn't call -refreshObject:mergeChanges:NO on an object with
    > changes (passing YES is okay).  The reason for this is if the object
    > is deleted or inserted, nuking that state is just plain weird.  The
    > graph of objects will not behave they way you would wish.  A nuked
    > inserted object is stuck in limbo, and a nuked deleted object can be
    > a stale pointer for any of the objects related to it across inverse
    > relationships.  Nuking an updated object also has a high probability
    > of corrupting the relationship state for other objects with
    > relationships to it.
    >
    > Since the objects on the main thread aren't supposed to have
    > changes, skipping the changed ones shouldn't be a burden.

    I should skip using -refreshObject:mergeChanges:NO on any object that
    is in NSUpdatedObjectsKey? Even though they don't have changes on the
    main thread? I suppose that makes sense, the original reason for doing
    this was to find a work-around for bug-5937572, which seems to work in
    experimentation (although now without lazy controllers). I'm probably
    just coding myself in a corner I don't want to be in anyways by doing
    this. Ideally I would like to simply use
    mergeChangesFromContextDidSaveNotification: although the merge seems
    to produce the previously mentioned inconsistencies. I even tried
    using -refreshObject:mergeChanges:YES, objectWithID:, and
    deleteObject: to emulate what
    mergeChangesFromContextDidSaveNotification: probably does on the
    userInfo of the notification, but it produced the exact same results
    to no surprise.

    >
    >
    >> At this point we can now look at ObjectA (which is the same as
    >> ObjectB except it is owned by the main thread's MOC) which will
    >> show that it has 10 friends. It should have 20 but the merge has
    >> deleted the 10 friends that were just inserted on the background
    >> thread, but just for ObjectA. Looking at those 10 objects that were
    >> deleted on the main thread you will find that they are still
    >> friends with ObjectA. This leaves the object graph in an
    >> inconsistent state.
    >
    > The object on the main thread seems to think it has local changes to
    > the "friends" relationship and during the merge this is trumping the
    > changes you've just made in the other context.  I'm still
    > investigating the issue with the new example.

    That makes sense. Although I wasn't able to see any changes with the
    public methods available to NSManagedObject, I'll assume that's
    happening somewhere private.

    I attached the Bug-2 project to bug-5989987 in Bug Reporter which may
    help in the investigation. Also here: http://www.dazys.com/Bug-2.zip

    --
    Michael
  • On Jun 5, 2008, at 7:49 PM, Michael Link wrote:

    >
    > On Jun 5, 2008, at 4:37 PM, Ben Trumbull wrote:
    >
    >>
    >> On Jun 3, 2008, at 8:15 PM, Michael Link wrote:
    >>
    >>>> Is there a reason you're not using Core Data's -
    >>>> mergeChangesFromContextDidSaveNotification: ?
    >>>>
    >>>> If the object is modified (inserted, update, or deleted), you
    >>>> should pass in YES to -refreshObject:mergeChanges:
    >>>
    >>> Actually I am using mergeChangesFromContextDidSaveNotification:, I
    >>> am calling refreshObject:mergeChanges: with NO on the
    >>> NSUpdatedObjectsKey objects just before as this avoids the problem
    >>> of Bug 5937572.
    >>
    >> You shouldn't call -refreshObject:mergeChanges:NO on an object with
    >> changes (passing YES is okay).  The reason for this is if the
    >> object is deleted or inserted, nuking that state is just plain
    >> weird.  The graph of objects will not behave they way you would
    >> wish.  A nuked inserted object is stuck in limbo, and a nuked
    >> deleted object can be a stale pointer for any of the objects
    >> related to it across inverse relationships.  Nuking an updated
    >> object also has a high probability of corrupting the relationship
    >> state for other objects with relationships to it.
    >>
    >> Since the objects on the main thread aren't supposed to have
    >> changes, skipping the changed ones shouldn't be a burden.
    >
    > I should skip using -refreshObject:mergeChanges:NO on any object
    > that is in NSUpdatedObjectsKey?

    No, by "changed" I mean returns YES to any of: -isDeleted, -
    isInserted, -isUpdated

    In your mergeChangesFromContextDidSaveNotification callback, you save
    the document.  If you do that first, and then call -
    refreshObject:mergeChanges:NO on all the objects in the notification
    that don't have changes, and then call the MOC's
    mergeChangesFromContextDidSaveNotification (since this will send
    notifications for any observing controllers), it appears to work
    around your first problem.

    Obviously, that's way more convoluted than it ought to be.

    - Ben
previous month june 2008 next month
MTWTFSS
            1
2 3 4 5 6 7 8
9 10 11 12 13 14 15
16 17 18 19 20 21 22
23 24 25 26 27 28 29
30            
Go to today