NSPersistentDocument Migration with Sandbox

  • Hi,

    I'm working on an update to a Core-Data document app, and have a new version of the document model.  I've got automatic migration (with a mapping model) of documents in the old format working fine when the app is run without sandboxing.  However, when running in the sandbox, migration fails at the end when the migrated data is prevented from being written to disk by the sandbox.  sandboxd gives this message on the console:

    deny file-write-create /Users/jimmcgowan/Desktop/.AGPS Examples.rtd.migrationdestination_41b5a6b5c6e848c462a8480cd24caef3

    And my app gives out:

    CoreData: error: (migration) migration failed with error Error Domain=NSCocoaErrorDomain Code=134110 "An error occured during persistent store migration." UserInfo=0x1006ab1b0 {NSUnderlyingError=0x100693de0 "The file couldn’t be saved.", reason=Can't add destination store}
    Error User Info: {
        NSUnderlyingError = "Error Domain=NSCocoaErrorDomain Code=512 \"The file couldn\U2019t be saved.\" UserInfo=0x100642d80 {NSUnderlyingException=Error validating url for store}";
        reason = "Can't add destination store";
    }
    Underlying error: Error Domain=NSCocoaErrorDomain Code=512 "The file couldn’t be saved." UserInfo=0x100642d80 {NSUnderlyingException=Error validating url for store}
    Error User Info: {
        NSUnderlyingException = "Error validating url for store";
    }

    I've spent a day looking into this and Googling around, but there doesn't seem to be much information out there.  I've found a couple of posts with others encountering this problem, but the solutions are either to use the temporary entitlement for absolute-path.read-write or to hijack the migration process to display a save dialogue to get user permission for the destination URL.  Neither of these are appropriate as the former will most likely mean MAS rejection, and the later creates a confusing user experience (opening a document shouldn't immediately present a save dialogue).

    Has anyone else run into this?  Any suggestions for a better workaround?

    Thanks,

    Jim
  • On 2013 May 08, at 21:20, Jim McGowan <jim_mcgowan...> wrote:

    > sandboxd gives this message on the console:
    >
    > deny file-write-create /Users/jimmcgowan/Desktop/.AGPS Examples.rtd.migrationdestination_41b5a6b5c6e848c462a8480cd24caef3

    I've never worked with a sandboxed document, but the above error says that Core Data is attempting to write to the regular Desktop, which of course is not allowed in the sandbox.  It shouldn't be doing that, unless your document was on the regular Desktop to begin with.

    When Core Data does a migration for me, it makes a copy of the old file in the same directory as the document is located, appending a tilde (~) to the name of the copy.  Presumably, the document in your sandboxed app is located somewhere in your sandbox.

    The question is: "Why is Core Data trying to create a file on the regular Desktop?"
  • On  9 May, 2013, at 1:56:29 PM HKT, Jerry Krinock <jerry...> wrote:
    >
    >> sandboxd gives this message on the console:
    >>
    >> deny file-write-create /Users/jimmcgowan/Desktop/.AGPS Examples.rtd.migrationdestination_41b5a6b5c6e848c462a8480cd24caef3
    >
    > I've never worked with a sandboxed document, but the above error says that Core Data is attempting to write to the regular Desktop, which of course is not allowed in the sandbox.  It shouldn't be doing that, unless your document was on the regular Desktop to begin with.

    The original document was on the desktop.  A user could have their documents anywhere.

    > The question is: "Why is Core Data trying to create a file on the regular Desktop?"
    >

    The document in the old format that this particular test used was located at ~/Desktop/AGPS Examples.rtd  After migration, core data tries to write the store to ~/Desktop/.AGPS Examples.rtd.migrationdestination_41b5a6b5c6e848c462a8480cd24caef3  The docs mention writing to a file with  '~' suffix, but this doesn't happen (unless writing to the path above is an intermediary step, but the docs make no mention of this).

    Core Data document migration will try to write the migrated store to the same directory as the original, but the sandbox blocks this.  So, does anyone know of any workaround?

    Jim
  • On 2013 May 09, at 23:26, Jim McGowan <jim_mcgowan...> wrote:

    > Core Data document migration will try to write the migrated store to the same directory as the original, but the sandbox blocks this.

    I think that it would help to have a higher-level understanding of the problem.  How about this…

    You have a sandboxed app in which the user has somehow opened a document outside your sandbox (because the user navigated to it in an Open dialog?).  I think that if migration was not required, the user could edit the document and click File > Save, and sandboxd would allow your user to re-save the document outside your sandbox, because it was there to begin with (is that correct?)  But Core Data writes temporary files during migrations in the document's directory, which is not allowed.

    This appears to be edge-case oversight in Apple's sandboxing design.  Either Core Data should write its temporary files (and tilde files) to an allowed location within your sandbox, or sandboxd should allow Core Data to do write these files in whatever directory the document currently resides.

    Have I stated this correctly?

    In the meantime, you need a workaround for this bug.  An ugly solution would be to override the migration methods, test if a file can be written in the document's directory before migration begins, and if not, inform the user that the document must be moved into the sandbox so that it can be updated.  This might not be too bad if users rarely open documents outside your sandbox.  Otherwise, well, I would throw this one at Apple.  DTS incidents are pretty cheap nowadays, the problem is clear, and I've already written it up for you :)
  • On May 10, 2013, at 07:12 , Jerry Krinock <jerry...> wrote:

    > I think that it would help to have a higher-level understanding of the problem.  How about this…
    >
    > You have a sandboxed app in which the user has somehow opened a document outside your sandbox (because the user navigated to it in an Open dialog?). […]

    > Have I stated this correctly?

    I don't think so. The *user* doesn't see files as being in or out of a sandbox. After all, the user isn't sandboxed -- your app is.

    By and large, *all* documents are outside an app's sandbox. Use of the Open dialog, or resolving a security-scoped bookmark, grants the app access to a document *as if* if it was inside the sandbox, but the document isn't moved in any way to get this access.

    An app wanting to migrate a document (including, say, a word processor converting an older document format into a newer one, or converting a format it only knows how to read into one it knows how to write) is eventually going to have to create a new file, and a sandboxed app is therefore going to have to ask for permission via the Save dialog. [I'm talking here of a migration that preserves the original file beside the migrated one.]

    In the case of Core Data, this means that a sandboxed app cannot use automatic migration, but must migrate manually.

    On May 8, 2013, at 21:20 , Jim McGowan <jim_mcgowan...> wrote:

    > to hijack the migration process to display a save dialogue to get user permission for the destination URL […] creates a confusing user experience (opening a document shouldn't immediately present a save dialogue)

    Yet, failing to get permission via a save dialog would defeat sandbox security. Malware invading your app would be able to hijack the migration process to create any file it wants, anywhere on the user's system.

    It's not clear that interacting with the user must be confusing. You could start with an alert that explains that the document format needs updating, offering a Save button that leads to the Save dialog. Annoying, perhaps, but not necessarily confusing.

    There are a couple of other approaches that might be preferable to you. The question is how well (or if) they work with NSPersistentDocument:

    1. Use post-Lion "autosave elsewhere" to migrate the Core Data store into a new untitled document in a temporary location. This would defer the Save dialog until the document window closes.

    2. Put your Core Data store inside a file package. Then do the migration within the package. However, you likely wouldn't be able to use NSPersistentDocument any more, since it doesn't sound like it supports packages directly.

    3. Migrate your Core Data store to a temporary file, then move the migrated file back to the original location, replacing the pre-migration file. Presumably, this is the same approach that NSDocument Save takes, but you'd need to implement it yourself, since NSPersistentDocument doesn't (AFAIK) do this. The replacement needs to be safely done (using the file system's "exchange objects" API), otherwise a crash or failure during replacement would effectively destroy the document.
  • On 11 May, 2013 2:07:18 HKT, Quincey Morris <quinceymorris...> wrote:

    > On May 10, 2013, at 07:12 , Jerry Krinock <jerry...> wrote:
    >
    >> I think that it would help to have a higher-level understanding of the problem.  How about this…
    >>
    >> You have a sandboxed app in which the user has somehow opened a document outside your sandbox (because the user navigated to it in an Open dialog?). […]

    Yes, all of a user's existing documents (from the old version on the app, in the old format) would be outside the sandbox.  This is just a regular document based app, so users' documents could be anywhere on any volume.

    > In the case of Core Data, this means that a sandboxed app cannot use automatic migration, but must migrate manually.

    This is what I'm beginning to suspect.

    >
    > It's not clear that interacting with the user must be confusing. You could start with an alert that explains that the document format needs updating, offering a Save button that leads to the Save dialog. Annoying, perhaps, but not necessarily confusing.
    >
    > There are a couple of other approaches that might be preferable to you. The question is how well (or if) they work with NSPersistentDocument:
    >
    > 1. Use post-Lion "autosave elsewhere" to migrate the Core Data store into a new untitled document in a temporary location. This would defer the Save dialog until the document window closes.

    This sounds like a promising approach.  Deferring the save dialogue until close seems like the most intuitive UX.  THis is what Pages does when you open an MS Word file.  The new version of this app uses autosave in place, but it could be possible to change this during the migration...

    >
    > 2. Put your Core Data store inside a file package. Then do the migration within the package. However, you likely wouldn't be able to use NSPersistentDocument any more, since it doesn't sound like it supports packages directly.
    >
    > 3. Migrate your Core Data store to a temporary file, then move the migrated file back to the original location, replacing the pre-migration file. Presumably, this is the same approach that NSDocument Save takes, but you'd need to implement it yourself, since NSPersistentDocument doesn't (AFAIK) do this. The replacement needs to be safely done (using the file system's "exchange objects" API), otherwise a crash or failure during replacement would effectively destroy the document.

    Thanks,
    Jim
  • For the sake of the archives, here is how I solved this issue:

    First off, I turned off automatic migration in my NSPersistentDocument subclass

    I overrode -openDocumentWithContentsOfURL:display:completionHandler: in a subclass on NSDocumentController.  In this method I check the UTI of the URL parameter and if it is a regular 'new' format document, go ahead and call super's implementation.

    If the UTI is for the older document format, I begin migration to a temporary file inside the sandboxed temporary directory (from NSTemporaryDirectory()).  I then open the migrated temporary doc without showing the UI (with -openDocumentWithContentsOfURL:tempURL display:NO completionHandler:…) and open a new blank doc with -openUntitledDocumentAndDisplay:error:    Making use of the fact that my model classes already implement NSPasteboardWriting, I copy of the the contents of the temp doc into the new doc.

    Then I show an alert in a sheet attached to the new doc window that explains to the user that the contents of the old doc have been upgraded, but the original file has not been altered.  The sheet has a 'don't show this again' switch.  The user can then save at their convenience.

    Jim

    On 11 May, 2013, at 10:40 AM, Jim McGowan <jim_mcgowan...> wrote:

    > On 11 May, 2013 2:07:18 HKT, Quincey Morris <quinceymorris...> wrote:
    >
    >> On May 10, 2013, at 07:12 , Jerry Krinock <jerry...> wrote:
    >>
    >>> I think that it would help to have a higher-level understanding of the problem.  How about this…
    >>>
    >>> You have a sandboxed app in which the user has somehow opened a document outside your sandbox (because the user navigated to it in an Open dialog?). […]
    >
    > Yes, all of a user's existing documents (from the old version on the app, in the old format) would be outside the sandbox.  This is just a regular document based app, so users' documents could be anywhere on any volume.
    >
    >> In the case of Core Data, this means that a sandboxed app cannot use automatic migration, but must migrate manually.
    >
    > This is what I'm beginning to suspect.
    >
    >>
    >> It's not clear that interacting with the user must be confusing. You could start with an alert that explains that the document format needs updating, offering a Save button that leads to the Save dialog. Annoying, perhaps, but not necessarily confusing.
    >>
    >> There are a couple of other approaches that might be preferable to you. The question is how well (or if) they work with NSPersistentDocument:
    >>
    >> 1. Use post-Lion "autosave elsewhere" to migrate the Core Data store into a new untitled document in a temporary location. This would defer the Save dialog until the document window closes.
    >
    > This sounds like a promising approach.  Deferring the save dialogue until close seems like the most intuitive UX.  THis is what Pages does when you open an MS Word file.  The new version of this app uses autosave in place, but it could be possible to change this during the migration...
    >
    >>
    >> 2. Put your Core Data store inside a file package. Then do the migration within the package. However, you likely wouldn't be able to use NSPersistentDocument any more, since it doesn't sound like it supports packages directly.
    >>
    >> 3. Migrate your Core Data store to a temporary file, then move the migrated file back to the original location, replacing the pre-migration file. Presumably, this is the same approach that NSDocument Save takes, but you'd need to implement it yourself, since NSPersistentDocument doesn't (AFAIK) do this. The replacement needs to be safely done (using the file system's "exchange objects" API), otherwise a crash or failure during replacement would effectively destroy the document.
    >
    > Thanks,
    > Jim
previous month may 2013 next month
MTWTFSS
    1 2 3 4 5
6 7 8 9 10 11 12
13 14 15 16 17 18 19
20 21 22 23 24 25 26
27 28 29 30 31    
Go to today