How does Finder determine when a file is busy being written to the disk?

  • When copying large files (over a GB each) from a network drive to a
    local drive Finder shows these files are busy by graying their
    filename out until they are finished writing to the local system.

    How can I detect this as well?

    I tried calling the NSFileManager fileAttributesAtPath:traverseLink:
    method to get the NSFileBusy attribute, but this attribute is null.
  • On Apr 7, 2010, at 4:31 PM, Lee Gillen wrote:

    > When copying large files (over a GB each) from a network drive to a
    > local drive Finder shows these files are busy by graying their
    > filename out until they are finished writing to the local system.

    I think the Finder only knows that because it's the one doing the
    copying. If something else is copying a file (like the 'cp' command)
    the Finder doesn't gray out the file.

    Unix doesn't really have a notion of a file being "busy", except via
    higher level mechanisms like exclusive locks. In general, I think it's
    difficult to figure out if some other process has opened a file. The
    'lsof' tool does it, but I think it uses privileged system calls.

    —Jens
  • On Wed, 7 Apr 2010 19:31:19 -0400, Lee Gillen said:

    > When copying large files (over a GB each) from a network drive to a
    > local drive Finder shows these files are busy by graying their
    > filename out until they are finished writing to the local system.

    Not sure if it still applies these days, but look through Finder.h, and
    read about kFirstMagicBusyFiletype and kMagicBusyCreationDate.

    --
    ____________________________________________________________
    Sean McBride, B. Eng                <sean...>
    Rogue Research                        www.rogue-research.com
    Mac Software Developer              Montréal, Québec, Canada
  • On Wed, Apr 7, 2010 at 7:53 PM, Jens Alfke <jens...> wrote:

    > I think the Finder only knows that because it's the one doing the copying.
    > If something else is copying a file (like the 'cp' command) the Finder
    > doesn't gray out the file.

    But I think what is happening is there is an FSEvent when the
    placeholder file is created at the destination. This file has a file
    size of 0 bytes. When the file is finished being written the file is
    swapped out with the placeholder file. This raises a 2nd FSEvent.
    Reading the file size on the second event shows the full file size.

    It looks to me like 'cp' doesn't show the file at all until it is
    finished writing.
  • That may be true, but it's not directly related to the Finder graying out the source file. As I said, it grays out the file when it itself starts copying it. It has no way of telling when some other process is doing the copy.

    --Jens {via iPad}

    On Apr 7, 2010, at 5:16 PM, Lee Gillen <lee.gillen...> wrote:

    > On Wed, Apr 7, 2010 at 7:53 PM, Jens Alfke <jens...> wrote:
    >
    >> I think the Finder only knows that because it's the one doing the copying.
    >> If something else is copying a file (like the 'cp' command) the Finder
    >> doesn't gray out the file.
    >
    > But I think what is happening is there is an FSEvent when the
    > placeholder file is created at the destination. This file has a file
    > size of 0 bytes. When the file is finished being written the file is
    > swapped out with the placeholder file. This raises a 2nd FSEvent.
    > Reading the file size on the second event shows the full file size.
    >
    > It looks to me like 'cp' doesn't show the file at all until it is
    > finished writing.
  • FYI, kMagicBusyCreationDate is 8:34:56am on 14th Feb 1946 (GMT).

    When copying an entire folder, it only sets the top-level folder's creation time to this, not all the sub-items.

    I suspect, but haven't tested, that anything going through FSCopyObjectXXXX will exhibit the same behaviour.

    Matt

    On 8 Apr 2010, at 01:13:24, Sean McBride wrote:

    > On Wed, 7 Apr 2010 19:31:19 -0400, Lee Gillen said:
    >
    >> When copying large files (over a GB each) from a network drive to a
    >> local drive Finder shows these files are busy by graying their
    >> filename out until they are finished writing to the local system.
    >
    > Not sure if it still applies these days, but look through Finder.h, and
    > read about kFirstMagicBusyFiletype and kMagicBusyCreationDate.
    >
    > --
  • On Wed, Apr 7, 2010 at 11:25 PM, Jens Alfke <jens...> wrote:
    > That may be true, but it's not directly related to the Finder graying out the source file. As I said, it grays out the file when it itself starts copying it. It has no way of telling when some other process is doing the copy.

    Unfortunately my theory is not true. Part of the file is written while
    Finder is copying the file and there is no swap.

    Is there anyway for my app to see that the file is being written to? I
    was thinking of looking at 'fuser' or trying to open the file in write
    mode as a test.

    I haven't tried looking at kMagicBusyCreationDate, is that recommended?
  • On Apr 8, 2010, at 10:53 AM, Lee Gillen wrote:

    > Is there anyway for my app to see that the file is being written to? I
    > was thinking of looking at 'fuser' or trying to open the file in write
    > mode as a test.

    I don't know what 'fuser' is. Opening the file in write mode will
    always succeed; Unix doesn't have the concept of a 'busy' file. If the
    Finder uses an exclusive lock on the file while it's being written to,
    then you could attempt to open the file with an exclusive lock, and it
    would fail if the Finder already had the file locked. (This is all
    standard Unix file stuff that you could learn more about from books or
    online references...)

    Maybe you should describe in detail what you're trying to do? There
    might be a better way to approach the problem.

    —Jens
  • On Thu, Apr 8, 2010 at 1:58 PM, Jens Alfke <jens...> wrote:

    > Maybe you should describe in detail what you're trying to do? There might be
    > a better way to approach the problem.

    My app watches a folder using FSEvents to see when files are added to
    it and then uploads those files to a server. It's kind of like a hot
    folder. The issue is that when users are copying large files,
    especially from a network drive, the upload will begin before the file
    has been completely written to the disk.

    'fuser' is a command line utility that tell you what app PID has the
    file open. I see that if I pass 'fuser' the file path I see that
    Finder has the file open.
  • Maybe poll the file size when you get your event and wait for it
    to stop changing.

    Paul Sanders.

    ----- Original Message -----
    From: "Lee Gillen" <lee.gillen...>
    To: "Jens Alfke" <jens...>
    Cc: "Apple Developer Mailing List" <Cocoa-dev...>
    Sent: Thursday, April 08, 2010 7:08 PM
    Subject: Re: How does Finder determine when a file is busy being
    written to the disk?

    My app watches a folder using FSEvents to see when files are
    added to
    it and then uploads those files to a server. It's kind of like a
    hot
    folder. The issue is that when users are copying large files,
    especially from a network drive, the upload will begin before
    the file
    has been completely written to the disk.

    'fuser' is a command line utility that tell you what app PID has
    the
    file open. I see that if I pass 'fuser' the file path I see that
    Finder has the file open.
  • On Apr 8, 2010, at 11:08 AM, Lee Gillen wrote:

    > My app watches a folder using FSEvents to see when files are added to
    > it and then uploads those files to a server. It's kind of like a hot
    > folder. The issue is that when users are copying large files,
    > especially from a network drive, the upload will begin before the file
    > has been completely written to the disk.

    Apple's Folder Actions feature runs into this issue too. I suspect
    they use a heuristic like waiting for the file's mod date and size to
    stop changing for a few seconds, but I'm not sure.

    You should ask on the darwin-userlevel or darwin-dev lists, where the
    real Unix graybeards hang out.

    —Jens
previous month april 2010 next month
MTWTFSS
      1 2 3 4
5 6 7 8 9 10 11
12 13 14 15 16 17 18
19 20 21 22 23 24 25
26 27 28 29 30    
Go to today