Obtain directory size.

  • Hello,
    Since NSFileManager has no capabilities of returning the size of a
    folder(a folder + contents), what would be a suitable & fast method
    for obtaining this besides enumerating as it's slow as christmas. I
    am looking for perhaps a carbon call to get this in one swoop.

    Thanks in advance,
    ~J
  • -----BEGIN PGP SIGNED MESSAGE-----
    Hash: SHA1

    man fts

    Very fast dir enum. I don't think there is a single call unless you
    fork du and parse it's results.

    HTH.

    On May 20, 2005, at 11:09 AM, Julian wrote:

    > Hello,
    > Since NSFileManager has no capabilities of returning the size of a
    > folder(a folder + contents), what would be a suitable & fast method
    > for obtaining this besides enumerating as it's slow as christmas. I
    > am looking for perhaps a carbon call to get this in one swoop.
    >
    > Thanks in advance,
    > ~J

    Brian Bergstrand
    <http://www.bergstrand.org/brian/>  PGP Key ID: 0xB6C7B6A2

    -----BEGIN PGP SIGNATURE-----
    Version: PGP Desktop 9.0.1 (Build 2130)

    iQA/AwUBQo4TaXnR2Fu2x7aiEQLB1gCgwuXveDhLMScV84pOTcgmhYhvi4YAmgKk
    D1vNOGMSDNUzvDqnbc3FP3bx
    =tjJS
    -----END PGP SIGNATURE-----
  • On May 20, 2005, at 9:09 AM, Julian wrote:

    > Hello,
    > Since NSFileManager has no capabilities of returning the size of a
    > folder(a folder + contents), what would be a suitable & fast method
    > for obtaining this besides enumerating as it's slow as christmas. I
    > am looking for perhaps a carbon call to get this in one swoop.

    I don't think there's anything that does it all for you.  I use
    Carbon to iterate, which is must faster than using NSEnumerator, but
    it still may not be the best way.

    Note my comment about the resource forks.  The application I'm using
    this in doesn't grab the resource forks so it doesn't care about
    their size.

    - (unsigned long long) fastFolderSizeAtFSRef:(FSRef*)theFileRef
    {
        FSIterator    thisDirEnum = NULL;
        unsigned long long totalSize = 0;

        // Iterate the directory contents, recursing as necessary
        if (FSOpenIterator(theFileRef, kFSIterateFlat, &thisDirEnum) ==
    noErr)
        {
            const ItemCount kMaxEntriesPerFetch = 256;
            ItemCount actualFetched;
            FSRef    fetchedRefs[kMaxEntriesPerFetch];
            FSCatalogInfo fetchedInfos[kMaxEntriesPerFetch];

            // DCJ Note right now this is only fetching data fork
    sizes... if we decide to include
            // resource forks we will have to add kFSCatInfoRsrcSizes

            OSErr fsErr = FSGetCatalogInfoBulk(thisDirEnum,
    kMaxEntriesPerFetch, &actualFetched,
                                            NULL, kFSCatInfoDataSizes |
    kFSCatInfoNodeFlags, fetchedInfos,
                                            fetchedRefs, NULL, NULL);
            while ((fsErr == noErr) || (fsErr == errFSNoMoreItems))
            {
                ItemCount thisIndex;
                for (thisIndex = 0; thisIndex < actualFetched; thisIndex++)
                {
                    // Recurse if it's a folder
                    if (fetchedInfos[thisIndex].nodeFlags &
    kFSNodeIsDirectoryMask)
                    {
                        totalSize += [self
    fastFolderSizeAtFSRef:&fetchedRefs[thisIndex]];
                    }
                    else
                    {
                        // add the size for this item
                        totalSize += fetchedInfos
    [thisIndex].dataLogicalSize;
                    }
                }

                if (fsErr == errFSNoMoreItems)
                {
                    break;
                }
                else
                {
                    // get more items
                    fsErr = FSGetCatalogInfoBulk(thisDirEnum,
    kMaxEntriesPerFetch, &actualFetched,
                                            NULL, kFSCatInfoDataSizes |
    kFSCatInfoNodeFlags, fetchedInfos,
                                            fetchedRefs, NULL, NULL);
                }
            }
            FSCloseIterator(thisDirEnum);
        }
        return totalSize;
    }
  • On May 20, 2005, at 12:09 PM, Julian wrote:

    > Since NSFileManager has no capabilities of returning the size of a
    > folder(a folder + contents), what would be a suitable & fast method
    > for obtaining this besides enumerating as it's slow as christmas. I
    > am looking for perhaps a carbon call to get this in one swoop.

    For a directory, call FSGetCatalogInfo, and ask for the valence.

    Heed the warning about the cost of computing this for volume formats
    which don't natively store this information.

    <http://developer.apple.com/documentation/Carbon/Reference/
    File_Manager/file_manager/chapter_1.5_section_29.html
    >

    Jim
  • On May 20, 2005, at 9:50 AM, Jim Correia wrote:

    > On May 20, 2005, at 12:09 PM, Julian wrote:
    >
    >
    >> Since NSFileManager has no capabilities of returning the size of a
    >> folder(a folder + contents), what would be a suitable & fast
    >> method for obtaining this besides enumerating as it's slow as
    >> christmas. I am looking for perhaps a carbon call to get this in
    >> one swoop.
    >>
    >
    > For a directory, call FSGetCatalogInfo, and ask for the valence.
    >
    > Heed the warning about the cost of computing this for volume
    > formats which don't natively store this information.
    >
    > <http://developer.apple.com/documentation/Carbon/Reference/
    > File_Manager/file_manager/chapter_1.5_section_29.html>

    The valence is just the *number* of items contained in the directory.
    I believe the original poster was asking for total size in terms of
    bytes on disk contained in the folder.

    NSFileManager & NSDirectoryEnumerator use FSIterators (indirectly) to
    do their work, but we don't (yet) allow only specific attributes to
    be returned from the -directoryAttributes or -fileAttributes calls.
    There's an enhancement request for that already, but I believe if you
    want this information you're going to have to walk the file structure
    somehow.

    .chris

    --
    Chris Parker
    Cocoa Frameworks
    Apple Computer, Inc.
  • On May 20, 2005, at 12:50 PM, Jim Correia wrote:

    > On May 20, 2005, at 12:09 PM, Julian wrote:
    >
    >
    >> Since NSFileManager has no capabilities of returning the size of a
    >> folder(a folder + contents), what would be a suitable & fast
    >> method for obtaining this besides enumerating as it's slow as
    >> christmas. I am looking for perhaps a carbon call to get this in
    >> one swoop.
    >>
    >
    > For a directory, call FSGetCatalogInfo, and ask for the valence.
    >
    > Heed the warning about the cost of computing this for volume
    > formats which don't natively store this information.
    >
    > <http://developer.apple.com/documentation/Carbon/Reference/
    > File_Manager/file_manager/chapter_1.5_section_29.html>

    I think I may have mis-interpreted your question. I thought you
    wanted to know how many files+folders were contained in a folder. If
    you want to know how much space (logical or physical) they take on
    disk, iterating (using on API set or another) is your only option.

    Jim
  • Julian wrote on Friday, May 20, 2005:
    > Since NSFileManager has no capabilities of returning the size of a
    > folder(a folder + contents), what would be a suitable & fast method
    > for obtaining this besides enumerating as it's slow as christmas.

    There isn't one.

    The OS doesn't track this information, which is why there is no API to get it "in one swoop."

    The quickest way of doing this, as far as I know, is to use FSGetCatalogInfoBulk to recursively fetch all of the data and resource fork lengths for every file in the subdirectory.  Set the catalog info flags so the file lengths are the only information returned (since you don't care about anything else, there's no point in having FSGetCatalogInfoBulk fetch/calculate any other file info values).

    --
    James Bucanek <mailto:<privatereply...>
  • Yes, what I was looking for was total size in bytes of a folder + all
    contents. Using Daniels, fastFolderSizeAtFSRef now makes me realize
    just how slow NSDirectoryEnumerator really is. :-)

    Thanks

    On May 20, 2005, at 1:00 PM, Chris Parker wrote:

    >
    > On May 20, 2005, at 9:50 AM, Jim Correia wrote:
    >
    >
    >> On May 20, 2005, at 12:09 PM, Julian wrote:
    >>
    >>
    >>
    >>> Since NSFileManager has no capabilities of returning the size of
    >>> a folder(a folder + contents), what would be a suitable & fast
    >>> method for obtaining this besides enumerating as it's slow as
    >>> christmas. I am looking for perhaps a carbon call to get this in
    >>> one swoop.
    >>>
    >>>
    >>
    >> For a directory, call FSGetCatalogInfo, and ask for the valence.
    >>
    >> Heed the warning about the cost of computing this for volume
    >> formats which don't natively store this information.
    >>
    >> <http://developer.apple.com/documentation/Carbon/Reference/
    >> File_Manager/file_manager/chapter_1.5_section_29.html>
    >>
    >
    > The valence is just the *number* of items contained in the
    > directory. I believe the original poster was asking for total size
    > in terms of bytes on disk contained in the folder.
    >
    > NSFileManager & NSDirectoryEnumerator use FSIterators (indirectly)
    > to do their work, but we don't (yet) allow only specific attributes
    > to be returned from the -directoryAttributes or -fileAttributes
    > calls. There's an enhancement request for that already, but I
    > believe if you want this information you're going to have to walk
    > the file structure somehow.
    >
    > .chris
    >
    > --
    > Chris Parker
    > Cocoa Frameworks
    > Apple Computer, Inc.
    >
    >
  • Looking more carefully at my own code, I would probably suggest
    changing this so that the allocations for the fetchedRefs and
    fetchedInfos are made outside of the recursion.

    I'm now afraid that this could easily exhaust the stack if a folder
    is sufficiently deep.  FSRefs and FSCatalogInfos aren't small!

    Daniel

    On May 20, 2005, at 9:42 AM, Daniel Jalkut wrote:

    >
    > const ItemCount kMaxEntriesPerFetch = 256;
    > ItemCount actualFetched;
    > FSRef    fetchedRefs[kMaxEntriesPerFetch];
    > FSCatalogInfo fetchedInfos[kMaxEntriesPerFetch];
    >
  • On 20 May 2005, at 17:09, Julian wrote:

    > Hello,
    > Since NSFileManager has no capabilities of returning the size of a
    > folder(a folder + contents), what would be a suitable & fast method
    > for obtaining this besides enumerating as it's slow as christmas. I
    > am looking for perhaps a carbon call to get this in one swoop.
    >
    > Thanks in advance,
    > ~J
    > _______________________________________________
    > Do not post admin requests to the list. They will be ignored.
    > Cocoa-dev mailing list      (<Cocoa-dev...>)
    > Help/Unsubscribe/Update your Subscription:
    > http://lists.apple.com/mailman/options/cocoa-dev/xcode.dev%
    > 40ntlworld.com
    >
    > This email sent to <xcode.dev...>
    >

    Check out the MoreFiles and MoreFilesX example code that you can find
    on the Apple web site.  There are many useful filemanager routines
    found in it.

    Cheers,

    Jason.
  • Usually not a problem. Although I would suggest using a number closer
    to the page size. I believe that is about 40 items or so per
    iteration. Some version of OS X (10.1? 10.2 and lower?) has a bug
    where anything over 42 or something would cause horribly bad results.
    I can't remember the exact number.

    FSRefs are 80bytes according to Files.h

    But I highly recommend calculating it based on the page size, or
    using 40 or so. It's also much faster. Yeah, I sadly did a bunch of
    Calculations on the speed of the different enumeration calls a long
    time back.

    Ack, at 5/20/05, Daniel Jalkut said:

    > Looking more carefully at my own code, I would probably suggest
    > changing this so that the allocations for the fetchedRefs and
    > fetchedInfos are made outside of the recursion.
    >
    > I'm now afraid that this could easily exhaust the stack if a folder
    > is sufficiently deep.  FSRefs and FSCatalogInfos aren't small!
    >
    > Daniel
    >
    > On May 20, 2005, at 9:42 AM, Daniel Jalkut wrote:
    >
    >>
    >> const ItemCount kMaxEntriesPerFetch = 256;
    >> ItemCount actualFetched;
    >> FSRef    fetchedRefs[kMaxEntriesPerFetch];
    >> FSCatalogInfo fetchedInfos[kMaxEntriesPerFetch];
    >>

    --

    Sincerely,
    Rosyna Keller
    Technical Support/Holy Knight/Always needs a hug

    Unsanity: Unsane Tools for Insanely Great People
  • On May 20, 2005, at 10:57 AM, Daniel Jalkut wrote:

    > On May 20, 2005, at 9:42 AM, Daniel Jalkut wrote:
    >
    >>
    >>   const ItemCount kMaxEntriesPerFetch = 256;
    >>   ItemCount actualFetched;
    >>   FSRef fetchedRefs[kMaxEntriesPerFetch];
    >>   FSCatalogInfo fetchedInfos[kMaxEntriesPerFetch];
    >
    > Looking more carefully at my own code, I would probably suggest
    > changing this so that the allocations for the fetchedRefs and
    > fetchedInfos are made outside of the recursion. 
    >
    > I'm now afraid that this could easily exhaust the stack if a folder is
    > sufficiently deep.  FSRefs and FSCatalogInfos aren't small!
    >
    > Daniel

    You actually do need storage for the FSRef and FSCatalogInfo at each
    recursion level, otherwise you're stomping on the [shared] storage.
    Probably better to malloc/free the storage in each call to
    fastFolderSizeAtFSRef instead, something like this:

    - (unsigned long long) fastFolderSizeAtFSRef:(FSRef*)theFileRef
    {
        unsigned long long totalSize = 0;
        FSIterator    thisDirEnum;

        if (FSOpenIterator(theFileRef, kFSIterateFlat, &thisDirEnum) ==
    noErr)
        {
            const ItemCount kMaxEntriesPerFetch = 256;

            struct SFetchedInfo
            {
                FSRef          fFSRefs[kMaxEntriesPerFetch];
                FSCatalogInfo  fInfos[kMaxEntriesPerFetch];
            };

            // might as well allocate all the storage with a single call...
            SFetchedInfo*    fetched = (SFetchedInfo*)
    malloc(sizeof(SFetchedInfo));

            if (fetched != NULL)
            {
                ItemCount  actualFetched;
                OSErr      fsErr = FSGetCatalogInfoBulk(thisDirEnum,
    kMaxEntriesPerFetch, &actualFetched,
                                        NULL, kFSCatInfoDataSizes |
    kFSCatInfoNodeFlags, fetched->fInfos,
                                        fetched->fFSRefs, NULL, NULL);
                [...]

                free(fetched);
            }

            FSCloseIterator(thisDirEnum);
        }

        return totalSize;
    }
previous month may 2005 next month
MTWTFSS
            1
2 3 4 5 6 7 8
9 10 11 12 13 14 15
16 17 18 19 20 21 22
23 24 25 26 27 28 29
30 31          
Go to today