temporary files and subsequent cleanup

  • I am creating a temporary file structure in which to store my
    application required files. That Cocoa method looks like this:

    // base:@"some/directory/path/in/file/system"
    // path:@"ScratchFolderTemplate.XXXXXX"
    - (NSString *)setupDirectory:(NSString *)base path:(NSString *)path
    {
    NSString *tempPath = nil;
    if (!base)
    {
      tempPath = NSTemporaryDirectory ();
      [tempPath retain];
    }
    else if (path)
    {
      tempPath = [base stringByAppendingPathComponent:path];
      char pathChars[PATH_MAX + 1];
      pathChars[PATH_MAX] = 0;
      [tempPath getFileSystemRepresentation:pathChars maxLength:(PATH_MAX
    + 1)];
      char *result = (char *) mkdtemp (pathChars);
      if (result == pathChars)
      {
      tempPath = [[NSFileManager defaultManager]
    stringWithFileSystemRepresentation:pathChars length:strlen (pathChars)];
      [tempPath retain];
      }
      else
      tempPath = nil;
    }
    return tempPath;
    }

    And it is functioning as I expect. And I cleanup as follows upon a
    clean exit from my application, which also works great.

    - (void)applicationWillTerminate:(NSNotification *)aNotification
    {
    // these files MUST be eliminated from the world upon program
    termination
    [[NSFileManager defaultManager] removeFileAtPath:tempApplication
    handler:self];
    }

    All of this is based on best, most recently discussed ideas garnered
    from this list.

    My question is this though. If my application fails, which I am
    certain it will never do - hah ;-) - I will have some temp files left
    over. Does Tiger+ have a daemon for temporary file removal that will
    care of these files from NSTemporaryDirectory() rooted temporary
    space here: /private/var/tmp/folders.501/temporaryitems/ and if so,
    when and how often does it run to take care of this? For instance,
    this directory on my machine has leftovers from some Apple
    applications, as follows:

    drwx------    2 chris  wheel    68 Oct 24 10:07
    TEMP_com.apple.iWork.Numbers_19857_214927641_1
    drwx------    3 chris  wheel  102 Oct 23 16:19
    com.apple.iWeb_19217_SFED_214863580_1
    drwx------    22 chris  wheel  748 Oct 23 21:40
    com.apple.iWeb_19317_SFED_214882158_1
    drwx------    12 chris  wheel  408 Oct 24 12:27
    com.apple.iWeb_19730_SFED_214922039_1
    drwx------  173 chris  wheel  5882 Oct 24 16:45
    com.apple.iWeb_21452_SFED_214951447_4
    drwx------    2 chris  wheel    68 Oct 22 22:26 com.apple.mail

    Bottom line question - will these be removed at some point in time by
    Tiger+ daemons?

    Thanks.

    Chris
  • On Oct 26, 2007, at 7:50 AM, Chris Heimark wrote:
    > My question is this though. If my application fails, which I am
    > certain it will never do - hah ;-) - I will have some temp files
    > left over. Does Tiger+ have a daemon for temporary file removal
    > that will care of these files from NSTemporaryDirectory() rooted
    > temporary space here: /private/var/tmp/folders.501/temporaryitems/
    > and if so, when and how often does it run to take care of this?

    There are three scripts

    /private/etc/daily
    /private/etc/weekly
    /private/etc/monthly

    that are intended to run periodically.  This snippet from daily
    cleans up /tmp

    if [ -d /tmp ]; then
        cd /tmp && {
        find -x . -fstype local -type f -atime +3 -ctime +3 -exec rm -f
    -- {} \;
        find -x -d . -fstype local ! -name . -type d -mtime +1 -exec
    rmdir -- {} \;
    \
            > /dev/null 2> &1; }
    fi

    It seems to be waiting three days from the last access time to rm the
    files.

    If the computer is sleeping or turned off, of course, the scripts
    won't run.

    You can view the log files generated by these scripts in the Console
    utility application.
    You can execute the scripts yourself by just typing their names in
    the Terminal if you like.

    --
    Brian Stern
    <brians99...>
  • On 26 Oct 2007, at 15:11, Brian Stern wrote:

    > On Oct 26, 2007, at 7:50 AM, Chris Heimark wrote:
    >> My question is this though. If my application fails, which I am
    >> certain it will never do - hah ;-) - I will have some temp files
    >> left over. Does Tiger+ have a daemon for temporary file removal
    >> that will care of these files from NSTemporaryDirectory() rooted
    >> temporary space here: /private/var/tmp/folders.501/temporaryitems/
    >> and if so, when and how often does it run to take care of this?
    >
    > There are three scripts
    >
    > /private/etc/daily
    > /private/etc/weekly
    > /private/etc/monthly

    [snip]

    > If the computer is sleeping or turned off, of course, the scripts
    > won't run.
    >
    > You can view the log files generated by these scripts in the
    > Console utility application.
    > You can execute the scripts yourself by just typing their names in
    > the Terminal if you like.

    You can also install anacron, which will make the scripts run at the
    first available opportunity subsequent to the time they were supposed
    to.  There's a package for Tiger here:

      <http://members.cox.net/18james/anacron-tiger.html>

    and one for Panther here:

      <http://alastairs-place.net/anacron.html>

    It looked like launchd was going to start doing this itself, so I
    don't know (yet) whether this is necessary for Leopard.

    Kind regards,

    Alastair.

    --
    http://alastairs-place.net
  • On Oct 26, 2007, at 10:20 AM, Alastair Houghton wrote:

    > On 26 Oct 2007, at 15:11, Brian Stern wrote:
    >
    >> On Oct 26, 2007, at 7:50 AM, Chris Heimark wrote:
    >>> My question is this though. If my application fails, which I am
    >>> certain it will never do - hah ;-) - I will have some temp files
    >>> left over. Does Tiger+ have a daemon for temporary file removal
    >>> that will care of these files from NSTemporaryDirectory() rooted
    >>> temporary space here: /private/var/tmp/folders.501/
    >>> temporaryitems/ and if so, when and how often does it run to take
    >>> care of this?
    >>
    >> There are three scripts
    >>
    >> /private/etc/daily
    >> /private/etc/weekly
    >> /private/etc/monthly

    I knew of these jobs. And upon (finally) examining them for the first
    time, I see that they reach into /tmp and other directory structures.

    I tried to trace how /tmp --> private/tmp ultimately points into /
    private/var/tmp/folders.501/temporaryitems/ and I gave up... Maybe
    that is why I stopped programming in Unix years ago...

    But I still wonder why, after running "sudo /private/etc/daily" (and
    later "sudo /private/etc/weekly") I still have files/directories
    older than 3 days present.

    Is it possible that other daemons are (or will be) at work here?
    After all this directory doesn't have any terribly old entries

    >
    >> If the computer is sleeping or turned off, of course, the scripts
    >> won't run.
    >>
    >> You can view the log files generated by these scripts in the
    >> Console utility application.
    >> You can execute the scripts yourself by just typing their names in
    >> the Terminal if you like.
    >
    > It looked like launchd was going to start doing this itself, so I
    > don't know (yet) whether this is necessary for Leopard.

    Isn't it launchd that makes the daily/weekly/monthly jobs run? Or is
    it entries in crontab that lead to their execution?

    >
    > Kind regards,
    >
    > Alastair.
  • On 26 Oct 2007, at 16:06, Chris Heimark wrote:

    > I knew of these jobs. And upon (finally) examining them for the
    > first time, I see that they reach into /tmp and other directory
    > structures.

    Yes.

    > I tried to trace how /tmp --> private/tmp ultimately points into /
    > private/var/tmp/folders.501/temporaryitems/ and I gave up... Maybe
    > that is why I stopped programming in Unix years ago...

    It doesn't.  So you wouldn't have had much luck.  /tmp points at /
    private/tmp.  It's the UNIX-style shared tmp folder, so if you create
    things in there you have to avoid conflicts (and security holes)
    caused by other users' access to that directory.  If you look in /
    private/tmp, I suspect you'll find some numbered folders with
    peoples' UIDs as their names, as that's quite a common way to address
    that problem.

    > But I still wonder why, after running "sudo /private/etc/
    > daily" (and later "sudo /private/etc/weekly") I still have files/
    > directories older than 3 days present.

    I don't think they actually clear out /var/tmp.  At least, on Tiger,
    it looks like /var/tmp is only cleared out when you restart the
    machine (from the /etc/rc script), and even then it only clears out /
    var/tmp/folders.*, so things in /var/tmp will remain indefinitely,
    which explains why I have files dated 2006 in that folder on my Mac
    Pro.  Arguably this is a bug.

    I think the /var/tmp folder, BTW, is the one that you get if you ask
    various Carbon/Cocoa APIs for a temporary folder.

    >>> If the computer is sleeping or turned off, of course, the scripts
    >>> won't run.
    >>>
    >>> You can view the log files generated by these scripts in the
    >>> Console utility application.
    >>> You can execute the scripts yourself by just typing their names
    >>> in the Terminal if you like.
    >>
    >> It looked like launchd was going to start doing this itself, so I
    >> don't know (yet) whether this is necessary for Leopard.
    >
    > Isn't it launchd that makes the daily/weekly/monthly jobs run? Or
    > is it entries in crontab that lead to their execution?

    On Panther and earlier, it was crontab that drove it.  On Tiger and
    later, it is done by launchd.  Unless, of course, you install
    anacron, in which case (at least for the packages I mentioned),
    anacron takes over the periodic job execution.

    I'm hoping, though I haven't checked, that Leopard's launchd is now
    capable of subsuming the functionality of anacron, as that behaviour
    makes the most sense for most desktop and laptop machines.

    Kind regards,

    Alastair.

    --
    http://alastairs-place.net
  • > I don't think they actually clear out /var/tmp.  At least, on
    > Tiger, it looks like /var/tmp is only cleared out when you restart
    > the machine (from the /etc/rc script), and even then it only clears
    > out /var/tmp/folders.*, so things in /var/tmp will remain
    > indefinitely, which explains why I have files dated 2006 in that
    > folder on my Mac Pro.  Arguably this is a bug.
    Looks like you are right. A reboot got me:

    [imacg5:~] chris% ls -al /private/var/tmp/folders.501/temporaryitems/
    ls: /private/var/tmp/folders.501/temporaryitems/: No such file or
    directory

    Few minutes later, I tried again:
    [imacg5:~] chris% ls -al /private/var/tmp/folders.501/temporaryitems/
    total 0
    drwxr-xr-x  2 chris  wheel  68 Oct 26 12:02 .
    drwx------  3 chris  wheel  102 Oct 26 12:02 ..
    [imacg5:~] chris%

    So apparently you never have to reboot ;-)

    >
    > I think the /var/tmp folder, BTW, is the one that you get if you
    > ask various Carbon/Cocoa APIs for a temporary folder.
    If by that, you mean NSTemporaryDirectory(), I get /private/var/tmp/
    folders.501/temporaryitems/. Do you mean lower level BSD calls?

    Thanks for your informed responses!

    Chris
  • If you are only storing files during a single run of your application
    (not between runs), you aren't sharing the files with anything else,
    you have a small number of temporary files AND you can keep them open
    at all times, you can probably use an old UNIX trick to ensure that
    they are always removed, even if your application crashes.  After
    opening the file, unlink it, and don't close the file handle.  The
    file is gone as far as anything else is concerned, but your app can
    still read and write to it.  As soon as the last file handle is
    closed, the file really goes away.

    I haven't tested this on OS X, but it works on the UNIXes I'm
    familiar with.

    --
    Steve Madsen <steve...>
    Light Year Software, LLC  http://lightyearsoftware.com
    ZingLists: Stay organized, and share lists online.  http://zinglists.com
  • On 26 Oct 2007, at 17:09, Chris Heimark wrote:

    >> I think the /var/tmp folder, BTW, is the one that you get if you
    >> ask various Carbon/Cocoa APIs for a temporary folder.
    > If by that, you mean NSTemporaryDirectory(), I get /private/var/tmp/
    > folders.501/temporaryitems/.

    Sorry, I should have been clearer.  The Carbon and Cocoa APIs give
    you a location *inside* /var/tmp (which is the same as /private/var/
    tmp).  I haven't looked to see exactly which APIs give which results
    and I'm not sure where it's actually documented anywhere either, but
    I'd noticed some time ago that that's where Carbon and Cocoa apps
    tended to put temporary files.

    > Do you mean lower level BSD calls?

    No, no.  The BSD/UNIX APIs will return paths in /tmp (which is
    symlinked to /private/tmp).  UNIX programs generally either hard-code
    "/tmp", or use the environment variable TMPDIR.  This kind of thing
    seems horrible to a lot of people who don't have a UNIX background,
    but since the filesystem layout is standardised, it isn't as nasty as
    it first feels.

    The biggest problem with the UNIX approach is that you have to be
    very careful when creating files in the shared /tmp folder to avoid
    creating security holes.  A common way to prevent them is to make a
    folder based on the user's UID and maybe some other information,
    setting the permission bits on the folder carefully, and then you
    create files inside that.  I think the /var/tmp thing that the Carbon
    and Cocoa temporary folder APIs do is simply an implementation of
    that that won't be interfered with by UNIX software (since that won't
    normally be looking in /var/tmp).

    Kind regards,

    Alastair.

    --
    http://alastairs-place.net
  • On 26 Oct 2007, at 17:25, Steve Madsen wrote:

    > If you are only storing files during a single run of your
    > application (not between runs), you aren't sharing the files with
    > anything else, you have a small number of temporary files AND you
    > can keep them open at all times, you can probably use an old UNIX
    > trick to ensure that they are always removed, even if your
    > application crashes.  After opening the file, unlink it, and don't
    > close the file handle.  The file is gone as far as anything else is
    > concerned, but your app can still read and write to it.  As soon as
    > the last file handle is closed, the file really goes away.
    >
    > I haven't tested this on OS X, but it works on the UNIXes I'm
    > familiar with.

    Yes, it works fine on OS X.  The actual implementation in terms of
    the filesystem is a bit freaky because HFS+ was never really designed
    to support those semantics, but it does work.

    All of this is thoroughly off topic, of course, but since most people
    have time to kill until we can talk about Leopard I'm not sure anyone
    will really care :-)

    Kind regards,

    Alastair.

    --
    http://alastairs-place.net
  • I am actually needing to be sure these files are "nuked", so I have
    been using the fileManager:willProcessPath: delegate method as a hook
    to actually "nuke" the files before they are deleted for me by the
    removeFileAtPath: method. I am building a "security" program which
    has a duty is to make sure it leaves no trace of its' own activities.

    > If you are only storing files during a single run of your application
    > (not between runs), you aren't sharing the files with anything else,
    > you have a small number of temporary files AND you can keep them open
    > at all times, you can probably use an old UNIX trick to ensure that
    > they are always removed, even if your application crashes.  After
    > opening the file, unlink it, and don't close the file handle.  The
    > file is gone as far as anything else is concerned, but your app can
    > still read and write to it.  As soon as the last file handle is
    > closed, the file really goes away.
    >
previous month october 2007 next month
MTWTFSS
1 2 3 4 5 6 7
8 9 10 11 12 13 14
15 16 17 18 19 20 21
22 23 24 25 26 27 28
29 30 31        
Go to today