Troublesome app when scaling it up, how to proceed?

  • I have an app that uses a lot of subprocesses, either via NSTask or
    via AuthExecuteWithPrivileges. Previously, this data-driven ap would
    have maybe a hundred subtasks in a list that were executed when it
    was their turn, though some parallelism is possible.

    Now, my app just ends doing the subprocesses. No subprocesses are in
    the ps list. Apparently, one process has died, but this has not
    reached the main app. Given the complex nature of my stuff
    (AuthExecuteWithPrivileges talk back through named pipes so I can get
    stdout and stderr back separately in a thread in my app)

    I am sorry this is all a bit short, but there is no way I can make a
    clear and short description of my problem. I suspect I may run out of
    some system resource, but I have no idea which one and I suspect some
    stuff that normally always works in the frameworks just stops working)

    Anyway, I was wondering what the best tools are to track this kind of
    a problem down. I have been looking at a few, but haven't seen an
    obvious one to help me find out what is happening.

    G
  • > Anyway, I was wondering what the best tools are to track this kind of
    > a problem down. I have been looking at a few, but haven't seen an
    > obvious one to help me find out what is happening.

      How about using the debugger? You should be able to determine
    exactly where the problem begins (or at the very least get a more
    specific idea). Have you tried this? I ask because you haven't
    mentioned whether you've tried that or anything else.

      You should at least be able to figure out what task is causing the problem ...

    --
    I.S.
  • >> Anyway, I was wondering what the best tools are to track this kind of
    >> a problem down. I have been looking at a few, but haven't seen an
    >> obvious one to help me find out what is happening.
    >
    > How about using the debugger? You should be able to determine
    > exactly where the problem begins (or at the very least get a more
    > specific idea). Have you tried this? I ask because you haven't
    > mentioned whether you've tried that or anything else.
    >
    > You should at least be able to figure out what task is causing the
    > problem ...

    I did try the debugger but that does not help me at all in this case.
    Something outside my app dies (e.g. running a couple of subprocesses in a
    pipe as in getting an md5 hash value for a file) and this is never picked
    up by my app. The app lives. It just does not pick up that something died.

    But my guess for a resource problem was right. I used setrlimit() to
    increase the number of max open files (from the 4096 I had set it to in
    the past) to 10240 and now it works again.

    So, my problem is 'solved' though I am pretty unhappy about this 'solution'.

    When my app and subprocesses run out of system resources, I am unable
    (either in the debugger or in any other way) to catch that problem.
    Somehow, it seems, maybe due to my apps architecture, these resources are
    not properly freed until I my subprocesses for a certain task have run (I
    know this because if succesful, I can run the same thing over and over
    again without running into the same problem, so it is not a leak, but a
    temporary shortage. Intuitively I think it is connected to delayed freeing
    of resources in the system for some reason). I run a set of processes (as
    in the pipe above), e.g. via NSTasks which are coupled with NSPipes (or
    more complex with a wrapper object around NSAuthWithPrivileges() if they
    have to run with privileges, somehow some subprocess runs into a resource
    shortage, dies, but this is not signalled to my app (which is programmed
    to pick up these signals and react to it) which is left in limbo. As far
    as my app is concerned, it is still waiting for the died subprocess.

    I think my app with its complex design with threads, DO to talk between
    threads, threads to write to and read from named pipes to work around the
    limitations of AuthExecuteWithPrivileges(), is at the end of its useful
    life.

    I am going to have a look at 10.5 to see if this might give me frameworks
    and tools to do a proper redsign. One never knows.

    Thanks for the feedback.

    G
  • Are your tasks all running the same program, perhaps with different
    arguments?

    If so, you might consider using distributed objects to perform your
    remote calculations in a single process, replacing your list of
    NSTasks with a list of NSInvocations representing messages to your
    'slave' process.  This would allow you to run the slave process in the
    debugger...

    dave

    On 23-Oct-07, at 1:06 PM, Gerben Wierda wrote:

    > I have an app that uses a lot of subprocesses, either via NSTask or
    > via AuthExecuteWithPrivileges. Previously, this data-driven ap would
    > have maybe a hundred subtasks in a list that were executed when it
    > was their turn, though some parallelism is possible.
    >
    > Now, my app just ends doing the subprocesses. No subprocesses are in
    > the ps list. Apparently, one process has died, but this has not
    > reached the main app. Given the complex nature of my stuff
    > (AuthExecuteWithPrivileges talk back through named pipes so I can
    > get stdout and stderr back separately in a thread in my app)
    >
    > I am sorry this is all a bit short, but there is no way I can make a
    > clear and short description of my problem. I suspect I may run out
    > of some system resource, but I have no idea which one and I suspect
    > some stuff that normally always works in the frameworks just stops
    > working)
    >
    > Anyway, I was wondering what the best tools are to track this kind
    > of a problem down. I have been looking at a few, but haven't seen an
    > obvious one to help me find out what is happening.
    >
  • On 23.10.2007, at 21:06, Gerben Wierda wrote:

    > I am sorry this is all a bit short, but there is no way I can make
    > a clear and short description of my problem. I suspect I may run
    > out of some system resource, but I have no idea which one and I
    > suspect some stuff that normally always works in the frameworks
    > just stops working)

    I had something like that with the system running out of file
    handles. fs_usage and lsof are your friends there.
    sample, Shark.app and ThreadViewer.app are also good candidates for
    attaching later to inspect the state.
  • You know, it's really easy to write code to MD5 a file without shelling out to a separate process to do it. The algorithm is open-source and above that it's just fopen/fread/fclose (or various Cocoa classes if you prefer). That also opens the door to much, much cleaner error checking than dealing with error codes coming back from a command-line app.

    -----Original Message-----
    From: cocoa-dev-bounces+jstiles=<blizzard.com...> on behalf of Gerben Wierda
    Sent: Wed 10/24/2007 7:55 AM
    To: I. Savant
    Cc: <cocoa-dev...>
    Subject: Re: Troublesome app when scaling it up, how to proceed?

    >> Anyway, I was wondering what the best tools are to track this kind of
    >> a problem down. I have been looking at a few, but haven't seen an
    >> obvious one to help me find out what is happening.
    >
    > How about using the debugger? You should be able to determine
    > exactly where the problem begins (or at the very least get a more
    > specific idea). Have you tried this? I ask because you haven't
    > mentioned whether you've tried that or anything else.
    >
    > You should at least be able to figure out what task is causing the
    > problem ...

    I did try the debugger but that does not help me at all in this case.
    Something outside my app dies (e.g. running a couple of subprocesses in a
    pipe as in getting an md5 hash value for a file) and this is never picked
    up by my app. The app lives. It just does not pick up that something died.

    But my guess for a resource problem was right. I used setrlimit() to
    increase the number of max open files (from the 4096 I had set it to in
    the past) to 10240 and now it works again.

    So, my problem is 'solved' though I am pretty unhappy about this 'solution'.

    When my app and subprocesses run out of system resources, I am unable
    (either in the debugger or in any other way) to catch that problem.
    Somehow, it seems, maybe due to my apps architecture, these resources are
    not properly freed until I my subprocesses for a certain task have run (I
    know this because if succesful, I can run the same thing over and over
    again without running into the same problem, so it is not a leak, but a
    temporary shortage. Intuitively I think it is connected to delayed freeing
    of resources in the system for some reason). I run a set of processes (as
    in the pipe above), e.g. via NSTasks which are coupled with NSPipes (or
    more complex with a wrapper object around NSAuthWithPrivileges() if they
    have to run with privileges, somehow some subprocess runs into a resource
    shortage, dies, but this is not signalled to my app (which is programmed
    to pick up these signals and react to it) which is left in limbo. As far
    as my app is concerned, it is still waiting for the died subprocess.

    I think my app with its complex design with threads, DO to talk between
    threads, threads to write to and read from named pipes to work around the
    limitations of AuthExecuteWithPrivileges(), is at the end of its useful
    life.

    I am going to have a look at 10.5 to see if this might give me frameworks
    and tools to do a proper redsign. One never knows.

    Thanks for the feedback.

    G
  • On 24 Oct 2007, at 17:59, John Stiles wrote:

    > You know, it's really easy to write code to MD5 a file without
    > shelling out to a separate process to do it. The algorithm is open-
    > source

    Moreover, there is a ready-made implementation of it built in to Mac
    OS X, in the OpenSSL library.  (See the man page for EVP_DigestInit()).

    I wouldn't be surprised to find that there are additional
    implementations of it in the various Apple frameworks also.

    Kind regards,

    Alastair.

    --
    http://alastairs-place.net
  • > Moreover, there is a ready-made implementation of it built in to Mac
    > OS X, in the OpenSSL library.  (See the man page for EVP_DigestInit()).
    >
    > I wouldn't be surprised to find that there are additional
    > implementations of it in the various Apple frameworks also.

    http://www.cocoadev.com/index.pl?MDFive

    --
    I.S.
  • > But my guess for a resource problem was right. I used setrlimit() to
    > increase the number of max open files (from the 4096 I had set it to in
    > the past) to 10240 and now it works again.

    Watch out, I think there is a system-wide limit, and I think it's around
    12,000 files total. If your app actually opens 10,000 files, it could cause
    problems for other apps.

    Do you really need to run so many processes in parallel? Can you not simply
    throttle the number of processes running at any given time?

    --
    Scott Ribe
    <scott_ribe...>
    http://www.killerbytes.com/
    (303) 722-0567 voice
  • >
    > Are your tasks all running the same program, perhaps with different
    > arguments?

    See other reply.

    >
    > If so, you might consider using distributed objects to perform your
    > remote calculations in a single process, replacing your list of
    > NSTasks with a list of NSInvocations representing messages to your
    > 'slave' process.  This would allow you to run the slave process in the
    > debugger...

    I am already using DO for various reasons.

    For one, my subprocess sometimes have to run in admin mode and I am using
    AuthExecuteWithPrivileges() for that. That function is really broken,
    because for processes started like that you cannot get the PID, you
    cannote get stderr seperately from stdout (it uses a bidirectional
    buffered pipe), you cannot pass an environment. So subprocesses like these
    go via a different route. The main program creates named pipes for stdin,
    stdout and stderr, passing the environment and pid, then starts a generic
    subprocess. This subprocess attaches stdout, stdin and stderr to the right
    named pipes, communicates the pid back, then execs the actual subprocess
    that needs to be run. Reading output from the stderr and stdout named
    pipes is done in separate threads of the main program. These programs
    communicate to the main thread using DO. Since the PID has been
    communicated back, I can kill these subprocesses from within the main
    program (which otherwise would have been troublesome as one does not know
    the pid).

    Another use of DO is that there is one special subsubprocess that has to
    communicate back to the main program as it steers several UI functions.
    This also is done by using DO. So, I am already using DO quite a bit.

    G

    >
    > dave
    >
    >
    > On 23-Oct-07, at 1:06 PM, Gerben Wierda wrote:
    >
    >> I have an app that uses a lot of subprocesses, either via NSTask or
    >> via AuthExecuteWithPrivileges. Previously, this data-driven ap would
    >> have maybe a hundred subtasks in a list that were executed when it
    >> was their turn, though some parallelism is possible.
    >>
    >> Now, my app just ends doing the subprocesses. No subprocesses are in
    >> the ps list. Apparently, one process has died, but this has not
    >> reached the main app. Given the complex nature of my stuff
    >> (AuthExecuteWithPrivileges talk back through named pipes so I can
    >> get stdout and stderr back separately in a thread in my app)
    >>
    >> I am sorry this is all a bit short, but there is no way I can make a
    >> clear and short description of my problem. I suspect I may run out
    >> of some system resource, but I have no idea which one and I suspect
    >> some stuff that normally always works in the frameworks just stops
    >> working)
    >>
    >> Anyway, I was wondering what the best tools are to track this kind
    >> of a problem down. I have been looking at a few, but haven't seen an
    >> obvious one to help me find out what is happening.
    >>
    >
    >
  • > You know, it's really easy to write code to MD5 a file without shelling
    > out to a separate process to do it. The algorithm is open-source and above
    > that it's just fopen/fread/fclose (or various Cocoa classes if you
    > prefer). That also opens the door to much, much cleaner error checking
    > than dealing with error codes coming back from a command-line app.

    I agree, but the md5 example was just an example. I use several other
    programs as well and there is not such a library/possibility for all.
    Secondly, as some of these processes have to run with privileges, I need
    to use AuthExecuteWithPrivileges() if I do not want to turn my entire app
    into a privileged operation (and I don't as I follow the principle that
    privileged operations should be kept to an absolute minimum).

    G
  • >> But my guess for a resource problem was right. I used setrlimit() to
    >> increase the number of max open files (from the 4096 I had set it to in
    >> the past) to 10240 and now it works again.
    >
    > Watch out, I think there is a system-wide limit, and I think it's around
    > 12,000 files total. If your app actually opens 10,000 files, it could
    > cause
    > problems for other apps.
    >
    > Do you really need to run so many processes in parallel? Can you not
    > simply
    > throttle the number of processes running at any given time?

    This is already built in. The processes do not run in parallel. The
    problem I am running into is that it seems that resources are not actually
    freed for some reason (even if I do explicitely close/free them).

    I am not at my desk right now, but maybe I can post an example later and
    people can tell me what I am doing wrong.

    G

    >
    > --
    > Scott Ribe
    > <scott_ribe...>
    > http://www.killerbytes.com/
    > (303) 722-0567 voice
    >
    >
    >
    >
  • On 24.10.2007, at 21:48, Scott Ribe wrote:

    >> But my guess for a resource problem was right. I used setrlimit() to
    >> increase the number of max open files (from the 4096 I had set it
    >> to in
    >> the past) to 10240 and now it works again.
    >
    > Watch out, I think there is a system-wide limit, and I think it's
    > around
    > 12,000 files total. If your app actually opens 10,000 files, it
    > could cause
    > problems for other apps.
    >
    > Do you really need to run so many processes in parallel? Can you
    > not simply
    > throttle the number of processes running at any given time?

    BTW, the number of processes is also limited.

    I had a bad apache2 config on my machine once that resulted in
    indefinitely spawning another instances under the hood. After a bunch
    of minutes it was not possible to open any other application
    reliable.. Most time it didn't succeed, sometimes I was lucky and
    able to get another process to start up. The total number of
    processes was not that high, 128 or so...

    Regards,
    Tom_E
  • On 25.10.2007, at 08:39, Gerben Wierda wrote:

    > go via a different route. The main program creates named pipes for
    stdin,
    > stdout and stderr, [...] Reading output from the stderr and stdout
    > named
    > pipes is done in separate threads of the main program. These programs
    > communicate to the main thread using DO.

    Pardon my ignorance, what has the suggestion "Use Distributed (NS)
    Objects INSTEAD of filehandles, i.e. pipes" to do with "I use DO
    ADDITIONALY to three filedescriptors INTERNAL"?

    Regards,
    Tom_E
  • > On 25.10.2007, at 08:39, Gerben Wierda wrote:
    >
    >> go via a different route. The main program creates named pipes for
    > stdin,
    >> stdout and stderr, [...] Reading output from the stderr and stdout
    >> named
    >> pipes is done in separate threads of the main program. These programs
    >> communicate to the main thread using DO.
    >
    > Pardon my ignorance, what has the suggestion "Use Distributed (NS)
    > Objects INSTEAD of filehandles, i.e. pipes" to do with "I use DO
    > ADDITIONALY to three filedescriptors INTERNAL"?

    Nothing in a stricly logical sense. I was just explaining (to people who
    informed me about DO and sent basic examples) that I already am aware of
    DO and am using it both internally between threads and in one special case
    between a special subprocess and my main program and that it does not
    solve my problem.

    I cannot use DO for communication because I want to communicate with the
    stdin, stdout, stderr of regular unix cli tools like file, tar, openssl,
    gpg, perl, sh, etc. Not all of these can be communicated with via DO
    directly (only perl using "use Foundation") or used via a linked library
    so I do have to communicate with stdin/stderr/stdout of normal cli tools
    anyway. It becomes more complex when one has to run privileged
    subprocesses from a non-privileged main program as the interface to
    privileged subprocesses (AuthExecuteWithPrivileges()) is seriously limited
    (no stderr, no pid, no environment passing).

    If there would be an NSPrivilegedTask it would become a lot easier. Now I
    had to program my own NSPrivilegedTask five years ago and doing that was
    far from simple and it required a special stub for starting the subprocess
    so I can connect to the named pipes in separate threads in the main
    program, report the pid, set the environment.

    But I am far from a day-to-day cocoa expert so it could be that I am
    overlooking something. As far as my current situation, my setup runs out
    of file handles even though I release everything when the subprocess ends
    and not more than a few are running in parallel at all times. They are
    kept open somehow, or at least they are not returned to the kernel for
    some reason, and I cannot figure out why.

    G
  • Trying to simply this to the known facts:

    * Your app is running out of file-handles and failing. You know this
    because you modified the limit and it stops failing. Correct?

    On 25 Oct 2007, at 10:28, Gerben Wierda wrote:

    > I cannot use DO for communication because I want to communicate
    > with the
    > stdin, stdout, stderr of regular unix cli tools like file, tar,
    > openssl,
    > gpg, perl, sh, etc.

    * You're running a lot of sub-processes and connecting to them with
    pipes. At least 3 per process (stdout, stdin, stderr) and possibly
    more (+named ones for privileged processes).

    > If there would be an NSPrivilegedTask it would become a lot easier.

    It might simplify your code, but it wouldn't solve your problem. That
    routine would still use file-handles to communicate. This is, I
    think, a red herring. If you are talking to stdin/out/err these are
    pipes however you connect to them, using Cocoa or not.

    So the question is 'why aren't they closing?'

    In this situation I would be using lsof to monitor the open file
    handles as the app runs. Something like

    lsof -p <pid> -n +r 5

    will show you all the open files of your process, and which ones are
    lasting longer than they should. Alternatively, using sharks 'System
    Trace' may help by allowing you to see what file-handles are used by
    what system call, and when they should have been released. That's not
    as easy to decode though.

    I seem to remember something about pipes or sockets needing to be
    closed from both ends for them to be released, and the process just
    dying isn't enough. I've seen 'zombie' pipes hang around before (and
    get cleaned up after a while) and I think it was something like this.
    (This is from general Unix programming, nothing Mac specific)
  • On 25 Oct 2007, at 07:39, Gerben Wierda wrote:

    > For one, my subprocess sometimes have to run in admin mode and I am
    > using
    > AuthExecuteWithPrivileges() for that. That function is really broken,
    > because for processes started like that you cannot get the PID, you
    > cannote get stderr seperately from stdout (it uses a bidirectional
    > buffered pipe), you cannot pass an environment. So subprocesses
    > like these
    > go via a different route. The main program creates named pipes for
    > stdin,
    > stdout and stderr, passing the environment and pid, then starts a
    > generic
    > subprocess. This subprocess attaches stdout, stdin and stderr to
    > the right
    > named pipes, communicates the pid back, then execs the actual
    > subprocess
    > that needs to be run. Reading output from the stderr and stdout named
    > pipes is done in separate threads of the main program. These programs
    > communicate to the main thread using DO. Since the PID has been
    > communicated back, I can kill these subprocesses from within the main
    > program (which otherwise would have been troublesome as one does
    > not know
    > the pid).

    On 25 Oct 2007, at 07:44, Gerben Wierda wrote:

    > I agree, but the md5 example was just an example. I use several other
    > programs as well and there is not such a library/possibility for all.
    > Secondly, as some of these processes have to run with privileges, I
    > need
    > to use AuthExecuteWithPrivileges() if I do not want to turn my
    > entire app
    > into a privileged operation (and I don't as I follow the principle
    > that
    > privileged operations should be kept to an absolute minimum).

    From reading this thread, it sounds like you may have adopted a
    somewhat baroque solution to whatever problem you're actually trying
    to solve.

    Perhaps a simpler mechanism would be to use AuthExecuteWithPrivileges
    () *once* to set the setuid bit on a helper tool that you then
    execute in the normal UNIX manner (i.e. fork()/exec() or similar)?
    Once the tool is setuid root, you don't need to use
    AuthExecuteWithPrivileges() again,  even if you need to pass an
    authorisation context to it.  That would mean you could avoid using
    DO, that you can set-up your sub-processes' file handles as you
    require, and that you can get their pids as usual for a UNIX program.

    As for the problem of running out of resources, it's hard without
    actually seeing what your program is doing to know why (or indeed
    whether) that's happening.  My guess is that the apparent complexity
    isn't helping any, though.

    Kind regards,

    Alastair.

    --
    http://alastairs-place.net
  • > On 25 Oct 2007, at 07:39, Gerben Wierda wrote:
    >
    >> For one, my subprocess sometimes have to run in admin mode and I am
    >> using
    >> AuthExecuteWithPrivileges() for that. That function is really broken,
    >> because for processes started like that you cannot get the PID, you
    >> cannote get stderr seperately from stdout (it uses a bidirectional
    >> buffered pipe), you cannot pass an environment. So subprocesses
    >> like these
    >> go via a different route. The main program creates named pipes for
    >> stdin,
    >> stdout and stderr, passing the environment and pid, then starts a
    >> generic
    >> subprocess. This subprocess attaches stdout, stdin and stderr to
    >> the right
    >> named pipes, communicates the pid back, then execs the actual
    >> subprocess
    >> that needs to be run. Reading output from the stderr and stdout named
    >> pipes is done in separate threads of the main program. These programs
    >> communicate to the main thread using DO. Since the PID has been
    >> communicated back, I can kill these subprocesses from within the main
    >> program (which otherwise would have been troublesome as one does
    >> not know
    >> the pid).
    >
    > On 25 Oct 2007, at 07:44, Gerben Wierda wrote:
    >
    >> I agree, but the md5 example was just an example. I use several other
    >> programs as well and there is not such a library/possibility for all.
    >> Secondly, as some of these processes have to run with privileges, I
    >> need
    >> to use AuthExecuteWithPrivileges() if I do not want to turn my
    >> entire app
    >> into a privileged operation (and I don't as I follow the principle
    >> that
    >> privileged operations should be kept to an absolute minimum).
    >
    > From reading this thread, it sounds like you may have adopted a
    > somewhat baroque solution to whatever problem you're actually trying
    > to solve.
    >
    > Perhaps a simpler mechanism would be to use AuthExecuteWithPrivileges
    > () *once* to set the setuid bit on a helper tool that you then
    > execute in the normal UNIX manner (i.e. fork()/exec() or similar)?

    That is a logical solution and it is in fact the preferred solution.
    However it is only safe if the helper tool is a specific tool doing a
    specific task. That is not the case here, as this system can startup *any*
    tool with admin privileges (think the same way that Installer.app runs a
    post or preflight script with admin privileges). Currently, Ápple's
    installer warns you about 'running a program' with privileges, do you
    agree? If you say yes, the post/pre script is run as admin. If that script
    creates a trojan somewhere, you have a security breach. I do not know how
    Apple has fixed that potential problem currently, so I do not know if it
    is possible to make a doctored .pkg with a doctored preflight that sets up
    a setuid sh or anything like that. Anyway, my system is like that in that
    the functionality that is being run as admin is not fully defined in
    advance (hence there are all kinds of checks with gpg/openssl to make sure
    nobody has doctored with the actual program that is being run).

    > Once the tool is setuid root, you don't need to use
    > AuthExecuteWithPrivileges() again,  even if you need to pass an
    > authorisation context to it.  That would mean you could avoid using
    > DO, that you can set-up your sub-processes' file handles as you
    > require, and that you can get their pids as usual for a UNIX program.

    A previous version worked a bit like this a few years ago, but I moved
    away from that approach to AEWP() because that approach made me run a lot
    of stuff with privileges (because it is all generics) which is not
    necessary. So, when I created this, I chose this setup to minimize the
    amount of stuff that runs with privileges to an absolute minimum.

    > As for the problem of running out of resources, it's hard without
    > actually seeing what your program is doing to know why (or indeed
    > whether) that's happening.  My guess is that the apparent complexity
    > isn't helping any, though.

    I agree fully. It has all become too complex. I am thinking also about
    dropping everything and starting from scratch, but that requires a usable
    solution for my functional demands or strongly minimizing the functional
    setup.

    G
  • > Trying to simply this to the known facts:
    >
    > * Your app is running out of file-handles and failing. You know this
    > because you modified the limit and it stops failing. Correct?

    Correct.

    > On 25 Oct 2007, at 10:28, Gerben Wierda wrote:
    >
    >> I cannot use DO for communication because I want to communicate
    >> with the
    >> stdin, stdout, stderr of regular unix cli tools like file, tar,
    >> openssl,
    >> gpg, perl, sh, etc.
    >
    > * You're running a lot of sub-processes and connecting to them with
    > pipes. At least 3 per process (stdout, stdin, stderr) and possibly
    > more (+named ones for privileged processes).
    >
    >> If there would be an NSPrivilegedTask it would become a lot easier.
    >
    > It might simplify your code, but it wouldn't solve your problem. That
    > routine would still use file-handles to communicate. This is, I
    > think, a red herring. If you are talking to stdin/out/err these are
    > pipes however you connect to them, using Cocoa or not.

    It might be that the way I do connect to them, release my objects etc,
    keeps them around longer than expected.

    > So the question is 'why aren't they closing?'

    Exactly. Or *maybe*, why are they closing (as I *think* I close
    everything) but are they not returned to the system?

    > In this situation I would be using lsof to monitor the open file
    > handles as the app runs. Something like
    >
    > lsof -p <pid> -n +r 5
    >
    > will show you all the open files of your process, and which ones are
    > lasting longer than they should. Alternatively, using sharks 'System
    > Trace' may help by allowing you to see what file-handles are used by
    > what system call, and when they should have been released. That's not
    > as easy to decode though.

    I know lsof (I think I used lsof the last time to see if it could help me
    find out what was happening) I'll look into Shark.

    > I seem to remember something about pipes or sockets needing to be
    > closed from both ends for them to be released, and the process just
    > dying isn't enough. I've seen 'zombie' pipes hang around before (and
    > get cleaned up after a while) and I think it was something like this.
    > (This is from general Unix programming, nothing Mac specific)

    I recall something like that as well. I suspect the result will be that if
    they are zombie pipes they will not show up with lsof -p.

    Suppose I run tar like this
    - my app creates named pipes a, b, c, d and e
    - my app starts reading from b and c in separate threads (these report the
    results back to my main thread using DO)
    - my app starts "helper tar"
    - helper connects stdin, stdout and stderr to named pipes a, b and c,
    reads the environment from d and reports pid back over e. d and e are
    closed when helper is done with them.
    - as soon as the pid has been reported, my app writes environment over to
    helper
    - d and e are closed by my app as soon as my app is done with them
    - as soon as environment has been received, helper closes d
    - helper now execs tar, which inherits environment and stdin, stdout and
    stderr which are connected to the named pipes a, b and c.
    - tar completes and dies

    could zombie pipes be created when apps like tar do not explicitely close
    stdout/stderr before exiting?

    G

    PS. I am aware that one can have two different discussions here. One about
    if this setup is at all the only way to do this and one that *if* you want
    to do it in this convoluted and complex manner. I'd like to discuss the
    second one now, not because I like my setup (I do not), but because I
    haven't been able to find a safe way to do it the preferred way when using
    a *generalized* helper. I would like to get rid of this, but while I will
    be investigating that angle I would like to solve my current riddle, if
    only out of technical curiosity.
  • Am 25.10.2007 um 16:49 schrieb Gerben Wierda:

    > could zombie pipes be created when apps like tar do not explicitely
    > close
    > stdout/stderr before exiting?

    In the case I had to debug the autorelease pool wasn't drained when
    the main app was in the background. The NSTasks were not released and
    the assiciated NSPipes neither, which lead to filehandle exhaustion
    after processing around 100 files with four subtaks á 2 handles.

    Debugged with lsof and ObjectAlloc, the fix I described AFAIR here.

    Regards,
    Tom_E
  • >
    > Am 25.10.2007 um 16:49 schrieb Gerben Wierda:
    >
    >> could zombie pipes be created when apps like tar do not explicitely
    >> close
    >> stdout/stderr before exiting?
    >
    > In the case I had to debug the autorelease pool wasn't drained when
    > the main app was in the background. The NSTasks were not released and
    > the assiciated NSPipes neither, which lead to filehandle exhaustion
    > after processing around 100 files with four subtaks á 2 handles.
    >
    > Debugged with lsof and ObjectAlloc, the fix I described AFAIR here.

    Interesting. Would that happen too when my process is still in the
    foreground (which it is) and the event loop (and thus autorelease
    effectuation) is still running periodically?

    G

    >
    > Regards,
    > Tom_E
    >
    >
  • On 25 Oct 2007, at 15:32, Gerben Wierda wrote:

    > That is a logical solution and it is in fact the preferred solution.
    > However it is only safe if the helper tool is a specific tool doing a
    > specific task.

    OK, so let's make the specific task be "checking signatures of and
    launching other programs".  If necessary you can check for rights
    using AuthorizationCopyRights() as well.

    I'm not sure what the security advantage is in not doing it that
    way?  Perhaps you're worried that the signature checks could be
    compromised somehow?  But if you run them as a non-root user, then
    surely it's easier for someone else to interfere with the checks?

    Kind regards,

    Alastair.

    --
    http://alastairs-place.net
  • On 25 Oct 2007, at 15:49, Gerben Wierda wrote:

    >
    >> In this situation I would be using lsof to monitor the open file
    >> handles as the app runs. Something like
    >>
    >> lsof -p <pid> -n +r 5
    >>
    >> will show you all the open files of your process, and which ones are
    >> lasting longer than they should. Alternatively, using sharks 'System
    >> Trace' may help by allowing you to see what file-handles are used by
    >> what system call, and when they should have been released. That's not
    >> as easy to decode though.
    >
    > I know lsof (I think I used lsof the last time to see if it could
    > help me
    > find out what was happening) I'll look into Shark.

    I just wanted to come back on this now that the Leopard embargo has
    lifted.

    Dtrace/Xray/Instruments (or whatever it's called now) should be able
    to help you on this too. Assuming you have a copy of Leopard around
    of course.
previous month october 2007 next month
MTWTFSS
1 2 3 4 5 6 7
8 9 10 11 12 13 14
15 16 17 18 19 20 21
22 23 24 25 26 27 28
29 30 31        
Go to today