Max arguments for NSTask?

  • I'm finding that if I try to launch a process using NSTask with too
    many arguments, I get this:

    *** NSTask: Task create for path /bin/pax failed: 7, "Argument list too
    long".

    and my app promptly hangs with the dreaded pinwheel of death. So, my
    question is, what is the maximum number of arguments that NSTask can
    take? Is this documented anywhere? Is there a way around it?

    Thanks,
    Charles
    _______________________________________________
    cocoa-dev mailing list | <cocoa-dev...>
    Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/cocoa-dev
    Do not post admin requests to the list. They will be ignored.
  • On Sun, Oct 27, 2002 at 12:37:48PM -0600, Charles Srstka wrote:
    > I'm finding that if I try to launch a process using NSTask with too
    > many arguments, I get this:
    >
    > *** NSTask: Task create for path /bin/pax failed: 7, "Argument list too
    > long".
    >
    > and my app promptly hangs with the dreaded pinwheel of death. So, my
    > question is, what is the maximum number of arguments that NSTask can
    > take? Is this documented anywhere? Is there a way around it?

    This is most likely not a NSTask limitation but a generic command-line
    invocation problem.  You can use sysctl to retrieve kern.argmax
    (CTL_KERN, KERN_ARGMAX in <sys/sysctl.h>).  Note that this is a
    character limit, not an argument limit; sysctl.h does not make this
    very clear.

    I mention some workarounds from the command-line context here:

    <http://www.cocoadev.com/index.pl?UsernameAsNSString>

    but they should be equally applicable to NSTask.

    --
    =Nicholas Riley <njriley...> | <http://www.uiuc.edu/ph/www/njriley>
            Pablo Research Group, Department of Computer Science and
      Medical Scholars Program, University of Illinois at Urbana-Champaign
    _______________________________________________
    cocoa-dev mailing list | <cocoa-dev...>
    Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/cocoa-dev
    Do not post admin requests to the list. They will be ignored.
  • On Sunday, October 27, 2002, at 01:37  PM, Charles Srstka wrote:

    > I'm finding that if I try to launch a process using NSTask with too
    > many arguments, I get this:
    >
    > *** NSTask: Task create for path /bin/pax failed: 7, "Argument list
    > too long".
    >
    > and my app promptly hangs with the dreaded pinwheel of death. So, my
    > question is, what is the maximum number of arguments that NSTask can
    > take? Is this documented anywhere? Is there a way around it?
    >
    >

    The best way around it in this case would be to write the files that
    you want pax to pack up to a file,a nd then use the flag for pax that
    reads that list from the files.

    There is a limit to the number of args (files usually) you can pass to
    a cli tool... that's why xargs exists (and that may be another option
    for you BTW..)
    _______________________________________________
    cocoa-dev mailing list | <cocoa-dev...>
    Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/cocoa-dev
    Do not post admin requests to the list. They will be ignored.
  • On Sunday, October 27, 2002, at 06:37  pm, Charles Srstka wrote:

    > and my app promptly hangs with the dreaded pinwheel of death. So, my
    > question is, what is the maximum number of arguments that NSTask can
    > take? Is this documented anywhere? Is there a way around it?

    Well, there's this in limits.h:

    #define _POSIX_ARG_MAX          4096

    Which you probably can't change easily.

      -- Finlay
    _______________________________________________
    cocoa-dev mailing list | <cocoa-dev...>
    Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/cocoa-dev
    Do not post admin requests to the list. They will be ignored.
  • On Sunday, October 27, 2002, at 06:59  pm, Finlay Dobbie wrote:

    >> and my app promptly hangs with the dreaded pinwheel of death. So, my
    >> question is, what is the maximum number of arguments that NSTask can
    >> take? Is this documented anywhere? Is there a way around it?
    >
    > Well, there's this in limits.h:
    >
    > #define _POSIX_ARG_MAX          4096

    As others have pointed out, this is probably incorrect. Sorry about
    that. :-)

      -- Finlay
    _______________________________________________
    cocoa-dev mailing list | <cocoa-dev...>
    Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/cocoa-dev
    Do not post admin requests to the list. They will be ignored.
  • On Sunday, October 27, 2002, at 12:59  PM, Scott Anguish wrote:

    >
    > On Sunday, October 27, 2002, at 01:37  PM, Charles Srstka wrote:
    >
    >> I'm finding that if I try to launch a process using NSTask with too
    >> many arguments, I get this:
    >>
    >> *** NSTask: Task create for path /bin/pax failed: 7, "Argument list
    >> too long".
    >>
    >> and my app promptly hangs with the dreaded pinwheel of death. So, my
    >> question is, what is the maximum number of arguments that NSTask can
    >> take? Is this documented anywhere? Is there a way around it?
    >
    > The best way around it in this case would be to write the files that
    > you want pax to pack up to a file,a nd then use the flag for pax that
    > reads that list from the files.

    Sounds great. I was hoping pax would have something like this, but I
    couldn't find any mention in the man pages (and still can't).

    Anyone know what flag would do this?

    If it doesn't exist, xargs does look promising.

    Thanks,
    Charles
    _______________________________________________
    cocoa-dev mailing list | <cocoa-dev...>
    Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/cocoa-dev
    Do not post admin requests to the list. They will be ignored.
  • On Sunday, October 27, 2002, at 02:23 PM, Charles Srstka wrote:
    >> The best way around it in this case would be to write the files that
    >> you want pax to pack up to a file,a nd then use the flag for pax that
    >> reads that list from the files.
    >
    > Sounds great. I was hoping pax would have something like this, but I
    > couldn't find any mention in the man pages (and still can't).
    >
    > Anyone know what flag would do this?
    >

    Huh.. nope, it doesn't have this..

    > If it doesn't exist, xargs does look promising.
    >

    that's an option.. but why in the hell use pax anyways... tar works
    and it doesn't replace symbolic links that the user has created with
    files that it feels are more appropriate.
    _______________________________________________
    cocoa-dev mailing list | <cocoa-dev...>
    Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/cocoa-dev
    Do not post admin requests to the list. They will be ignored.
  • On Sunday, October 27, 2002, at 03:01  PM, Scott Anguish wrote:

    > On Sunday, October 27, 2002, at 02:23 PM, Charles Srstka wrote:
    >>> The best way around it in this case would be to write the files
    >>> that you want pax to pack up to a file,a nd then use the flag for
    >>> pax that reads that list from the files.
    >>
    >> Sounds great. I was hoping pax would have something like this, but I
    >> couldn't find any mention in the man pages (and still can't).
    >>
    >> Anyone know what flag would do this?
    >>
    >
    > Huh.. nope, it doesn't have this..

    Damn. Oh well, thanks for trying. I guess I may have to use xargs,
    although I don't like the way it executes the command multiple times.

    >> If it doesn't exist, xargs does look promising.
    >>
    >
    > that's an option.. but why in the hell use pax anyways... tar works
    > and it doesn't replace symbolic links that the user has created with
    > files that it feels are more appropriate.

    Unfortunately, I have no choice but to use pax for what I do. However,
    tar doesn't seem to have this option either...

    Charles
    _______________________________________________
    cocoa-dev mailing list | <cocoa-dev...>
    Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/cocoa-dev
    Do not post admin requests to the list. They will be ignored.
  • On Sunday, October 27, 2002, at 04:33 PM, Charles Srstka wrote:

    >>> If it doesn't exist, xargs does look promising.
    >>>
    >>
    >> that's an option.. but why in the hell use pax anyways... tar works
    >> and it doesn't replace symbolic links that the user has created with
    >> files that it feels are more appropriate.
    >
    > Unfortunately, I have no choice but to use pax for what I do. However,
    > tar doesn't seem to have this option either...
    >
    OK.. tar in this case is infact Pax (barf).. why would you have no
    choice but to use pax?  Unless you're pulling apart .pkg files..

    gnutar.. that'll do it.. --files-from filename

    with pax you could just iterate over the files, adding/extracting them
    to/from an existing archive...
    _______________________________________________
    cocoa-dev mailing list | <cocoa-dev...>
    Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/cocoa-dev
    Do not post admin requests to the list. They will be ignored.
  • At 12:53 PM -0600 10/27/02, Nicholas Riley wrote:
    > This is most likely not a NSTask limitation but a generic command-line
    > invocation problem.  You can use sysctl to retrieve kern.argmax
    > (CTL_KERN, KERN_ARGMAX in <sys/sysctl.h> ).  Note that this is a
    > character limit, not an argument limit; sysctl.h does not make this
    > very clear.

    What I want to know is why a modern operating system would have such
    a limitation in the first place.

    It's not like memory allocation is that hard a problem.

      -- Chris

    --
    Chris Hanson                      |  Email: <cmh...>
    bDistributed.com, Inc.            |  Phone: +1-847-372-3955
    Making Business Distributed      |  Fax:  +1-847-589-3738
    http://bdistributed.com/          |  Personal Email: <cmh...>
    _______________________________________________
    cocoa-dev mailing list | <cocoa-dev...>
    Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/cocoa-dev
    Do not post admin requests to the list. They will be ignored.
  • On Sun, Oct 27, 2002 at 04:14:44PM -0600, Chris Hanson wrote:
    > At 12:53 PM -0600 10/27/02, Nicholas Riley wrote:
    >> This is most likely not a NSTask limitation but a generic command-line
    >> invocation problem.  You can use sysctl to retrieve kern.argmax
    >> (CTL_KERN, KERN_ARGMAX in <sys/sysctl.h>).  Note that this is a
    >> character limit, not an argument limit; sysctl.h does not make this
    >> very clear.
    >
    > What I want to know is why a modern operating system would have such
    > a limitation in the first place.

    I don't know of a Unix-derived OS that doesn't have such a limitation.

    > It's not like memory allocation is that hard a problem.

    It's a design decision, IMO.  The approach of dumping lots of
    filenames on the command doesn't scale to thousands of files, because
    you have to enumerate them all, potentially filling up megabytes of
    memory and holding the whole thing in argv for the duration of the
    program's execution.  Unix has pipes to facilitate interprocess
    communication, and xargs simply functions as a gateway from pipes to
    command line arguments, thereby using a lot less memory and resources
    than the naive approach of simply making the command line buffer
    bounded by memory.

    --
    =Nicholas Riley <njriley...> | <http://www.uiuc.edu/ph/www/njriley>
            Pablo Research Group, Department of Computer Science and
      Medical Scholars Program, University of Illinois at Urbana-Champaign
    _______________________________________________
    cocoa-dev mailing list | <cocoa-dev...>
    Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/cocoa-dev
    Do not post admin requests to the list. They will be ignored.
  • > What I want to know is why a modern operating system would have such a
    > limitation in the first place.
    >
    > It's not like memory allocation is that hard a problem.

    An arbitrary limit also guards against processes requesting
    outrageously large or infinite amounts of resources, accidentally or
    otherwise. There are hard limits on the numbers of many other items
    throughout posix, why not exe invocations?

    --Mark

    "Today's lesson: Time. Imagine a donut, fired from a cannon at the
    speed of light while rotating. Time is like that, except without the
    cannon and the donut."
    _______________________________________________
    cocoa-dev mailing list | <cocoa-dev...>
    Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/cocoa-dev
    Do not post admin requests to the list. They will be ignored.
  • Thank you all for your help.

    I have just one more question: Are the spaces between the arguments
    counted in the total characters?

    Thanks,
    Charles

    On Sunday, October 27, 2002, at 04:59  PM, Nicholas Riley wrote:

    > On Sun, Oct 27, 2002 at 04:14:44PM -0600, Chris Hanson wrote:
    >> At 12:53 PM -0600 10/27/02, Nicholas Riley wrote:
    >>> This is most likely not a NSTask limitation but a generic
    >>> command-line
    >>> invocation problem.  You can use sysctl to retrieve kern.argmax
    >>> (CTL_KERN, KERN_ARGMAX in <sys/sysctl.h>).  Note that this is a
    >>> character limit, not an argument limit; sysctl.h does not make this
    >>> very clear.
    >>
    >> What I want to know is why a modern operating system would have such
    >> a limitation in the first place.
    >
    > I don't know of a Unix-derived OS that doesn't have such a limitation.
    >
    >> It's not like memory allocation is that hard a problem.
    >
    > It's a design decision, IMO.  The approach of dumping lots of
    > filenames on the command doesn't scale to thousands of files, because
    > you have to enumerate them all, potentially filling up megabytes of
    > memory and holding the whole thing in argv for the duration of the
    > program's execution.  Unix has pipes to facilitate interprocess
    > communication, and xargs simply functions as a gateway from pipes to
    > command line arguments, thereby using a lot less memory and resources
    > than the naive approach of simply making the command line buffer
    > bounded by memory.
    >
    > --
    > =Nicholas Riley <njriley...> |
    > <http://www.uiuc.edu/ph/www/njriley>
    > Pablo Research Group, Department of Computer Science and
    > Medical Scholars Program, University of Illinois at Urbana-Champaign
    > _______________________________________________
    > cocoa-dev mailing list | <cocoa-dev...>
    > Help/Unsubscribe/Archives:
    > http://www.lists.apple.com/mailman/listinfo/cocoa-dev
    > Do not post admin requests to the list. They will be ignored.
    _______________________________________________
    cocoa-dev mailing list | <cocoa-dev...>
    Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/cocoa-dev
    Do not post admin requests to the list. They will be ignored.
  • At 5:23 PM -0600 10/27/02, Mark Levin wrote:
    >> What I want to know is why a modern operating system would have
    >> such a limitation in the first place.
    >>
    >> It's not like memory allocation is that hard a problem.
    >
    > An arbitrary limit also guards against processes requesting
    > outrageously large or infinite amounts of resources, accidentally or
    > otherwise. There are hard limits on the numbers of many other items
    > throughout posix, why not exe invocations?

    There shouldn't be hard limits on most other items either.

    If it's important to limit the size of the argument buffer a process
    can be assigned, it should be a modifiable limit, not one compiled
    into the kernel as a constant.

      -- Chris

    --
    Chris Hanson                      |  Email: <cmh...>
    bDistributed.com, Inc.            |  Phone: +1-847-372-3955
    Making Business Distributed      |  Fax:  +1-847-589-3738
    http://bdistributed.com/          |  Personal Email: <cmh...>
    _______________________________________________
    cocoa-dev mailing list | <cocoa-dev...>
    Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/cocoa-dev
    Do not post admin requests to the list. They will be ignored.
  • in /usr/include/limits.h:
    #if !defined _ANSI_SOURCE
    #define _POSIX_ARG_MAX 4096

    On Sunday, October 27, 2002, at 11:14  PM, Chris Hanson wrote:
    >>
    >> An arbitrary limit also guards against processes requesting
    >> outrageously large or infinite amounts of resources, accidentally or
    >> otherwise. There are hard limits on the numbers of many other items
    >> throughout posix, why not exe invocations?
    Hasan Diwan
    OpenPGP KeyID: 0xBE42DCA6
    Fingerprint: 1CB0 47E3 0A24 DAC1 DCCA  4225 F166 40C2 BE42 DCA6
    http://www.cs.rpi.edu/~diwanh/gpg.key

    [demime 0.98b removed an attachment of type application/pgp-signature which had a name of PGP.sig]
    _______________________________________________
    cocoa-dev mailing list | <cocoa-dev...>
    Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/cocoa-dev
    Do not post admin requests to the list. They will be ignored.
previous month october 2002 next month
MTWTFSS
  1 2 3 4 5 6
7 8 9 10 11 12 13
14 15 16 17 18 19 20
21 22 23 24 25 26 27
28 29 30 31      
Go to today