NSOperationQueue

  • A question about NSOperationQueue:

    Is there a way to add operations to the queue so that they "jump the queue" ahead of any operations that are not running and have not been run yet?

    Here's my use case:

    I have a browser that browses folders full of SVG files. To create a thumbnail preview of the file, it needs to be parsed and converted but it's really slow to do this, so I encapsulate this in an operation and stick it on a concurrent operation queue. The queue limits the number of concurrent ops to a small handful. As each thumbnail is generated, it refreshes the browser and the thumbnail pops into view.

    If I'm scrolling through the browser, it may have queued up a large number of these, but ideally I would prefer if it gave priority to those that are actually visible in the window, which are the ones most recently added to the queue. Instead, it adds them to the end which means I often have to wait a long time for the window to show the thumbnails. Setting a higher priority on these items doesn't help because the operations all end up with the same priority.

    An NSOperationQueue that simply iterated in reverse would do it, or a way to prepend the objects to the head of the queue.

    --Graham
  • On Jun 1, 2012, at 8:07 PM, Graham Cox <graham.cox...> wrote:

    > A question about NSOperationQueue:
    >
    > Is there a way to add operations to the queue so that they "jump the queue" ahead of any operations that are not running and have not been run yet?

    Give them a higher priority. You should be able to alter the priorities as the user scrolls, and NSOperationQueue will do the right thing.

    --Kyle Sluder
  • On 02/06/2012, at 1:12 PM, Kyle Sluder wrote:

    > Give them a higher priority. You should be able to alter the priorities as the user scrolls, and NSOperationQueue will do the right thing.

    I tried this but it doesn't work - a bit of thought about how the ops are queued will show why no meaningful priority value can be assigned.

    At the moment that the operations are queued, there are some operations in the queue not yet run, and some running. The code that creates the operations doesn't know which ones are needed more urgently (the latest ones), so it can only assign a high priority to all of them, so they all end up with the same (high) priority and so we're back to square one.

    What I'm considering is actually adding them to a priority "holding queue" of my own then moving them to the operation queue when it has some capacity available. I'm sure it will work but I was hoping there might have been a ready solution given this sort of scenario.

    --Graham
  • On Jun 1, 2012, at 8:23 PM, Graham Cox <graham.cox...> wrote:

    >
    > On 02/06/2012, at 1:12 PM, Kyle Sluder wrote:
    >
    >> Give them a higher priority. You should be able to alter the priorities as the user scrolls, and NSOperationQueue will do the right thing.
    >
    >
    > I tried this but it doesn't work - a bit of thought about how the ops are queued will show why no meaningful priority value can be assigned.
    >
    > At the moment that the operations are queued, there are some operations in the queue not yet run, and some running. The code that creates the operations doesn't know which ones are needed more urgently (the latest ones), so it can only assign a high priority to all of them, so they all end up with the same (high) priority and so we're back to square one.

    Maintain a mapping of objects to be previewed and the operations that generate those previews. As the user scrolls, figure out which placeholders have been scrolled on/off screen and modify the related operations' priorities appropriately. Altering a running operation's priority won't have an effect.

    --Kyle Sluder
  • >> At the moment that the operations are queued, there are some operations
    in the queue not yet run, and some running. The code that creates the
    operations doesn't know which ones are needed more urgently (the latest
    ones), so it can only assign a high priority to all of them, so they all end
    up with the same (high) priority and so we're back to square one.
    >
    > Maintain a mapping of objects to be previewed and the operations that
    generate those previews. As the user scrolls, figure out which placeholders
    have been scrolled on/off screen and modify the related operations'
    priorities appropriately. Altering a running operation's priority won't have
    an effect.

    Potentially you could end up queuing up hundreds (or thousands) of thumbnail
    generation requests (depending on how many SVG files the user has). If the
    user was to close the browser to do something else, or switch to another
    app, you would continue generating those thumbnails. My solution, in an iOS
    app FWIW, was to cancel the thumbnail generation operations for the
    thumbnails that scrolled out of view.
  • On 02/06/2012, at 2:36 PM, Julius Oklamcak wrote:

    > Potentially you could end up queuing up hundreds (or thousands) of thumbnail
    > generation requests (depending on how many SVG files the user has). If the
    > user was to close the browser to do something else, or switch to another
    > app, you would continue generating those thumbnails. My solution, in an iOS
    > app FWIW, was to cancel the thumbnail generation operations for the
    > thumbnails that scrolled out of view.

    Yes, this is an issue I'm needing to consider.

    Right now, Kyle's suggestion, while it certainly sounds interesting and workable, is difficult for a couple of reasons. Chief among these is that the view in question is an IKImageBrowserView, and I'm supplying it with images on demand through its dataSource protocol. This protocol does not keep you up to date with view changes as scrolling takes place, so it's currently quite difficult to keep track of the set of visible items in order to prioritise them. I'm hoping I can avoid having to subclass the view.

    At the moment, when the datasource gets asked for an image, I kick off the asynchronous fetch if needed and return nil. When the image arrives, I am able to check if it's still visible and refresh the given cell of the view, which requests the image again. Another source of difficulty is that the image supplier item (implementing the IKImageBrowserItem protocol) is not strongly linked to the object that handles the file parse to generate the thumbnail. It is its delegate, and gets called back when the image is available, but it cannot actively go to that object and cancel it - it simply doesn't know what it is. This isn't a showstopper - I could make the browser item own the thumbnail generator for a while, but that's a bit of a redesign.

    My simple priority stack sorts out the problem of prioritising the visible items without really needing to know what they are, though I think Kyle's idea would work better if I could make it work. But it doesn't bother cancelling items no longer required.

    Need to think this through a bit more - any ideas or experiences of something similar welcomed.

    --Graham
  • On Jun 1, 2012, at 10:23 PM, Graham Cox wrote:

    > On 02/06/2012, at 1:12 PM, Kyle Sluder wrote:
    >
    >> Give them a higher priority. You should be able to alter the priorities as the user scrolls, and NSOperationQueue will do the right thing.
    >
    >
    > I tried this but it doesn't work - a bit of thought about how the ops are queued will show why no meaningful priority value can be assigned.
    >
    > At the moment that the operations are queued, there are some operations in the queue not yet run, and some running. The code that creates the operations doesn't know which ones are needed more urgently (the latest ones), so it can only assign a high priority to all of them, so they all end up with the same (high) priority and so we're back to square one.

    Setting the priority seems to work, in my testing:

    #import <Foundation/Foundation.h>

    int main(int argc, const char * argv[]) {
        @autoreleasepool {
            NSOperationQueue *queue = [[NSOperationQueue alloc] init];
            const NSUInteger numberOfOperations = 5;

            [queue setMaxConcurrentOperationCount:1];

            for(NSUInteger i = 0; i < numberOfOperations; i++) {
                NSOperation *operation = [NSBlockOperation blockOperationWithBlock:^{
                    NSLog(@"Running operation number %lu", (unsigned long)i);

                    sleep(5);
                }];

                [operation setQueuePriority:NSOperationQueuePriorityNormal];

                [queue addOperation:operation];
            }

            double delayInSeconds = 10.0;
            dispatch_time_t popTime = dispatch_time(DISPATCH_TIME_NOW, delayInSeconds * NSEC_PER_SEC);
            dispatch_after(popTime, dispatch_get_main_queue(), ^{
                NSLog(@"Adding prioritized operation");

                NSOperation *prioritizedOperation = [NSBlockOperation blockOperationWithBlock:^{
                    NSLog(@"Running prioritized operation");
                }];

                [prioritizedOperation setQueuePriority:NSOperationQueuePriorityVeryHigh];

                [queue addOperation:prioritizedOperation];
            });

            CFRunLoopRun();
        }

        return 0;
    }

    produces the following output:

    2012-06-02 09:51:14.780 test[2508:1e03] Running operation number 0
    2012-06-02 09:51:19.784 test[2508:1e03] Running operation number 1
    2012-06-02 09:51:24.779 test[2508:403] Adding prioritized operation
    2012-06-02 09:51:24.787 test[2508:1e03] Running prioritized operation
    2012-06-02 09:51:24.790 test[2508:1e03] Running operation number 2
    2012-06-02 09:51:29.793 test[2508:1e03] Running operation number 3
    2012-06-02 09:51:34.797 test[2508:1e03] Running operation number 4

    Trying this with some concurrency in place by changing maxConcurrentOperationCount to 5 and numberOfOperations to 25 results in this output:

    2012-06-02 09:55:04.995 test[2560:1e03] Running operation number 0
    2012-06-02 09:55:04.996 test[2560:1f03] Running operation number 1
    2012-06-02 09:55:05.003 test[2560:2003] Running operation number 2
    2012-06-02 09:55:05.003 test[2560:2503] Running operation number 3
    2012-06-02 09:55:05.007 test[2560:2c03] Running operation number 4
    2012-06-02 09:55:10.000 test[2560:2f03] Running operation number 5
    2012-06-02 09:55:10.005 test[2560:1e03] Running operation number 6
    2012-06-02 09:55:10.006 test[2560:2503] Running operation number 7
    2012-06-02 09:55:10.007 test[2560:2003] Running operation number 8
    2012-06-02 09:55:10.010 test[2560:2d03] Running operation number 9
    2012-06-02 09:55:14.992 test[2560:403] Adding prioritized operation
    2012-06-02 09:55:15.005 test[2560:2f03] Running prioritized operation
    2012-06-02 09:55:15.007 test[2560:2c07] Running operation number 10
    2012-06-02 09:55:15.008 test[2560:1f03] Running operation number 11
    2012-06-02 09:55:15.011 test[2560:1e03] Running operation number 12
    2012-06-02 09:55:15.012 test[2560:2003] Running operation number 13
    2012-06-02 09:55:15.013 test[2560:2d03] Running operation number 14
    2012-06-02 09:55:20.010 test[2560:2c07] Running operation number 15
    2012-06-02 09:55:20.012 test[2560:1f03] Running operation number 16
    2012-06-02 09:55:20.014 test[2560:2507] Running operation number 17
    2012-06-02 09:55:20.014 test[2560:2003] Running operation number 18
    2012-06-02 09:55:20.016 test[2560:1e03] Running operation number 19
    2012-06-02 09:55:25.014 test[2560:2c07] Running operation number 20
    2012-06-02 09:55:25.015 test[2560:2507] Running operation number 21
    2012-06-02 09:55:25.015 test[2560:1f03] Running operation number 22
    2012-06-02 09:55:25.018 test[2560:1e03] Running operation number 23
    2012-06-02 09:55:25.021 test[2560:2d03] Running operation number 24

    So, it would seem that setting the queue priority does seem to be capable of effectively bumping operations to the front of the queue like you want.

    Charles
  • On 6/2/12 4:57 PM, Charles Srstka wrote:
    > On Jun 1, 2012, at 10:23 PM, Graham Cox wrote:
    >
    >> On 02/06/2012, at 1:12 PM, Kyle Sluder wrote:
    >>
    >>> Give them a higher priority. You should be able to alter the priorities as the user scrolls, and NSOperationQueue will do the right thing.
    >>
    >>
    >> I tried this but it doesn't work - a bit of thought about how the ops are queued will show why no meaningful priority value can be assigned.
    >>
    >> At the moment that the operations are queued, there are some operations in the queue not yet run, and some running. The code that creates the operations doesn't know which ones are needed more urgently (the latest ones), so it can only assign a high priority to all of them, so they all end up with the same (high) priority and so we're back to square one.
    >
    > Setting the priority seems to work, in my testing:

    Prioritizing doesn't solve the LIFO problem the OP has. When you add a new
    prioritized operation you need to de-prioritize the ones that are already in the
    queue (in order to make sure your new operation executes first). Those
    de-prioritized operations execute in FIFO order.

    I had the same problem and ended up maintaining my own pool which adds the most
    recently added operation to the queue first (with only one operation in the
    queue at any time). It's ridiculously complicated, even a generic integer based
    priority would do. I fail to understand why someone thought it's a good idea to
    limit it to 5 distinct settings.

    Regards
    Markus
    --
    __________________________________________
    Markus Spoettl
  • On Jun 2, 2012, at 8:38 AM, Markus Spoettl <ms_lists...> wrote:

    > On 6/2/12 4:57 PM, Charles Srstka wrote:
    >> On Jun 1, 2012, at 10:23 PM, Graham Cox wrote:
    >>
    >>> On 02/06/2012, at 1:12 PM, Kyle Sluder wrote:
    >>>
    >>>> Give them a higher priority. You should be able to alter the priorities as the user scrolls, and NSOperationQueue will do the right thing.
    >>>
    >>>
    >>> I tried this but it doesn't work - a bit of thought about how the ops are queued will show why no meaningful priority value can be assigned.
    >>>
    >>> At the moment that the operations are queued, there are some operations in the queue not yet run, and some running. The code that creates the operations doesn't know which ones are needed more urgently (the latest ones), so it can only assign a high priority to all of them, so they all end up with the same (high) priority and so we're back to square one.
    >>
    >> Setting the priority seems to work, in my testing:
    >
    > Prioritizing doesn't solve the LIFO problem the OP has. When you add a new prioritized operation you need to de-prioritize the ones that are already in the queue (in order to make sure your new operation executes first). Those de-prioritized operations execute in FIFO order.

    I'm still not seeing the problem. Just set their queuePriority property. The order of operations is not fixed at the time of enqueueing, and NSOperation's properties are thread safe for a reason.

    --Kyle Sluder
  • On Jun 2, 2012, at 10:55 AM, Kyle Sluder wrote:

    > On Jun 2, 2012, at 8:38 AM, Markus Spoettl <ms_lists...> wrote:
    >
    >> On 6/2/12 4:57 PM, Charles Srstka wrote:
    >>> On Jun 1, 2012, at 10:23 PM, Graham Cox wrote:
    >>>
    >>>> On 02/06/2012, at 1:12 PM, Kyle Sluder wrote:
    >>>>
    >>>>> Give them a higher priority. You should be able to alter the priorities as the user scrolls, and NSOperationQueue will do the right thing.
    >>>>
    >>>>
    >>>> I tried this but it doesn't work - a bit of thought about how the ops are queued will show why no meaningful priority value can be assigned.
    >>>>
    >>>> At the moment that the operations are queued, there are some operations in the queue not yet run, and some running. The code that creates the operations doesn't know which ones are needed more urgently (the latest ones), so it can only assign a high priority to all of them, so they all end up with the same (high) priority and so we're back to square one.
    >>>
    >>> Setting the priority seems to work, in my testing:
    >>
    >> Prioritizing doesn't solve the LIFO problem the OP has. When you add a new prioritized operation you need to de-prioritize the ones that are already in the queue (in order to make sure your new operation executes first). Those de-prioritized operations execute in FIFO order.
    >
    > I'm still not seeing the problem. Just set their queuePriority property. The order of operations is not fixed at the time of enqueueing, and NSOperation's properties are thread safe for a reason.

    It’s even easier than that, since there’s no need to de-prioritize anything. You just have everything’s priority set to NSOperationQueuePriorityNormal at the time of enqueuing, and give NSOperationQueuePriorityHigh or NSOperationQueuePriorityVeryHigh to the ones that need to be bumped to the front. The run-of-the-mill operations can stay normal; the only ones you need to change the priority for are the ones you want to prioritize.

    The OP mentioned that this won’t work because he’s giving all the operations the highest priority level, but there’s really no reason to do so. Just give the normal operations normal priority.

    Charles
  • On Jun 2, 2012, at 9:04 AM, Charles Srstka <cocoadev...> wrote:

    >
    > It’s even easier than that, since there’s no need to de-prioritize anything. You just have everything’s priority set to NSOperationQueuePriorityNormal at the time of enqueuing, and give NSOperationQueuePriorityHigh or NSOperationQueuePriorityVeryHigh to the ones that need to be bumped to the front. The run-of-the-mill operations can stay normal; the only ones you need to change the priority for are the ones you want to prioritize.
    >
    > The OP mentioned that this won’t work because he’s giving all the operations the highest priority level, but there’s really no reason to do so. Just give the normal operations normal priority.

    The OP's point is that the priority of operations changes over time, based on where the user has scrolled: onscreen previews need to be generated before offscreen ones.

    The solution to changing priorities is… just change the priority.

    --Kyle Sluder
  • On 6/2/12 5:55 PM, Kyle Sluder wrote:
    > On Jun 2, 2012, at 8:38 AM, Markus Spoettl<ms_lists...>  wrote:
    >
    >> On 6/2/12 4:57 PM, Charles Srstka wrote:
    >>> On Jun 1, 2012, at 10:23 PM, Graham Cox wrote:
    >>>
    >>>> On 02/06/2012, at 1:12 PM, Kyle Sluder wrote:
    >>>>
    >>>>> Give them a higher priority. You should be able to alter the priorities as the user scrolls, and NSOperationQueue will do the right thing.
    >>>>
    >>>>
    >>>> I tried this but it doesn't work - a bit of thought about how the ops are queued will show why no meaningful priority value can be assigned.
    >>>>
    >>>> At the moment that the operations are queued, there are some operations in the queue not yet run, and some running. The code that creates the operations doesn't know which ones are needed more urgently (the latest ones), so it can only assign a high priority to all of them, so they all end up with the same (high) priority and so we're back to square one.
    >>>
    >>> Setting the priority seems to work, in my testing:
    >>
    >> Prioritizing doesn't solve the LIFO problem the OP has. When you add a new prioritized operation you need to de-prioritize the ones that are already in the queue (in order to make sure your new operation executes first). Those de-prioritized operations execute in FIFO order.
    >
    > I'm still not seeing the problem. Just set their queuePriority property. The order of operations is not fixed at the time of enqueueing, and NSOperation's properties are thread safe for a reason.

    I'm still not seeing how queuePriority would solve the issue. Say you have a
    queue with maxConcurrentOperationCount == 1

    1) You add operation (A), it starts executing

    2) You add operation (B), (C), (D), (E), (F), (G), (H) and (I) in that order.
    All wait

    Please explain how you suggest to use queuePriority in order to ensure the
    waiting operations will execute in the reverse order:

    (I), (H), (G), (F), (E), (D), (C), (B).

    Regards
    Markus
    --
    __________________________________________
    Markus Spoettl
  • On Jun 2, 2012, at 11:54 AM, Markus Spoettl wrote:

    > I'm still not seeing how queuePriority would solve the issue. Say you have a queue with maxConcurrentOperationCount == 1
    >
    > 1) You add operation (A), it starts executing
    >
    > 2) You add operation (B), (C), (D), (E), (F), (G), (H) and (I) in that order. All wait
    >
    > Please explain how you suggest to use queuePriority in order to ensure the waiting operations will execute in the reverse order:
    >
    > (I), (H), (G), (F), (E), (D), (C), (B).

    Well, you could have each operation, as its first action, set the last operation in the queue to have high priority.  If you don't like mixing that LIFO knowledge into the operation, you can have a separate operation class wrap another operation.  It would re-prioritize the next operation and then call through to the wrapped operation's -start method.

    That said, I don't know that OP requires strict LIFO order.  He just wants the operations associated with images which are scrolled into view to have higher priority than those which aren't.  That can just be accomplished by setting the priority of operations as their associated images are scrolled into and out of view.  The problem, as I understand it, is that IKImageView doesn't make it easy to detect that.

    Regards,
    Ken
  • On Jun 2, 2012, at 11:54 AM, Markus Spoettl wrote:

    > I'm still not seeing how queuePriority would solve the issue. Say you have a queue with maxConcurrentOperationCount == 1
    >
    > 1) You add operation (A), it starts executing
    >
    > 2) You add operation (B), (C), (D), (E), (F), (G), (H) and (I) in that order. All wait
    >
    > Please explain how you suggest to use queuePriority in order to ensure the waiting operations will execute in the reverse order:
    >
    > (I), (H), (G), (F), (E), (D), (C), (B).

    Who cares if they execute in reverse order? Pedantry like the exact order that the operations execute in is not important in practice. The important issue here is that some operations — i.e. loading the objects on screen — need to have a higher priority than other operations — i.e. loading objects that aren’t on screen. Giving the important operations a high priority solves this quite well.

    If A still executing is a problem (I think it wouldn’t be unless the operation was particularly lengthy), then just cancel and re-add A, maybe giving A some way of saving and restoring its state.

    Charles
  • On Jun 2, 2012, at 9:54 AM, Markus Spoettl <ms_lists...> wrote:

    >
    > I'm still not seeing how queuePriority would solve the issue. Say you have a queue with maxConcurrentOperationCount == 1

    OP expressed his desire to have a queue with maxConcurrentOperationCount > 1.

    >
    > Please explain how you suggest to use queuePriority in order to ensure the waiting operations will execute in the reverse order:

    Who said anything about order?
  • On Jun 2, 2012, at 11:54 AM, Markus Spoettl wrote:

    > I'm still not seeing how queuePriority would solve the issue. Say you have a queue with maxConcurrentOperationCount == 1
    >
    > 1) You add operation (A), it starts executing
    >
    > 2) You add operation (B), (C), (D), (E), (F), (G), (H) and (I) in that order. All wait
    >
    > Please explain how you suggest to use queuePriority in order to ensure the waiting operations will execute in the reverse order:
    >
    > (I), (H), (G), (F), (E), (D), (C), (B).

    Thinking about this a little more, if you *do* want to execute your operations in LIFO order, you can do that pretty easily as well using dependencies:

    [B addDependency:C];
    [C addDependency:D];
    .
    .
    [H addDependency:I];

    Each time you add an operation to the queue, make it a dependency of the last operation you added that isn’t running yet — this will ensure LIFO order. I wouldn’t do this for the problem the OP is trying to solve, though.

    Charles
  • On 6/2/12 7:27 PM, Kyle Sluder wrote:
    > On Jun 2, 2012, at 9:54 AM, Markus Spoettl<ms_lists...>  wrote:
    >
    >>
    >> I'm still not seeing how queuePriority would solve the issue. Say you have a queue with maxConcurrentOperationCount == 1
    >
    > OP expressed his desire to have a queue with maxConcurrentOperationCount>  1.

    Well, maybe, it doesn't change anything. maxConcurrentOperationCount = 1 just
    makes it easier to explain things.

    >>
    >> Please explain how you suggest to use queuePriority in order to ensure the waiting operations will execute in the reverse order:
    >
    > Who said anything about order?

    The OP did:

    > Is there a way to add operations to the queue so that they "jump the queue" ahead of any operations that are not running and have not been run yet?

    Markus
    --
    __________________________________________
    Markus Spoettl
  • On Jun 2, 2012, at 12:32 PM, Markus Spoettl wrote:

    >> Who said anything about order?
    >
    > The OP did:
    >
    >> Is there a way to add operations to the queue so that they "jump the queue" ahead of any operations that are not running and have not been run yet?

    That statement isn’t expressing a desire for strict LIFO order; he just wants some operations to jump ahead of the rest. This can be done by giving those operations a higher priority.

    If I have eight operations A, B, C, D, E, F, G, H, and I want A, B and C to run first (because they correspond to on-screen objects), then I don’t really care if the order is:

    ABCDEFGH
    CBAHGFED
    BACFDHEG

    etc.

    These orders are all equivalent in practical terms. What’s important is that A, B, and C get finished before D, E, F, G, and H.

    Charles
  • On 03/06/2012, at 3:03 AM, Ken Thomases wrote:

    > That said, I don't know that OP requires strict LIFO order.  He just wants the operations associated with images which are scrolled into view to have higher priority than those which aren't.  That can just be accomplished by setting the priority of operations as their associated images are scrolled into and out of view.  The problem, as I understand it, is that IKImageView doesn't make it easy to detect that.

    This.

    The problem is not that setting the priority of an operation doesn't work, it's that manipulating the priority based on what is visible at a given moment is hard.

    The exact order of execution is unimportant - but it's "nicer" if what's actually visible right now is given a higher priority than others that might be queued but have scrolled out of view. Having a LIFO execution order helps somewhat with that.

    With IKImageBrowserView, the view requests an item from its dataSource, presumably because it has become visible (though it seems it asks for others out of view as well, by some logic of its own - perhaps it tries to fill in as many as it can for future display when things are otherwise idle - I can't really tell). This request can kick off an asynchronous request to generate the image.

    Because of queuing these requests, and the relatively long time they take to run, by the time the actual task to generate the thumbnail comes to execute, it may no longer be needed, or can be given a lower priority. This could be because the user has scrolled the view. The difficulty is that the code responsible for queuing and generating the thumbnails has no knowledge of the view that asked for them to be created. However, it does have a delegate, so I've extended the delegate protocol so that it can ask it "do you still need this?" just before it executes. The delegate can return yes or no based on its visibility within the view, and the task is either discarded if it hasn't run yet and it isn't needed, or its priority can be bumped up if it is.  This works, but is becoming somewhat complex, but so far I haven't managed to think of a better/cleaner/more elegant solution.

    --Graham
  • On Jun 2, 2012, at 8:16 PM, Graham Cox wrote:

    > The problem is not that setting the priority of an operation doesn't work, it's that manipulating the priority based on what is visible at a given moment is hard.

    How is it hard? You just call setQueuePriority: on the operations for the visible objects!

    Charles
  • On 03/06/2012, at 4:13 PM, Charles Srstka wrote:

    > On Jun 2, 2012, at 8:16 PM, Graham Cox wrote:
    >
    >> The problem is not that setting the priority of an operation doesn't work, it's that manipulating the priority based on what is visible at a given moment is hard.
    >
    > How is it hard? You just call setQueuePriority: on the operations for the visible objects!

    define "the visible objects" when what you have at hand is not a visual object at all, but an object whose job is to make an image from some data.

    Clearly those who think this is trivial or easy have not actually tried doing this, and are not understanding what the problem is here. Of course it's easy to call -setQueuePriority:, that isn't the problem. The problem is how to decide what it should be set to, when.

    --Graham
  • On Jun 2, 2012, at 11:23 PM, Graham Cox <graham.cox...> wrote:

    >
    > define "the visible objects" when what you have at hand is not a visual object at all, but an object whose job is to make an image from some data

    If IKImageView's architecture is proving to be a problem, perhaps it's time to consider ditching IKImageView.

    >
    > Clearly those who think this is trivial or easy have not actually tried doing this, and are not understanding what the problem is here. Of course it's easy to call -setQueuePriority:, that isn't the problem. The problem is how to decide what it should be set to, when.

    Well, since we implemented it ourselves, our iPad document picker, which prioritized previews of visible items, works quite well.

    --Kyle Sluder
  • On 03/06/2012, at 4:54 PM, Kyle Sluder wrote:
    > If IKImageView's architecture is proving to be a problem, perhaps it's time to consider ditching IKImageView.

    I'm certainly going to give that some serious thought.

    In fact my scheme is now working fairly well, at least when I run it from Xcode. But run it directly from Finder (either as a debug or a release build) and it gets into all sorts of trouble, which is most puzzling. Deep inside the IKImageBrowserView I get this crash:

    Crashed Thread:  8

    Exception Type:  EXC_CRASH (SIGABRT)
    Exception Codes: 0x0000000000000000, 0x0000000000000000

    Application Specific Information:
    objc[2081]: garbage collection is OFF
    Assertion failed: (!"CantHappenCB"), function CantHappenCB, file /private/var/tmp/PSNormalizerCore/PSNormalizerCore-10.8.2~1/packages/produtil/sources/psrip.c, line 231.

    Thread 8 Crashed:
    0  libsystem_kernel.dylib           0x00007fff8700ece2 __pthread_kill + 10
    1  libsystem_c.dylib                 0x00007fff882517d2 pthread_kill + 95
    2  libsystem_c.dylib                 0x00007fff88242a7a abort + 143
    3  libsystem_c.dylib                 0x00007fff882755de __assert_rtn + 146
    4  libnserverlite.dylib             0x000000010c351d52 0x10c34a000 + 32082
    5  libnserverlite.dylib             0x000000010c557b8d 0x10c34a000 + 2153357
    6  libnserverlite.dylib             0x000000010c53a36c 0x10c34a000 + 2032492
    7  libnserverlite.dylib             0x000000010c350522 0x10c34a000 + 25890
    8  libnserverlite.dylib             0x000000010c51c8e3 0x10c34a000 + 1911011
    9  libnserverlite.dylib             0x000000010c51bd88 0x10c34a000 + 1908104
    10  libnserverlite.dylib             0x000000010c51b6f2 0x10c34a000 + 1906418
    11  libnserverlite.dylib             0x000000010c351fc9 0x10c34a000 + 32713
    12  libnserverlite.dylib             0x000000010c34b724 NormalizerServerInit + 1512
    13  com.apple.normalizer.privateframework    0x0000000106e74ba3 doNormalizeWithMutex + 3588
    14  com.apple.normalizer.privateframework    0x0000000106e74faa normalize + 200
    15  com.apple.CoreGraphics           0x00007fff872bde14 sync_convert + 462
    16  com.apple.CoreGraphics           0x00007fff8727ac89 CGPSConverterConvert + 11
    17  com.apple.AppKit                 0x00007fff8c69b31f -[NSEPSImageRep initWithData:] + 297
    18  com.apple.AppKit                 0x00007fff8c69adf5 +[NSEPSImageRep imageRepWithData:] + 47
    19  com.apple.AppKit                 0x00007fff8c33c2be +[NSImageRep _imageRepsWithContentsOfURL:expandImageContentNow:giveUpOnNetworkURLsWithoutGoodExtensions:] + 998
    20  com.apple.AppKit                 0x00007fff8c33be9a __-[NSImageURLReferencingRepProvider representations]_block_invoke_1 + 48
    21  com.apple.Foundation             0x00007fff8f2d5347 _NSFaultInObject + 35
    22  com.apple.AppKit                 0x00007fff8c30cae7 -[NSImageURLReferencingRepProvider representations] + 80
    23  com.apple.AppKit                 0x00007fff8c33bd7e __-[NSImage size]_block_invoke_1 + 353
    24  com.apple.AppKit                 0x00007fff8c2c7825 -[NSImage _usingRepProviderPerformBlock:] + 37
    25  com.apple.AppKit                 0x00007fff8c2f56c2 -[NSImage size] + 129
    26  com.apple.AppKit                 0x00007fff8c6fe681 -[NSImage isValid] + 149
    27  com.apple.imageKit               0x00007fff8ef281fa -[NSImage(IKAdditions) IKFixDPI] + 31
    28  com.apple.imageKit               0x00007fff8ef2836a -[NSImage(IKAdditions) IKSize] + 35
    29  com.apple.imageKit               0x00007fff8ef2cce8 -[IKImageWrapper _size] + 1666
    30  com.apple.imageKit               0x00007fff8ef2c649 -[IKImageWrapper size] + 140
    31  com.apple.imageKit               0x00007fff8f020b93 -[IKThumbnailBuilder computeThumbnail] + 75
    32  com.apple.imageKit               0x00007fff8ef4eb08 -[IKImageBrowserView(ImageBrowserImport) startScrollPrefetchTask] + 629
    33  com.apple.CoreFoundation         0x00007fff8ac9cfb1 -[NSObject performSelector:] + 49
    34  com.apple.imageKit               0x00007fff8efeeddd -[IKTaskManager taskLoop] + 776
    35  com.apple.Foundation             0x00007fff8f2f272a -[NSThread main] + 68
    36  com.apple.Foundation             0x00007fff8f2f26a2 __NSThread__main__ + 1575
    37  libsystem_c.dylib                 0x00007fff8824f8bf _pthread_start + 335
    38  libsystem_c.dylib                 0x00007fff88252b75 thread_start + 13

    This is for items which are not even part of my scheme, but are returned as a simple path to the image - in this case an EPS image. The assertion suggests "it can't happen", but it does, every bloody time. It never happens when I run it with the debugger, so it's proving exceedingly hard to do anything about.

    > Well, since we implemented it ourselves, our iPad document picker, which prioritized previews of visible items, works quite well.

    I have another home-rolled view which does something similar and it works fine as well. It's just the ImageKit class that's a bitch and a half. The only reason I'm persisting with it is that for all the OTHER image types (notwithstanding the above crash), it handles it all for you. Replacing it is a big job.

    --Graham
  • On Jun 3, 2012, at 1:23 AM, Graham Cox wrote:

    > define "the visible objects" when what you have at hand is not a visual object at all, but an object whose job is to make an image from some data.

    I’ll go with whatever you meant when you said:

    On Jun 2, 2012, at 12:34 AM, Graham Cox wrote:

    > When the image arrives, I am able to check if it's still visible and refresh the given cell of the view

    Since you’ve evidently already figured out a way to determine what images are visible in the view, just set up an NSMapTable or an NSDictionary mapping image URLs to operations, and then you know which operations correspond to the visible images. Prioritize those.

    Although, given what you’ve said thus far, I’m beginning to think that Kyle may be right and that IKImageBrowserView might not be the best tool for what you’re trying to do.

    Charles
  • On 03/06/2012, at 5:29 PM, Charles Srstka wrote:

    > Since you’ve evidently already figured out a way to determine what images are visible in the view, just set up an NSMapTable or an NSDictionary mapping image URLs to operations, and then you know which operations correspond to the visible images. Prioritize those.

    That's still not the problem.

    The problem is you do not get, from moment to moment, information from the view that the visible items have changed. Given such a change, there are numerous ways I could go through the operations and reprioritise them.

    Right now, what I actually do is to hold the operations in a queue of my own, and only feed them to the actual NSOperationQueue when a timer fires and the queue has some capacity. At that point I am able to reprioritise the pending operations (including those already queued but not yet executing). But I have to do it by effectively polling through them with a timer, rather than doing it on some sort of notification that the view has moved, which is a bit kludgey.

    >
    > Although, given what you’ve said thus far, I’m beginning to think that Kyle may be right and that IKImageBrowserView might not be the best tool for what you’re trying to do.
    >

    Indeed. I have turned off my SVG stuff altogether and I still get the deep crash. I cut down the number of acceptable image types to just JPEG, PNG, and PDF and I don't get the crash, but I get new weird stuff happening, e.g. suddenly I get:

    -[NSWindowController loadWindow]: failed to load window nib file 'MainDocument'.

    after scrolling around the IKImageBrowserView for a while and I try to open a new document. This smells to me of memory corruption, since otherwise these two parts of the app are unrelated, and for these image types, I defer 100% to Cocoa's implementation.

    I think the evidence points to IKImageBrowserView still being buggy even after all this time, so I think I'm going to have to ditch it, if only to obtain some stability in my app and some sanity in myself.

    --Graham
previous month june 2012 next month
MTWTFSS
        1 2 3
4 5 6 7 8 9 10
11 12 13 14 15 16 17
18 19 20 21 22 23 24
25 26 27 28 29 30  
Go to today