Best practice for Core Data "ordered" NSArray problem?

  • Sorry to everybody who is sick of this question.. I know it's been
    asked before but I'm wondering whether there's any news on this
    subject..

    Core Data uses a relational database back-end, relational databases
    do not have a "natural" order, therefore the results returned by a
    query are unsorted unless you use a sorting descriptor. Creating an
    actual array (you now where things are in sequence irrespective of an
    external sorting order) thus means adding an index to the model,
    which makes life difficult for NSArrayController and demands lots of
    custom coding, etc..

    As this is a very common problem, I assume that there are now some
    mature solutions out there to deal with this generically?

    Does anybody know of a generic NSArrayController that can deal with
    the internal index issue, row re-ordering, etc?

    Something I don't know yet is whether this problem is specific to
    using the SQLite store. If I use an xml store or an in-memory store
    is the order guaranteed to be preserved?

    Thanks for your patience..

    Best regards,

    Frank
  • On Sep 28, 2007, at 8:31 AM, Frank Reiff wrote:

    > Sorry to everybody who is sick of this question.. I know it's been
    > asked before but I'm wondering whether there's any news on this
    > subject..
    >
    > Core Data uses a relational database back-end, relational databases
    > do not have a "natural" order, therefore the results returned by a
    > query are unsorted unless you use a sorting descriptor. Creating an
    > actual array (you now where things are in sequence irrespective of
    > an external sorting order) thus means adding an index to the model,
    > which makes life difficult for NSArrayController and demands lots of
    > custom coding, etc..

    I asked this same question through another source at Apple not too
    long ago.

    > As this is a very common problem, I assume that there are now some
    > mature solutions out there to deal with this generically?

    Actually this is probably not the case.  The difficulty (as I
    understand it) is that while there are many, many people who want to
    have ordered collections in their databases, each program has subtle
    differences in it's requirements that make this a difficult problem to
    solve in a generic fashion.  For example, my application likes to
    order small groups of objects (say 20 at the most) while another
    program might need to order 100,000 objects.  Each of these two
    applications might use a different strategy.

    > Something I don't know yet is whether this problem is specific to
    > using the SQLite store. If I use an xml store or an in-memory store
    > is the order guaranteed to be preserved?

    You're thinking about this the wrong way, in my opinion.  The idea is
    that a Core Data object graph does not have an intrinsic mechanism for
    managing ordered collections of objects in Tiger. How that object
    graph is stored is completely irrelevant.  In other words, it doesn't
    matter if the store is SQLite, XML, in-memory, or recorded on papyrus
    by trained squirrels.  The fact that that you have to provide your own
    ordering mechanism doesn't change.

    Scott
  • On 2007 Sep, 28, at 6:31, Frank Reiff wrote:

    > As this is a very common problem, I assume that there are now some
    > mature solutions out there to deal with this generically?

    I'm working on this now, but one lesson I've learned so far is to try
    and use Core Data "as is" if possible.  In other words, at each
    interface where you think you need an (ordered) array, consider
    carefully if you can somehow use the Core Data set-with-position-
    attribute instead.
  • > Creating an
    > actual array (you now where things are in sequence irrespective of an
    > external sorting order) thus means adding an index to the model,
    > which makes life difficult for NSArrayController and demands lots of
    > custom coding, etc..

      I'm not sure I agree with this assessment. Whether or not you have
    an "index" attribute in your model doesn't really make anything more
    "difficult" for an NSArrayController than not using Core Data at all.
    In either case, you set the sort descriptor and you're done (for
    simple retrieval / display).

      The controller layer's job is to apply the logic when drag and drop
    reordering occurs, which takes exactly as much work for a Core Data
    app as a non-Core Data app with Bindings. There are several very
    well-written examples on the web  of how to accomplish this with a
    table view. It's a bit obtuse with an outline view and
    NSTreeController, granted, but you don't appear to be using either.

      The bottom line is, the clearest route is a "sortOrder" attribute
    and some (relatively basic) custom controller code and, speaking from
    experience, it works perfectly.

    --
    I.S.
  • The basic fact is that an array is an array and a set is a set  Sure,
    the implementation could vary, depending on performance constraints.  What
    he (and others) have asked for is any solution (or reference to a solution).
    A lot of us would like to see it.  I haven't tackled core data or BigTen
    yet, so I don't know what is involved.  I just know I'm going to have to
    crack that nut sometime soon.  Fortunately, my current stuff is mostly in
    the small array ( <100 ) category.

    > Actually this is probably not the case.  The difficulty (as I
    > understand it) is that while there are many, many people who want to
    > have ordered collections in their databases, each program has subtle
    > differences in it's requirements that make this a difficult problem to
    > solve in a generic fashion.  For example, my application likes to
    > order small groups of objects (say 20 at the most) while another
    > program might need to order 100,000 objects.  Each of these two
    > applications might use a different strategy.
    >
  • Hi,

    Thanks for your comments. Hand-coding is the only way then..

    Of course, Core Data is neither the first nor  the only object
    relational mapping tool and the ordering problem is the same for all
    implementations.

    In my previous job, I used the Hibernate (open source Java) framework
    and that had a very elegant way of doing this; when you have a set,
    you define your attribute as a java.util.Set and if you have an
    array, you define it as a java.util.List. In the case of a list, you
    can define the mapping in a variety of ways (including the lame "I'm
    a set impersonating a list" mode of Core Data). The most obvious one
    is to simply tell Hibernate what the order-by-attribute field is and
    it does the rest.. sounds simple enough? If you are really worried
    about performance (say you have a large list), you do a bit of
    dynamic configuration, plug in your own policy/ strategy class and
    you're done.

    It mystifies me why Apple can't do the same thing. In fact I was so
    SURE that this was merely an oversight (not ready in time) in Tiger
    and that it would be the brand new ueber-feature of Core Data in
    Leopard... looks like I'm wrong again..

    Seriously, it's possible to do this generically. No it won't perform
    miracles performance-wise, but on today's machines it should scale to
    a couple of thousand records without becoming noticeably slow. If
    Java can do it, so can Cocoa.

    For my particular example, nothing else but an actual array will do:
    I'm storing the order in which particular operations are to be
    carried out.

    In my humble opinion, arrays (or rather ordered lists) are not a sign
    of bad design; they are a common feature of many programming tasks
    and not supporting them directly is a sign of an incomplete
    framework, not faulty thinking on the part of the framework users.

    It is Apple who have got it wrong. They seem to have started with the
    relational database and then build an object model on top; kind of a
    "relational object mapping" framework rather than a "object
    relational mapping" framework. At least concerning this particular
    feature.

    Gives us arrays, damn it!

    Best regards,

    Frank

    > The basic fact is that an array is an array and a set is a set
    > Sure,
    > the implementation could vary, depending on performance
    > constraints.  What
    > he (and others) have asked for is any solution (or reference to a
    > solution).
    > A lot of us would like to see it.  I haven't tackled core data or
    > BigTen
    > yet, so I don't know what is involved.  I just know I'm going to
    > have to
    > crack that nut sometime soon.  Fortunately, my current stuff is
    > mostly in
    > the small array ( <100 ) category.
  • Hi,

    Given that I have offended at least one subscriber with my flippant
    "damn", I've probably offended lots more.. sorry about that.

    It was supposed to be a light hearted "hot d*mn" and wasn't supposed
    to offend anybody. In future, I'll stick to most decidedly sober and
    appropriate language only.

    Best regards,

    Frank

    On 1 Oct 2007, at 18:58, Frank Reiff wrote:

    > Hi,
    >
    > Thanks for your comments. Hand-coding is the only way then..
    >
    > Of course, Core Data is neither the first nor  the only object
    > relational mapping tool and the ordering problem is the same for
    > all implementations.
    >
    > In my previous job, I used the Hibernate (open source Java)
    > framework and that had a very elegant way of doing this; when you
    > have a set, you define your attribute as a java.util.Set and if you
    > have an array, you define it as a java.util.List. In the case of a
    > list, you can define the mapping in a variety of ways (including
    > the lame "I'm a set impersonating a list" mode of Core Data). The
    > most obvious one is to simply tell Hibernate what the order-by-
    > attribute field is and it does the rest.. sounds simple enough? If
    > you are really worried about performance (say you have a large
    > list), you do a bit of dynamic configuration, plug in your own
    > policy/ strategy class and you're done.
    >
    > It mystifies me why Apple can't do the same thing. In fact I was so
    > SURE that this was merely an oversight (not ready in time) in Tiger
    > and that it would be the brand new ueber-feature of Core Data in
    > Leopard... looks like I'm wrong again..
    >
    > Seriously, it's possible to do this generically. No it won't
    > perform miracles performance-wise, but on today's machines it
    > should scale to a couple of thousand records without becoming
    > noticeably slow. If Java can do it, so can Cocoa.
    >
    > For my particular example, nothing else but an actual array will
    > do: I'm storing the order in which particular operations are to be
    > carried out.
    >
    > In my humble opinion, arrays (or rather ordered lists) are not a
    > sign of bad design; they are a common feature of many programming
    > tasks and not supporting them directly is a sign of an incomplete
    > framework, not faulty thinking on the part of the framework users.
    >
    > It is Apple who have got it wrong. They seem to have started with
    > the relational database and then build an object model on top; kind
    > of a "relational object mapping" framework rather than a "object
    > relational mapping" framework. At least concerning this particular
    > feature.
    >
    > Gives us arrays, damn it!
    >
    > Best regards,
    >
    > Frank
    >
    >
    >> The basic fact is that an array is an array and a set is a
    >> set  Sure,
    >> the implementation could vary, depending on performance
    >> constraints.  What
    >> he (and others) have asked for is any solution (or reference to a
    >> solution).
    >> A lot of us would like to see it.  I haven't tackled core data or
    >> BigTen
    >> yet, so I don't know what is involved.  I just know I'm going to
    >> have to
    >> crack that nut sometime soon.  Fortunately, my current stuff is
    >> mostly in
    >> the small array ( <100 ) category.

previous month september 2007 next month
MTWTFSS
          1 2
3 4 5 6 7 8 9
10 11 12 13 14 15 16
17 18 19 20 21 22 23
24 25 26 27 28 29 30
Go to today