CoreData Indexing is it possible?

  • Dear Cocoa Gurus,    first I have to say that Core Data is awesome :-)

    When I took my Comp. Science class about database they taught me that by
    building an index on a specific table attribute the search can
    be dramatically speeded up (if the search involves that attribute).

    Is there a way to do the same think with a core data database? I have a
    table with 10 attributes, but the user can search only by using one. It
    would be great to optimize the table for the retrieval of that specific
    attribute. Is that possible?

    Thank you!

    Giovanni Donelli
  • well, if you use sqlite as a coredata backend, that is certainly
    possible to do (at least by using sqlite-native calls, not sure if
    coredata has native interface for that)

    On 9/16/07, Giovanni Donelli <giovanni.donelli...> wrote:
    > Dear Cocoa Gurus,    first I have to say that Core Data is awesome :-)
    >
    > When I took my Comp. Science class about database they taught me that by
    > building an index on a specific table attribute the search can
    > be dramatically speeded up (if the search involves that attribute).
    >
    > Is there a way to do the same think with a core data database? I have a
    > table with 10 attributes, but the user can search only by using one. It
    > would be great to optimize the table for the retrieval of that specific
    > attribute. Is that possible?
    >
    > Thank you!
    >
    > Giovanni Donelli
    >

    --
    Alexey Zakhlestin
    http://blog.milkfarmsoft.com/
  • It is possible to add a index using the Model Designer: create an
    attribute key called "_NSCreateIndex" without a value for the
    attribute for which you want the index to be created.

    Some remarks:

    - Inserts and saves might become slower as the index has be to
    maintained.
    - Don't expect noticeable improvements for data sets under 200k objects.

      Kind regards,

    Diederik

    --
    Diederik Hoogenboom
    Obvious Matter - DiskLibrary for Mac
    http://www.obviousmatter.com

    On 17-sep-2007, at 12:58, Alexey Zakhlestin wrote:

    > well, if you use sqlite as a coredata backend, that is certainly
    > possible to do (at least by using sqlite-native calls, not sure if
    > coredata has native interface for that)
    >
    > On 9/16/07, Giovanni Donelli <giovanni.donelli...> wrote:
    >> Dear Cocoa Gurus,    first I have to say that Core Data is
    >> awesome :-)
    >>
    >> When I took my Comp. Science class about database they taught me
    >> that by
    >> building an index on a specific table attribute the search can
    >> be dramatically speeded up (if the search involves that attribute).
    >>
    >> Is there a way to do the same think with a core data database? I
    >> have a
    >> table with 10 attributes, but the user can search only by using
    >> one. It
    >> would be great to optimize the table for the retrieval of that
    >> specific
    >> attribute. Is that possible?
    >>
    >> Thank you!
    >>
    >> Giovanni Donelli
    >>
    >
    >
    > --
    > Alexey Zakhlestin
    > http://blog.milkfarmsoft.com/
    >
  • That's great! Thank you for your reply. Is that a documented functionality?
    I have google it and it seems there is no documentation on it, is it a
    private API?

    once again, thank you!
    Giovanni

    On 9/17/07, Diederik Hoogenboom <mailings...> wrote:
    >
    > It is possible to add a index using the Model Designer: create an
    > attribute key called "_NSCreateIndex" without a value for the attribute for
    > which you want the index to be created.
    >
    > Some remarks:
    >
    >
    > - Inserts and saves might become slower as the index has be to
    > maintained.
    > - Don't expect noticeable improvements for data sets under 200k objects.
    >
    >
    > Kind regards,
    >
    >
    > Diederik
    >
    >
    > --
    > Diederik Hoogenboom
    > *Obvious Matter - DiskLibrary for Mac*
    > http://www.obviousmatter.com
    >
    >
    >
    >
    > On 17-sep-2007, at 12:58, Alexey Zakhlestin wrote:
    >
    > well, if you use sqlite as a coredata backend, that is certainly
    > possible to do (at least by using sqlite-native calls, not sure if
    > coredata has native interface for that)
    >
    >
    > On 9/16/07, Giovanni Donelli <giovanni.donelli...> wrote:
    >
    > Dear Cocoa Gurus,    first I have to say that Core Data is awesome :-)
    >
    >
    > When I took my Comp. Science class about database they taught me that by
    > building an index on a specific table attribute the search can
    > be dramatically speeded up (if the search involves that attribute).
    >
    >
    > Is there a way to do the same think with a core data database? I have a
    > table with 10 attributes, but the user can search only by using one. It
    > would be great to optimize the table for the retrieval of that specific
    > attribute. Is that possible?
    >
    >
    > Thank you!
    >
    >
    > Giovanni Donelli
    >
    >
    >
    >
    >
    >
    >
    > --
    > Alexey Zakhlestin
    > http://blog.milkfarmsoft.com/
    >
    >
    >
    >
    >
  • AFAIK it is not mentioned in any public documentation. It has been
    mentioned once or twice on this list though.
    It is not really an API; it is more a "tag" in your model that will
    trigger the index to be created when creating your store.

    Diederik

    --
    Diederik Hoogenboom
    Obvious Matter - DiskLibrary for Mac
    http://www.obviousmatter.com

    On 17-sep-2007, at 23:33, Giovanni Donelli wrote:

    > That's great! Thank you for your reply. Is that a documented
    > functionality?
    >
    > I have google it and it seems there is no documentation on it, is
    > it a private API?
    >
    > once again, thank you!
    > Giovanni
    >
    >
    > On 9/17/07, Diederik Hoogenboom < <mailings...> wrote:
    > It is possible to add a index using the Model Designer: create an
    > attribute key called "_NSCreateIndex" without a value for the
    > attribute for which you want the index to be created.
    >
    >
    > Some remarks:
    >
    >
    > - Inserts and saves might become slower as the index has be to
    > maintained.
    > - Don't expect noticeable improvements for data sets under 200k
    > objects.
    >
    >
    > Kind regards,
    >
    >
    > Diederik
    >
    >
    > --
    > Diederik Hoogenboom
    > Obvious Matter - DiskLibrary for Mac
    > http://www.obviousmatter.com
    >
    >
    >
    >
    > On 17-sep-2007, at 12:58, Alexey Zakhlestin wrote:
    >
    >> well, if you use sqlite as a coredata backend, that is certainly
    >> possible to do (at least by using sqlite-native calls, not sure if
    >> coredata has native interface for that)
    >>
    >>
    >> On 9/16/07, Giovanni Donelli < <giovanni.donelli...> wrote:
    >>> Dear Cocoa Gurus,    first I have to say that Core Data is
    >>> awesome :-)
    >>>
    >>>
    >>> When I took my Comp. Science class about database they taught me
    >>> that by
    >>> building an index on a specific table attribute the search can
    >>> be dramatically speeded up (if the search involves that attribute).
    >>>
    >>>
    >>> Is there a way to do the same think with a core data database? I
    >>> have a
    >>> table with 10 attributes, but the user can search only by using
    >>> one. It
    >>> would be great to optimize the table for the retrieval of that
    >>> specific
    >>> attribute. Is that possible?
    >>>
    >>>
    >>> Thank you!
    >>>
    >>>
    >>> Giovanni Donelli
    >>>
    >>>
    >>
    >>
    >>
    >>
    >> --
    >> Alexey Zakhlestin
    >> http://blog.milkfarmsoft.com /
    >>
    >>
    >
    >
    >
  • On Sep 18, 2007, at 1:28 AM, Diederik Hoogenboom wrote:

    > AFAIK it is not mentioned in any public documentation. It has been
    > mentioned once or twice on this list though.
    > It is not really an API; it is more a "tag" in your model that will
    > trigger the index to be created when creating your store.
    >

      That's not really the point, though. It's undocumented and starts
    with an underscore (Apple's convention for "private"), therefore it's
    subject to change and can break your application (or, in this case,
    *potentially* corrupt your users' documents) in future versions of OS X.

      I would steer *well clear* of this approach without official
    support from Apple.

    --
    I.S.
  • on 2007-09-18 7:03 AM, I. Savant at <idiotsavant2005...> wrote:

    > On Sep 18, 2007, at 1:28 AM, Diederik Hoogenboom wrote:
    >
    >> AFAIK it is not mentioned in any public documentation. It has been
    >> mentioned once or twice on this list though.
    >> It is not really an API; it is more a "tag" in your model that will
    >> trigger the index to be created when creating your store.
    >>
    >
    > That's not really the point, though. It's undocumented and starts
    > with an underscore (Apple's convention for "private"), therefore it's
    > subject to change and can break your application (or, in this case,
    > *potentially* corrupt your users' documents) in future versions of OS X.
    >
    > I would steer *well clear* of this approach without official
    > support from Apple.

    I implemented indexes a couple of years ago in a demo core data app. You can
    download the project with source from
    <http://www.quecheesoftware.com/downloads/WareroomDemo.html>.

    Maintenance of indexes when deleting objects in this sample is very simple
    and will not scale well.

    --

    Bill Cheeseman - <bill...>
    Quechee Software, Quechee, Vermont, USA
    www.quecheesoftware.com

    PreFab Software - www.prefabsoftware.com
  • Hi Bill, thanks for the source code.
    Before I dig into it, could you please tell me whether you used
    the _NSCreateIndex call, or you implemented yourself a different mechanism
    to index a db. If so what is it based on? Just a general idea, I will then
    look up at the details on your webpage

    Thank you very much,
    Giovanni

    On 9/18/07, Bill Cheeseman <bill...> wrote:
    >
    > on 2007-09-18 7:03 AM, I. Savant at <idiotsavant2005...> wrote:
    >
    >> On Sep 18, 2007, at 1:28 AM, Diederik Hoogenboom wrote:
    >>
    >>> AFAIK it is not mentioned in any public documentation. It has been
    >>> mentioned once or twice on this list though.
    >>> It is not really an API; it is more a "tag" in your model that will
    >>> trigger the index to be created when creating your store.
    >>>
    >>
    >> That's not really the point, though. It's undocumented and starts
    >> with an underscore (Apple's convention for "private"), therefore it's
    >> subject to change and can break your application (or, in this case,
    >> *potentially* corrupt your users' documents) in future versions of OS X.
    >>
    >> I would steer *well clear* of this approach without official
    >> support from Apple.
    >
    > I implemented indexes a couple of years ago in a demo core data app. You
    > can
    > download the project with source from
    > <http://www.quecheesoftware.com/downloads/WareroomDemo.html>.
    >
    > Maintenance of indexes when deleting objects in this sample is very simple
    > and will not scale well.
    >
    > --
    >
    > Bill Cheeseman - <bill...>
    > Quechee Software, Quechee, Vermont, USA
    > www.quecheesoftware.com
    >
    > PreFab Software - www.prefabsoftware.com
    >
  • on 2007-09-18 9:05 AM, Giovanni Donelli at <giovanni.donelli...> wrote:

    > Before I dig into it, could you please tell me whether you used the
    > _NSCreateIndex call, or you implemented yourself a different mechanism to
    > index a db. If so what is it based on? Just a general idea, I will then look
    > up at the details on your webpage

    I did NOT use _NSCreateIndex. (I don't know if it existed two years ago,
    when I wrote this demo, but I certainly wasn't aware of it.)

    I implemented indexes in the demo only because the demo is really about how
    to make a Core Data application scriptable. AppleScript generally expects
    objects to be indexed (although this certainly isn't necessary to
    AppleScript). For that reason, I didn't have any special need for the
    indexes to reflect any particular aspect of the data. I was therefore able
    to pick a very simple protocol, which was to increment the last-index
    counter every time a new object was added. On deletion, I decrement all
    indexes beyond the deleted item, which obviously won't scale well.

    --

    Bill Cheeseman - <bill...>
    Quechee Software, Quechee, Vermont, USA
    www.quecheesoftware.com

    PreFab Software - www.prefabsoftware.com
  • On Sep 18, 2007, at 12:08 PM, Aaron Burghardt wrote:

    On Sep 18, 2007, at 7:03 AM, I. Savant wrote:

    > On Sep 18, 2007, at 1:28 AM, Diederik Hoogenboom wrote:
    >
    >> AFAIK it is not mentioned in any public documentation. It has been
    >> mentioned once or twice on this list though.
    >> It is not really an API; it is more a "tag" in your model that will
    >> trigger the index to be created when creating your store.
    >>
    >
    > That's not really the point, though. It's undocumented and starts
    > with an underscore (Apple's convention for "private"), therefore
    > it's subject to change and can break your application (or, in this
    > case, *potentially* corrupt your users' documents) in future
    > versions of OS X.
    >
    > I would steer *well clear* of this approach without official support
    > from Apple.
    >

    While I agree with this advice in most circumstances, I also think
    using _NSCreateIndex is a reasonable thing to do.

    In this case, the index won't change SQLite/Core Data's ability to
    access the file--SQLite can read and write the tables regardless of
    the indices in the database, and this fact won't change.

    The second potential concern is that Apple changes Core Data and
    changes it's behavior or ignores the key.  If it ignores it,
    performance will be affected without the index, but otherwise the data
    will be accessible.  As for changing the behavior, it's hard to
    imagine a change that wouldn't also affect every existing Core Data app.

    Lastly, Malcolm Crawford is one who has suggested using it:

    http://www.cocoabuilder.com/archive/message/cocoa/2005/6/22/139661

    Aaron
previous month september 2007 next month
MTWTFSS
          1 2
3 4 5 6 7 8 9
10 11 12 13 14 15 16
17 18 19 20 21 22 23
24 25 26 27 28 29 30
Go to today