Ordered Core Data
-
I'm thinking about using Core Data to develop a new application.
Unfortunately, it involves elements that need to be saved in order --
an order that users might rearrange by dragging. Core Data doesn't
guarantee anything about ordering (other than sorting by value). In
fact, the OutlineEdit example doesn't read files in the same order it
saves them.
I suspect using an internal index can solve this, but I'm a little
concerned about performance: what if the user adds an element at the
very beginning of the list? This means that each element has to have
its index incremented. I know Core Data is relatively fast, but this
still seems like it could be asking for trouble, since the data set
could be arbitrarily large.
Any thoughts on solutions (or actual performance)?
David Dunham A Sharp, LLC
Voice/Fax: 206 783 7404 http://a-sharp.com
Efficiency is intelligent laziness. -
Am 11.06.2005 um 19:19 schrieb David Dunham:
> I'm thinking about using Core Data to develop a new application.
> Unfortunately, it involves elements that need to be saved in order
> -- an order that users might rearrange by dragging.
> I suspect using an internal index can solve this, but I'm a little
> concerned about performance: what if the user adds an element at
> the very beginning of the list?
As the data representation in memory (obviously) doesn't change, at
worst a couple of pointers get shuffled around. This is why you
usually read -arrangedObjects and not -managedObjectContext (they
differ, as I've learned a few weeks ago here).
> This means that each element has to have its index incremented.
Isn't there another field you can sort the data by without the risk
of having conflicts?
If there isn't, how about creating the index in steps of 100? This
way you can put new items in between and have to rewrite the whole
index rarely.
Above you said you'd have to _save_ the data in order. Since you have
few influence how Core Data stores away the batch of information, I'm
not sure where you can take advantage of it's capabilities. But
perhaps you just have to make sure the data is viewed correctly
sorted always. This is easily done, see that recent thread "Sorting
Core Data data".
HTH,
Markus
- - - - - - - - - - - - - - - - - - -
Dipl. Ing. Markus Hitter
http://www.jump-ing.de/ -
On Jun 11, 2005, at 10:19 AM, David Dunham wrote:
> I suspect using an internal index can solve this, but I'm a little
> concerned about performance: what if the user adds an element at
> the very beginning of the list? This means that each element has to
> have its index incremented. I know Core Data is relatively fast,
> but this still seems like it could be asking for trouble, since the
> data set could be arbitrarily large.
>
It's not clear why performance using Core Data would be significantly
different from performance if you were using custom subclasses of
NSObject that used an index? That is, if you need to support
ordering, then you have to provide a mechanism to do so whether or
not you use Core Data. In what respect might Core Data impede that
mechanism?
mmalc -
On 11 Jun 2005, at 19:03, mmalcolm crawford wrote:
> It's not clear why performance using Core Data would be
> significantly different from performance if you were using custom
> subclasses of NSObject that used an index? That is, if you need to
> support ordering, then you have to provide a mechanism to do so
> whether or not you use Core Data. In what respect might Core Data
> impede that mechanism?
Without Core Data, I can do without a per-element index, and just
keep everything in an NSArray. NSArrays don't forget the order of
insertion, the way Core Data does.
David Dunham A Sharp, LLC
Voice/Fax: 206 783 7404 http://a-sharp.com
Efficiency is intelligent laziness. -
On Jun 12, 2005, at 5:08 PM, David Dunham wrote:
>> It's not clear why performance using Core Data would beSorry I misunderstood your original.
>> significantly different from performance if you were using custom
>> subclasses of NSObject that used an index? That is, if you need
>> to support ordering, then you have to provide a mechanism to do so
>> whether or not you use Core Data. In what respect might Core Data
>> impede that mechanism?
> Without Core Data, I can do without a per-element index, and just
> keep everything in an NSArray.
>
In that case, I still don't think there should be a significant
(compared to the rest of the application) overhead. Would you be
doing anything more than getting and setting the index instance
variable?
One other issue would be, at what stage are you actually setting the
indexes. If you have just a single ordered table managed by an array
controller, then you might be able to defer updating the indexes
until just before save...
mmalc -
On 12 Jun 2005, at 17:47, mmalcolm crawford wrote:
> In that case, I still don't think there should be a significant
> (compared to the rest of the application) overhead. Would you be
> doing anything more than getting and setting the index instance
> variable?
I guess I would, come to think of it.
> One other issue would be, at what stage are you actually setting
> the indexes. If you have just a single ordered table managed by an
> array controller, then you might be able to defer updating the
> indexes until just before save...
I'd been hoping to get all that free undo that Core Data provides,
which I don't think this approach provides (since I'd still need to
undo any array changes).
I think I may have been missing the likelihood that the managed
object context could simply keep things around in memory, much like
an NSArray would, and that the overhead is therefore fairly low.
David Dunham A Sharp, LLC
Voice/Fax: 206 783 7404 http://a-sharp.com
"People seem to misinterpret complexity as sophistication" -- Niklaus
Wirth -
On Jun 11, 2005, at 1:19 PM, David Dunham wrote:
> I suspect using an internal index can solve this, but I'm a little
> concerned about performance: what if the user adds an element at
> the very beginning of the list? This means that each element has to
> have its index incremented. I know Core Data is relatively fast,
> but this still seems like it could be asking for trouble, since the
> data set could be arbitrarily large.
>
> Any thoughts on solutions (or actual performance)?
"Premature optimization..."
Protect yourself by encapsulating your unknown from the rest of your
code. Then implement the unknown in the simplest way that could
possibly work. If it doesn't perform for you, optimize within the
encapsulated space as necessary. But more than likely, YAGNI.
Erik -
On 13 Jun 2005, at 18:41, Erik Price wrote:
>> Any thoughts on solutions (or actual performance)?
>
> "Premature optimization..."
I'm treating it as selection of algorithm, which is always the most
important part of optimization. (This isn't something I can just whip
together with bindings.) It's sorta like the difference between
choosing a stable sort or a non-stable one. (BTW I'll bet Core Data
sorting isn't stable, since they can't normally get you information
in the order you arrange it.)
David Dunham A Sharp, LLC
Voice/Fax: 206 783 7404 http://a-sharp.com
"People seem to misinterpret complexity as sophistication" -- Niklaus
Wirth -
I am getting a bit tired of this apparently deliberate obfuscation.
The problem here, and it is a huge one, is that there is not a
managed-object friendly version of NSArray, and it is a gaping hole
in the implementation of CoreData. ***There is simply no transparent
way of representing an ordered collection of arbitrary managed
objects.***
It is *obviously* not as simple as getting/setting an index variable
which is why I find this kind of response so annoying.
Let us assume we have 10,000 objects, each with an index variable.
We want to insert an object at the beginning.
We must now riffle down through our ordered collection and increment
the index variable for all 10,000 objects. Of course we can start
with indices 100 apart or whatever but it's a miserable way to have
to think.
There are ways around this, obviously, but they *all* involve either
(a) rolling your own datastructures or (b) creating NSMutableArrays
on the fly as temporary ivars in a managed object, with code to
unroll them into indexed sets on save.
As for (a) it is ridiculous that in a IDE like Cocoa, a user should
have to write their own black/red tree implementation simply to be
able to preserve the order of a collection of arbitrary elements (I
soon found that linked lists weren't nearly fast enough).
As for (b) it is obviously a kludge, and every time I raise it as a
possibility I am warned off.
This single lacuna has caused me endless headaches and I still have
not had a really good solution proposed.
On 12-Jun-05, at 8:47 PM, mmalcolm crawford wrote:
> In that case, I still don't think there should be a significant
> (compared to the rest of the application) overhead. Would you be
> doing anything more than getting and setting the index instance
> variable?
-
It all boils down to the fact that the relational database model
doesn't lend itself nicely to ordered collections of instances of
entities with insertion and removal.
It's not a core data problem. It's a relational database problem.
You're going to find the same issues on any other RDB. I know I have.
Usually it means you should think if that is really the right design
to go. And if you really, really, really have to do it, then your
life sucks a little more and you have to do it manually.
If the main part of your application's data model has those
characteristics, maybe you should think about not using Core Data, or
you can just go and implement the manual indexing which from what
I've heard about the performance of SQLite shouldn't be that bad.
Hope that helps.
Oscar Morales Vivo
On Jun 14, 2005, at 09:04 , Johnny Deadman wrote:
> I am getting a bit tired of this apparently deliberate obfuscation.
>
> The problem here, and it is a huge one, is that there is not a
> managed-object friendly version of NSArray, and it is a gaping hole
> in the implementation of CoreData. ***There is simply no
> transparent way of representing an ordered collection of arbitrary
> managed objects.***
>
> It is *obviously* not as simple as getting/setting an index
> variable which is why I find this kind of response so annoying.
>
> Let us assume we have 10,000 objects, each with an index variable.
>
> We want to insert an object at the beginning.
>
> We must now riffle down through our ordered collection and
> increment the index variable for all 10,000 objects. Of course we
> can start with indices 100 apart or whatever but it's a miserable
> way to have to think.
>
> There are ways around this, obviously, but they *all* involve
> either (a) rolling your own datastructures or (b) creating
> NSMutableArrays on the fly as temporary ivars in a managed object,
> with code to unroll them into indexed sets on save.
>
> As for (a) it is ridiculous that in a IDE like Cocoa, a user should
> have to write their own black/red tree implementation simply to be
> able to preserve the order of a collection of arbitrary elements (I
> soon found that linked lists weren't nearly fast enough).
>
> As for (b) it is obviously a kludge, and every time I raise it as a
> possibility I am warned off.
>
> This single lacuna has caused me endless headaches and I still have
> not had a really good solution proposed.
>
>
> On 12-Jun-05, at 8:47 PM, mmalcolm crawford wrote:
>
>
>> In that case, I still don't think there should be a significant
>> (compared to the rest of the application) overhead. Would you be
>> doing anything more than getting and setting the index instance
>> variable?
>>
>
> _______________________________________________
> MacOSX-dev mailing list
> <MacOSX-dev...>
> http://www.omnigroup.com/mailman/listinfo/macosx-dev
>
-
On Jun 14, 2005, at 6:04 AM, Johnny Deadman wrote:
> I am getting a bit tired of this apparently deliberate obfuscation.There is no intent at obfuscation. There is simply no way to give a
>
concrete answer without knowing more about a given application. The
overhead that using an index incurs is a relative quantity, and
depends on what else the application is doing.
> The problem here, and it is a huge one, is that there is not aPlease file an enhancement request.
> managed-object friendly version of NSArray, and it is a gaping hole
> in the implementation of CoreData. ***There is simply no
> transparent way of representing an ordered collection of arbitrary
> managed objects.***
>
> It is *obviously* not as simple as getting/setting an indexI don't see anything complex about that. It's tedious, but to me
> variable which is why I find this kind of response so annoying.
> Let us assume we have 10,000 objects, each with an index variable.
> We want to insert an object at the beginning.
> We must now riffle down through our ordered collection and
> increment the index variable for all 10,000 objects.
>
it's simple.
> There are ways around this, obviously, but they *all* involveI don't recall having seen you raise (b) as a possibility before.
> either (a) rolling your own datastructures or (b) creating
> NSMutableArrays on the fly as temporary ivars in a managed object,
> with code to unroll them into indexed sets on save.
> As for (a) it is ridiculous that in a IDE like Cocoa, a user should
> have to write their own black/red tree implementation simply to be
> able to preserve the order of a collection of arbitrary elements (I
> soon found that linked lists weren't nearly fast enough).
> As for (b) it is obviously a kludge, and every time I raise it as a
> possibility I am warned off.
>
mmalc -
One solution I'm toying with on the insertion-at-beginning-
of-10,000-items problem is, on awakeFromInsert, set the index to -1.
On save, if there're any set to -1, I reorder all, starting the -1's
at 0. It isn't pretty and it's still just an idea for now, but it's
one I'm toying with. Unfortunately, one problem that jumps out at me
is what to do about a reverse-order sort. It's even uglier (in my
mind) to find the highest-number++. Again, just a first thought. I've
written 0 lines of code to attempt this so far.
Maybe if someone here beats me to trying it out, they could let us
all know how it works. Admittedly, it might be a total waste of time,
but then so are most things I try ... ;-)
I agree with Johnny, however. An ordered collection mechanism that
works seamlessly with Core Data and Cocoa Bindings would be a hunk of
solid gold for many developers. It's also very likely a pipe dream
(at least for 10.4), given the relational database issues Oscar
mentions. I imagine it would take a pretty big change to Cocoa and
Core Data which is too late to change for Tiger ...
Again, I may be talking out of my butt, being completely wrong
about everything I just said (I defer to the list on that issue), but
this *is* a pretty big speed bump that seems like it wants to be
simple. ;-)
On Jun 14, 2005, at 9:19 AM, Óscar Morales Vivó wrote:
> It all boils down to the fact that the relational database model
> doesn't lend itself nicely to ordered collections of instances of
> entities with insertion and removal.
>
> It's not a core data problem. It's a relational database problem.
> You're going to find the same issues on any other RDB. I know I
> have. Usually it means you should think if that is really the right
> design to go. And if you really, really, really have to do it, then
> your life sucks a little more and you have to do it manually.
>
> If the main part of your application's data model has those
> characteristics, maybe you should think about not using Core Data,
> or you can just go and implement the manual indexing which from
> what I've heard about the performance of SQLite shouldn't be that bad.
>
> Hope that helps.
>
> Oscar Morales Vivo
>
> On Jun 14, 2005, at 09:04 , Johnny Deadman wrote:
>
>
>> I am getting a bit tired of this apparently deliberate obfuscation.
>>
>> The problem here, and it is a huge one, is that there is not a
>> managed-object friendly version of NSArray, and it is a gaping
>> hole in the implementation of CoreData. ***There is simply no
>> transparent way of representing an ordered collection of arbitrary
>> managed objects.***
>>
>> It is *obviously* not as simple as getting/setting an index
>> variable which is why I find this kind of response so annoying.
>>
>> Let us assume we have 10,000 objects, each with an index variable.
>>
>> We want to insert an object at the beginning.
>>
>> We must now riffle down through our ordered collection and
>> increment the index variable for all 10,000 objects. Of course we
>> can start with indices 100 apart or whatever but it's a miserable
>> way to have to think.
>>
>> There are ways around this, obviously, but they *all* involve
>> either (a) rolling your own datastructures or (b) creating
>> NSMutableArrays on the fly as temporary ivars in a managed object,
>> with code to unroll them into indexed sets on save.
>>
>> As for (a) it is ridiculous that in a IDE like Cocoa, a user
>> should have to write their own black/red tree implementation
>> simply to be able to preserve the order of a collection of
>> arbitrary elements (I soon found that linked lists weren't nearly
>> fast enough).
>>
>> As for (b) it is obviously a kludge, and every time I raise it as
>> a possibility I am warned off.
>>
>> This single lacuna has caused me endless headaches and I still
>> have not had a really good solution proposed.
>>
>>
>> On 12-Jun-05, at 8:47 PM, mmalcolm crawford wrote:
>>
>>
>>
>>> In that case, I still don't think there should be a significant
>>> (compared to the rest of the application) overhead. Would you be
>>> doing anything more than getting and setting the index instance
>>> variable?
>>>
>>>
>>
>> _______________________________________________
>> MacOSX-dev mailing list
>> <MacOSX-dev...>
>> http://www.omnigroup.com/mailman/listinfo/macosx-dev
>>
>>
>
> _______________________________________________
> Do not post admin requests to the list. They will be ignored.
> Cocoa-dev mailing list (<Cocoa-dev...>)
> Help/Unsubscribe/Update your Subscription:
> http://lists.apple.com/mailman/options/cocoa-dev/dev%
> 40silentalcove.net
>
> This email sent to <dev...>
>
-
On Jun 14, 2005, at 9:49 AM, mmalcolm crawford wrote:
>> ***There is simply no transparent way of representing an ordered
>> collection of arbitrary managed objects.***
> Please file an enhancement request.
Filed on the day Tiger was released I think... marked as 'dupe' FWIW
>> It is *obviously* not as simple as getting/setting an index
>> variable which is why I find this kind of response so annoying.
>> Let us assume we have 10,000 objects, each with an index variable.
>> We want to insert an object at the beginning.
>> We must now riffle down through our ordered collection and
>> increment the index variable for all 10,000 objects.
>>
> I don't see anything complex about that. It's tedious, but to me
> it's simple.
(1) it is more than 'simply getting/setting an index variable', which
is the obfuscation I was talking about
(2) it is too slow for anything but trivial applications. If the time
to insert an arbitrary item in a list of N item scales with N, you
are in trouble.
>> There are ways around this, obviously, but they *all* involve
>> either (a) rolling your own datastructures or (b) creating
>> NSMutableArrays on the fly
>> [snip]
> I don't recall having seen you raise (b) as a possibility before.
I certainly have off-list, maybe not with you though. It may be that
this turns out to be the best interim solution, despite the
kludginess, since it offloads the overhead of maintaining the ordered
list onto tried-and-trusted structures like NSArray. -
Am 14.06.2005 um 15:04 schrieb Johnny Deadman:
> The problem here, and it is a huge one, is that there is not a
> managed-object friendly version of NSArray, and it is a gaping hole
> in the implementation of CoreData. ***There is simply no
> transparent way of representing an ordered collection of arbitrary
> managed objects.***
Without re-reseaching the issue in depth, the default object store
for an NSManagedObjectContext is an NSMutableArray and you are free
to re-arrange this array to your hearts contents. This shouldn't
affect functionality of the levels above and below the
NSManagedObjectContext.
Even data exchange with the external store should continue to work
since each NSManagedObject maintains an unique identifier which is
invisible to the upper layers. For comparison: you have to set the
identifier explicitely in the similar EOF framework.
However, you get more stuff "for free" if you can use orderedObjects
instead of the base array. orderedObjects behave a lot like an
NSArray and are in fact an array of proxy objects, as far as I found
out so far.
> We want to insert an object at the beginning.
If you insist in using an NSArray, not even an NSMutableArray, much
less an NSManagedObjectContext, you have to roll your own storage
handling methods, obviously.
> This single lacuna has caused me endless headaches and I still have
> not had a really good solution proposed.
"Never fight the frameworks"
Markus
- - - - - - - - - - - - - - - - - - -
Dipl. Ing. Markus Hitter
http://www.jump-ing.de/ -
On Jun 14, 2005, at 6:04 AM, Johnny Deadman wrote:
> The problem here, and it is a huge one, is that there is not a
> managed-object friendly version of NSArray, and it is a gaping hole
> in the implementation of CoreData. ***There is simply no
> transparent way of representing an ordered collection of arbitrary
> managed objects.***
That is correct and, yes, it is a limitation in the 1.0 release of
Core Data.
In evaluating the feature set that could be engineered for a 1.0,
every feature was considered in light of both engineering effort and
return on investment of said engineering effort.
Unfortunately, direct modeling and maintenance of ordered
relationships fell of the list of 1.0 features for a number of reasons.
First, it is very important to note that the automatic maintenance of
ordered relationships has nothing to do with the display of ordered
sets of data within the user interface. In most cases, if the
developer is displaying a set of objects in the UI, it is via a table
view, a browser, an outline view, or some other UI widget that will
either have an implied order or will allow the user to re-order the
contents on the fly by clicking on a column header. Furthermore,
for table view and table view like widgets, the contents are very
often qualified by a predicate or the contents are the result of a
fetch request, which can have a sort order that incurs relatively
little overhead.
Conclusion: Maintaining order of an entity's instances within the
object model contributes little to most user interfaces.
Secondly, offering a generic solution for maintaining ordered
relationships is rife with caveats that make it extremely difficult
to do in a universally useful fashion. Some various random issues:
- can an object be a member of a relationship more than once?
- No? What happens when the object is inserted a second time?
- should it support an order implied by an attribute's value?
- what about duplicate values?
- should there be a secondary value?
- can a relationship have multiple sort orders?
- if a relationship TO an object should be modified by a change
to that object, is there an efficient way to handle the change if the
relationship hasn't been faulted?
- does this mean that every relationship must be aware of
its inverse?
Conclusion: Modeling ordered relationships is considerably more
complex than just adding support for relationship containment within
an array.
Given that maintaining order on a relationship is just not that hard
of a problem to solve when viewed from a per-app perspective (many of
the above requirements are mutually exclusive, therefore any single
use of ordered relationships will require answers to a limited number
of the above questions). Re-applying indices across all objects in
a relationship will incur effectively no overhead in the in-memory,
XML, or Binary store cases. For the SQL store, it will be
significantly slower to modify the 10,000 objects, but any of a
number of approaches to the implementation/modeling can yield
efficient solutions without adding a significant performance penalty.
Finally, thank you for filing the enhancement request. Even though
it was returned as a dupe, the specific use case information that you
may have included in the bug -- any details about the problem you
were hoping to solve -- is captured and evaluated as a part of
determining features contained in the next release of the product.
b.bum -
On Jun 14, 2005, at 8:50 AM, Bill Bumgarner wrote:
> On Jun 14, 2005, at 6:04 AM, Johnny Deadman wrote:
>> The problem here, and it is a huge one, is that there is not a
>> managed-object friendly version of NSArray, and it is a gaping
>> hole in the implementation of CoreData. ***There is simply no
>> transparent way of representing an ordered collection of arbitrary
>> managed objects.***
> That is correct and, yes, it is a limitation in the 1.0 release of
> Core Data.
Just to avoid the confusion -- it is a limitation in 1.0, but not a
huge limitation.... -
On Jun 14, 2005, at 6:04 AM, Johnny Deadman wrote:
> The problem here, and it is a huge one, is that there is not a
> managed-object friendly version of NSArray, and it is a gaping hole
> in the implementation of CoreData. ***There is simply no
> transparent way of representing an ordered collection of arbitrary
> managed objects.***
Relationships are not collections. Relationships are relations.
Relations can be used to represent containment, or they can be used
to represent association.
-- Chris -
On Jun 14, 2005, at 7:28 AM, Markus Hitter wrote:
> Without re-reseaching the issue in depth, the default object store
> for an NSManagedObjectContext is an NSMutableArray and you are free
> to re-arrange this array to your hearts contents. This shouldn't
> affect functionality of the levels above and below the
> NSManagedObjectContext.
This is incorrect in a number of ways. There is no such thing as a
"default object store for an NSManagedObjectContext."
NSMutableArrays are not used to represent relationships in Core Data,
and with only a couple of exceptions any NSMutableSet you get from
Core Data should *not* be manipulated directly.
The only time you should mutate a set representing a relationship
directly in client code is when it was returned by -
mutableSetValueForKey:. The *only* exception is when implementing
mutable-set mutation accessors in your managed objects, and then you
should *only* do so in the same manner as the templates that the
Xcode design tools will generate for you, within a -
willChangeValueForKey:withSetMutation:usingObjects: and -
didChangeValueForKey:withSetMutation:usingObjects: pair.
-- Chris -
On Jun 14, 2005, at 5:38 PM, Chris Hanson wrote:
>> The problem here, and it is a huge one, is that there is not a
>> managed-object friendly version of NSArray, and it is a gaping
>> hole in the implementation of CoreData. ***There is simply no
>> transparent way of representing an ordered collection of arbitrary
>> managed objects.***
>>
>
> Relationships are not collections. Relationships are relations.
> Relations can be used to represent containment, or they can be used
> to represent association.
I assume you are being pedantic about RDBMS in some way that escapes me.
Take this (ridiculously oversimplified) example. Let us say I wish to
use CoreData to model the scenes in a screenplay.
I naturally create two entities called Screenplay and Scene.
Screenplay naturally has a relationship 'scenes' to Scene entities.
Each scene may appear in an arbitrary number of Screenplays in
different positions.
Each screenplay must know, manipulate and persist the order of its
Scenes.
The Scene wishes to know nothing about Screenplays, indices, linked
lists, or anything else. It is just a Scene and cares not for whether
it is in a screenplay or on the dark side of the moon.
It is plain from this description that the ordering of the scenes is
an attribute of the Screenplay, and a fundamental one.
How should this be represented in CoreData? Pedantically speaking?
You see immediately why using an index attribute for the Scene is
totally wrong.
Several other options present themselves, all bad. One could create a
new entity 'IndexedItem' with an 'index' attribute and a to-one
relationship to the Scene entity, and add a new relationship
'indexedScenes' which points to IndexedItems which in turn point to
the Scenes. Ugh. But we want to generalize this so that we can index
anything so we have to create an entity 'IndexableItem' and make that
the parent of any entity we wish to index. Now our entity diagram is
being twisted to facilitate the implementation. Yuk.
(This is how I persist my temporary NSMutableArrays, however, for
want of a better way).
Or you can start writing linked lists which do much the same thing
except that they maintain pointers to 'nextItem' instead of keeping
an index, but it's just as ugly.
<RANT>
Isn't it obvious that this should be easier? I am hearing a lot of
flim flam but no solutions that are not horrible.
I am talking about Screenplays, but the same problem exists in
hundreds of situations. Queues, waypoints, fill in the blanks. So
much of the conversation on this topic assumes that ordering a data
set is just something for the user interface, but in many models the
ordering of the items in a collection is a fundamental property. I
simply do not believe that the RDBMS world does not have good
solutions for these problems that could be implemented in CoreData.
</RANT> -
On Jun 14, 2005, at 4:27 PM, John Brownlow wrote:
> Or you can start writing linked lists which do much the same thing
> except that they maintain pointers to 'nextItem' instead of keeping
> an index, but it's just as ugly.
Actually, depending on context of implementation, using a linked list
is both quite easy and very efficient. Obviously, inserting/deleting
objects into a linked list is trivial and fast if you have both a
next and a previous pointer. Given that Core Data will maintain the
inverse relationship automatically, it will also maintain the linked
list pointers automatically.
Given that many UI type usage contexts will require all of the
objects to be faulted into memory anyway, walking the "next"
relations and filling an NSMutableArray for display purposes is about
three lines of code.
So, for the particular example of Screenplay -> Scenes, a linked list
might very well be a totally acceptable way of modeling this.
In this particular example, the amount of overhead in the form of
additional lines of code incurred by the developer is pretty trivial.
Yeah -- it is a bummer that Core Data doesn't have direct modeling of
ordered relationships. See my other email for why that decision was
made.
b.bum -
On 14-Jun-05, at 7:44 PM, Bill Bumgarner wrote:
> Actually, depending on context of implementation, using a linked
> list is both quite easy and very efficient. Obviously, inserting/
> deleting objects into a linked list is trivial and fast if you have
> both a next and a previous pointer. Given that Core Data will
> maintain the inverse relationship automatically, it will also
> maintain the linked list pointers automatically.
Yes, I know. I did it after (I think you or) John suggested it off
list. It is quite a general solution but... The problem is that
accessing list items is really slow. I run through the list
frequently while the user is typing and it slowed everything down. I
implemented some caching of the list as an NSArray and lazy updating
and that has just about brought the access times under control but it
is messier now, and more error-prone, and I feel a heck of a lot less
abstracted from the implementation.
Using a b-tree in my case didn't really help because the tree tends
to decompose into a linked list anyway. -
On 15/06/2005, at 03:22, Johnny Deadman wrote:
>
> On 14-Jun-05, at 7:44 PM, Bill Bumgarner wrote:
>
>
>> Actually, depending on context of implementation, using a linked
>> list is both quite easy and very efficient. Obviously, inserting/
>> deleting objects into a linked list is trivial and fast if you
>> have both a next and a previous pointer. Given that Core Data
>> will maintain the inverse relationship automatically, it will also
>> maintain the linked list pointers automatically.
>>
>
> Yes, I know. I did it after (I think you or) John suggested it off
> list. It is quite a general solution but... The problem is that
> accessing list items is really slow.
Quite useless for random access with large data sets. I have planned
on using Core Data as a solution for an application in which strict
ordering plays a large role in content semantics. Now I seriously
doubt I'll use it, because of its deficiencies in this regard.
Does anybody know if DataCrux (http://treehouseideas.com) has a
better solution to these problems?
Sharon -
On Jun 14, 2005, at 8:30 PM, Sharon Rosner wrote:
> On 15/06/2005, at 03:22, Johnny Deadman wrote:
>> On 14-Jun-05, at 7:44 PM, Bill Bumgarner wrote:
>>> Actually, depending on context of implementation, using a linked
>>> list is both quite easy and very efficient. Obviously, inserting/
>>> deleting objects into a linked list is trivial and fast if you
>>> have both a next and a previous pointer. Given that Core Data
>>> will maintain the inverse relationship automatically, it will
>>> also maintain the linked list pointers automatically.
>> Yes, I know. I did it after (I think you or) John suggested it off
>> list. It is quite a general solution but... The problem is that
>> accessing list items is really slow.
> Quite useless for random access with large data sets. I have
> planned on using Core Data as a solution for an application in
> which strict ordering plays a large role in content semantics. Now
> I seriously doubt I'll use it, because of its deficiencies in this
> regard.
>
> Does anybody know if DataCrux (http://treehouseideas.com) has a
> better solution to these problems?
Given that DataCrux is a relatively thin layer on top of SQLite, you
will run in to the same problems in terms of random data access.
Of course, I don't understand what "quite useless for random access
with large data sets" actually means.
Core Data uses Predicates to define Fetch Requests that allow for the
construction and evaluation of arbitrary queries against the store or
stores, including merging in changes that may be present in the in-
memory object graph state. This has proven to be extremely useful
for random access to large data sets. While Core Data does not
offer the ability to evaluate hand tuned SQL, doing so has rarely
proven to fix a performance problem that hasn't better been solved
through other, more straightforward, means.
What are your requirements? I.e. how many objects of the type
requiring a strict ordering will typically be present? How will they
be ordered? How often will they be reordered? etc..?
b.bum -
Am 14.06.2005 um 23:44 schrieb Chris Hanson:
> On Jun 14, 2005, at 7:28 AM, Markus Hitter wrote:
>
>> Without re-reseaching the issue in depth, the default object store
>> for an NSManagedObjectContext is an NSMutableArray and you are
>> free to re-arrange this array to your hearts contents. This
>> shouldn't affect functionality of the levels above and below the
>> NSManagedObjectContext.
>
> This is incorrect in a number of ways.
You are right, I mixed up "NSManagedObjectContext" with
"NSArrayController". In all tutorials I read so far, they use an
NSArrayController, bound to an NSManagedObjectContext, to talk to the
persistent store. The NSArrayController is the class using the
NSMutableArray (inherited from NSObjectController) and the array
(arrangedObjects) of proxy objects.
Markus
- - - - - - - - - - - - - - - - - - -
Dipl. Ing. Markus Hitter
http://www.jump-ing.de/ -
On Jun 15, 2005, at 1:41 AM, Markus Hitter wrote:
> In all tutorials I read so far, they use an NSArrayController,It may be useful to work through this one:
> bound to an NSManagedObjectContext, to talk to the persistent store.
>
<http://developer.apple.com/documentation/Cocoa/Conceptual/
CoreDataUtilityTutorial/index.html>
mmalc -
Am 15.06.2005 um 11:05 schrieb mmalcolm crawford:
> On Jun 15, 2005, at 1:41 AM, Markus Hitter wrote:> CoreDataUtilityTutorial/index.html>
>
>> In all tutorials I read so far, they use an NSArrayController,
>> bound to an NSManagedObjectContext, to talk to the persistent store.
>>
> It may be useful to work through this one:
> <http://developer.apple.com/documentation/Cocoa/Conceptual/
Thanks for the link, mmalcolm, but I don't feel the need to get rid
of the NSArrayController, currently. I've solved problems which sound
similar to what the others are struggling with and I neither need
enourmous amounds of code (five lines for the sorting stuff) nor do I
see performance issues.
Perhaps I should send a copy of my project to those others so they
can push a lot of data into and measure the performance the way they
like it.
Have fun,
Markus
- - - - - - - - - - - - - - - - - - -
Dipl. Ing. Markus Hitter
http://www.jump-ing.de/ -
...snip...
> How many attributes per object? ...relationships?
>
> I'm trying to get a relative feel for the data.
>
> Is it a time series data kind of application?
Not really. Although it does have an indirect relationship to time.
It's really a content creation application, much like a
word-processor, but the content is not text. Objectsts have a few
different kinds but they are basically very similar - about 10
attributes per object. Relationships are currently not needed.
> OK -- will this be done in large batches or typically one at a time?
Either way. The user will be able to perform operations on large
selections or on individual objects.
> This is starting to sound a lot like a time-series kind of a
> problem. This is something that neither relational stores nor Core
> Data are terribly well equipped to handle.
>
> I'll ask the Core Data folks tomorrow if they have any ideas -- my
> office is next to theirs.
>
> b.bum
>
Thanks for all your help.
s.
