Core Data performance advice... creating relationships.

FROM : Martin Linklater
DATE : Mon Jan 14 22:43:14 2008

Hi. I'm converting a large database over to Core Data and I'm running 
in to some performance problems. I've read the performance part of the 
Core Data docs but I'm still not sure what to do to speed up my code. 
My basic problem is as follows:

I have created all my entities and am in the process of creatingp the 
relationships. I have two entities, each with around 400,000 entries 
each. 'Foo' and 'Bar' are the two entities. They reference each other 
using a common 'ID' integer. I have created a one to many relationship 
from Foo to Bar (rel), along with the corresponding inverse.

'Foo" <------>> 'Bar'

My algorithm for creating these relationships is to fetch every entry 
in 'Foo', and enumerate through the resulting array building a fetch 
request for 'Bar' in the form of 'All entities in Bar where ID == x'. 
Then when I get that result, I set 'Foo.rel' to the NSArray returned 
by that fetch request.

This technique has been working OK for smaller data sets but now that 
I am linking two very large tables together I am seeing terrible 
performance - My code is only creating around 20 relationships per 
second on my new Macbook Pro.

Rather than blindly stumble around trying to finding some performance 
enhancements I thought I would ask the good people on this list for 
some advice. Can anyone point me in the right direction ?

PS - I'm using an SQLite store, which when populated with these two 
tables is around 100MB on disk. I'm also a bit of a Core Data newbie 
so go easy on me 8)....

Thank you.