Re: sorting large tables

FROM : Daniel Child
DATE : Thu Mar 13 21:42:56 2008

That makes a lot of sense to me, and that's the situation I'm trying 
to get to. But right now all I have is the original (unsorted) raw 
data. So I need to load it into memory and sort it.

Since the sorting operation hangs some kazillion compare:s into the 
process, I can't exactly trace my way through the debugger. 
ObjectAlloc showed that I had a lot of strings and a couple of large 
arrays, which makes sense. Each record is a series of strings, and 
there are two arrays, one with the raw data, one which is becoming a 
sorted array (returned by Apple's sortArrayUsingFunction:). I don't 
see how to cut corners midway through the process.

In answer to your question, each record is a "GenericRecord" which 
contains an array of fields and a numfields count (which varies by 
record). The table is a "GenericTable" that has an ivar that holds an 
array of GenericRecords and a Description object that describes the 
makeup of the fields. There's not much extra.

So the issue is how to get the data sorted so that I can archive it 
in a useful order, at which point I can use your strategy of loading 
bits at a time....

On Mar 13, 2008, at 3:21 PM, Mike Engber wrote:

> Lately I've been working with large table views, hundreds of 
> thousands of records.
>
> My approach has been to take advantage of the fact that the data 
> source only has to provide rows in sorted order - and not all of 
> the rows at once, just the ones that are requested - generally the 
> visible rows.
>
> I do not keep all of my items in a sorted NSArray. They're stored 
> in my own data structure.
>
> So, I guess the question is - what is the data structure you're 
> using for your records and do you really need to keep them in big 
> NSArray. A big NSArray of NSObjects has a fair amount of overhead - 
> something you may want to avoid if you can.