Re: Best Way To Lookup From a Huge Table

FROM : John Stiles
DATE : Fri Mar 21 17:13:40 2008

Michael Ash wrote:
> On Thu, Mar 20, 2008 at 7:58 PM, E. Wing <<email_removed>> wrote:
> 

>> You really should profile to find your bottlenecks, especially when
>>  the STL is concerned. My personal experience has been that gcc poorly
>>  optimizes STL code automatically for you and you must go in and
>>  profile to eliminate the real bottlenecks.
>>
>>  A real world case I dealt with a couple years back, we were loading in
>>  a data set containing around 36,000 objects which contained lots of
>>  fields of numbers and strings. We were slurping a Lua file and copying
>>  all data into a C++ hash_map (extension). We needed to do full copies
>>  of the data so I'm sure we had maps with full-blown std::strings and
>>  not pointers to std::strings.
>>
>>  The performance was slow for us on a 1.3GHz Powerbook under Tiger/gcc
>>  4.0. It took over a minute and a half to load. The knee-jerk reaction
>>  was to blame Lua, but when we built and ran the same code under Visual
>>  Studio 7.1/Windows XP on an almost comparable system, the runtime was
>>  under half a second.
>>
>>  So we used Shark. In about 7 iterations (maybe 20 minutes), we got the
>>  Mac down to about half a second.
>>   

>
> For what it's worth, I wrote a quick test program creating 50,000
> random key/value pairs of NSStrings of around 500 characters each,
> then inserted them into an NSDictionary. My quick code compiled with
> no optimizations took 200 milliseconds to insert all 50,000 strings
> into an NSMutableDictionary. This is admittedly on a much faster
> machine (a 2.66GHz Mac Pro) and your objects may have been
> substantially more complex to hash or compare, but the above does not
> build a convincing case for the STL option in my mind.

You're only comparing apples and oranges if you don't port your test
code to STL and test it yourself.
There's simply no such thing as a fair comparison between a G4 running
on battery power and an Intel Xeon.