Re: Best Way To Lookup From a Huge Table

FROM : Michael Ash
DATE : Fri Mar 21 05:48:49 2008

On Thu, Mar 20, 2008 at 7:58 PM, E. Wing <<email_removed>> wrote:
> You really should profile to find your bottlenecks, especially when
>  the STL is concerned. My personal experience has been that gcc poorly
>  optimizes STL code automatically for you and you must go in and
>  profile to eliminate the real bottlenecks.
>
>  A real world case I dealt with a couple years back, we were loading in
>  a data set containing around 36,000 objects which contained lots of
>  fields of numbers and strings. We were slurping a Lua file and copying
>  all data into a C++ hash_map (extension). We needed to do full copies
>  of the data so I'm sure we had maps with full-blown std::strings and
>  not pointers to std::strings.
>
>  The performance was slow for us on a 1.3GHz Powerbook under Tiger/gcc
>  4.0. It took over a minute and a half to load. The knee-jerk reaction
>  was to blame Lua, but when we built and ran the same code under Visual
>  Studio 7.1/Windows XP on an almost comparable system, the runtime was
>  under half a second.
>
>  So we used Shark. In about 7 iterations (maybe 20 minutes), we got the
>  Mac down to about half a second.


For what it's worth, I wrote a quick test program creating 50,000
random key/value pairs of NSStrings of around 500 characters each,
then inserted them into an NSDictionary. My quick code compiled with
no optimizations took 200 milliseconds to insert all 50,000 strings
into an NSMutableDictionary. This is admittedly on a much faster
machine (a 2.66GHz Mac Pro) and your objects may have been
substantially more complex to hash or compare, but the above does not
build a convincing case for the STL option in my mind.

Mike