Re: BLOBs, MySQL, hex. Oh my

FROM : David Wilson
DATE : Thu May 08 06:19:39 2008

On Wed, May 7, 2008 at 11:12 PM, Ben Einstein <<email_removed>> wrote:
> John's suggestion of getting rid of strlen was spot on; the code below works
> extremely well if using [NSData length] a single time before the loop.
>
>  What I'm still curious about is the difference in processing between the
> NSImage and the zip archive. The same code and the same number of bytes but
> orders of magnitude slower.
>


Is there still a difference in processing speed after swapping to
using the length function? To be honest, I'm with John- the prior code
couldn't possibly have been working. I think it's likely that either
the images had a null byte very early in their data (not at all
surprising) and as such were doing almost no work, or that the zip
archives had *no* null bytes and the loop was therefore moving
straight on past the end of the archive until it hit a null byte
somewhere in memory- the only thing suggesting against this is that
I'd have expected the occasional segfault if that were happening.

Part of your code appends exactly one byte to an NSData every time
through the loop. If the source data is 600k, you're doing 300k calls
to append bytes to the NSData object, which is about as inefficient as
it's possible to get. I'd strongly suggest grabbing the length of the
source data, halving that, malloc()'ing an appropriate-length
character array, and then writing the bytes directly into the
character array to subsequently be stuck into your output object. (If
you wanted to go even fancier you could probably operate on a word at
a time - 8 source hex characters - but that may not be necessary after
you stop calling appendBytes.) I'm willing to bet that if you sharked
right now, that would be your bottleneck.

On that note, John suggested sharking in his first response, and your
response to him was "I know what's happening, so I don't think Shark
or Instruments can help any." I would very strongly suggest that this
response is very wrong- a shark session would have very rapidly shown
both the strlen call and the appenddata calls to be extreme hotspots
in your code, which might have pointed you to the problem.


--
- David T. Wilson
david.t.<email_removed>