Instruments, how to fix leak in C Lib

  • Hi,

    through Instruments I found a memory leak outside my code in a pqlib, the postgres client library.

    This is what Leaks prints out:

    Bytes Used    # Leaks        Symbol Name
      84.81 KB    100.0%    48         PQgetResult
      84.81 KB    100.0%    48          pqParseInput3
      80.00 KB      94.3%    40           pqResultAlloc
      80.00 KB      94.3%    40           malloc
      4.00 KB      4.7%    4           pqAddTuple
      4.00 KB      4.7%    4           malloc
        832 Bytes    0.9%    4           PQmakeEmptyPGresult
        832 Bytes    0.9%    4           malloc

    I thought that inside PQgetResult there ought to be a call to pqParseInput3, inside of which I would then e.g. find a call to pqResultAlloc. But this is not the case. So I thought maybe the compiled version of the C-library I am using here does not match the sourcecode I obtained. I recompiled the library from source, built the Objective-C framework and rebuilt my app and did a new trace, but the leaks' trees were identical also now that I excluded the source for error of there being a difference in version.

    Please can someone help how to read this?

    Cheers
  • Le 2 mai 2012 à 09:59, Alexander Reichstadt a écrit :

    > Hi,
    >
    > through Instruments I found a memory leak outside my code in a pqlib, the postgres client library.
    >
    > This is what Leaks prints out:
    >
    > Bytes Used    # Leaks        Symbol Name
    > 84.81 KB    100.0%    48         PQgetResult
    > 84.81 KB    100.0%    48          pqParseInput3
    > 80.00 KB      94.3%    40           pqResultAlloc
    > 80.00 KB      94.3%    40           malloc
    > 4.00 KB      4.7%    4           pqAddTuple
    > 4.00 KB      4.7%    4           malloc
    > 832 Bytes    0.9%    4           PQmakeEmptyPGresult
    > 832 Bytes    0.9%    4           malloc
    >
    >
    > I thought that inside PQgetResult there ought to be a call to pqParseInput3, inside of which I would then e.g. find a call to pqResultAlloc. But this is not the case. So I thought maybe the compiled version of the C-library I am using here does not match the sourcecode I obtained. I recompiled the library from source, built the Objective-C framework and rebuilt my app and did a new trace, but the leaks' trees were identical also now that I excluded the source for error of there being a difference in version.
    >
    > Please can someone help how to read this?
    >

    Are you properly using this library ?
    If you alloc some PQ structure and does not free them, the leaks will appears in the libpq, but your code will be the culprit.
    Make sure you properly call PQclear on all result you got from a libpq call, and PQfinish on all connections objects.

    -- Jean-Daniel
  • > Bytes Used    # Leaks        Symbol Name
    > 84.81 KB    100.0%    48        PQgetResult
    > 84.81 KB    100.0%    48        pqParseInput3
    > 80.00 KB      94.3%    40          pqResultAlloc
    > 80.00 KB      94.3%    40          malloc
    > 4.00 KB      4.7%    4          pqAddTuple
    > 4.00 KB      4.7%    4          malloc
    > 832 Bytes    0.9%    4          PQmakeEmptyPGresult
    > 832 Bytes    0.9%    4          malloc
    >
    > I thought that inside PQgetResult there ought to be a call to pqParseInput3, inside of which I would then e.g. find a call to pqResultAlloc. But this is not the case. So I thought maybe the compiled version of the C-library I am using here does not match the sourcecode I obtained. I recompiled the library from source, built the Objective-C framework and rebuilt my app and did a new trace, but the leaks' trees were identical also now that I excluded the source for error of there being a difference in version.

    Symbolication picks the first known symbol before the address in question.  It's the consequence of a tragic design flaw in debug info formats to date.

    It's more obvious when this goes wrong in Crash Reporter or Shark where you can see the offsets (e.g. "CGContextLock + 38623824").

    It happens as a result of stripping of debug info from the built library.  This is normal practice for release distributions.  That you see it when you build from source yourself implies that you either didn't actually get your new version of the library loaded as expected, you didn't build it with full debug info to begin with, or that at some point some of the info was stripped again.
  • Look at Valgrind - it works on the Mac now. What you do is have a demo
    app or even a GUI that launches, runs, then quits. You enter your app
    and args as arguments to valgrind when you run it.

    Valgrind monitors every malloc, and when the app/program quits tells you
    not only what leaks you have, but the stack trace that got you to the
    allocation. Absolutely amazing. I've used it with several open source
    libraries I support.

    David
  • On 2 May 2012, at 10:55 AM, Wade Tregaskis wrote:
    > Symbolication picks the first known symbol before the address in question.  It's the consequence of a tragic design flaw in debug info formats to date.
    >
    > It's more obvious when this goes wrong in Crash Reporter or Shark where you can see the offsets (e.g. "CGContextLock + 38623824").
    >
    > It happens as a result of stripping of debug info from the built library.  This is normal practice for release distributions.  That you see it when you build from source yourself implies that you either didn't actually get your new version of the library loaded as expected, you didn't build it with full debug info to begin with, or that at some point some of the info was stripped again.

    DWARF debug info, which is the standard on OSX now, is rich enough to solve this problem. Unfortunately Apple's "external dsyms" convention means that the debug info of open-source libraries usually gets lost before installation, and Apple doesn't include it in system frameworks either.

    (Tail-call elimination is another reason that call stacks might not precisely match the structure of the source code, but from a glance at the libpq sources I think it's more likely that the lack of accurate debug info is the reason.)

    Anyway, Jean-Daniel is probably right in suggesting that the bug isn't in libpq but in its caller--- odds are that some code is calling PQgetResult, and then failing to fulfill its obligation to release the resources returned to it.
  • On May 2, 2012, at 11:38 AM, David Hoerl wrote:

    > Valgrind monitors every malloc, and when the app/program quits tells you not only what leaks you have, but the stack trace that got you to the allocation. Absolutely amazing. I've used it with several open source libraries I support.

    Valgrind is cool, but the ‘leaks’ tool that’s always shipped with OS X can do what you describe above. You just have to set the environment variable MallocStackLogging when running your app/tool so ‘leaks’ can grab the backtraces:

    env MallocStackLogging=1 ./MyTool &
    leaks MyTool

    One could argue that Instruments makes the ‘leaks’ tool obsolete, but I find it’s so quick and easy to use that I still turn to it. Especially since you can use it on an already-running app (although you lose the backtraces if the app wasn’t launched specially as above.)

    Valgrind can do a lot more, including detecting illegal memory accesses — if you read or write even one byte past the end of a malloc block, it’ll tell you, and show you the stack trace and the position/size of the block. This has been really helpful in debugging some low-level bit-twiddly code I’m currently working on!

    —Jens
previous month may 2012 next month
MTWTFSS
  1 2 3 4 5 6
7 8 9 10 11 12 13
14 15 16 17 18 19 20
21 22 23 24 25 26 27
28 29 30 31      
Go to today