Virtual memory increases from temporary allocations.

  • I have an application that collects data every few seconds, updates
    the model objects with data, and table views are updated through
    cocoa bindings with the data.  Using ObjectAlloc and MallocDebug, I
    can see that memory allocation reaches a steady state and that the
    object and memory allocations do not increase, but when using top and
    vmmap the virtual memory pages increase and the memory usage appears
    to balloon. (Also, leaks doesn't report any with each update.)

    First, I would like to confirm what I believe is going on.  Each time
    my data updates, temporary objects and new objects for the data are
    created.  A one minute difference of the virtual memory allocation
    using vmmap produces the following,

    costarica:~ brian$ vmmap -pages -d 60 1710
    Virtual Memory Map of process 1710 (StatusMonitors)
    Output report format:  2.0

    ========= Regions in 1st snapshot not in 2nd ==========
    ==== Non-writable regions for process 1710

    ==== Writable regions for process 1710
    VM_ALLOCATE ?          04c0a000-04d01000 [  247] rw-/rwx SM=COW
    MALLOC (freed?)        04db6000-04db7000 [    1] rw-/rwx SM=PRV

    ========= Regions in 2nd snapshot not in 1st ==========
    ==== Non-writable regions for process 1710

    ==== Writable regions for process 1710
    MALLOC (freed?)        04c0a000-04c0b000 [    1] rw-/rwx SM=PRV
    MALLOC (freed?)        04db6000-04dbe000 [    8] rw-/rwx SM=PRV
    IOKit                  04dbe000-04dbf000 [    1] rw-/rwx SM=PRV
    MALLOC (freed?)        04dbf000-04dd5000 [    22] rw-/rwx SM=PRV
    VM_ALLOCATE ?          04dd5000-04ecc000 [  247] rw-/rwx SM=COW
    ==== Legend
    SM=sharing mode:
            COW=copy_on_write PRV=private NUL=empty ALI=aliased
            SHM=shared ZER=zero_filled S/A=shared_alias

    If I understand this correctly, malloc() calls vm_allocate() to
    create a block of pages to use, uses them for the temporary objects,
    frees them, and then calls vm_deallocate() which are the MALLOC
    (freed?) entries. Now, from what I've been able to find out, the
    MALLOC (freed?) pages are cleaned up by the dynamic_pager daemon
    which only runs when the kernel tells it to when the free pages
    threshold has been reached.  So at the next update the malloc/free
    cycle above repeats, the free pages threshold is not met, and
    applications like top and Activity Monitor report that the
    deallocated pages plus the new pages in their memory values which
    makes my application appear to balloon out of control.  Is this the
    correct analysis?

    Finally, is there a way other than running a bunch of applications to
    fill memory to force the deallocate of vm pages? What are some
    strategies for stopping this viscous cycle other than reducing the
    allocation of temporary objects?

    Regards,

    Brian Smith
    http://www.suaveware.com
  • As a follow up, this was a memory leak. The hint was the IOKit entry
    in the vmmap output. I call some mach functions to get kernel
    statistics and one of them allocates an array of structures, but it
    uses virtual memory allocation functions to do it. I have to call
    vm_deallocate to free the page it created.This showed up as IOKit
    pages and because malloc is not used it doesn't show up in
    MallocDebug, ObjectAlloc or any malloc based tool. Does anybody know
    of a why to trap in gdb on virtual memory allocations? It would have
    been easier to have gotten a backtrace to the allocations that
    created these pages.  I found this by searching on vm_deallocate to
    find out more about what it does and found some sample code that did
    close to what I'm doing.

    Brian

    On Oct 23, 2006, at 8:34 PM, Brian Smith wrote:

    > I have an application that collects data every few seconds, updates
    > the model objects with data, and table views are updated through
    > cocoa bindings with the data.  Using ObjectAlloc and MallocDebug, I
    > can see that memory allocation reaches a steady state and that the
    > object and memory allocations do not increase, but when using top
    > and vmmap the virtual memory pages increase and the memory usage
    > appears to balloon. (Also, leaks doesn't report any with each update.)
    >
    > First, I would like to confirm what I believe is going on.  Each
    > time my data updates, temporary objects and new objects for the
    > data are created.  A one minute difference of the virtual memory
    > allocation using vmmap produces the following,
    >
    > costarica:~ brian$ vmmap -pages -d 60 1710
    > Virtual Memory Map of process 1710 (StatusMonitors)
    > Output report format:  2.0
    >
    > ========= Regions in 1st snapshot not in 2nd ==========
    > ==== Non-writable regions for process 1710
    >
    > ==== Writable regions for process 1710
    > VM_ALLOCATE ?          04c0a000-04d01000 [  247] rw-/rwx SM=COW
    > MALLOC (freed?)        04db6000-04db7000 [    1] rw-/rwx SM=PRV
    >
    > ========= Regions in 2nd snapshot not in 1st ==========
    > ==== Non-writable regions for process 1710
    >
    > ==== Writable regions for process 1710
    > MALLOC (freed?)        04c0a000-04c0b000 [    1] rw-/rwx SM=PRV
    > MALLOC (freed?)        04db6000-04dbe000 [    8] rw-/rwx SM=PRV
    > IOKit                  04dbe000-04dbf000 [    1] rw-/rwx SM=PRV
    > MALLOC (freed?)        04dbf000-04dd5000 [    22] rw-/rwx SM=PRV
    > VM_ALLOCATE ?          04dd5000-04ecc000 [  247] rw-/rwx SM=COW
    > ==== Legend
    > SM=sharing mode:
    > COW=copy_on_write PRV=private NUL=empty ALI=aliased
    > SHM=shared ZER=zero_filled S/A=shared_alias
    >
    >
    > If I understand this correctly, malloc() calls vm_allocate() to
    > create a block of pages to use, uses them for the temporary
    > objects, frees them, and then calls vm_deallocate() which are the
    > MALLOC (freed?) entries. Now, from what I've been able to find out,
    > the  MALLOC (freed?) pages are cleaned up by the dynamic_pager
    > daemon which only runs when the kernel tells it to when the free
    > pages threshold has been reached.  So at the next update the malloc/
    > free cycle above repeats, the free pages threshold is not met, and
    > applications like top and Activity Monitor report that the
    > deallocated pages plus the new pages in their memory values which
    > makes my application appear to balloon out of control.  Is this the
    > correct analysis?
    >
    > Finally, is there a way other than running a bunch of applications
    > to fill memory to force the deallocate of vm pages? What are some
    > strategies for stopping this viscous cycle other than reducing the
    > allocation of temporary objects?
    >
    >
    > Regards,
    >
    > Brian Smith
    > http://www.suaveware.com
    >
    >
    > _______________________________________________
    > MacOSX-dev mailing list
    > <MacOSX-dev...>
    > http://www.omnigroup.com/mailman/listinfo/macosx-dev
    >
previous month october 2006 next month
MTWTFSS
            1
2 3 4 5 6 7 8
9 10 11 12 13 14 15
16 17 18 19 20 21 22
23 24 25 26 27 28 29
30 31          
Go to today