[Leo] Strange Core Data vs. Spotlight issue

  • On my application (Notae), on two different machines, I'm seeing
    Spotlight's mds process take over the sytem when I change notes.  If I
    turn off Spotlight indexing then the issue is mediated.  This only
    happens on Leopard; Tiger works perfectly well for this.

    However the very strange thing is that I don't use Spotlight at all.
    I use Core Data in a document-based application that's not saving
    files when this is happening.

    Changing notes involves changing a selection on an array controller
    which triggers an observer that then changes out the note view and
    loads an NSData from the store to fuel that view.  Nothing terribly
    complicated, really, and certainly nothing that uses Spotlight.

    Does anyone have a clue as to where mds could be getting into the
    mix?  I'm not even sure how to start to debug this one.  I've already
    done the "sysadmin" thing and destroyed the Spotlight database and
    rebuilt it as well as tried several machines.  It appears to be a code
    issue.

    Adam Knight
    Notae: Organize your notes, organize your life - http://www.codepoetry.net/products/notae/
  • At 9:52 AM -0800 11/8/07, <cocoa-dev-request...> wrote:
    > However the very strange thing is that I don't use Spotlight at all.
    > I use Core Data in a document-based application that's not saving
    > files when this is happening.

    That's strange ... inconceivable.  Core Data doesn't do anything
    special with or for Spotlight.

    > Changing notes involves changing a selection on an array controller
    > which triggers an observer that then changes out the note view and
    > loads an NSData from the store to fuel that view.  Nothing terribly
    > complicated, really, and certainly nothing that uses Spotlight.
    >
    > Does anyone have a clue as to where mds could be getting into the
    > mix?  I'm not even sure how to start to debug this one.  I've already
    > done the "sysadmin" thing and destroyed the Spotlight database and
    > rebuilt it as well as tried several machines.  It appears to be a code
    > issue.

    Have you checked the console log ?  I've found if an app logs
    constantly (sometimes from throwing exceptions) that it can really
    degrade the system performance.

    What other I/O operations is your app doing besides reading an NSData
    ?  Are you creating or deleting a lot of files ?

    You could also try stepping along through gdb and seeing what area of
    code appears to trigger spotlight.  Without symbols, frustrating, but
    you might be able to gleam something.
    --

    -Ben
  • On Nov 8, 2007, at 2:27 PM, Ben Trumbull wrote:

    > At 9:52 AM -0800 11/8/07, <cocoa-dev-request...> wrote:
    >> However the very strange thing is that I don't use Spotlight at
    >> all.  I use Core Data in a document-based application that's not
    >> saving files when this is happening.
    >
    > That's strange ... inconceivable.  Core Data doesn't do anything
    > special with or for Spotlight.

    That's what I was thinking, but the system performed otherwise.

    > Have you checked the console log ?  I've found if an app logs
    > constantly (sometimes from throwing exceptions) that it can really
    > degrade the system performance.

    Yeah, nothing there.

    > What other I/O operations is your app doing besides reading an
    > NSData ?  Are you creating or deleting a lot of files ?

    Absolutely nothing with the filesystem goes on when changing notes
    other than a possible CD fault.  Though, in theory, it should keep
    that object in memory for a period of time but this issue appears even
    when going to and fro between two notes that should already be in
    memory.  Indeed, it happened on an unsaved store as well (which is
    even less possible...).

    > You could also try stepping along through gdb and seeing what area
    > of code appears to trigger spotlight.  Without symbols, frustrating,
    > but you might be able to gleam something.

    That does appear to be the next route.  Well, that and a lot of time
    in Instruments.  However a co-worker of mine doesn't see this issue at
    all, so it's getting more and more strange that my two computers see
    it (one an upgrade install from Tiger and one a non-preserving archive
    install from a previous Leo).  I'll keep poking around, I suppose.
    Very strange...

    Adam Knight
    codepoetry - http://www.codepoetry.net/products/
  • On Nov 8, 2007, at 13:11, Adam Knight wrote:

    > On Nov 8, 2007, at 2:27 PM, Ben Trumbull wrote:
    >
    >> At 9:52 AM -0800 11/8/07, <cocoa-dev-request...> wrote:
    >>> However the very strange thing is that I don't use Spotlight at
    >>> all.  I use Core Data in a document-based application that's not
    >>> saving files when this is happening.
    >>
    >> That's strange ... inconceivable.  Core Data doesn't do anything
    >> special with or for Spotlight.
    >
    > That's what I was thinking, but the system performed otherwise.
    >
    >> Have you checked the console log ?  I've found if an app logs
    >> constantly (sometimes from throwing exceptions) that it can really
    >> degrade the system performance.
    >
    > Yeah, nothing there.
    >
    >> What other I/O operations is your app doing besides reading an
    >> NSData ?  Are you creating or deleting a lot of files ?
    >
    > Absolutely nothing with the filesystem goes on when changing notes
    > other than a possible CD fault.  Though, in theory, it should keep
    > that object in memory for a period of time but this issue appears
    > even when going to and fro between two notes that should already be
    > in memory.  Indeed, it happened on an unsaved store as well (which
    > is even less possible...).
    >
    >> You could also try stepping along through gdb and seeing what area
    >> of code appears to trigger spotlight.  Without symbols,
    >> frustrating, but you might be able to gleam something.
    >
    > That does appear to be the next route.  Well, that and a lot of time
    > in Instruments.  However a co-worker of mine doesn't see this issue
    > at all, so it's getting more and more strange that my two computers
    > see it (one an upgrade install from Tiger and one a non-preserving
    > archive install from a previous Leo).  I'll keep poking around, I
    > suppose.  Very strange...
    >

    You might also want to try fsusage to check on what mds is opening
    while it thrashes. See if it's your notes file or a different one that
    seems to be causing the choking, or if it's something else and your
    just noticing the problem while in your application. In either case,
    you might want to track down spotlight importers and see if you've got
    any overlap on the files they try to import.

    +Melissa
  • On Nov 8, 2007, at 3:46 PM, Melissa J. Turner wrote:

    > You might also want to try fsusage to check on what mds is opening
    > while it thrashes. See if it's your notes file or a different one
    > that seems to be causing the choking, or if it's something else and
    > your just noticing the problem while in your application. In either
    > case, you might want to track down spotlight importers and see if
    > you've got any overlap on the files they try to import.
    >
    > +Melissa
    >

    I followed something similar to that and ran the DTrace script
    filebyproc.d and discovered that every time I switch notes I'm hitting
    a fault and/or loading a NIB file to get the viewer (text, web, or
    PDF) and that every time I'm doing those things Spotlight is opening
    up the following files about forty times each:

      1  17600                      open:entry mds .
      1  17600                      open:entry mds .
      1  17600                      open:entry mds 0.indexGroups
      1  17600                      open:entry mds 0.indexIds
      1  17600                      open:entry mds 0.indexHead
      1  17600                      open:entry mds 0.indexPostings
      1  17600                      open:entry mds 0.indexPositions
      1  17600                      open:entry mds 0.indexDirectory
      1  17600                      open:entry mds 0.indexCompactDirectory
      1  17600                      open:entry mds 0.indexArrays
    [repeats 45 times]

    So it's not Core Data after all.  However, for some reason, accessing
    those files causes mds to go haywire.  Well, at least I have a head
    start on the issue and can try to see what's causing that.  This is
    really strange.  I suppose it's updating the access time or something,
    but even erasing the Spotlight database (by hand, from the install CD)
    didn't resolve this.  Oh well, I'll go file a bug...

    Thanks for the lead!

    Adam Knight
    Notae: Organize your notes, organize your life - http://www.codepoetry.net/products/notae/
previous month november 2007 next month
MTWTFSS
      1 2 3 4
5 6 7 8 9 10 11
12 13 14 15 16 17 18
19 20 21 22 23 24 25
26 27 28 29 30    
Go to today