Proposal for metadata interoperability on OS X

FROM : ab_lists
DATE : Tue Jul 08 09:52:48 2008

A lot of discussion on different application user forums seem to be 
going on regarding the exchange of metadata between different 
applications. Apple has provided parts of a possible solution in the 
latest versions of Mac OS X but nothing that can be seen as the final 
verdict.

First some background: with the advent of Spotlight, a lot of metadata 
is made available by application developers for use in Spotlight 
importers. It is relatively easy to extract this from your own data 
files and hand them over to Spotlight. Apple has also put a lot of 
work in providing a long list of keywords that can be used by these 
importers to store this data in the Spotlight database.

There are however problems with types of applications that are 
primarily using third party file formats (such as PDF for example) and 
that want to add application specific metadata to these files. In this 
case you cannot add these in most cases without some cumbersome 
workarounds.

Another problem is with types of applications that want to import 
metadata from third parties where it is difficult to parse the file-
format.

One mechanism that is currently used by some applications to resolve 
this is the use of Finder/Spotlight comments. Elaborate formatting 
options try to make some order out of what is essentially a free-form 
text string. Moreover, this text string is under user control and can 
be changed by her at any point in time, thereby destroying potentially 
vital data.

What is needed is a mechanism that allows application developers to 
add metadata to files without having to touch the actual file data. In 
the Mac OS days resource forks were used for this purpose, but these 
caused problems with foreign filesystems. In OS X (since Tiger) there 
is a useful mechanism called eXtended ATTRibutes that allows for 
metadata to be tacked on files. And since Leopard there is a way to 
preserve this while using the Cocoa or Unix file manipulation classes/
functions and even when storing these on non-HFS disks.

What is missing is a standardized way to set and interpret the 
metadata. What I'm proposing is to use the benefits of the Spotlight 
indexing mechanism, i.e. a dictionary of standard keywords with 
arbitrary values and use this on top of the extended attributes. This 
would allow for transparent transfer of metadata between applications, 
yet retain the use of Spotlight-based keyword searching. This would 
even work with extra keywords that might have been defined for 
Spotlight because the file type and application are known so these 
could be loaded from the application's bundle dictionary.

An example Objective-C class to implement part of this: Uli Kusterer's 
UKXattrMetadataStore class that can be found at <http://codebeach.org/code/show/15
>. Missing from this is some error checking with regards to the 
limitations of xattr and a way to map keywords to localized 
descriptions (these can be found from third-party Spotlight schemas 
but seem to be hidden for the Apple keywords).

I'm looking to start a discussion on this list that can be of benefit 
to all of us and hopefully Apple will take notice and may take our 
ideas to heart while they're working on 10.6.

Annard Brouwer
(contractor for DEVONtechnologies LLC and therefore very much involved 
in this subject at the moment)