NSBitmapImageRep byte/color component ordering

  • Hi all,

    I'm working on an audio/video metadata tagging application.  One of
    the metadata types it can tag a file with is cover/album art.  Users
    can get art in one of three ways: selecting an image file on their
    hard drive, picking a frame from the video file itself to serve as the
    cover art (using QTMovieView from QTKit), or fetching the cover image
    from imdb.com or amazon.com (using an embedded WebView from WebKit for
    this).  No problems so far...

    I've also implemented a feature to automatically "trim" a border off
    the image.  This is mainly useful for images retrieved from
    amazon.com, which has a large white border around the actual cover art
    images.  Here's where the problem lies.  The image is shown in my UI
    in a NSImageView.  I pull the underlying NSImage out, get the
    NSBitmapImageRep's bitmap data (using NSBitmapImageRep::bitmapData),
    convert it to 32bpp (RGBA), run my border trimmer on the data, and
    then pack it into a new NSBitmapImageRep (created with NULL as the
    bitmap data, and then memcpy()ing the data in) which gets set as the
    representation for a new NSImage.

    When I do all this when the source image is a frame from the movie
    itself, it works fine.  However, when I do this for an image fetched
    from amazon.com, the post-trimmed image has the colors messed up; it
    looks like the raw bitmap data is BGRA (or maybe ABGR, not sure), but
    the new NSBitmapImageRep I create expects RGBA, so, in the final
    image, the green and red components are swapped.

    I can think of two possible ways to solve this problem, neither of
    which appears to be possible:

    1.  Convert the color order myself before copying the image data to
    the new NSBitmapImageRep, or
    2.  Tell the new NSBitmapImageRep what color order to expect.

    However, I can't seem to figure out how to 1) determine the color
    order in the source NSBitmapImageRep, or 2) tell the destination
    imagerep what color order the data is in.  Any ideas?  Am I missing
    something here?  I looked around in the list archives(*), and I found
    a couple similar posts, but with no useful answers.

    Thanks,
    Brian

    * These:
    http://lists.apple.com/archives/cocoa-dev/2002/Jan/msg00350.html (no answer)
    http://lists.apple.com/archives/cocoa-dev/2004/Jul/msg00560.html
    (possibly useful answer, but no hint as to how to do it)
    http://lists.apple.com/archives/cocoa-dev/2006/Feb/msg00185.html (no answer)
  • On 1 Nov 2007, at 00:06, Brian Tarricone wrote:

    > When I do all this when the source image is a frame from the movie
    > itself, it works fine.  However, when I do this for an image fetched
    > from amazon.com, the post-trimmed image has the colors messed up; it
    > looks like the raw bitmap data is BGRA (or maybe ABGR, not sure), but
    > the new NSBitmapImageRep I create expects RGBA, so, in the final
    > image, the green and red components are swapped.
    >
    > I can think of two possible ways to solve this problem, neither of
    > which appears to be possible:
    >
    > 1.  Convert the color order myself before copying the image data to
    > the new NSBitmapImageRep, or
    > 2.  Tell the new NSBitmapImageRep what color order to expect.
    >
    > However, I can't seem to figure out how to 1) determine the color
    > order in the source NSBitmapImageRep, or 2) tell the destination
    > imagerep what color order the data is in.  Any ideas?  Am I missing
    > something here?  I looked around in the list archives(*), and I found
    > a couple similar posts, but with no useful answers.

    As far as I'm aware, NSBitmapImageRep only supports RGBA and ARGB (for
    RGB data, that is).  You can tell which using the -bitmapFormat
    method, and you can set which using the bitmapFormat: argument when
    creating a bitmap image rep.

    If you're certain that the data you're seeing is BGRA, where are you
    getting it from?  If something is returning an NSBitmapImageRep
    containing BGRA data, AFAIK that's wrong and you should file a bug
    report.

    Hopefully, though, it's just ARGB rather than RGBA and you can check
    or tweak the bitmapFormat in your code.

    Kind regards,

    Alastair.

    --
    http://alastairs-place.net
  • On Nov 1, 2007 3:43 AM, Alastair Houghton <alastair...> wrote:
    > On 1 Nov 2007, at 00:06, Brian Tarricone wrote:
    >
    >> When I do all this when the source image is a frame from the movie
    >> itself, it works fine.  However, when I do this for an image fetched
    >> from amazon.com, the post-trimmed image has the colors messed up; it
    >> looks like the raw bitmap data is BGRA (or maybe ABGR, not sure), but
    >> the new NSBitmapImageRep I create expects RGBA, so, in the final
    >> image, the green and red components are swapped.
    >
    > As far as I'm aware, NSBitmapImageRep only supports RGBA and ARGB (for
    > RGB data, that is).  You can tell which using the -bitmapFormat
    > method, and you can set which using the bitmapFormat: argument when
    > creating a bitmap image rep.

    The source data is actually a 24bpp image, no alpha channel, which I
    get from an NSImage using -TIFFRepresentation.  I'm converting it to
    RGBA just by taking the raw pixels (using -bitmapData), creating a new
    buffer, and copying the pixels into the new array, adding a 0xff alpha
    value as the 4th uchar for every 3 uchars (taking into account row
    padding, etc.).  After I run the 32bpp data through my 'trimmer', I
    pack the data into a new NSBitmapImageRep that I create as follows:

    imgRep = [[NSBitmapImageRep alloc] initWithBitmapDataPlanes:NULL
                                                        pixelsWide:width
                                                        pixelsHigh:height
                                                      bitsPerSample:8
                                                    samplesPerPixel:4
                                                          hasAlpha:YES
                                                          isPlanar:NO
                                                    colorSpaceName:colorspace
                                                      bitmapFormat:format
                                                        bytesPerRow:0
                                                      bitsPerPixel:0];

    Then I copy the raw pixel data int [imgRep bitmapData], adding row
    padding if necessary, determined by comparing [imgRep bytesPerRow]
    with (4 * width * height).  'colorspace' and 'format' are taken from
    the source imgrep -- I've tried toggling NSAlphaFirstBitmapFormat in
    'format', but the colors are still wrong, and the alpha is noticeably
    messed up (i.e., not uniform 0xff opaque) without that flag set.

    > If you're certain that the data you're seeing is BGRA, where are you
    > getting it from?

    An NSImage, -TIFFRepresentation.  (The NSImage was created with
    -initFromContentsOfURL:.)  Note that this source NSImage displays
    correctly in a NSImageView.  No, I'm not 100% sure it's BGR, but
    that's the only explanation I can come up with for the blue and red
    colors being reversed when I copy the pixels to the new imgrep (which
    presumably expects ARGB/RGBA only).  Gonna start sampling known pixels
    from the input to see what's going on.

    > If something is returning an NSBitmapImageRep
    > containing BGRA data, AFAIK that's wrong and you should file a bug
    > report.

    If I can confirm the colors are swapped, I'll do just that; thanks.

    > Hopefully, though, it's just ARGB rather than RGBA and you can check
    > or tweak the bitmapFormat in your code.

    Or I'm just doing something else incorrectly, hopefully ^_^.  I'm
    going to try modifying everything (ugh) to just work on 24bpp images
    so I don't do the intermediate conversion.  I don't see why that could
    be a problem, since it works for some source images but not others,
    but I'm running out of ideas here.

    Cheers,
    Brian
  • >> If you're certain that the data you're seeing is BGRA, where are you
    >> getting it from?
    >
    > An NSImage, -TIFFRepresentation.  (The NSImage was created with
    > -initFromContentsOfURL:.)  Note that this source NSImage displays
    > correctly in a NSImageView.  No, I'm not 100% sure it's BGR, but
    > that's the only explanation I can come up with for the blue and red
    > colors being reversed when I copy the pixels to the new imgrep (which
    > presumably expects ARGB/RGBA only).  Gonna start sampling known pixels
    > from the input to see what's going on.

    Uncompressed TIFFs are BGR. Try -[NSBitmapImageRep initWithData:].

    -Ben
  • On Nov 1, 2007 11:33 AM, Benjamin Stiglitz <stig...> wrote:
    >>> If you're certain that the data you're seeing is BGRA, where are you
    >>> getting it from?
    >>
    >> An NSImage, -TIFFRepresentation.  (The NSImage was created with
    >> -initFromContentsOfURL:.)  Note that this source NSImage displays
    >> correctly in a NSImageView.  No, I'm not 100% sure it's BGR, but
    >> that's the only explanation I can come up with for the blue and red
    >> colors being reversed when I copy the pixels to the new imgrep (which
    >> presumably expects ARGB/RGBA only).  Gonna start sampling known pixels
    >> from the input to see what's going on.
    >
    > Uncompressed TIFFs are BGR.

    It doesn't seem consistent though, AFAICT.  This returns a 24bpp RGB image:

    QTMovieView *movieView = ....;  // video is loaded into this
    NSImage *img = [movieView currentFrameImage];
    NSBitmapImageRep *qtImgRep = [img
    TIFFRepresentationUsingCompression:NSTIFFCompressionNone

                                                              factor:0];

    Whereas this returns a 24bpp BGR image:

    NSURL *url = ...;  // loaded with URL point to jpeg image via http
    NSImage *img = [[NSImage alloc] initWithContentsOfURL:url];
    NSBitmapImageRep *urlImgRep = [img
    TIFFRepresentationUsingCompression:NSTIFFCompressionNone

                                                              factor:0];

    (Sorry, I said before I was just using -[NSImage TIFFRepresentation],
    but I made a mistake!)

    How can I tell the difference?

    > Try -[NSBitmapImageRep initWithData:].

    How does -initWithData: know the image format (bpp, alpha or no alpha,
    etc.)?  I have raw pixels here; no tiff/bmp/etc. header.

    Thanks,
    Brian
  • In the meantime, I've "solved" my problem by using -[NSImage
    TIFFRepresentationWithCompression:factor:] to get TIFF data, which I
    then feed to a CGImageSource -> CGImage, then draw to a
    CGBitmapContext (which I've created by telling it the endianness I
    want), then taking the raw bitmap data back out, applying my "border
    trimming", and then stuffing it in a new NSBitmapImageRep.  Weirdly,
    the image seems to lose some quality in the colors (maybe a colorspace
    issue?), but for now it at least works.

    Thanks to Alastair and Ben for their help.

        -brian

    On Oct 31, 2007 5:06 PM, Brian Tarricone <kelnos...> wrote:
    > Hi all,
    >
    > I'm working on an audio/video metadata tagging application.  One of
    > the metadata types it can tag a file with is cover/album art.  Users
    > can get art in one of three ways: selecting an image file on their
    > hard drive, picking a frame from the video file itself to serve as the
    > cover art (using QTMovieView from QTKit), or fetching the cover image
    > from imdb.com or amazon.com (using an embedded WebView from WebKit for
    > this).  No problems so far...
    >
    > I've also implemented a feature to automatically "trim" a border off
    > the image.  This is mainly useful for images retrieved from
    > amazon.com, which has a large white border around the actual cover art
    > images.  Here's where the problem lies.  The image is shown in my UI
    > in a NSImageView.  I pull the underlying NSImage out, get the
    > NSBitmapImageRep's bitmap data (using NSBitmapImageRep::bitmapData),
    > convert it to 32bpp (RGBA), run my border trimmer on the data, and
    > then pack it into a new NSBitmapImageRep (created with NULL as the
    > bitmap data, and then memcpy()ing the data in) which gets set as the
    > representation for a new NSImage.
    >
    > When I do all this when the source image is a frame from the movie
    > itself, it works fine.  However, when I do this for an image fetched
    > from amazon.com, the post-trimmed image has the colors messed up; it
    > looks like the raw bitmap data is BGRA (or maybe ABGR, not sure), but
    > the new NSBitmapImageRep I create expects RGBA, so, in the final
    > image, the green and red components are swapped.
    >
    > I can think of two possible ways to solve this problem, neither of
    > which appears to be possible:
    >
    > 1.  Convert the color order myself before copying the image data to
    > the new NSBitmapImageRep, or
    > 2.  Tell the new NSBitmapImageRep what color order to expect.
    >
    > However, I can't seem to figure out how to 1) determine the color
    > order in the source NSBitmapImageRep, or 2) tell the destination
    > imagerep what color order the data is in.  Any ideas?  Am I missing
    > something here?  I looked around in the list archives(*), and I found
    > a couple similar posts, but with no useful answers.
    >
    > Thanks,
    > Brian
    >
    > * These:
    > http://lists.apple.com/archives/cocoa-dev/2002/Jan/msg00350.html (no answer)
    > http://lists.apple.com/archives/cocoa-dev/2004/Jul/msg00560.html
    > (possibly useful answer, but no hint as to how to do it)
    > http://lists.apple.com/archives/cocoa-dev/2006/Feb/msg00185.html (no answer)
    >
previous month november 2007 next month
MTWTFSS
      1 2 3 4
5 6 7 8 9 10 11
12 13 14 15 16 17 18
19 20 21 22 23 24 25
26 27 28 29 30    
Go to today