Image scaling

  • I think I'm missing something here.

    I need to take an image and scale it (very cleanly) down while keeping
    the metadata.  The source is a JPEG file on-disk (typically from a
    digital camera).  The destination is an NSData object in memory that
    is JPEG-encoded with the same metadata.

    I tried this using CIImage and a CILanczosScaleTransform filter then
    using NSBitmapImageRep's initWithCIImage method.  This worked very
    well and made a great image, but it looks like Apple's initWithCIImage
    code leaks a whole copy of the image outside of my reach.  I'm working
    with too many images for this to be good.  Also, this method drops the
    file metadata entirely (EXIF, etc.) and doesn't appear to give me the
    chance to add it back in.

    So then I tried using a CGImageRef with
    CGImageSourceCreateThumbnailAtIndex and copying the metadata with
    CGImageSourceCopyPropertiesAtIndex.  This works well except that the
    scaled image is blurry and not at all professional.  I tried fiddling
    with CGContextSetInterpolationQuality to no avail.  Almost there.

    It appears that NSImage does not scale the backing image, but only the
    display one.  Tricks involving drawing onto a new context are not
    thread-safe.  Thus, I cannot use that.

    The only solution that's coming to mind is one I really dread:

    * Get the JPEG data with a CGImageSourceRef
    * Store the metadata
    * Make a CIImage (from the original file or the CGImage)
    * Do the transform that works
    * Make the CIImage a CGImage without using NSBitmapImageRep (Can I
    even do this? I haven't found this path.)
    * Copy the metadata to a CGImageDestinationRef item along with the
    CGImage
    * Generate the JPEG data object using CGImageDestinationCreateWithData

    This seems overly complicated just to scale an image properly.
    Everything appears highly optimized for drawing on-screen rather than
    to a data structure in a threaded app.

    Can anyone tell me how I can easily scale an image with a good/
    excellent quality in a thread-safe manner in Mac OS X?

    Adam Knight
    Co-Founder; CTO
    Barton Springs Software
  • On Jan 4, 2008, at 4:25 PM, Adam Knight wrote:

    > I tried this using CIImage and a CILanczosScaleTransform filter then
    > using NSBitmapImageRep's initWithCIImage method.  This worked very
    > well and made a great image, but it looks like Apple's
    > initWithCIImage code leaks a whole copy of the image outside of my
    > reach.  I'm working with too many images for this to be good.  Also,
    > this method drops the file metadata entirely (EXIF, etc.) and
    > doesn't appear to give me the chance to add it back in.

    I take this back -- even this method produces slightly blurry scaled
    images.  Is there anything in the system that can make a sharp, proper
    scaled image?

    Examples:
    http://www.bssware.com/~ahknight/good_scale.jpg (Image uploaded to
    Flickr in full size, the resulting image at 500px wide)
    http://www.bssware.com/~ahknight/bad_scale.jpg (Scaled using either
    CG or CI to 500 pixels wide)

    Adam Knight
    Co-Founder; CTO
    Barton Springs Software
  • I'd suggest that if you are trying to demonstrate an image's sharpness,
    a lossy format like JPG is not the format to use :)

    FWIW it's entirely possible that Flickr is explicitly running a
    sharpening filter on the image before scaling it down. I am not 100%
    sure but I am fairly positive that Google's photo service (Picasa) does
    this when you scale down images from within its iPhoto plugin.

    Adam Knight wrote:
    > On Jan 4, 2008, at 4:25 PM, Adam Knight wrote:
    >
    >> I tried this using CIImage and a CILanczosScaleTransform filter then
    >> using NSBitmapImageRep's initWithCIImage method.  This worked very
    >> well and made a great image, but it looks like Apple's
    >> initWithCIImage code leaks a whole copy of the image outside of my
    >> reach.  I'm working with too many images for this to be good.  Also,
    >> this method drops the file metadata entirely (EXIF, etc.) and doesn't
    >> appear to give me the chance to add it back in.
    >
    > I take this back -- even this method produces slightly blurry scaled
    > images.  Is there anything in the system that can make a sharp, proper
    > scaled image?
    >
    > Examples:
    > http://www.bssware.com/~ahknight/good_scale.jpg (Image uploaded to
    > Flickr in full size, the resulting image at 500px wide)
    > http://www.bssware.com/~ahknight/bad_scale.jpg (Scaled using
    > either CG or CI to 500 pixels wide)
    >
    >
    > Adam Knight
    > Co-Founder; CTO
    > Barton Springs Software
  • I have code that creates a JPEG thumbnail in an NSData using
    NSBitmapImageRep and drawing into a new context, in a very threaded context
    (often several threads doing it at the same time on different directories).
    It was, admittedly, leaking like heck down in the OS somewhere, but I
    wrapped it in my own autorelease pool and that seems to have cured it.  I
    have had a thread work on 1M+ images and not run out of memory.

    Not sure how to help you with the metadata, I explicitly wanted to strip it
    for my app, so I can't help you there.  But let me know (off list) if you
    want to see my code, and I'll happily provide it.  I'm not 1000% sure it's
    rock solid, but I've been using it for a few months now in testing, and I
    can't break it...

    Chris

    > From: Adam Knight <adam.knight...>
    > Subject: Image scaling
    >
    > I think I'm missing something here.
  • Yep, the one on flickr definitely looks to me like it was processed with an
    unsharp mask. Anyway, Lanczos is about the best image scaling algorithm
    you'll find.

    --
    Scott Ribe
    <scott_ribe...>
    http://www.killerbytes.com/
    (303) 722-0567 voice
  • And I forgot the most important point. I believe that NS image classes do
    some caching of images for performance, behind your back, out of your
    control. It's possible that accounts for the perceived leak that you saw,
    and that if you use your initial method to process a large number of images,
    the cache will get flushed before the "leak" grows too large.

    Of course it's also possible there's a bug: in your code with
    retain/release, or no autorelease pool in a thread; or less likely in Cocoa
    functions.

    --
    Scott Ribe
    <scott_ribe...>
    http://www.killerbytes.com/
    (303) 722-0567 voice
  • On Jan 4, 2008 2:25 PM, Adam Knight <adam.knight...> wrote:
    > I think I'm missing something here.
    >
    > I need to take an image and scale it (very cleanly) down while keeping
    > the metadata.  The source is a JPEG file on-disk (typically from a
    > digital camera).  The destination is an NSData object in memory that
    > is JPEG-encoded with the same metadata.
    >
    > I tried this using CIImage and a CILanczosScaleTransform filter then
    > using NSBitmapImageRep's initWithCIImage method.  This worked very
    > well and made a great image, but it looks like Apple's initWithCIImage
    > code leaks a whole copy of the image outside of my reach.

    Consider using vImage (supports "Lanczos3" and "Lanczos5").

    <http://developer.apple.com/documentation/Performance/Conceptual/vImage/vima
    ge_geometry/chapter_6_section_2.html#//apple_ref/doc/uid/TP30001001-CH207-T
    PXREF103
    >

    -Shawn
  • On Jan 4, 2008, at 5:51 PM, John Stiles wrote:
    > I'd suggest that if you are trying to demonstrate an image's
    > sharpness, a lossy format like JPG is not the format to use :)
    >
    > FWIW it's entirely possible that Flickr is explicitly running a
    > sharpening filter on the image before scaling it down. I am not 100%
    > sure but I am fairly positive that Google's photo service (Picasa)
    > does this when you scale down images from within its iPhoto plugin.

    That's actually exactly what's happening.  I did some searching and
    they boost saturation and run a sharpness filter on the thumbnails.
    So, using Core Image, I did the same things after running the Lanczos
    filter and managed to get a decent result.  Thanks for the lead. :)

    Adam Knight
    Co-Founder; CTO
    <adam.knight...>
previous month january 2008 next month
MTWTFSS
  1 2 3 4 5 6
7 8 9 10 11 12 13
14 15 16 17 18 19 20
21 22 23 24 25 26 27
28 29 30 31      
Go to today