Visualization strategy/choosing a framework
-
Hello everyone,
I've been lurking here for a while as I've been learning the Cocoa ropes,
and I'm at the point where I need to put the (excellent) Hillegass theory
into practice. My app is an interactive particle swarm optimization (
http://en.wikipedia.org/wiki/Particle_swarm_optimization for the curious)
that generates N-dimensional coordinates representing individuals in a very
compelling, organic way. For visualization, of course, I'm mainly concerned
with two- and three-dimensional instances of these swarms.
My trouble is in deciding which graphics framework to pursue for the
visualization. Since all I need are representations of points, the path of
least resistance for 2-D visualization seems to be using a simple loop with
calls to [NSBezierPath fillrect:]. However, I suspect this pattern would not
extend well to rendering a scene with 3-D instances of the swarms.
Beyond the use of Bezier paths, my knowledge falls short pretty quickly. I
often hear people sing the praises of Core Animation, but the emphasis on
layering in the documentation makes me doubt its capability to draw such a
3-D scene.
I also see Quartz mentioned as a friendlier wrapper for OpenGL, but again I
get the impression that it's more geared toward 2-D rendering than anything
else. Is a straight dive into full OpenGL the way to go here? I realize that
these frameworks have similarities, and that picking the "wrong" one won't
at all be a waste of my time, but I'd be thrilled to hear thoughts on their
comparative virtues before I take the plunge either way.
Cheers, and a Happy New Year to you all,
Adam -
On 31 Dec 08, at 19:19, Adam Foltzer wrote:
> My trouble is in deciding which graphics framework to pursue for the
> visualization. Since all I need are representations of points, the
> path of
> least resistance for 2-D visualization seems to be using a simple
> loop with
> calls to [NSBezierPath fillrect:]. However, I suspect this pattern
> would not
> extend well to rendering a scene with 3-D instances of the swarms.
It won't. Incidentally, take a look at the NSRectFillList() function
if all you're after is drawing a bunch of rectangles - it's likely to
be a lot faster than NSBezierPath once N gets large.
> Beyond the use of Bezier paths, my knowledge falls short pretty
> quickly. I
> often hear people sing the praises of Core Animation, but the
> emphasis on
> layering in the documentation makes me doubt its capability to draw
> such a
> 3-D scene.
My understanding of Core Animation is that it's designed primarily for
presenting visual effects within applications. In particular, it
doesn't have any "true 3D" effects; the closest it's got are some
effects which emulate 3D layering in various ways. See the Animation
Overview document[1] for more details.
[1]: http://developer.apple.com/documentation/GraphicsImaging/Conceptual/Animati
on_Overview/Animation_Overview.pdf
> I also see Quartz mentioned as a friendlier wrapper for OpenGL
It isn't. Quartz is a 2D graphics library and has no 3D capabilities -
if what you're after is 3D rendering of any variety, go directly to
OpenGL; do not pass GO; do not collect $200. -
On 01/01/2009, at 1:19 PM, Adam Foltzer wrote:
> I also see Quartz mentioned as a friendlier wrapper for OpenGL, but
> again I
> get the impression that it's more geared toward 2-D rendering than
> anything
> else. Is a straight dive into full OpenGL the way to go here? I
> realize that
> these frameworks have similarities, and that picking the "wrong" one
> won't
> at all be a waste of my time, but I'd be thrilled to hear thoughts
> on their
> comparative virtues before I take the plunge either way.
If you want to do 3D visualisation with good performance then OpenGL
would be by far the best option.
--
Rob Keniger -
OpenGL has a steep learning curve, but it’s the only truly 3D
framework out of all of these mentioned. Quartz is 2D-only, and only
wraps OpenGL as a matter of performance and/or convenience;
CoreAnimation is a really clever system for compositing 2D layers in
3D; managing the drawing yourself using NSBezierPath will quickly
become mind-numbing.
Fortunately, OpenGL is also one of the best-documented frameworks on
the system—once you’ve got past the initial bootstrapping and can draw
something, anything, in your NSOpenGLView, you’ll find that a Google
search can answer practically any question you might have about
drawing, projections, picking, optimizations (after you’ve profiled,
of course (: ), etc, because the vast majority of OpenGL knowledge
earned on SGI/Windows/Linux/anywhere else is applicable to OpenGL on
Mac OS X, too.
Sincerely,
Rob
On 31-Dec-08, at 10:19 PM, Adam Foltzer wrote:
> Hello everyone,
>
> I've been lurking here for a while as I've been learning the Cocoa
> ropes,
> and I'm at the point where I need to put the (excellent) Hillegass
> theory
> into practice. My app is an interactive particle swarm optimization (
> http://en.wikipedia.org/wiki/Particle_swarm_optimization for the
> curious)
> that generates N-dimensional coordinates representing individuals in
> a very
> compelling, organic way. For visualization, of course, I'm mainly
> concerned
> with two- and three-dimensional instances of these swarms.
>
> My trouble is in deciding which graphics framework to pursue for the
> visualization. Since all I need are representations of points, the
> path of
> least resistance for 2-D visualization seems to be using a simple
> loop with
> calls to [NSBezierPath fillrect:]. However, I suspect this pattern
> would not
> extend well to rendering a scene with 3-D instances of the swarms.
>
> Beyond the use of Bezier paths, my knowledge falls short pretty
> quickly. I
> often hear people sing the praises of Core Animation, but the
> emphasis on
> layering in the documentation makes me doubt its capability to draw
> such a
> 3-D scene.
>
> I also see Quartz mentioned as a friendlier wrapper for OpenGL, but
> again I
> get the impression that it's more geared toward 2-D rendering than
> anything
> else. Is a straight dive into full OpenGL the way to go here? I
> realize that
> these frameworks have similarities, and that picking the "wrong" one
> won't
> at all be a waste of my time, but I'd be thrilled to hear thoughts
> on their
> comparative virtues before I take the plunge either way.
>
> Cheers, and a Happy New Year to you all,
> Adam
-
> My app is an interactive particle swarm optimization (
> http://en.wikipedia.org/wiki/Particle_swarm_optimization for the curious)
> that generates N-dimensional coordinates representing individuals in a very
> compelling, organic way. For visualization, of course, I'm mainly concerned
> with two- and three-dimensional instances of these swarms.
>
> My trouble is in deciding which graphics framework to pursue for the
> visualization. Since all I need are representations of points, the path of
> least resistance for 2-D visualization seems to be using a simple loop with
> calls to [NSBezierPath fillrect:]. However, I suspect this pattern would not
> extend well to rendering a scene with 3-D instances of the swarms.
>
> Beyond the use of Bezier paths, my knowledge falls short pretty quickly. I
> often hear people sing the praises of Core Animation, but the emphasis on
> layering in the documentation makes me doubt its capability to draw such a
> 3-D scene.
>
> I also see Quartz mentioned as a friendlier wrapper for OpenGL, but again I
> get the impression that it's more geared toward 2-D rendering than anything
> else. Is a straight dive into full OpenGL the way to go here? I realize that
> these frameworks have similarities, and that picking the "wrong" one won't
> at all be a waste of my time, but I'd be thrilled to hear thoughts on their
> comparative virtues before I take the plunge either way.
>
So I think a quick simple distinction between Quartz, Cocoa, Core
Animation, and OpenGL would be useful. Simplistically, Quartz is the
2D drawing framework on the Mac. It does a lot of in software though
which has its strength and weaknesses. Strengths are that drawing
looks consistent on all different Macs and it provides some nice
features like anti-aliasing, rounded caps, etc, for things it draws.
Cocoa is the highest level framework which provides the easiest APIs
to use. It's drawing functionality uses Quartz underneath the hood.
Core Animation focuses less on how to actually draw underlying
primitives, but how to move them in space. You can think of Core
Animation as a rectangle engine. You draw things in rectangles and
move them around. To actually draw stuff in the rectangles, you would
generally use one of the other frameworks. Core Animation leverages
OpenGL to cache your rectangles on the video card so if the underlying
view content doesn't change, drawing can be very fast and unload the
burden off the CPU. Core Animation also leverages OpenGL to transform
these rectangles in space (position, rotation, scale). Core Animation
also provides animation functionality that lets you describe where and
how you want to animate an object, and Core Animation will animate it
for you without you having to worry about interpolating it over time
or poll the event loop.
OpenGL is your direct access to the video card. You don't get a lot of
stuff for free, but it is very fast. And it also does 3D. Unlike
Quartz, you are not guaranteed to see exactly the same thing on
different video cards.
Concerning 2D, I don't have a sense of what you plan to actually draw
and how much stuff you intend to draw. If you are planning to draw so
much that you expect to push the graphics system, OpenGL will be your
best bet. However, if you don't think you will be in this situation,
Quartz/Core Animation/Cocoa may be easier and draw things prettier
with less effort.
Scott Stevenson has a nice Core Animation based demo called NanoLife
which may be of interest to you.
http://theocacao.com/document.page/555
For 3D, Core Animation has some 3D capabilities, but it really isn't
intended for it. (In the above NanoLife link, I think somebody posted
a link to a modification that makes the spores align in a spherical
layout.) If you are serious about 3D, I would recommend OpenGL.
If you go the OpenGL route, GL_POINT_SPRITE might be one of the
primitives you want to consider to represent your object.
One last thing, you might also look into Quartz Composer. It basically
draws OpenGL, but let's you express things with wires and boxes
instead of code.
-Eric -
Thanks very much for your input, everyone. This seems to be the key theme:
On Thu, Jan 1, 2009 at 3:48 AM, Andrew Farmer <andfarm...> wrote:
> It isn't. Quartz is a 2D graphics library and has no 3D capabilities - if
> what you're after is 3D rendering of any variety, go directly to OpenGL; do
> not pass GO; do not collect $200.
>
So, I just ordered a copy of the OpenGL "SuperBible". I suspect I'll have
some frustrations ahead of me, but it should be worthwhile to learn the
low-level API (and probably make picking up CA trivial afterwards).
Eric, thanks particularly for the disambiguation between the technologies.
I've read over the high-level intro documentation for each, but without
having put any of it into much practice, I was still fuzzy about the
distinctions -- crystal clear now. As for Quartz Composer, I was looking
into that, but I would eventually like to use this visualization on the
phone, which (AFAIK) cannot render compositions.
Off to do some refactoring of my Swarm class; at the risk of spawning a
tangent, am I just completely missing a Cocoa data structure that's suited
to matrices of scalar values? I had a working implementation using nested
NSMutableArrays, but the code wound up looking disgusting (my fault, not the
API's ;-), so I rewrote it with good ol' C arrays, malloc, and free. It has
no leaks now, and is really, really fast, but I feel like I shouldn't have
to reinvent the wheel in order to do matrix operations.
Cheers and thanks again,
Adam -
Hi,
This isn't my area of expertise, but I just have one point to make:
On Dec 31, 2008, at 9:19 PM, Adam Foltzer wrote:
> I also see Quartz mentioned as a friendlier wrapper for OpenGL, but
> again I
> get the impression that it's more geared toward 2-D rendering than
> anything
> else.
I think you've confused two similarly named things. Quartz is the
name of the 2D graphics engine of Mac OS X. Quartz Composer is a
different tool. Wikipedia <http://en.wikipedia.org/wiki/
Quartz_Composer> describes it as:
> [...] a node-based visual programming language [...] for processing
> and rendering graphical data.
I googled a bit and see several demos on YouTube of Quartz Composer
being used for particle animation type stuff. I can't tell if it's
exactly what you're looking for, but it's probably worth checking out.
Cheers,
Ken -
Hi Ken,
The Particle Swarm patch in QC is exactly the sort of visual effect I'm
looking for, but unfortunately it seems like a closed box in terms of being
able to place each particle yourself. You can decide where the swarm is and
how it behaves, but not in a way that would let my algorithm place each
entity.
The more I look at QC, the more I'm certain it could do exactly what I want
on the desktop, but that leaves the phone out. I may still do a QC version
though, just for kicks :)
Cheers,
Adam
On Thu, Jan 1, 2009 at 5:13 PM, Ken Thomases <ken...> wrote:
> Hi,
>
> This isn't my area of expertise, but I just have one point to make:
>
> On Dec 31, 2008, at 9:19 PM, Adam Foltzer wrote:
>
> I also see Quartz mentioned as a friendlier wrapper for OpenGL, but again
>> I
>> get the impression that it's more geared toward 2-D rendering than
>> anything
>> else.
>>
>
> I think you've confused two similarly named things. Quartz is the name of
> the 2D graphics engine of Mac OS X. Quartz Composer is a different tool.
> Wikipedia <http://en.wikipedia.org/wiki/Quartz_Composer> describes it as:
>
> [...] a node-based visual programming language [...] for processing and
>> rendering graphical data.
>>
>
> I googled a bit and see several demos on YouTube of Quartz Composer being
> used for particle animation type stuff. I can't tell if it's exactly what
> you're looking for, but it's probably worth checking out.
>
> Cheers,
> Ken
>
>
-
Hi Adam,
Happy New Year to you too!
Go for OpenGL. If you choose something else, you might run into
problems along the road.
I know OpenGL can handle both 2D and 3D, and you get a fairly good
support from samples on the Web.
-Take a look at the CocoaGLView at http://developer.apple.com/
samplecode, you're up and running in a few minutes.
The advantage of using OpenGL is that you can...
* Make cross-platform development.
* Start out easy without any texture-fancy stuff, and then, if you
like to, you can add textures later on.
* Get the graphics card to do the work, rather than making the CPU
burn hot.
A last thing: Never use NSBezierPath, unless you're writing a program
like Illustrator. ;)
...First time I tried to make 200 pixels with NSBezierPath, I had 1
FPS; I quickly found out that NSRectFill was faster.
Love,
Jens
On Jan 1, 2009, at 04:19, Adam Foltzer wrote:
> Hello everyone,
>
> I've been lurking here for a while as I've been learning the Cocoa
> ropes,
> and I'm at the point where I need to put the (excellent) Hillegass
> theory
> into practice. My app is an interactive particle swarm optimization (
> http://en.wikipedia.org/wiki/Particle_swarm_optimization for the
> curious)
> that generates N-dimensional coordinates representing individuals in
> a very
> compelling, organic way. For visualization, of course, I'm mainly
> concerned
> with two- and three-dimensional instances of these swarms.
>
> My trouble is in deciding which graphics framework to pursue for the
> visualization. Since all I need are representations of points, the
> path of
> least resistance for 2-D visualization seems to be using a simple
> loop with
> calls to [NSBezierPath fillrect:]. However, I suspect this pattern
> would not
> extend well to rendering a scene with 3-D instances of the swarms.
>
> Beyond the use of Bezier paths, my knowledge falls short pretty
> quickly. I
> often hear people sing the praises of Core Animation, but the
> emphasis on
> layering in the documentation makes me doubt its capability to draw
> such a
> 3-D scene.
>
> I also see Quartz mentioned as a friendlier wrapper for OpenGL, but
> again I
> get the impression that it's more geared toward 2-D rendering than
> anything
> else. Is a straight dive into full OpenGL the way to go here? I
> realize that
> these frameworks have similarities, and that picking the "wrong" one
> won't
> at all be a waste of my time, but I'd be thrilled to hear thoughts
> on their
> comparative virtues before I take the plunge either way.
>
> Cheers, and a Happy New Year to you all,
> Adam
>
-
On Wed, Dec 31, 2008 at 7:19 PM, Adam Foltzer <acfoltzer...> wrote:
> Hello everyone,
>
> I've been lurking here for a while as I've been learning the Cocoa ropes,
> and I'm at the point where I need to put the (excellent) Hillegass theory
> into practice. My app is an interactive particle swarm optimization (
> http://en.wikipedia.org/wiki/Particle_swarm_optimization for the curious)
> that generates N-dimensional coordinates representing individuals in a very
> compelling, organic way. For visualization, of course, I'm mainly concerned
> with two- and three-dimensional instances of these swarms.
>
> My trouble is in deciding which graphics framework to pursue for the
> visualization. Since all I need are representations of points, the path of
> least resistance for 2-D visualization seems to be using a simple loop with
> calls to [NSBezierPath fillrect:]. However, I suspect this pattern would not
> extend well to rendering a scene with 3-D instances of the swarms.
>
> Beyond the use of Bezier paths, my knowledge falls short pretty quickly. I
> often hear people sing the praises of Core Animation, but the emphasis on
> layering in the documentation makes me doubt its capability to draw such a
> 3-D scene.
>
> I also see Quartz mentioned as a friendlier wrapper for OpenGL, but again I
> get the impression that it's more geared toward 2-D rendering than anything
> else. Is a straight dive into full OpenGL the way to go here? I realize that
> these frameworks have similarities, and that picking the "wrong" one won't
> at all be a waste of my time, but I'd be thrilled to hear thoughts on their
> comparative virtues before I take the plunge either way.
I would recommend you take a look at the open source VTK toolkit,
which can do fast (OpenGL-based) 2D and 3D visualization. The 3D
visualizations can be made particularly attractive. VTK is C++ but can
be integrated with a Cocoa UI pretty easily (see
http://www.macdevcenter.com/pub/a/mac/2003/02/11/dev_osx.html?page=1
and http://www.macresearch.org/installing_vtk_on_mac_os_x to help you
get started).
Barry
>
> Cheers, and a Happy New Year to you all,
> Adam
>
-
On 01 Jan 09, at 10:56, Adam Foltzer wrote:
> Off to do some refactoring of my Swarm class; at the risk of
> spawning a
> tangent, am I just completely missing a Cocoa data structure that's
> suited
> to matrices of scalar values? I had a working implementation using
> nested
> NSMutableArrays, but the code wound up looking disgusting (my fault,
> not the
> API's ;-), so I rewrote it with good ol' C arrays, malloc, and free.
> It has
> no leaks now, and is really, really fast, but I feel like I
> shouldn't have
> to reinvent the wheel in order to do matrix operations.
The Cocoa collection classes are primarily oriented towards dealing
with ObjC objects. If all you're working with is homogeneous arrays of
scalar values (or structures consisting of scalar values), you
probably are best off using C arrays.
That's not to say that Apple has nothing to offer. If you're
performing SIMD-esque operations on large matrices or vectors, the
Accelerate framework might have some tools you'd be interested in.
It's listed under Carbon for some reason, but it's just as applicable
to Cocoa applications.
If that doesn't do it, but you're fine with targeting Intel machines
only, dropping down to the SSE primitives may also be an option to
consider. -
> Off to do some refactoring of my Swarm class; at the risk of spawning a
> tangent, am I just completely missing a Cocoa data structure that's suited
> to matrices of scalar values? I had a working implementation using nested
> NSMutableArrays, but the code wound up looking disgusting (my fault, not the
> API's ;-), so I rewrote it with good ol' C arrays, malloc, and free. It has
> no leaks now, and is really, really fast, but I feel like I shouldn't have
> to reinvent the wheel in order to do matrix operations.
Core Animation provides CATransform3D and a few functions for
manipulating the 4x4 matrix (see CATransform3D.h). However, it is
fairly limited. As you fear, most OpenGL work ends up reinventing this
(usually in C++) . I recommend you find an open source one to your
liking and use it.
-Eric -
If it doesn't require a Cocoa app, there's also tools tuned for 2D/3D
visualization like Processing. It's very easy to get into, but since
it's based on Java, you can dig deeper. There are several libraries
that increase its usefulness, including one which apparently adds
OpenGL acceleration, and there is Mobile Processing as well. It's
probably not be the fastest thing, however.
John
On Jan 2, 2009, at 8:30 AM, Adam Foltzer wrote:
> Hi Ken,
>
> The Particle Swarm patch in QC is exactly the sort of visual effect
> I'm
> looking for, but unfortunately it seems like a closed box in terms
> of being
> able to place each particle yourself. You can decide where the swarm
> is and
> how it behaves, but not in a way that would let my algorithm place
> each
> entity.
>
> The more I look at QC, the more I'm certain it could do exactly what
> I want
> on the desktop, but that leaves the phone out. I may still do a QC
> version
> though, just for kicks :)
-
To the list this time :D
---------- Forwarded message ----------
From: Adam Foltzer <acfoltzer...>
Date: Tue, Jan 6, 2009 at 10:00 AM
Subject: Re: Visualization strategy/choosing a framework
To: Andrew Farmer <andfarm...>
I found the Accelerate framework after posting this, and have had some good
success with it on my Intel Mac. For the iPhone, I found this bit of code
for accessing the VFP unit, since Accelerate is not available there:
http://code.google.com/p/vfpmathlibrary/ . I may wind up porting more
functions from Accelerate to ARM -F assembly, since nearly everything the
algorithm does is SIMD. The instruction set for the VFP seems capable enough
to reimplement many of the veclib.h functions, which makes me wonder why
Apple hasn't done it (at least not publicly ;)
Yes, I'm also getting sidetracked, but this project is all about practice,
and this seems like an informative rabbit hole :)
Cheers,
Adam
On Tue, Jan 6, 2009 at 1:53 AM, Andrew Farmer <andfarm...> wrote:
> On 01 Jan 09, at 10:56, Adam Foltzer wrote:
>
>> Off to do some refactoring of my Swarm class; at the risk of spawning a
>> tangent, am I just completely missing a Cocoa data structure that's suited
>> to matrices of scalar values? I had a working implementation using nested
>> NSMutableArrays, but the code wound up looking disgusting (my fault, not
>> the
>> API's ;-), so I rewrote it with good ol' C arrays, malloc, and free. It
>> has
>> no leaks now, and is really, really fast, but I feel like I shouldn't have
>> to reinvent the wheel in order to do matrix operations.
>>
>
> The Cocoa collection classes are primarily oriented towards dealing with
> ObjC objects. If all you're working with is homogeneous arrays of scalar
> values (or structures consisting of scalar values), you probably are best
> off using C arrays.
>
> That's not to say that Apple has nothing to offer. If you're performing
> SIMD-esque operations on large matrices or vectors, the Accelerate framework
> might have some tools you'd be interested in. It's listed under Carbon for
> some reason, but it's just as applicable to Cocoa applications.
>
> If that doesn't do it, but you're fine with targeting Intel machines only,
> dropping down to the SSE primitives may also be an option to consider.
>
-
Just wanted to update for anyone who might be interested -- OpenGL turned
out to be perfect for this. After only the first three chapters of the
OpenGL SuperBible (and a little help from the Hillegass OpenGL chapter),
I've got a visualization up and running with great performance. Now I need
to keep reading so I can make it look halfway decent :)
Thanks again to everyone who chimed in.
Cheers,
Adam
On Tue, Jan 6, 2009 at 10:31 AM, Adam Foltzer <acfoltzer...> wrote:
> To the list this time :D
>
>
> ---------- Forwarded message ----------
> From: Adam Foltzer <acfoltzer...>
> Date: Tue, Jan 6, 2009 at 10:00 AM
> Subject: Re: Visualization strategy/choosing a framework
> To: Andrew Farmer <andfarm...>
>
>
> I found the Accelerate framework after posting this, and have had some good
> success with it on my Intel Mac. For the iPhone, I found this bit of code
> for accessing the VFP unit, since Accelerate is not available there:
> http://code.google.com/p/vfpmathlibrary/ . I may wind up porting more
> functions from Accelerate to ARM -F assembly, since nearly everything the
> algorithm does is SIMD. The instruction set for the VFP seems capable enough
> to reimplement many of the veclib.h functions, which makes me wonder why
> Apple hasn't done it (at least not publicly ;)
>
> Yes, I'm also getting sidetracked, but this project is all about practice,
> and this seems like an informative rabbit hole :)
>
> Cheers,
> Adam
>
>
> On Tue, Jan 6, 2009 at 1:53 AM, Andrew Farmer <andfarm...> wrote:
>
>> On 01 Jan 09, at 10:56, Adam Foltzer wrote:
>>
>>> Off to do some refactoring of my Swarm class; at the risk of spawning a
>>> tangent, am I just completely missing a Cocoa data structure that's
>>> suited
>>> to matrices of scalar values? I had a working implementation using nested
>>> NSMutableArrays, but the code wound up looking disgusting (my fault, not
>>> the
>>> API's ;-), so I rewrote it with good ol' C arrays, malloc, and free. It
>>> has
>>> no leaks now, and is really, really fast, but I feel like I shouldn't
>>> have
>>> to reinvent the wheel in order to do matrix operations.
>>>
>>
>> The Cocoa collection classes are primarily oriented towards dealing with
>> ObjC objects. If all you're working with is homogeneous arrays of scalar
>> values (or structures consisting of scalar values), you probably are best
>> off using C arrays.
>>
>> That's not to say that Apple has nothing to offer. If you're performing
>> SIMD-esque operations on large matrices or vectors, the Accelerate framework
>> might have some tools you'd be interested in. It's listed under Carbon for
>> some reason, but it's just as applicable to Cocoa applications.
>>
>> If that doesn't do it, but you're fine with targeting Intel machines only,
>> dropping down to the SSE primitives may also be an option to consider.
>>
>
>
>
