Tutorial: Paint Brushes, Trailing Object Effects, and More with the Snapshot Canvas

Graphics 2.0 Demo Contest Winners
Share on Facebook0Share on Google+1Tweet about this on TwitterShare on LinkedIn1

gfx2-iconToday, I’m going to show you how to use snapshots to achieve the sorts of tricks you might be used to doing if you did a lot of traditional CPU-based computer graphics.

One of the old school tricks is to be able to touch pixels on the frame buffer directly, or modify the pixels of an image directly. In a GPU world, you cannot do that b/c passing memory between the CPU and GPU is extremely expensive.

Versatility of Snapshots

Snapshots to the rescue!

Normally, you use snapshots as a one-time operation to cache a single rendered result. They work by adding objects to the snapshot’s ‘group’ property, and then calling ‘invalidate()’ each time you want to render the group’s children to the texture. This lets you achieve cool effects like this Mode7 demo.

We recently added some new canvas features to let you manipulate the snapshot texture in new and interesting ways.

Here’s a video showing you a trailing brush effect that renders onto the snapshot. Underneath the snapshot is a background image of the world:

One way to achieve this affect is to keep track of every brush image we draw and then fade it out over time. That’s pretty complex and unwieldy to write code for.

A far easier way approach is to draw the brush image onto the snapshot for every touch event. The trick is to draw a black translucent rectangle in between each touch, thus causing previously drawn images to appear to fade away. It’s a common technique from the CPU days, but now you can achieve it on the GPU!

In Corona, the way to do this is to use the new canvas feature of snapshot, combined with support for Porter-Duff blend modes:

local w = display.viewableContentWidth
local h = display.viewableContentHeight

local background = display.newImage( "world.jpg", w*0.5, h*0.5 )

local snapshot = display.newSnapshot( w,h )
snapshot:translate( w * 0.5, h * 0.5 )
snapshot.canvasMode = "discard"

function listener( event )
   local x,y = event.x - snapshot.x, event.y - snapshot.y

   if ( event.phase == "began" or event.phase == "moved" ) then
      local r = display.newRect( 0, 0, w, h )
      r:setFillColor( 0, .98 )
      r.blendMode = "dstIn" -- enables snapshot to render over background

      local o = display.newImage( "brush.png", x, y )
      o:setFillColor( 1, 0, 1 )

      snapshot.canvas:insert( r )
      snapshot.canvas:insert( o )
      snapshot:invalidate( "canvas" ) -- accumulate changes w/o clearing

Runtime:addEventListener( "touch", listener )

Incidentally, if you take out the black rectangles between each touch, you can also build a simple paint brush program (code).

The Snapshot Canvas

The new snapshot.canvas lets you draw onto the snapshot texture without clearing between invalidates. In order to render these objects to the snapshot, you invalidate with the “canvas” parameter, e.g. snapshot:invalidate( "canvas" ).

We’ve also added a snapshot.canvasMode property that lets you control what happens to the children between invalidates. Normally, the canvas group is emptied, and the children are appended to the snapshot’s main group. This ensure that your snapshot texture isn’t lost, which sometimes happens when your app is suspended. If you don’t care to preserve your edits, you can throw away the children via the "discard" mode.

Porter-Duff Blend Modes

The Porter-Duff blend modes are something we’ve also added. The normal blend mode that we’re all used to corresponds to “srcOver”, but there are a ton of other modes like "clear", "xor", "dstIn", "src", "dst", etc that let you do a lot of amazing things.

In the example above, we could have used the normal blend mode. However, if you do that, the background of the snapshot will become black and you won’t be able to see the image of the world behind the snapshot. The key is to only fade out the portions of the snapshot that are already opaque. And so in the example above, we use the "dstIn" which multiplies the snapshot texture with the alpha of the black rectangle.

In Summary

As you can see, snapshots are an incredibly powerful and versatile tool. And yes, there are some obvious next steps for snapshots (e.g. using them as textures for other objects), so we’ve put that on our roadmap. I think what’s really amazing is that this example only used a couple of features, so we’ve only scratched the surface of what’s possible!

Share on Facebook0Share on Google+1Tweet about this on TwitterShare on LinkedIn1

This entry has 15 replies

  1. David Condolora says:

    For trivia fans, the “Porter” part of the Porter-Duff Blend Modes name is for Tom Porter, who worked at the Lucasfilm Computer Division in the 80s, which eventually became…Pixar. He’s still there to this day, as part of Pixar’s senior leadership team.

    Here’s a link to the paper he co-authored: http://graphics.pixar.com/library/indexAuthorPorter.html

  2. Jacques says:

    Im curious to know how much processing power all these new api functions take. They do look great but if you end up using a bunch of them to get a decent looking realtime result is it going to kill the app with lag once you have other things going on? Would be nice to know from anyone trying it out if most of these new 2.0 effects can be used effectively in an game/app that isn’t similar to a paint program.

  3. Jen says:

    This is really nice, starting to get the hang of the new graphics engine. This tutorial in particular gave me a boost. thanks!

  4. Ed Johnson says:

    A gem of post on advanced graphics technique and usage rarely found elsewhere! Enjoyed reading the 2 references on the Pixar link suggested by David. Thanks and keep advancing the Corona SDK’s cutting-edge capabilities.

  5. Andy says:

    this is very cool, but i’ve noticed if you have an effect (filter) on the snapshot it only gets applied when you invalidate the snapshot , not when you invalidate the canvas… would it be possible to add effects to the canvas? that way you could have cool iterative effects, like gaussian blurs or blooms.

    • Walter says:

      Yes, you can add an effect on the snapshot’s fill just like you would on a normal rectangle. Try adding the following before the function listener is declared:

      snapshot.fill.effect = “filter.sepia”

      You can also set effects on a per-object basis if you don’t want a snapshot-wide effect.

      • Andy says:

        right, but the effect will only be run when you call
        not when you call
        is that correct?

        it would be very cool if you could run the effect iteratively on the canvas without clearing it, for example progressively blurring effects as you fade them using the rectangle trick…..

        • Andy says:

          also, :), in your example code the snapshot fade never quite goes to zero opacity, but seems to stop at a small opacity value.. this is fine over black, but i’m trying to use this as an effect layer over the top of my game, and it leaves ghost images of the effects. i can send you an image and submit a report if you think it’s a bug not a feature :)

        • Walter says:

          I think we’re talking about different things.

          If you set the effect as I show, you do NOT need to call invalidate b/c the effect gets applied to as a post-processing step on the entire texture.

          If you want to apply effects on individual objects that are added to the snapshot group, just set them on an per-object basis. In this case, yes, you would need to call invalidate to ensure objects you added are drawn into the snapshot texture on the next render pass.

          • Andy says:

            i want to apply the effect on the snapshot per frame, without clearing the result of the prior effect. leading to compounded effects….

            eg i want some particles to start out as sharp images , but then fade and increasingly blur.

            frame 1:
            i draw some particles
            frame 2:
            blur the snapshot –
            *draw a rectangle 5% alpha * – to fade
            draw some more particles
            frame 3
            blur the snapshot:
            *draw a rectangle 5% alpha *
            draw some more particles

            the particles from frame 1 are now blurred and faded twice, the effect is compounding.

            does that make sense?

            in your suggested setup, the result of the filter isn’t passed back into the snapshot, so the blur wouldn’t be additive.

          • Tom says:

            Did this ever get resolved? I’m seeing the same issue. I’m using the technique to render firework trails, but they never quite fade out.

  6. Tony says:

    Thanks for for throwing this stuff out here and I look forward to digging deeper into 2.0’s power. In this example, is there any kind of memory build up? If not, why? Thanks again!

    • Walter says:

      The drawing happens on a single texture so there’s little impact. In this particular example, I further reduce the memory impact via the “discard” mode. Normally (discard off), the display objects are saved into the group. When discard is on, these objects are thrown away after each render to texture.

  7. Alex says:

    I keep getting the error (running today’s build) while trying to reproduce the code above. Any ideas why?

    Line: 6

    Attempt to call field ‘newSnapshot’ (a nil value)

  8. Rajat says:

    Nice tutorial, it did helped us building a full painting app however there seems to be an issue with the snapshot object on android. As soon as you return after the app has been suspended the snapshot data is lost (This does not occur on iOS).
    This issue is present in your demo as well.

    The only solution that comes as an option here is to listen for the system events (applicationSuspend, applicationResume) and capture a png file on “applicationSuspend” and load that png once the “applicationResume” fires in to the snapshot. However whenever you try to save a snapshot or a group containing a snapshot on “applicationSuspend” and return to the application, you’ll return with a blank black screen.

    Any ideas anybody how to make a workaround on this issue

Leave a reply

You may use these HTML tags and attributes: <a href="" title="" rel=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>