01 November 2013
Tutorial: Paint Brushes, Trailing Object Effects, and More with the Snapshot Canvas
Today, I’m going to show you how to use snapshots to achieve the sorts of tricks you might be used to doing if you did a lot of traditional CPU-based computer graphics.
One of the old school tricks is to be able to touch pixels on the frame buffer directly, or modify the pixels of an image directly. In a GPU world, you cannot do that b/c passing memory between the CPU and GPU is extremely expensive.
Versatility of Snapshots
Snapshots to the rescue!
Normally, you use snapshots as a one-time operation to cache a single rendered result. They work by adding objects to the snapshot’s ‘group’ property, and then calling ‘invalidate()’ each time you want to render the group’s children to the texture. This lets you achieve cool effects like this Mode7 demo.
We recently added some new canvas features to let you manipulate the snapshot texture in new and interesting ways.
Here’s a video showing you a trailing brush effect that renders onto the snapshot. Underneath the snapshot is a background image of the world:
One way to achieve this affect is to keep track of every brush image we draw and then fade it out over time. That’s pretty complex and unwieldy to write code for.
A far easier way approach is to draw the brush image onto the snapshot for every touch event. The trick is to draw a black translucent rectangle in between each touch, thus causing previously drawn images to appear to fade away. It’s a common technique from the CPU days, but now you can achieve it on the GPU!
In Corona, the way to do this is to use the new canvas feature of snapshot, combined with support for Porter-Duff blend modes:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 |
local w = display.viewableContentWidth local h = display.viewableContentHeight local background = display.newImage( "world.jpg", w*0.5, h*0.5 ) local snapshot = display.newSnapshot( w,h ) snapshot:translate( w * 0.5, h * 0.5 ) snapshot.canvasMode = "discard" function listener( event ) local x,y = event.x - snapshot.x, event.y - snapshot.y if ( event.phase == "began" or event.phase == "moved" ) then local r = display.newRect( 0, 0, w, h ) r:setFillColor( 0, .98 ) r.blendMode = "dstIn" -- enables snapshot to render over background local o = display.newImage( "brush.png", x, y ) o:setFillColor( 1, 0, 1 ) snapshot.canvas:insert( r ) snapshot.canvas:insert( o ) snapshot:invalidate( "canvas" ) -- accumulate changes w/o clearing end end Runtime:addEventListener( "touch", listener ) |
Incidentally, if you take out the black rectangles between each touch, you can also build a simple paint brush program (code).
The Snapshot Canvas
The new snapshot.canvas
lets you draw onto the snapshot texture without clearing between invalidates. In order to render these objects to the snapshot, you invalidate with the “canvas” parameter, e.g. snapshot:invalidate( "canvas" )
.
We’ve also added a snapshot.canvasMode
property that lets you control what happens to the children between invalidates. Normally, the canvas group is emptied, and the children are appended to the snapshot’s main group. This ensure that your snapshot texture isn’t lost, which sometimes happens when your app is suspended. If you don’t care to preserve your edits, you can throw away the children via the "discard"
mode.
Porter-Duff Blend Modes
The Porter-Duff blend modes are something we’ve also added. The normal blend mode that we’re all used to corresponds to “srcOver”, but there are a ton of other modes like "clear"
, "xor"
, "dstIn"
, "src"
, "dst"
, etc that let you do a lot of amazing things.
In the example above, we could have used the normal blend mode. However, if you do that, the background of the snapshot will become black and you won’t be able to see the image of the world behind the snapshot. The key is to only fade out the portions of the snapshot that are already opaque. And so in the example above, we use the "dstIn"
which multiplies the snapshot texture with the alpha of the black rectangle.
In Summary
As you can see, snapshots are an incredibly powerful and versatile tool. And yes, there are some obvious next steps for snapshots (e.g. using them as textures for other objects), so we’ve put that on our roadmap. I think what’s really amazing is that this example only used a couple of features, so we’ve only scratched the surface of what’s possible!
David Condolora
Posted at 06:52h, 01 NovemberFor trivia fans, the “Porter” part of the Porter-Duff Blend Modes name is for Tom Porter, who worked at the Lucasfilm Computer Division in the 80s, which eventually became…Pixar. He’s still there to this day, as part of Pixar’s senior leadership team.
Here’s a link to the paper he co-authored: http://graphics.pixar.com/library/indexAuthorPorter.html
Jacques
Posted at 18:33h, 01 NovemberIm curious to know how much processing power all these new api functions take. They do look great but if you end up using a bunch of them to get a decent looking realtime result is it going to kill the app with lag once you have other things going on? Would be nice to know from anyone trying it out if most of these new 2.0 effects can be used effectively in an game/app that isn’t similar to a paint program.
Jen
Posted at 20:13h, 01 NovemberThis is really nice, starting to get the hang of the new graphics engine. This tutorial in particular gave me a boost. thanks!
Ed Johnson
Posted at 05:12h, 03 NovemberA gem of post on advanced graphics technique and usage rarely found elsewhere! Enjoyed reading the 2 references on the Pixar link suggested by David. Thanks and keep advancing the Corona SDK’s cutting-edge capabilities.
Andy
Posted at 13:10h, 04 Novemberthis is very cool, but i’ve noticed if you have an effect (filter) on the snapshot it only gets applied when you invalidate the snapshot , not when you invalidate the canvas… would it be possible to add effects to the canvas? that way you could have cool iterative effects, like gaussian blurs or blooms.
Walter
Posted at 13:31h, 04 NovemberYes, you can add an effect on the snapshot’s fill just like you would on a normal rectangle. Try adding the following before the function listener is declared:
snapshot.fill.effect = “filter.sepia”
You can also set effects on a per-object basis if you don’t want a snapshot-wide effect.
Andy
Posted at 19:16h, 04 Novemberright, but the effect will only be run when you call
snapshot.invalidate()
not when you call
snapshot.invalidate(“canvas”)
is that correct?
it would be very cool if you could run the effect iteratively on the canvas without clearing it, for example progressively blurring effects as you fade them using the rectangle trick…..
Andy
Posted at 19:27h, 04 Novemberalso, :), in your example code the snapshot fade never quite goes to zero opacity, but seems to stop at a small opacity value.. this is fine over black, but i’m trying to use this as an effect layer over the top of my game, and it leaves ghost images of the effects. i can send you an image and submit a report if you think it’s a bug not a feature 🙂
Walter
Posted at 15:24h, 05 NovemberI think we’re talking about different things.
If you set the effect as I show, you do NOT need to call invalidate b/c the effect gets applied to as a post-processing step on the entire texture.
If you want to apply effects on individual objects that are added to the snapshot group, just set them on an per-object basis. In this case, yes, you would need to call invalidate to ensure objects you added are drawn into the snapshot texture on the next render pass.
Andy
Posted at 16:27h, 05 Novemberi want to apply the effect on the snapshot per frame, without clearing the result of the prior effect. leading to compounded effects….
eg i want some particles to start out as sharp images , but then fade and increasingly blur.
frame 1:
i draw some particles
frame 2:
blur the snapshot –
*draw a rectangle 5% alpha * – to fade
draw some more particles
frame 3
blur the snapshot:
*draw a rectangle 5% alpha *
draw some more particles
the particles from frame 1 are now blurred and faded twice, the effect is compounding.
does that make sense?
in your suggested setup, the result of the filter isn’t passed back into the snapshot, so the blur wouldn’t be additive.
Tom
Posted at 03:10h, 05 DecemberDid this ever get resolved? I’m seeing the same issue. I’m using the technique to render firework trails, but they never quite fade out.
Tony
Posted at 15:43h, 04 NovemberThanks for for throwing this stuff out here and I look forward to digging deeper into 2.0’s power. In this example, is there any kind of memory build up? If not, why? Thanks again!
Walter
Posted at 15:22h, 05 NovemberThe drawing happens on a single texture so there’s little impact. In this particular example, I further reduce the memory impact via the “discard” mode. Normally (discard off), the display objects are saved into the group. When discard is on, these objects are thrown away after each render to texture.
Alex
Posted at 15:53h, 05 NovemberI keep getting the error (running today’s build) while trying to reproduce the code above. Any ideas why?
main.lua
Line: 6
Attempt to call field ‘newSnapshot’ (a nil value)
Rajat
Posted at 06:07h, 03 DecemberNice tutorial, it did helped us building a full painting app however there seems to be an issue with the snapshot object on android. As soon as you return after the app has been suspended the snapshot data is lost (This does not occur on iOS).
This issue is present in your demo as well.
The only solution that comes as an option here is to listen for the system events (applicationSuspend, applicationResume) and capture a png file on “applicationSuspend” and load that png once the “applicationResume” fires in to the snapshot. However whenever you try to save a snapshot or a group containing a snapshot on “applicationSuspend” and return to the application, you’ll return with a blank black screen.
Any ideas anybody how to make a workaround on this issue