Cel Painting in Photoshop
© 2009 Robin Wood
Hi! This short tutorial will explain how to paint in the lines with no worries or mistakes using Photoshop.
I’ll be using CS4, but this will work in any version of Photoshop that has layers.
The first thing you need to do is change your sketch or drawing into outlines on a transparent ground. I have another tutorial about that, if you’re not sure how to do it. There’s a link to it on the page. [http://www.robinwood.com/Catalog/Technical/PhotoshopTuts/PhotoTutPages/WhiteOutPS.html]
It’s possible to follow this technique using the Multiply blend mode, but I’ve found that using outlines on transparency gives you a cleaner image, and more flexibility, since you can manipulate the outlines as well.
The Outline layer will go on the top of your layer stack.
As you can see, I've got that ready here. Notice the checkerboard.
Then all you have to do is take the Magic Wand, and click in a section that you would like to fill. [Contiguous is Enabled, and Sample All Layers is disabled, in Magic Wand Options.]
If more than one section will be treated as a single color, hold down the Shift key and click to select them all.
Expand the selection by going to Select > Modify > Expand...
That brings up a dialog that lets you choose how much you want to increase it.
Choose a width that will take the selection to about the middle of your outlines.
In this case, that’s about 4 pixels (we’re looking at the image at 33.33% size, as you can see here in the corner.)
This will make sure that there are no gaps between the outline and the color, which is likely to happen otherwise because of anti-aliasing.
With the selection still active, make a New Layer below the Outline Layer.
Hold down the Command key (that's ctrl on a PC) while you click the New Layer icon at the bottom of the layer panel to make a layer below the currently selected layer.
Hold down the Option key, and tap Delete (that's alt and backspace on a PC) to fill the selection with the foreground color.
Command D (ctrl D on a PC) to drop the selection. 
Check the piece, especially in small areas or narrow angles, and make sure that there’s color in all of them.
If there are any that need touch up, just paint it in there. Shouldn’t take more than a minute or two to do that.
Now, if you want flat color, and this is the color you wanted, you’re done.
If you wanted flat color but this isn't the color you wanted, you can use the Hue and Saturation dialog to tweak it. Tap Command U, or ctrl U on a PC to get that.
Then just use the Hue slider to change the color, like this.
Or the Saturation. [Saturation slider]
Or the Lightness. [Lightness slider]
This works really well on a solid color, and won’t give you the low contrast you can get on an image.
But if you want to add shading, or a pattern, or do other things, that’s easy, and won’t require you to be careful about lines, or go back and erase things, or anything!
Let me show you how.
Just make a new layer, using the New Layer icon at the bottom of the Panel, and then hold down the Option key (that's alt on a PC) and click the line between the two layers.
When you are in the right place, your icon will change to two overlapping circles, kind of like the Mastercard symbol, like you see here.
When it does that, click, and you’ll have a Clipping Layer.
As you can see, the Layer Thumbnail moves to the right, and you get this bent arrow to show you that the layer is clipped to the one below it.
The pixels on this layer will only be revealed where there are active pixels in the clipping layer, at the bottom here.
Now you can paint freely on this layer, and nothing you do will show past the lines, ever!
Since it’s just a layer, you can name it, or change the Blend Mode, or whatever you need to do!
Let’s call this one Highlight, change the Blend Mode to Screen, and reduce the opacity to about 50%.
Then we'll grab the Brush tool (just tap B to get it) get white paint, and paint in some highlights, just like that, with no thought for the edges.
The thing that’s really great about Clipping Layers, and the reason that they’re better than a simple layer mask for things like this, is that you can add as many layers as you like, and they’ll all be clipped by the one on the bottom.
So we can add another layer, Clip it, call it Shadow, set to Multiply blend mode, tap X to swap the foreground and background colors and paint some shadows.
Let's reduce the opacity of these just a little.
We can use the Smudge tool, and never worry about what’s happening on the edges.
You can even add Fill and Adjustment layers, say a Pattern layer, and it, too will stay in the lines!
If you add it under the others, it will automatically be added to the clipping layers. Otherwise, just clip it, like we did before.
[Adds seamless Tie Dye pattern.]
And there we go – Sixties Bluenose.
Okay, that’s a bit much, let’s change that to canvas, like that. 
And then paste in a stripe that I have ready on the clipboard. [Command V to Paste, ctrl V on a PC] And there it is.
We can even distort it, so it matches the bell of the sail. (You won't have the Warp tool unless you're using CS, but if you are, then it's a great thing to have.) 
You can get as complex or as simple as you like, and you’ll never have to think about staying in the lines again.
Now If you want to go outside the lines, all you have to do is just draw on the bottom layer, and whatever you have on the layers above will show.
Just repeat this for each section, and you’ll have a beautifully colored image, with no tears!
Remember to go back to the Outline layer when you use the Magic Wand, to grab your section, and you’ll be fine. [Or enable "Sample All Layers" in the Magic Wand options.]
This is Robin Wood, and I hope you found this helpful.