Oni's Coloring Tutorial

Back to Step 1 and 2
New!: I now have a "Wet" tutorial up here.
Step 3: Shading
 

Ok, on this one I intended to do "two-layer" shading. I sometimes use one layer for very simple pieces, or three if I want a lot of shading variety. Two generally works fine though. Ok, so I go to the Skin layer, and I create two new layers above it, "s1, and s2. I then group these to the skin layer by hitting "crtl-g". This is ESSENTIAL. What this does it make it so that anything I draw on s1 or s2 will ONLY show up if they overlap the Skin layer.

[edit: I now typically only use two skin layers when I'm working with an inked image. For color-only, I typically use three skin layers, and the afore mentioned "m" layer for lips, so the hiarachy would be Skin-> s1, s2, s3, m. I often also use three layers on the clothing color, but two is generally sufficient for smaller details, since they don't need as much variety between the colors.

The s1 layer is for the soft tones, the smooth variations. The s2 layer is for the darker tones, the shadows. The s3 layer is for the REALLY dark shadows, the cast shadows, the sharp edges, like under the chin, the nostrils, the eyelid lines, creases in the knees and elbows, stuff that would generally be black lines when linework is involved.]
 

OK, so the first shading part I'll do is on S2, where I'll use a darker shade of skin tone. This will be the deep shadows for now. Generally it's a good idea to make this color slightly more red, and slightly less saturated than the base color.

[edit: This is important because, usually, if the shading is the same tonal range as the lighter areas (like the same orangey tone), then they will look flat and unnaturally orange. You basically want to move yellows towards red and purples, and greens towards blues and purples.]
 
More key commands! "Ctrl-[ and Ctrl-]" let you move up and down the layer list. Very good for only moving up one or two layers. Also, just "[ and ]" move between adjacent brush sizes. The fact that this feature doesn't work quite right in CS is the main reason I still use 5.5.
 
I made the color block in the top picture with my 65x60% (65 pixel, 60 hardness) brush.
 
I then made a new layer and painted that little bit around the cheek with the 65x25% brush, to give it a softer edge. I then grouped it to the layers below and collapsed it back into the s2 layer.
 
I then applied a layer mask and erased the whole thing. Remember that the layer mask is now saving the original block of color. So now I can paint in the bottom edge of the shadow, but the chin-line stays fully intact.  
 
I paint back this line with the 65x60 brush, and then erase back out like this with the 65x0 brush. Then I use the 100x0 brush to do that bit on her left side. I'm done with that block, so I apply the layer mask and make a new layer. I find it easier when dealing with the topmost layer of a stack to created a "throwaway" layer at the top, group it, and then put all the work layers below that ones, just so I don't have to hit "ctrl-g" after making each layer.
 
1 2 3
 
 
 
 
When I got to the hand, I made this color blob:   And then made a layer mask and erased it out to this:
 

I actually did this over several layer masks, making the mask, getting one finger right, then applying the mask and making a new mask, and repeating. The final "fade" of it, BTW, was done using the 900x0 brush. I use it instead of gradients whenever possible.

[edit: My technique for this has actually changed. Instead of painting the "negative" space like this, I would now paint the arm as a separate form, on a layer that I would name something like "arm mask", which I would generally leave orange for contrast. I would leave this layer visible, and then below that layer, so that I could only see the shadows coming out from under that layer, I would draw those cast shadows. The advantage here is thta I don't have top make and apply the mask so many times, it's cleaner and simpler.

Then, once I have the shadows how I want them, I would compress them into a single layer (but probably seperate from the adjacent layers still), select that layer, and then I would drag and drop the "arm mask" layer onto the layer mask icon at the bottom of the layers panel. What this does is create a layer mask on the selected layer (that has the shadows), in the shape of the opaque portions of the mask layer. Magic, huh? This might end up being the opposite of the effect you want, if so just hit "crtl-i" to invert the mask and you'll get the result you want. Apply, and boom, a nice, clean line between the arm and what's below (or any other bodypart that you need sharply defined from its surroundings). You'll want to save this layer mask though, it'll come in handy when you have to shade the arm.]
 
So this is the s2 layer for now. It ended up taking about 30-some layers to do, and then I collapsed them all together. My whole point on the s2 layer right now is to define the body parts. If I was keeping the linework this part would be less important, since the lines would do that for me, but I find it's hard to do the soft shadows without this step. Once I'm done the soft shading (s1) I'll come back to this layer and add the areas that need more definition.
 
On to s1. I pick a color roughly in between the other two, a little more saturated than this one, a little lighter, maybe a bit orangier
 
 

This is what I got going over the face with soft tones. At this point I generally shift gears and do all the face stuff before moving on to the body. I create "x1 and x2" layers for each of the "ex" layers, group them, and get to coloring. I do this because the face is the most important part of the piece, and making sure it looks right shaded is quite important.

Remember, the names of the layers are not important, this is just a naming scheme I use to keep everything ordered in my head. Feel free to use your own naming system, it makes no difference outside your own mind. ;)
This is also where I add my first highlights. I do this on a layer I call, appropriately enough, "highlights", and that I place right above everything else, including the linework if I have any. In this case, I apply a white highlight to the eyes, and a pink one to the lips.
 
So this is what I get:
 
At this point I decide that I should probably darken the right side of her face a bit. I make a copy of the s1 layer (by dragging it only the "new layer" icon), trim out the parts to her left of the big hair strand, hit "/" and paint in the s2 color. I then move the layer up above s2, erase out a little bit near the chin, and add a bit of shadow to her left eyelid before merging it all back into s2. OK, I think I'm done with the face for now, on to the body.
 
 
 
Don't forget to do this part on a new layer (between s1 and s2), so as to not mess up the careful work you just did on the face. Once you get the body done you'll merge it all back into s1 though. At this point I generally hide the ex1 layer, since the breasts are easier to work on without the nipples breaking up the curves.
 

I generally do the breasts after the face, since they're important too. ;) I do it like this, blocking out a huge blob of color, basically "fleshing out" one breast and to hell with everything around it. I then make a layer mask and erase out the other breast, then apply that and make a new mask, erasing out until I have most of the upper torso defined. Be careful with this process, and make back-up copies of the layers before doing anything risky. If you make a big mistake, and it's too far back in the history to correct then you might have to start the whole process over, or at least have a lot of damage control to do.

[edit: like with above, there's a slightly better way to do this, sometimes. While I still sometimes do it this way, other times I might instead make a layer above that dark layer from the first frame there, and I name this new layer "x", just so I can keep it, and everything above it, mentally separated from the one below. On this layer, instead of painting with the s1 color, I'll pain with the base tone, essentially painting in negative. This is often a more intuitive process than erasing out is. Once I have the shape looking like it does in the second picture, I'd merge all the base-tone layers into "x", then select the s1-tones layer, and do the "layer mask" trick I mentioned above to cut the x layer out of the s1-tones layer. Then you can throw the x layer away, it's served it's purpose.]
 
I did this part erasing and painting with brushes mostly in the 100x0 to 400x0 range.
 
 
So this is the result, it was all done in one layer by erasing chunks out of that blob in multiple layer masks (I probably created and applied about 10 or more to get through this part). The shape of this part depends on the pose, in this case I wanted to get it so that it would blend right with the layers I'll be doing next. You don't want hard edges or unnatural edges when you'd done with this step. Remember that if you erase too much you can create a new layer, paint in the part that was messed up, and then merge it back down and continue.

I do the right arm easy enough, but the left will be tricky without linework. I start by blobbing in all the color between the torso shading and the knees. I then mask it and erase out the left arm. Before applying it, I copy the layer, mask and all, then I select the mask and hit "ctrl-i", which is "invert colors". In a layer mask, this makes the invisible parts visible, and vise-versa. By applying both masks, I now have one layer with just the arm, and the other with everything but the arm. If you used a soft brush there will be a slight halo, however,.I'll correct that by hand painting in to fill the gaps. Layer mask the arm and start erasing and painting the shade into it, like with the torso. You might have to make multiple copies of this layer, since fingers are twitchy.

[edit: you can use the arm mask from above to make this a little easier.]

 
So this is the finished arm. It took about 4 copies of the arm layer, one to get the basic fingers, one to get the forearm, and then two to get the knuckles and gradation of the fingers just right. I merge those all together and start on the lower body.
 
 
I find it easiest to start by erasing the thighs. They make nice, simple ovoid shapes. Go nuts with the big brushes on the thighs. I used the 4-900's for most of the shading here.
 
Now the belly's always hard, at least for me. I consider myself far from a master of the secrets of the female belly. The basic principle's the same though, mask, erase out, paint back in. I suggest doing the outer borders of the belly first, and then doing the center line on a separate layer. Avoid overemphasizing the abs, it's very easy to go from "sexily tight", like Britney, to "muscle girl". Going for a "one pack" or "two pack" (2 vertical muscle groups) tends to look better than going for a "six pack".
 
This is what I ended up with:
 
 
At this point I decided to merge this layer with the torso one, mask that, and continue to work on the mid-torso for a bit. A few more passes leaves me with this:
 
Now, looking it over, I think it's clear that the darker curves need more shadow to them, so it's back to s2 for another pass. So about 14 work layers later, I end up with this:
 
Now it's time to add the nails and shade the nipples. I go back to EX1 and draw in the nails. I think they look all right. I then go in and shade everything. Remember to drop back to S2 and add drop shadows under the nails. So that's that:
 
Now on to hair. Hair uses the same basic technique (two shading layers), but used slightly differently. I use a very large, hard brush (150x98 usually), paint in a line, and then erase back out with a slight softer one (150x85 usually). I repeat this over and over to build the shapes of various hairs. It's an anime inspired technique. Once I've done this, I tend to do another layer of it in the same color that takes up more space, and then use a very large brush to erase part of it, like so:
j
 
Normally I merge this layer with the h1 layer, but I'll keep it in the psd for you guys to check out. I've labeled it "hg". Next I do the dark shadows. Since this is such dark hair already, I create a temp layer above it all that's transparent white to lighten everything. I'll delete it when I'm done.
 

OK, next I'll highlight the hair. I use a Color Dodge layer for this (selected from the drop down menu on the Layers pallet that defaults to "Normal". I call this layer "lights" btw.A color dodge layer makes it so that blacks painted onto it are invisible, but light colors overbrighten the area under it. Be careful, because a little dodging can go a LONG way. Go into the PSD for this tutorial and change the Lights layer from "Color Dodge" back to "normal", you'll see how translucent and light the actual highlights are, but they have a great effect when applied.

One trick I picked up is to make the bottom color dodge layer black, and then I can group normal layers above it and they'll be considered "dodge" layers as well, since they inherit the properties of the layer they're grouped to. When I'm done putting in the color dodge, I can remove the black layer and compress them all together, then make that color dodge. I'm painting with white onto this layer, but depending on the hair color other colors might give a better look, experiment..

 
Next, the rest of the body needs highlights. I do this in a very very light version of the base skintone, maybe a little orangier, on the highlights level (although on other pieces I do this on the Lights layer).Play with the HSV on the highlight layer to see where I applied highlights, I generally use them sparingly. The brighter and harder edged the highlights are, the "shinier" the character will look. I also decided to give her some eyeshadow, which I put on a multiply layer above s2.
 
I also decided at this point to "pinken up" the s1 and s2 layers a bit, by using "adjust hue and saturation" to color shift those layers to the left a bit.
 
So that's it. Tweak it until it looks good, flatten it, and save it as a jpg (or whatever).
 
I hope you found that helpful, and that you at least picked up a few pointers. If you have any further questions, feel free to e-mail me at ohoni@hotmail.com
 
PS, one fun thing to play with, and one reason that I love layers, is that using the PSD you can change her hair and skin color pretty easlly. Give it a shot. If the shading was all on the same layer as the base colors, this would not end up looking very good, because the contrast and tonal range of the shading varies depending on the color. For example, if you merge all three hair layers, and shift it until the base color is a light blonde, the shading will look very dirty and green. Properly, the darker the shadows are on blonde hair, the more red they need to be. By keeping the shading on different layers, you can adjust these at will.