decoration left 1
decoration left 2
decoration rigth

Show/Hide search bar
black cat logo variable logo
[2 Apr 2013]

Non-photorealistic Rendering (Cel shading)

Non-photorealistic rendering can be used in stylized computer games, animated films, interactive comics, for technical illustrations, for simulation of artistic drawing and hatching, etc. In many cases such rendering reduces amount of information that users have to perceive.
This article describes how to create cel shading effect (Toon shading) with help of OpenGL and GLSL. This effect adds a silhouette to an image and reduces number of colors that are used.
There're different methods to create cel shading effect. In simplest case, mesh is visualized twice: to create silhouette and to create stylized colors. Two different shaders are required to accomplish this.
Creation of silhouette
First of all to create silhouette we should render enlarged mesh. This can be done with help of simple shader that assigns constant color (color of silhouette - black) for each rendered fragment and shifts each vertex of the mesh along normal of the vertex by constant value (size of silhouette). You should allow rendering only of back-faces of the mesh. First image depicts slightly enlarged mesh that is rendered with black color.
Visualize same mesh once more with same shader. But now set shift along normal to 0 and constant color to white, disable writes to depth buffer and enable visualization of front-faces of the mesh. Result is depicted on second image. As you can see, result of second draw call is overlayed over result of first draw call (if you need not only silhouette, but also effect of decreased number of colors, then this second draw call isn't required).
By shifting each vertex along normal and additionally in constant direction you can achive effect of "fat" mesh (as on third image).
Rendering of enlarged black cat mesh with enabled rendering of back-faces
Silhouette of cat: white cat mesh over enlarged black cat mesh
Silhouette of "fat cat"
Decreased number of colors
Effect of decreased number of colors is calculated in fragment shader. Calculate each component of standard Blinn-Phong lighting (ambient, diffuse and specular) and add them together. Clamp total value to range [0, 1]. With help of this value you can access to 1D texture that contains a set of colors for cel shading. High total lighting value will sample colors from right part of the texture, and low lighting intensities will sample colors from left part of the texture. Number of different colors on cel shading results will be same as number of colors in this texture.
Texture with mapping of lighting intensity to cel shading color

Cel shading effect can be achieved without usage of intensity to color texture. You can define base color for cell chading and quantize value of total intensity. For example by following statement: shadeIntensity = ceil(intensity * numShades)/numShades;

Ceil() function rounds input argument upward and returns the smallest integral value that is not less than input argument. NumShades - number of different colors that are required for cel shading.

Quantization of intensity

ShadeIntensity value can be used for modulation of cell shading base color, as on first image. If mesh has diffuse texture, then you can modulate sampled color by value of ShadeIntendity, as on second image. Or you can mix cel shading base color with color sampled from diffuse texture, and modulate result with shadeIntensity value, as on third image. finalColor = baseColor * shadeIntensity; // modulation base color
finalColor = modelTexture * shadeIntensity; // modulation of color from texture
finalColor = baseColor * modelTexture * intensity; // mixed modulation

Modulation of base color with shadeIntensity
Modulation of texture color with shadeIntensity
Mixed modulation

Effect of metallic cartoon
There are many other ways to produce non-realistic images. And many of them are quite easy to implement. For example metallic cartoon effect. You can add this effect to your cel shading. Only slight modification of shader is required. If angle between normal and direction to camera is higher than defined value, then fragments' intensity is decreased. For example: float metallic = dot(v_normal, v_directionToCamera); // cos of angle between N and V
// if cos > 0.6 - full intensity
// if cos < 0.4 - low intensity
// if cos is within [0.4, 0.6] - interpolate values
metallic = smoothstep(0.4,0.6,metallic); // smooth interpolation between values
// shift matallic value from range [0, 1] to range[0.5, 1]
metallic = metallic/2 + 0.5; // shift metallic intensity by 0.5 = metallic * u_baseColor; // modulate final color
Metallic effect
OpenGL Calls
void renderTestCat()
   // RENDERING OF SILHOUETTE ///////////////////////////////////
   glUseProgram(CEL_SILHOUETTE);// select shader for rendering of silhouette

   // BIND VERTEX ARRAY OBJECT (buffers with mesh data)
   glBindBuffer(GL_ELEMENT_ARRAY_BUFFER, indexBuffer);

   glUniformMatrix4f("u_mvp_matrix",GL_FALSE, modelMatrix);

   glEnable(GL_CULL_FACE); // enable culling
   glCullFace (GL_CCW); // enable culling of front faces
   glDepthMask(GL_TRUE); // enable writes to Z-buffer
   glUniform3f("u_color1", vec3(0,0,0)); // black colour
   glUniform1f("u_offset1", 0.65f); // offset along normal
   glDrawElements (GL_TRIANGLES, numIndices, GL_UNSIGNED_INT, NULL); // draw mesh

   glCullFace (GL_CW); // enable culling of back faces
   glDepthMask(GL_FALSE); // disable writes to Z-buffer
   glUniform3f("u_color1", vec3(1,1,1)); // white color
   glUniform1f("u_offset1", 0.0f); // no offset
   glDrawElements (GL_TRIANGLES, numIndices, GL_UNSIGNED_INT, NULL);

   // RENDERING OF COLORED MESH /////////////////////////////////

   glUseProgram(CEL_SHADES); // select shader for rendering of colors


   glUniform1fv("u_numShades",numShades); // number of shades
   glUniform3fv("u_baseColor",vec3(1.0f,0.62f,0.0)); // base color

   glEnable (GL_TEXTURE_2D);
   glActiveTexture (GL_TEXTURE0);
   glBindTexture (GL_TEXTURE_2D, colorTexture->texture);

   // BIND VERTEX ARRAY OBJECT (buffers with mesh data) AND RENDER MESH
   glBindBuffer(GL_ELEMENT_ARRAY_BUFFER, indexBuffer);
   glDrawElements (GL_TRIANGLES, numIndices, GL_UNSIGNED_INT, NULL); // draw mesh

Shader for silhouette rendering
// VERTEX SHADER /////////////////

#version 330

// vertex attributes layout(location = 0) in vec3 i_position;
layout(location = 1) in vec3 i_normal;

uniform mat4 u_mvp_mat; // model-view-projection matrix

uniform float u_offset1; // offset along normal

void main(void){
   vec4 tPos   = vec4(i_position + i_normal * u_offset1, 1.0);
   gl_Position = u_mvp_mat * tPos;

// FRAGMENT SHADER ///////////////

#version 330

uniform vec3 u_color1;

layout(location = 0) out vec4 o_FragColor;

void main(void){
   o_FragColor = vec4(u_color1, 1.0);
Shader that decreases number of colors
// VERTEX SHADER /////////////////

#version 330

// vertex attributes layout(location = 0) in vec3 i_position;
layout(location = 1) in vec3 i_normal;
layout(location = 2) in vec2 i_texcoord1;

uniform mat4 u_viewProj_mat; // view-projection matrix
uniform mat4 u_model_mat; // model matrix
uniform mat3 u_normal_mat; // normal matrix

uniform vec3 u_light_position;
uniform vec3 u_camera_position;

// inputs for fragment shader
out vec3 v_normal;
out vec2 v_texcoord1;
out vec3 v_directionToLight;
out vec3 v_directionToCamera;

void main(void){
   vec4 worldPos = u_model_mat * vec4(i_position, 1.0);
   v_normal = u_normal_mat * i_normal;
   v_texcoord1 = i_texcoord1;

   vec3 vectorToLight = u_light_position -;
   v_directionToLight = normalize( vectorToLight);
   v_directionToCamera = normalize( u_camera_position - );

   gl_Position = u_viewProj_mat * worldPos;

// FRAGMENT SHADER ///////////////
#version 330

uniform sampler2D u_colorTexture; // diffuse texture
uniform vec3 u_baseColor; // shading color
uniform float u_numShades; // number of shades

// inputs from vertex shader
in vec3 v_normal;
in vec2 v_texcoord1;
in vec3 v_directionToLight;
in vec3 v_directionToCamera;

layout(location = 0) out vec4 o_FragColor;

// calculate diffuse component of lighting
float diffuseSimple(vec3 L, vec3 N){
   return clamp(dot(L,N),0.0,1.0);

// calculate specular component of lighting
float specularSimple(vec3 L,vec3 N,vec3 H){
      return pow(clamp(dot(H,N),0.0,1.0),64.0);
   return 0.0;

void main(void){
   // sample color from diffuse texture
   vec3 colfromtex = texture( u_colorTexture, v_texcoord1 ).rgb;

   // calculate total intensity of lighting
   vec3 halfVector = normalize( v_directionToLight + v_directionToCamera );
   float iambi = 0.1;
   float idiff = diffuseSimple(v_directionToLight, v_normal);
   float ispec = specularSimple(v_directionToLight,v_normal, halfVector);
   float intensity = iambi + idiff + ispec;

   // quantize intensity for cel shading
   shadeIntensity = ceil(intensity * u_numShades)/ u_numShades;

   // use base color = u_baseColor*shadeIntensity ;
   // or use color from texture = colfromtex*shadeIntensity ;
   // or use mixed colors = u_baseColor * colfromtex*shadeIntensity ;

   o_FragColor.w = 1.0;

Sun and Black Cat- Igor Dykhta (igor dykhta email) 2007-2014