Creating a black-and-white pencil sketch

In order to obtain a pencil sketch (that is, a black-and-white drawing) of the camera frame, we will make use of two image blending techniques, known as dodging and burning. These terms refer to techniques employed during the printing process in traditional photography; photographers would manipulate the exposure time of a certain area of a darkroom print in order to lighten or darken it. Dodging lightens an image, whereas burning darkens it.

Areas that were not supposed to undergo changes were protected with a mask. Today, modern image editing programs, such as Photoshop and Gimp, offer ways to mimic these effects in digital images. For example, masks are still used to mimic the effect of changing exposure time of an image, wherein areas of a mask with relatively intense values will expose the image more, thus lightening the image. OpenCV does not offer a native function to implement these techniques, but with a little insight and a few tricks, we will arrive at our own efficient implementation that can be used to produce a beautiful pencil sketch effect.

If you search on the Internet, you might stumble upon the following common procedure to achieve a pencil sketch from an RGB color image:

Whereas steps 1 to 3 are straightforward, step 4 can be a little tricky. Let's get that one out of the way first.

((B[idx] == 255) ? B[idx] : min(255, ((A[idx] << 8) / (255-B[idx]))))

def dodgeNaive(image, mask):
    # determine the shape of the input image
    width,height = image.shape[:2]

    # prepare output argument with same size as image
    blend = np.zeros((width,height), np.uint8)

    for col in xrange(width):
        for row in xrange(height):

            # shift image pixel value by 8 bits
            # divide by the inverse of the mask
            tmp = (image[c,r] << 8) / (255.-mask)

            # make sure resulting value stays within bounds
            if tmp > 255:
                tmp = 255
            blend[c,r] = tmp
    return blend

import cv2

def dodgeV2(image, mask):
    return cv2.divide(image, 255-mask, scale=256)

import cv2

def burnV2(image, mask):
    return 255 – cv2.divide(255-image, 255-mask, scale=256)

Pencil sketch transformation

With these tricks in our bag, we are now ready to take a look at the entire procedure. The final code will be in its own class in the filters module. After we have converted a color image to grayscale, we aim to blend this image with its blurred negative:

1. We import the OpenCV and numpy modules:

   import cv2
   import numpy as np

2. Instantiate the PencilSketch class:

   class PencilSketch:
       def __init__(self, (width, height), bg_gray='pencilsketch_bg.jpg'):

   The constructor of this class will accept the image dimensions as well as an optional background image, which we will make use of in just a bit. If the file exists, we will open it and scale it to the right size:

   self.width = width
   self.height = height
   
   # try to open background canvas (if it exists)
   self.canvas = cv2.imread(bg_gray, cv2.CV_8UC1)
   if self.canvas is not None:
       self.canvas = cv2.resize(self.canvas, (self.width, self.height))

3. Add a render method that will perform the pencil sketch:

   def renderV2(self, img_rgb):

4. Converting an RGB image (imgRGB) to grayscale is straightforward:

   img_gray = cv2.cvtColor(img_rgb, cv2.COLOR_RGB2GRAY)

   Note that it does not matter whether the input image is RGB or BGR.

5. We then invert the image and blur it with a large Gaussian kernel of size (21,21):

   img_gray_inv = 255 – img_gray
   img_blur = cv2.GaussianBlur(img_gray_inv, (21,21), 0, 0)

6. We use our dodgeV2 dodging function from the aforementioned code to blend the original grayscale image with the blurred inverse:

   img_blend = dodgeV2(mg_gray, img_blur)
   return cv2.cvtColor(img_blend, cv2.COLOR_GRAY2RGB)

The resulting image looks like this:

Did you notice that our code can be optimized further?

A Gaussian blur is basically a convolution with a Gaussian function. One of the beauties of convolutions is their associative property. This means that it does not matter whether we first invert the image and then blur it, or first blur the image and then invert it.

"Then what matters?" you might ask. Well, if we start with a blurred image and pass its inverse to the dodgeV2 function, then within that function, the image will get inverted again (the 255-mask part), essentially yielding the original image. If we get rid of these redundant operations, an optimized render method would look like this:

def render(img_rgb):
    img_gray = cv2.cvtColor(img_rgb, cv2.COLOR_BGR2GRAY)
    img_blur = cv2.GaussianBlur(img_gray, (21,21), 0, 0)
    img_blend = cv2.divide(img_gray, img_blur, scale=256)
    return img_blend

For kicks and giggles, we want to lightly blend our transformed image (img_blend) with a background image (self.canvas) that makes it look as if we drew the image on a canvas:

if self.canvas is not None:
    img_blend = cv2.multiply(img_blend, self.canvas, scale=1./256)
return cv2.cvtColor(img_blend, cv2.COLOR_GRAY2BGR)

And we're done! The final output looks like what is shown here: