Codebrawl entry - Content aware image crop

README.md

Content-aware image cropping

How to use

Syntax is as follows:

ruby cropper.rb input output width height noise

How it works

The idea is to calculate the energy of each pixel (something like how important this pixel is for the whole image) and then crop the part that has the highes energy.

The energy function in this case is edge detection via the Sobel operator. You can swap in any other energy function, as long as it returns a grayscale canvas.

After getting the grayscale image, the total energy of each column and row are calculated. Then, it takes lines from the left or right (or top and bottom) of the image, depending on which has the smaller total energy. What is left is a high energy image.

This works pretty good if there is one large edgy thing (like the cat or dog), but breaks down if there are a lot of small edges (like the dirt in the duck image). For this cases, the cacrop method takes an additional noise operator, which makes it ignore values below a certain threshold. All images have been generated with a noise value of 70.

The three images are all very hard to crop to 100x100. You may want to try with 150x150, which gives much better results

cacrop.rb

module CACrop

  # This tries to crop the most energy dense parts of the image as determined
  # by the Sobel operator.
  def cacrop width, height, noise = 0
    
    # Get the edge detected image
    edges = edge_detection

    # This calculates arrays of the sums of each pixel in each row / column.
    # This is a simple measure of the activity in each row / column
    columns = (0...self.width).map{|i| edges.column(i).map{|c| ChunkyPNG::Color.r(c)}.map{|c| c > noise ? c : 0}.inject(&:+)}
    rows = (0...self.height).map{|i| edges.row(i).map{|c| ChunkyPNG::Color.r(c)}.map{|c| c > noise ? c : 0 }.inject(&:+)} 
    
    
    x = take_smallest_until columns, width
    y = take_smallest_until rows, height

    crop(x, y, width, height)

  end

  # Takes the smalles value from the end or the beginning of the array.
  # This continues until the size of the array is right.
  # It returns the numbers of dropped values from the left.
  def take_smallest_until arr, goal
    res = []
    arr.first < arr.last ? res << arr.shift : arr.pop while arr.size > goal
    res.size
  end
end

cat_cropped.png

cropper.rb

require "rubygems"
require "chunky_png"
require "pry"
require_relative "sobel"
require_relative "cacrop"

input = ARGV[0]
output = ARGV[1]
width = (ARGV[2] || 100).to_i
height = (ARGV[3] || 100).to_i
noise = (ARGV[4] || 0).to_i

image = ChunkyPNG::Image.from_file(input)
image.extend(EdgeDetection)
image.extend(CACrop)

image = image.cacrop width, height, noise
image.save(output)

dog_cropped.png

duck_cropped.png

sobel.rb

module EdgeDetection
  def edge_detection
    res = ChunkyPNG::Image.new(width, height)
    (1...width - 1).each do |x|
      (1...height - 1).each do |y|
        x_edge = 
          (-1 * ChunkyPNG::Color.grayscale_teint(self[x - 1, y - 1])) +
          (-2 * ChunkyPNG::Color.grayscale_teint(self[x - 1, y    ])) +
          (-1 * ChunkyPNG::Color.grayscale_teint(self[x - 1, y + 1])) +
          ( 1 * ChunkyPNG::Color.grayscale_teint(self[x + 1, y - 1])) +
          ( 2 * ChunkyPNG::Color.grayscale_teint(self[x + 1, y    ])) +
          ( 1 * ChunkyPNG::Color.grayscale_teint(self[x + 1, y + 1]))

        y_edge = 
          (-1 * ChunkyPNG::Color.grayscale_teint(self[x - 1, y - 1])) +
          (-2 * ChunkyPNG::Color.grayscale_teint(self[x    , y - 1])) +
          (-1 * ChunkyPNG::Color.grayscale_teint(self[x + 1, y - 1])) +
          ( 1 * ChunkyPNG::Color.grayscale_teint(self[x - 1, y + 1])) +
          ( 2 * ChunkyPNG::Color.grayscale_teint(self[x    , y + 1])) +
          ( 1 * ChunkyPNG::Color.grayscale_teint(self[x + 1, y + 1]))

        edge = ((x_edge.abs / 4) + (y_edge.abs / 4)) / 2
        res[x,y] = ChunkyPNG::Color.grayscale(edge)
      end
    end
    res
  end
end