Abstract art with parttree and friends

Background

One fun application of tree-based methods is abstracting over art and other images. For some really striking examples, take a look at designer Dimitris Ladopoulos’ portfolio. This vignette will show you how to implement the same basic ideas using parttree and a few friends. Here are the packages that we’ll be using.

library(parttree) # This package
library(rpart)    # For decision trees
library(magick)   # For reading and manipulating images
library(imager)   # Another image library, with some additional features

op = par(mar = c(0,0,0,0)) # Remove plot margins
magick:::magick_threads(2)
#> [1] 2

While the exact details will vary depending on the image at hand, the essential recipe for this type of art abstraction is as follows:

  1. Convert the image to a matrix (or data frame), where rows and columns correspond, respectively, to the X and Y coordinates of individual pixels. In other words, each cell in our matrix (data frame) represents the colour values of an individual pixel.
  2. Split the data by primary (RGB) colour channels. We should now have three matrices (data frames), where each cell represents the red/green/blue colour channel value of an individual pixel.
  3. Run a tree model on our three RGB datasets. In each case, we are trying to predict the relevant colour channel value as a function of the X and Y coordinates.
  4. Use the predicted values to plot the abstracted art piece! (Okay, this step requires a bit more work, but we’re about to see how it works in practice…)

Example 1: Peale’s “Portrait of Rosalba”

Our first example, will mimic one of Dimitri Ladopoulos’s aforementioned portrait pieces. Specifically, we will abstract over a close-up (eyes only) of Rembrandt Peale’s portrait of his daughter, Rosalba.

We can download a digital image of the original artwork from Wikimedia. I’ll download a high-res version to show off, but feel free to go for a lower resolution if you want to reduce modeling and plotting times.1 However, I will crop the image around Rosalba’s eyes to get a close-up and reduce the overall complexity of the exercise.2

# Download the image
rosalba = image_read("https://upload.wikimedia.org/wikipedia/commons/a/aa/Rembrandt_Peale_-_Portrait_of_Rosalba_Peale_-_Google_Art_Project.jpg")

# Crop around the eyes
rosalba = image_crop(rosalba, "850x400+890+1350")
# rosalba = image_crop(rosalba, "750x350+890+1350")

# Convert to cimg (better for in-memory manipulation)
rosalba = magick2cimg(rosalba)

# Display
rosalba
#> Image. Width: 850 pix Height: 400 pix Depth: 1 Colour channels: 3
plot(rosalba, axes = FALSE)