TDM 40100: Project 6 — 2023

Motivation: Images are everywhere, and images are data! We will take some time to dig more into working with images as data in this next series of projects, particularly the processes.

Context: We are about to dive straight into a series of projects that emphasize working with images (with other fun things mixed in). We will start out with a straightforward task, that will involve lots of visual, manual analyses of images after you modify them to be easier to analyze. Then, in future projects, we will start to use computer vision to do this analysis for of us.

Scope: Python, images, openCV, skimage

Learning Objectives

  • Use numpy, skimage, and openCV to process images.

Make sure to read about, and use the template found here, and the important information about projects submissions here.

Dataset(s)

The following questions will use the following dataset(s):

  • /anvil/projects/tdm/data/images/ballpit.jpg

Questions

Question 1 (2 pts)

  1. Write code to read in the image and display it in your notebook.

  2. Write code to find the shape of the image.

Let’s ease into things by first taking a look at the image we are going to analyze for this project. First, read up on this matplotlib documentation on image processing, and then write a small snippet of Python code in order to read in our image and display it in our notebook.

Don’t forget to run the below code in order to import the proper libraries for this project.

 
import cv2
import matplotlib.pyplot as plt
import matplotlib.image as mpimg
import numpy as np

The functions imread and imshow, from matplotlib.image and matplotlib.pyplot respectively, will be useful for this question.

If you take a look at img, you will find it is simply a multidimensional numpy.ndarray, with a somewhat strange shape. We will discuss this shape more in Question 2, but for now you can note that the first two dimensions given are the height and width of the image, in pixels. Keep the third one in mind for now, we will discuss it later.

For the last part of this question, write some code to print the shape of the image. What are the dimensions of the image? How many pixels wide and tall is it? .Items to submit

  • Code to read in and display our image.

  • Code used to print shape, and height and width of our image.

Question 2 (2 pts)

  1. Using openCV with two different methods, grayscale the image.

  2. Find the shape of the grayscale image.

  3. Write one to two sentences explaining any differences between this image shape and the shape you identified in the previous question.

Now that we are familiar with the image we are working with, let’s get started modifying it with the end goal of eventually making it easy to manually count how many balls of each color are in our image.

First off, let’s convert our image to grayscale. This is a good first step when analyzing an image, as it can give you an idea of the 'black-white contrast' for an image, which is often very useful in something referred to as 'contour-edge detection'. We will learn more about contour-edge detection, how to perform it, and what it is useful for later on in this course.

Read through this short article on how to do grayscaling of images with openCV. Then, using two different methods, convert the image to grayscale. Note that both of the methods of question are contained in the article provided.

The functions imread and cvtColor from openCV will be useful for this question, with the latter in conjunction with the cv2.COLOR_BGR2GRAY constant.

Once you’ve done this, print the image along with the shape of the image. How does this shape differ from the shape of the original image? What do you think the dimensions of the grayscale image represent?

Items to submit

  • Code to grayscale the image, two different ways

  • Printed image, grayscaled,

  • Shape of grayscaled image, and explanation of what the dimensions represent.

Question 3 (2 pts)

  1. Code to split the image into red, green, and blue color channels.

  2. Code to display each channel (should be grayscale).

  3. 1-2 sentences about our grayscale images and their usefulness in determining colors.

While we are on the topic of color, let’s take a look at the color channels of our image and how we can best analyze them individually. After all, outside of edge detection, you will likely want to talk about the different colors present in images you are analyzing.

Read through this short article on how to split an image into its color channels with openCV. Then, write some code to split our image into its red, green, and blue color channels. Then, display each of the channels individually. You should see three grayscale images, each with slight but clearly noticeable differences from the others.

From these images, do you think that it would be possible to determine which color ball is most common? Write a sentence or two discuss why or why not.

Items to submit

  • Code to split the image into its RGB color channels, and display each channel.

  • 1-2 sentences about our grayscale images and their usefulness in determining colors.

Question 4 (2 pts)

  1. Code to recolor our images into their respective colors.

  2. Code to display each channel (should be colored).

  3. Result of running provided code snippet to create red mask.

You may notice when you first attempt this question that the colors are not matching up with what you expect. This is due to a difference in formatting between openCV and matplotlib, where openCV uses BGR instead of RGB. You can fix this by using the cv2.cvtColor function with the cv2.COLOR_BGR2RGB constant, similar to how you used it to grayscale images in question 2.

Next, write some code to recolor each of the channels with its respective color, and display the colored images. You should see three images, each with a different color tint. Note that the colors may not be exactly what you expect, but they should be close. This can be done by creating another channel (a simple numpy array) of all zeroes, and then copying your channel into the proper dimension of the numpy array before displaying it with imshow as usual.

Here is an example of how to do this with the red channel, if you’re getting stuck:

blank = 255 * (r_c.copy() * 0)

# r_c represents the red channel from the last question
red_image = cv2.merge([blank, blank, r_c])
plt.imshow(plt.imshow(cv2.cvtColor(red_image, cv2.COLOR_BGR2RGB)), plt.title('Red Channel'))

Finally, run the following code after you have shown your color images. This will create something called a color mask, which you will find is much more useful in determing the most common color of ball in our image.

# Define lower and upper bounds for red color in BGR format
lower_red = np.array([100, 0, 0])  # Lower bound
upper_red = np.array([255, 100, 100])  # Upper bound

# Create a mask for red pixels
red_mask = cv2.inRange(img, lower_red, upper_red)

# Apply the red mask to the original image
red_pixels = cv2.bitwise_and(img, img, mask=red_mask)

plt.figure(figsize=(12, 4))  # Create a larger figure for better visualization
plt.subplot(131), plt.imshow(red_pixels), plt.title('Red Masked')
plt.subplot(132), plt.imshow(img), plt.title('Original Image')
Items to submit

  • Code to recolor each channel, and display each channel.

  • 1-2 sentences about our colored images, their usefulness/shortcomings in analyzing color, and how they could be improved upon.

Question 5 (2 pts)

  1. Code to create a color mask for green and blue color masks.

  2. Code to display each color mask.

As a wrap-up for this project, create your own color masks for green and blue. In this project, we won’t worry about creating a mask for the less 'basic' colors in our image, as this will be more of a focus in the next project, but please feel free to experiment on your own. Additionally, note that the color limits you choose for your masks may not be perfect, but you should be able to get a good grasp for the relative presence of each color of ball in our image. Do your best to create realistic bounds, and be sure to print your final masks in your Jupyter notebook.

Items to submit

  • Code to create a color mask for green and blue masks.

  • Code to display each color mask.

Submitting your Work

Nicely done, you’ve made it to the end of Project 6! This is likely a very new topic for many of you, so please take the time to get things right now and learn all of the core concepts before we move on to more advanced topics in the next project. Unlike for most of your other projects, it is actually okay if you get the 'File to large to display' error in Gradescope. We will be excusing it for this project due to the nature of wanting to display a lot of images in our notebook. Just make sure that if you redownload your .ipynb file from Gradescope, it contains everything you expect it to.

You must double check your .ipynb after submitting it in gradescope. A very common mistake is to assume that your .ipynb file has been rendered properly and contains your code, markdown, and code output, when in fact it does not. Please take the time to double check your work. See here for instructions on how to double check this.

You will not receive full credit if your .ipynb file does not contain all of the information you expect it to, or it does not render properly in gradescope. Please ask a TA if you need help with this.
Items to submit

  • firstname-lastname-project06.ipynb.

Please make sure to double check that your submission is complete, and contains all of your code and output before submitting. If you are on a spotty internet connection, it is recommended to download your submission after submitting it to make sure what you think you submitted, was what you actually submitted.

In addition, please review our submission guidelines before submitting your project.