Making an IP camera with the Raspberry Pi Including on-board motion detection, and a password protected web server for viewing the camera stream

Instructions.md

Making an IP camera with the Raspberry Pi

Including on-board motion detection

Requirements

• Raspberry Pi (512MB rev2 recommended)
• Raspberry Pi Camera board
• SD Card (min 2BG, 8GB recommended for good measure. Class 10)

Optionally, a wifi adapter verified to work with raspberry pi ( I used Edimax Wireless Nano USB Adapter - http://www.amazon.com/gp/product/B005CLMJLU/ )

Prepare the Pi

Image raspbian wheezy to an SD card:

• Run diskutil list
• Identify the SD card
• Unmount the disk: run diskutil unmountDisk <disk> (ex: diskutil unmountDisk /dev/disk2)
• Image the disk: sudo dd bs=1m if=2013-05-25-wheezy-raspbian.img of=<disk>
• Remove hidden os x files
  • In terminal: cd /Volumes/boot (disk may be named something other than boot)
    • rm -rf .Trashes
    • rm -rf .fseventsd

Attach peripherals and boot the Pi

• Attach the camera
• Attach ethernet
• Insert the SD card
• Insert wifi adapter if desired
• Attach power

Find the IP address of the pi, and SSH into it.

• ssh pi@192.168.12.201
• default password is "raspberry"

Run raspi-config

• sudo raspi-config
• Expand filesystem
• Change user password
• Internationalization Options
• Enable camera
• Advanced Options
  • hostname
  • memory split (I recommend 16 if running headless)
• Finish and reboot
• SSH back in once it's booted

Set the timezone

• sudo dpkg-reconfigure tzdata

Change the root password

• sudo passwd root

Fetch any updates

• sudo apt-get update
• sudo apt-get upgrade -y

Reboot for good measure

• sudo shutdown -r now

Test the camera

• SSH into your raspberry pi: ssh pi@192.168.12.201
• run raspistill -o test_image.jpg.
• You should see the red light on your camera light up for a moment and then it should take a photo.
• ls should reveal a test_image.jpg You can copy this file somewhere if you want to test further

Install PIP

• wget https://bitbucket.org/pypa/setuptools/raw/bootstrap/ez_setup.py
• wget https://raw.github.com/pypa/pip/master/contrib/get-pip.py
• sudo python ez_setup.py
• sudo python get-pip.py

Install libjpeg8

• sudo apt-get install libjpeg8-dev
• sudo ln -s /usr/lib/arm-linux-gnueabihf/libjpeg.so /usr/lib/

Install PIL

• sudo apt-get install python-dev
• sudo pip install PIL
• Reboot since we just installed the world sudo shutdown -r now

Create yapimotion

• mkdir ~/ip_camera
• nano ~/ip_camera/yapimotion.py

Paste this in as the contents of the file:

#modified from http://www.raspberrypi.org/phpBB3/viewtopic.php?f=43&t=45235&p=436209

import os
import shutil
import time, datetime
import signal, sys
import numpy as np
from PIL import Image, ImageChops

SRCFILE = '/raspicam/still.jpg'
DSTDIR = '/home/pi/ip_camera/motion_data/'
MASKFILE = '/home/pi/ip_camera/mask.jpg'
USE_MASK = False
MOTIONTHRESHOLD = 20
resize = (100, 100)
# yet another pi motion
class yaPiMotion():
    def __init__(self):
        # Take current time as a timestamp and get the first image
        self.snapshot_timestamp = time.time()
        self.img1 = self.loadImage(SRCFILE)         
        self.snapshot_size = 0
		self.snapshot_timestamp2 = None
        self.img2 = None
		self.snapshot_size2 = 0
                  
        if USE_MASK: 
            self.marr = np.array( self.loadImage( MASKFILE ) )  # Mask numpy array           
            self.img1 = self.applyMask(self.img1)
            #self.img1.save('test_nmask.jpg')   # Uncomment for mask testing

    def applyMask(self, image):
        arr = np.array(image)    # Convert image to numpy array
        arr[self.marr==0] = 0    # Copy black colour (zeros) from mask array to image array
        return Image.fromarray(arr)  # Convert array back to an image and return it
    
    def signalHandler(self, signal, frame):
        # Clean stuff here, if needed     
        sys.exit(0)
    
    # Get files modification time
    def modificationDate(self, filename):
        try:
            t = os.path.getmtime(filename)
        except OSError:
            t = None          
        return t

    def sizeOf(self, filename):
		try:
		   s = os.path.getsize(filename)
		except OSError:
		   s = None
		return s

    def loadImage(self, image_file):
        try:     
            im = Image.open(image_file) 
            im.thumbnail( resize )
            gray = im.convert('L')    # Convert to grayscale
            #im = gray.convert('RGB')  # Convert back, so we have a rgb grayscale image (duh?)
        except IOError:
            im = None
        return im
    
    # http://stackoverflow.com/questions/5524179/how-to-detect-motion-between-two-pil-images-wxpython-webcam-integration-example
    def imageEntropy(self, img1, img2):
        try:
            img = ImageChops.difference(img1, img2)
        except AttributeError:
        	return 0
        
        w,h = img.size
        a = np.array(img).reshape((w*h,3))                        # Figure out, how to do this to grayscale image
        h,e = np.histogramdd(a, bins=(16,)*3, range=((0,256),)*3) # image, it could be a little faster 
        prob = h/np.sum(h) # normalize
        prob = prob[prob>0] # remove zeros
        
        ret = -np.sum(prob*np.log2(prob)) * 100
        #if ret > 0: img.save('./diff/diff_' + self.timeStamp()) # Uncomment this line, if you want to see triggered motion    
        
        return ret

    def timeStamp(self):
        ts = time.time()
        timestp = datetime.datetime.fromtimestamp(ts).strftime('%H.%M.%S.%f.jpg')
        print timestp
        return timestp        

    def dirPath(self):
		ts = time.time()
		dirpath = datetime.datetime.fromtimestamp(ts).strftime('%Y-%m-%d/')
		return dirpath

    def detect(self):
        ret = 0
        # Take timestamp from file, so we can check if there's any change
        self.snapshot_timestamp2 = self.modificationDate(SRCFILE)
        self.snapshot_size2 = self.sizeOf(SRCFILE)
		if self.snapshot_timestamp2 is not None or self.img1 is not None:
            # There's an image available, but check if it's new         
            if self.snapshot_timestamp2 is not self.snapshot_timestamp or self.snapshot_size2 is not self.snapshot_size:
                self.img2 = self.loadImage(SRCFILE)
                if USE_MASK: self.img2 = self.applyMask(self.img2)
                
                ret = self.imageEntropy(self.img1, self.img2)
                self.img1 = self.img2
                self.snapshot_timestamp = self.snapshot_timestamp2 
				self.snapshot_size = self.snapshot_size2

        else: # No image available, try again
            self.snapshot_timestamp = time.time()
            self.img1 = self.loadImage(SRCFILE)                     
            if USE_MASK: self.img1 = self.applyMask(self.img1)
        
        return ret    

if __name__ == "__main__":
    y = yaPiMotion()

    while (True):
        m = y.detect()
        print("Detect: " + str( m ) )
        if m > MOTIONTHRESHOLD: # There's been detected motion, do something	
		    if not os.path.exists(DSTDIR + y.dirPath()):            
		        os.makedirs(DSTDIR + y.dirPath())
            shutil.copy2(SRCFILE, DSTDIR + y.dirPath() + y.timeStamp())  # Copy image to a safe place
        time.sleep(0.2)

Save it, and continue on.

Setting up the ram disk

In an effort to limit the amount of times your images are written to disk, let's set up a ram disk for the tmp directory. This is important as SD cards have relatively limited write cycles

• sudo nano /etc/fstab
• Add this line to the end of that file:
  • tmpfs /raspicam tmpfs defaults,noatime,mode=1777 0 0
  • Once you added the line, press ctrl+x it will prompt you to save, say Yes. 
• Make the directiry: sudo mkdir /raspicam
• Mount the new ram disk: sudo mount -a

This creates a virtual disk in ram for the /raspicam directory.

Installing nginx

• sudo apt-get install nginx
• sudo service nginx start

Configure nginx

• sudo nano /etc/nginx/sites-available/default

Find the line that is something like: root /var/www; and change it to root /home/pi/ip_camera/html_root;

Add a location block as so:

location /camera/ {
	alias /raspicam/;
}

Restart nginx sudo service nginx restart

Add basic auth to the nginx install

Install the tools to generate htpasswd files:

• sudo apt-get install apache2-utils

Add a user and password:

• htpasswd -c /home/pi/ip_camera/.htpasswd <username>
  • ex: htpasswd -c /home/pi/ip_camera/.htpasswd johnathan
• enter the password you want when prompted.

Configure nginx to use the newly created password file

• sudo nano /etc/nginx/sites-available/default

Add these lines inside the server { block:

auth_basic "Restricted";
auth_basic_user_file /home/pi/ip_camera/.htpasswd;

Restart nginx sudo service nginx restart

Make a script to start the services

Let's make a scripte to start things for us.

Run nano ~/ip_camera/start_ip_camera.sh which will create a file called start_ip_camera in your home directory.

Paste this in as the contents of the file:

#!/bin/bash

if ps ax | grep -v grep | grep raspistill > /dev/null
then
    echo "raspistill already running"
else
    raspistill -w 1920 -h 1080 -q 10 -o /raspicam/still.jpg -tl 200 -t 9999999 -th 0:0:0 &
fi

if ps ax | grep -v grep | grep yapimotion > /dev/null
then
    echo "yapimotion already running"
else
    python /home/pi/ip_camera/yapimotion.py > /dev/null &
fi

Add execute privileges to the file: chmod +x ~/ip_camera/start_ip_camera.sh

And run it: ~/ip_camera/start_ip_camera.sh

You should now be able to access your camera by visiting http:///camera/still.jpg

Add fast image refreshing for video-like interface

• mkdir ~/ip_camera/html_root
• nano ~/ip_camera/html_root/index.html

Paste this as the contents:

<html>
<head>
  <script language="JavaScript">
    window.refreshTimeout = null;
    window.page_is_visible = true;
  
    function refreshIt(element) {
      if(document.hasFocus()){
        delay = 500;
      }else if(window.page_is_visible){
        delay = 2500;
      }else{
        console.log("page not visible. Not updating");
        return;
      }
      console.log("Delay " + delay);
      element.src = element.src.split('?')[0] + '?' + new Date().getTime();
      window.refreshTimeout = setTimeout(function() {
        refreshIt(element);
      }, delay);
    }
    
    (function() {
        var hidden = "hidden";

        // Standards:
        if (hidden in document)
            document.addEventListener("visibilitychange", onchange);
        else if ((hidden = "mozHidden") in document)
            document.addEventListener("mozvisibilitychange", onchange);
        else if ((hidden = "webkitHidden") in document)
            document.addEventListener("webkitvisibilitychange", onchange);
        else if ((hidden = "msHidden") in document)
            document.addEventListener("msvisibilitychange", onchange);
        // IE 9 and lower:
        else if ('onfocusin' in document)
            document.onfocusin = document.onfocusout = onchange;
        // All others:
        else
            window.onpageshow = window.onpagehide 
                = window.onfocus = window.onblur = onchange;

        function onchange (evt) {
            var v = true, h = false, //v = 'visible', h = 'hidden',
                evtMap = {
                    focus:v, focusin:v, pageshow:v, blur:h, focusout:h, pagehide:h 
                };

            evt = evt || window.event;
            if (evt.type in evtMap)
                window.page_is_visible = evtMap[evt.type];
            else        
                window.page_is_visible = this[hidden] ? false : true;
                
            clearTimeout(window.refreshTimeout);
            refreshIt(document.getElementById('v'));            
        }
    })();
  </script>
</head>
<body style="margin: 0;" onload="refreshIt(document.getElementById('v'))">
  <img src="/camera/still.jpg" width="100%" id="v">
</body>

Have it restart automatically (and on boot)

Now we have a working script to start our IP camera! However, we don't want to have to run this every time something crashes, or the raspberry pi restarts. Since the script is pretty fast to run if the processes are running, and we want to minimize any downtime, we'll have cron run it once per minute. To do this, open up your crontab with crontab -e and paste this line at the bottom:

* * * * * /home/pi/ip_camera/start_ip_camera.sh

Save the crontab, and you should be done! You can test this by rebooting and seeing if it starts automatically, or by killing the process and seeing if it restarts within 1 minute.

Get WiFi working

Open sudo nano /etc/network/interfaces

You'll likely see some lines towards the end of that file similar to these:

allow-hotplug wlan0
iface wlan0 inet manual
wpa-roam /etc/wpa_supplicant/wpa_supplicant.conf
iface default inet dhcp

Change those to be like this:

allow-hotplug wlan0
auto wlan0
iface wlan0 inet dhcp
    wpa-ssid "ssid"
    wpa-psk "password"

Reboot the pi and it should be connected to WiFi now (Note: the IP address it uses to connect with WiFi is different than the one for ethernet)

Other / Optional tasks

Disable camera LED

Personally I don't want this camera to have a bright red LED on the front. Luckily fixing this is pretty easy!

Open your boot config file: sudo nano /boot/config.txt At the bottom, add this: disable_camera_led=1

Save, and reboot. When the system comes back up the camera LED will be disabled.

Store the motion images on a network drive

You'll need to edit your fstab file sudo nano /etc/fstab Add an entry to the file: //192.168.12.1/4TB\040EXT/my_folder_for_camera_images /home/pi/ip_camera/motion_data cifs password=xxxxxx,_netdev,rw,uid=pi,gid=users,defaults 0 0

Set your own network share IP, name, and path, and replace the password as needed. If a username is required, simply add username=xxxxxx, before the password=xxxxxx. Note: All spaces in your file paths must be replaced with \040

Run sudo mount -a to mount this. It will automount on future reboots

Changing the resolution, quality, or framerate of the stream

To change the resolution of your stream, edit the startup script and look for this line:

raspistill -w 1920 -h 1080 -q 5 -o /tmp/stream/image.jpg -tl 100 -t 9999999 -th 0:0:0 &

The -w 1920 -h 1080 is specifying the image width and height. You may change these to the size image you desire.

The -q 5 is the quality of the jpg. This is a scale from 0 to 100. Note: This will drastically change the size of your image, and thus the bandwidth requirements for the camera operation.

-tl 100 is saying take a frame every 100ms. This gives us a framerate of approximately 10fps. For a framerate of 3fps you could set it to -tl 333.