Create a gist now

Instantly share code, notes, and snippets.

PFC2 Install Guide: Setting up an OpenAg dev environment

[Update 17 Mar 2017: Gordon has been working lately on changing and simplifying the installation process for openag_brain. Also, openag_ui has developed a lot further than the screenshots below show. I haven't been following the changes closely, but at some point, these instructions will become obsolete. Maybe that's already happened.]

PFC2 Install Guide: Setting up an OpenAg dev environment

This is how I installed the OpenAg Personal Food Computer v2 (PFC2) software stack on a Raspberry Pi 3 and Arduino Mega. The Pi 3, SD card, and Arduino cost me less than $100 USD.

My goal was to get the main software components--openag_brain, openag_python, and the UI--running cheaply so I could experiment with them. As you can see in the screenshots below, it worked.

With this gist, my aim is to describe my installl procedure in enough detail for you to reproduce my results.

Materials

To follow along with this guide, you will need:

  1. Raspberry Pi 3 with a 2.5A USB power adapter (I used this kit).

  2. A micro SD card (I used this one).

  3. An Arduino Mega or a compatible clone (I used this one) and a USB A to B cable.

  4. A Mac computer running macOS El Capitan with an internal SD card. Other computers will work, but you might need to adapt the instructions.

  5. A wifi network where you can add devices just by using the right SSID and password. For example, on a managed school network, the access points might ignore your Pi 3 until you provide its wifi MAC address to your network administrator. This guide assumes you have a simple wifi network.

Download and expand the Raspbian Jessie Lite SD card image

  1. Download Raspbian Jessie Lite from the RaspberryPi.org download page: https://www.raspberrypi.org/downloads/raspbian/

  2. Use the Terminal app to verify the SHA-1 hash of your downloaded file. I used the "January 2017" of Jessie Lite, and the official download page says that its SHA-1 hash should be 0ed4dff4248c4c8b437a5d5feed4013197660c8a.

    $ cd ~/Downloads
    $ openssl sha -sha1 2017-01-11-raspbian-jessie-lite.zip
    SHA1(2017-01-11-raspbian-jessie-lite.zip)= 0ed4dff4248c4c8b437a5d5feed4013197660c8a
    
  3. Unzip the SD card image:

    $ unzip 2017-01-11-raspbian-jessie-lite.zip 
    Archive:  2017-01-11-raspbian-jessie-lite.zip
      inflating: 2017-01-11-raspbian-jessie-lite.img
    
  4. Check that the unzipped file size seems right:

    $ du -sh 2017-01-11-raspbian-jessie-lite.*
    1.3G  2017-01-11-raspbian-jessie-lite.img
    292M  2017-01-11-raspbian-jessie-lite.zip
    

Write the SD card image

To make a Raspbian micro SD card, follow the official instructions at https://www.raspberrypi.org/documentation/installation/installing-images/README.md

Tips for good results:

  1. Back up your Mac before you start this because sudo dd ... is dangerous.

  2. Find a good, non-counterfeit micro SD card. I used a new 16GB SanDisk Ultra that came with a regular size SD card adapter.

  3. The internal SD card readers on Macs running El Capitan often don't work after waking up from sleep mode. I've found that restarting the Mac usually makes the SD card reader start working again.

  4. To format as FAT32 using Disk Utility, select the disk, click the "Erase" button in the toolbar, Pick "MS-DOS (FAT)" from the "Format" list, and pick "Master Boot Record" from the "Scheme" list.

These are the Terminal commands I used, but you may need to use a different disk identifier:

$ # Find my SD card's disk identifier
$ diskutil list
...
/dev/disk3 (internal, physical):
#:                       TYPE NAME                    SIZE       IDENTIFIER
0:     FDisk_partition_scheme                        *15.9 GB    disk3
1:                 DOS_FAT_32 UNTITLED                15.9 GB    disk3s1
$
$ # Unmount the SD card
$ diskutil unmountDisk disk3
Unmount of all volumes on disk3 was successful
$
$ # Write the disk image--this may take a couple minutes
$ sudo dd bs=1m if=2017-01-11-raspbian-jessie-lite.img of=/dev/rdisk3
Password:
1326+0 records in
1326+0 records out
1390411776 bytes transferred in 152.531932 secs (9115546 bytes/sec)

Enable SSH and configure wifi

This is how I set my Raspberry Pi 3 up for headless ssh access over wifi by editing configuration files on the SD card's boot partition. This way I didn't have to use a dedicated keyboard and monitor for the Pi 3--less stuff to buy and more space on my desk.

The Pi 3's wifi configuration lives in /etc/wpa_supplicant. Linux workstations can mount the SD card's ext4 root partition to modify /etc, but it's a hassle to do that from my Mac. Fortunately, there's also a way to put wifi configuration on the SD card's FAT32 boot partition that my Mac can access easily. As of May 2016, if you put wifi configuration in /boot/wpa_supplicant.conf, it will get moved to /etc/wpa_supplicant/ on the next boot. For the announcement about this, check https://www.raspberrypi.org/blog/another-update-raspbian/ and scroll down to the "Tweaks" section.

Raspbian used to come with ssh enabled by default, but that changed in November of 2016: https://www.raspberrypi.org/blog/a-security-update-for-raspbian-pixel/. To enable ssh, you need to create a file called "ssh" on the SD card's boot partition.

If your sudo dd worked, you should already have the SD card's FAT32 boot partition mounted as "boot" in the finder and "/Volumes/boot" in Terminal. So, to configure wifi and ssh, do this:

  1. Use your favorite text editor to make /Volumes/boot/wpa_supplicant.conf look like this, but use the right SSID and password for your network:

    network={
     ssid="YOUR_SSID"
     psk="YOUR_PASSWORD"
    }
    

    See http://raspberrypi.stackexchange.com/questions/10251/prepare-sd-card-for-wifi-on-headless-pi

  2. Create a file called ssh on the SD card's boot partition:

    $ touch /Volumes/boot/ssh
    $ ls
    COPYING.linux          bootcode.bin           kernel7.img
    LICENCE.broadcom       cmdline.txt            overlays
    LICENSE.oracle         config.txt             ssh
    bcm2708-rpi-b-plus.dtb fixup.dat              start.elf
    bcm2708-rpi-b.dtb      fixup_cd.dat           start_cd.elf
    bcm2708-rpi-cm.dtb     fixup_db.dat           start_db.elf
    bcm2709-rpi-2-b.dtb    fixup_x.dat            start_x.elf
    bcm2710-rpi-3-b.dtb    issue.txt              wpa_supplicant.conf
    bcm2710-rpi-cm3.dtb    kernel.img
    

Boot the Pi 3, log in, configure local settings, and apply the latest updates

  1. Eject the SD card from your Mac, take the micro SD card out of the adapter, insert it into your Pi 3, and power it up with a good 2.5A USB power adapter.

  2. Find your Pi's hostname or IP address. The January 2017 Jessie Lite includes avahi, so there's a good chance the raspberrypi.local hostname will work without any configuration. If you understand how to log into your wifi router and check DHCP assignments, that's probably the quickest way to find the IP address. RaspberryPi.org also suggests using a display or scanning your local network with NMAP. See https://www.raspberrypi.org/documentation/remote-access/ip-address.md

  3. Log in with the default user and password (pi & raspberry):

    $ ssh pi@raspberrypi.local
    pi@raspberrypi.local's password: 
    
    The programs included with the Debian GNU/Linux system are free software;
    the exact distribution terms for each program are described in the
    individual files in /usr/share/doc/*/copyright.
    
    Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
    permitted by applicable law.
    Last login: Sun Jan 29 00:55:47 2017 from my-mac.my-local-domain.net
    
    SSH is enabled and the default password for the 'pi' user has not been changed.
    This is a security risk - please login as the 'pi' user and type 'passwd' to set a new password.
    
    pi@raspberrypi:~ $
    
  4. Set the locale, timezone, and wifi country:

    $ sudo raspi-config
    

    This will launch an interactive configuration menu that is documented at https://www.raspberrypi.org/documentation/configuration/raspi-config.md. Use the menus to:

    • Expand the root file system. [edit: Raspbian does this automatically now]
    • Under "Internationalisation Options", change your locale if needed. The default is en_GB.UTF-8 UTF-8. I changed my locale to en_US.UTF-8 UTF-8. When you are prompted about setting the "Default locale for the system environment", set it to your locale.
    • Under "Internationalisation Options", set your wifi country. I set mine to "US United States".
    • Under "Internationalisation Options", set your timezone. I set mine to "US" on the first screen and "Central" on the second screen.
    • Back on the main menu, pick "Finish", and then "Yes" when it asks if you want to reboot.
  5. Wait a minute for the Pi 3 to reboot, then log back in with ssh. If it looks like ssh is stuck after the reboot, just hit the return key. That should make ssh exit and get you back to a bash prompt on your local workstation. From there just use the same ssh command you used before. For me, it was ssh pi@raspberrypi.local.

  6. Upgrade your packages to get the latest security updates, then reboot again.

    $ sudo apt-get update
    $ sudo apt-get upgrade
    $ sudo reboot
    
  7. Log back in with ssh. Now you're ready to start installing OpenAg software.

Install OpenAg Brain

  1. Get git so you can clone the install script:

    $ sudo apt-get install git
    
  2. Use the "Installing globally" method to install ROS, dependencies, openag_python, and openag_brain:

    $ cd ~
    $ git clone https://github.com/OpenAgInitiative/openag_brain_install_rpi.git
    Cloning into 'openag_brain_install_rpi'...
    remote: Counting objects: 72, done.
    remote: Compressing objects: 100% (12/12), done.
    remote: Total 72 (delta 6), reused 0 (delta 0), pack-reused 60
    Unpacking objects: 100% (72/72), done.
    Checking connectivity... done.
    $ cd openag_brain_install_rpi
    $ ./install.sh
    ...
    

    The install.sh script took 59 minutes to finish on my Raspberry Pi 3, and it generated about 14,500 lines of output. I've posted the full log in another gist: https://gist.github.com/wsnook/47c0b7004a1bb306c1734ced4da9dcba.

  3. Connect your Ardunio Mega to one of the Pi 3's USB ports. I used a SunFounder Mega2560 clone.

  4. Activate the catkin workspace (catkin is part of ROS):

    pi@raspberrypi:~ $ source ~/catkin_ws/devel/setup.bash
    
  5. Load the default fixture with rosrun to begin using the default software modules and Arduino firmware:

    $ rosrun openag_brain load_fixture default
    Applying fixture /home/pi/catkin_ws/src/openag_brain/fixtures/default.json
    environment  [####################################]  100%
    software_module_type  [####################################]  100%
    firmware_module_type  [####################################]  100%
    software_module  [####################################]  100%
    
  6. Use rosrun to start openag_brain. Among other things, this will flash the Arduino with firmware specified in the default fixture:

    $ rosrun openag_brain main
    Initializing the database
    Applying CouchDB configuration  [####################################]  100%
    Creating databases  [####################################]  100%
    Pushing design documents
    Generating launch file
    Processing module "sensor_info_publisher" from server
    Processing module "arduino_handler" from server
    Processing module "topic_filter_1" from server
    Processing module "recipe_handler_1" from server
    Processing module "api" from server
    Processing module "topic_connector" from server
    Processing module "video_writer_1" from server
    Processing module "image_persistence_1" from server
    Processing module "sensor_persistence_1" from server
    Processing module "expand_diagnostics" from server
    Spawning software modules
    Unable to register with master node [http://localhost:11311]: master may not be running yet. Will keep trying.
    ... logging to /home/pi/.ros/log/29a6ffae-e5fe-11e6-990c-b827eb23a528/roslaunch-raspberrypi-15833.log
    Checking log directory for disk usage. This may take awhile.
    Press Ctrl-C to interrupt
    Done checking log file disk usage. Usage is <1GB.
    
    started roslaunch server http://raspberrypi:45927/
    
    SUMMARY
    ========
    
    PARAMETERS
     * /arduino_handler/serial_port: /dev/ttyACM0
     * /categories: ['sensors', 'actu...
     * /environments/environment_1/image_persistence_1/min_update_interval: 3600
     * /environments/environment_1/sensor_persistence_1/max_update_interval: 600
     * /environments/environment_1/sensor_persistence_1/min_update_interval: 5
     * /environments/environment_1/video_writer_1/timelapse_scaling_factor: 86400
     * /rosdistro: indigo
     * /rosversion: 1.11.20
    
    NODES
      /environments/environment_1/
         image_persistence_1 (openag_brain/image_persistence.py)
         recipe_handler_1 (openag_brain/recipe_handler.py)
         sensor_persistence_1 (openag_brain/sensor_persistence.py)
         topic_filter_1 (openag_brain/topic_filter.py)
         video_writer_1 (openag_brain/video_writer.py)
      /
         api (openag_brain/api.py)
         arduino_handler (openag_brain/handle_arduino.py)
         expand_diagnostics (openag_brain/expand_diagnostics.py)
         sensor_info_publisher (openag_brain/sensor_info_publisher.py)
         topic_connector (openag_brain/topic_connector.py)
    
    auto-starting new master
    process[master]: started with pid [15849]
    ROS_MASTER_URI=http://localhost:11311
    
    setting /run_id to 29a6ffae-e5fe-11e6-990c-b827eb23a528
    process[rosout-1]: started with pid [15868]
    started core service [/rosout]
    process[sensor_info_publisher-2]: started with pid [15875]
    process[arduino_handler-3]: started with pid [15887]
    process[environments/environment_1/topic_filter_1-4]: started with pid [15888]
    process[environments/environment_1/recipe_handler_1-5]: started with pid [15889]
    process[environments/environment_1/video_writer_1-6]: started with pid [15891]
    process[environments/environment_1/image_persistence_1-7]: started with pid [15892]
    process[environments/environment_1/sensor_persistence_1-8]: started with pid [15893]
    process[api-9]: started with pid [15896]
    process[topic_connector-10]: started with pid [15897]
    process[expand_diagnostics-11]: started with pid [15900]
    [WARN] [WallTime: 1485679050.164984] Not specified whether Arduino should be flashed on startup. Defaulting to True.
    [WARN] [WallTime: 1485679054.340629] Cloning into '/tmp/tmpwB_yOZ/lib/rosserial_arduino_libs'...
    [WARN] [WallTime: 1485679097.539406] Copyright (c) 2000-2005 Brian Dean, http://www.bdmicro.com/
    [WARN] [WallTime: 1485679097.541449] Copyright (c) 2007-2009 Joerg Wunsch
    [WARN] [WallTime: 1485679097.590572] User configuration file does not exist or is not a regular file, skipping
    [WARN] [WallTime: 1485679097.592759] Using Port                    : /dev/ttyACM0
    [WARN] [WallTime: 1485679097.594433] Using Programmer              : wiring
    [WARN] [WallTime: 1485679097.596058] Overriding Baud Rate          : 115200
    [WARN] [WallTime: 1485679097.949021] AVR Part                      : ATmega2560
    [WARN] [WallTime: 1485679097.951185] Chip Erase delay              : 9000 us
    [WARN] [WallTime: 1485679097.953215] PAGEL                         : PD7
    [WARN] [WallTime: 1485679097.955166] BS2                           : PA0
    [WARN] [WallTime: 1485679097.957096] RESET disposition             : dedicated
    [WARN] [WallTime: 1485679097.958795] RETRY pulse                   : SCK
    [WARN] [WallTime: 1485679097.960472] serial program mode           : yes
    [WARN] [WallTime: 1485679097.962169] parallel program mode         : yes
    [WARN] [WallTime: 1485679097.964188] Timeout                       : 200
    [WARN] [WallTime: 1485679097.965910] StabDelay                     : 100
    [WARN] [WallTime: 1485679097.967649] CmdexeDelay                   : 25
    [WARN] [WallTime: 1485679097.969556] SyncLoops                     : 32
    [WARN] [WallTime: 1485679097.971572] ByteDelay                     : 0
    [WARN] [WallTime: 1485679097.973183] PollIndex                     : 3
    [WARN] [WallTime: 1485679097.974890] PollValue                     : 0x53
    [WARN] [WallTime: 1485679097.976693] Memory Detail                 :
    [WARN] [WallTime: 1485679097.978683] Block Poll               Page                       Polled
    [WARN] [WallTime: 1485679097.980744] Memory Type Mode Delay Size  Indx Paged  Size   Size #Pages MinW  MaxW   ReadBack
    [WARN] [WallTime: 1485679097.982812] ----------- ---- ----- ----- ---- ------ ------ ---- ------ ----- ----- ---------
    [WARN] [WallTime: 1485679097.984698] eeprom        65    10     8    0 no       4096    8      0  9000  9000 0x00 0x00
    [WARN] [WallTime: 1485679097.986527] flash         65    10   256    0 yes    262144  256   1024  4500  4500 0x00 0x00
    [WARN] [WallTime: 1485679097.988323] lfuse          0     0     0    0 no          1    0      0  9000  9000 0x00 0x00
    [WARN] [WallTime: 1485679097.990285] hfuse          0     0     0    0 no          1    0      0  9000  9000 0x00 0x00
    [WARN] [WallTime: 1485679097.992018] efuse          0     0     0    0 no          1    0      0  9000  9000 0x00 0x00
    [WARN] [WallTime: 1485679097.994113] lock           0     0     0    0 no          1    0      0  9000  9000 0x00 0x00
    [WARN] [WallTime: 1485679097.996381] calibration    0     0     0    0 no          1    0      0     0     0 0x00 0x00
    [WARN] [WallTime: 1485679097.998201] signature      0     0     0    0 no          3    0      0     0     0 0x00 0x00
    [WARN] [WallTime: 1485679098.003283] Programmer Type : Wiring
    [WARN] [WallTime: 1485679098.004465] Description     : Wiring
    [WARN] [WallTime: 1485679098.005580] Programmer Model: AVRISP
    [WARN] [WallTime: 1485679098.007438] Hardware Version: 15
    [WARN] [WallTime: 1485679098.008920] Firmware Version Master : 2.10
    [WARN] [WallTime: 1485679098.010680] Vtarget         : 0.0 V
    [WARN] [WallTime: 1485679098.012079] SCK period      : 1.1 us
    [WARN] [WallTime: 1485679098.014203] Reading | ################################################## | 100% 0.01s
    [WARN] [WallTime: 1485679099.784864] Writing | ################################################## | 100% 1.75s
    [WARN] [WallTime: 1485679101.071393] Reading | ################################################## | 100% 1.28s
    
  7. Check that CouchDB is running by using your Mac's browser to open the CouchDB administrative interface at http://raspberrypi.local:5984/_utils/. Check the comments below for screenshots.

  8. Load the OpenAg UI in your Mac's browser by opening https://openaginitiative.github.io/openag_ui/. This will load a single page javascript app that lets your browser make API calls to CouchDB on your Raspberry Pi. Enter a name for your PFC (Personal Food Computer) and the hostname or IP address for your Raspberry Pi. Click the "Save" link in the corner to log in. See the comments below for screenshots.

Review

If you've followed along successfully to this point, here's what you've accomplished:

  1. Obtained a Raspberry Pi 3 and an Arduino Mega (or a compatible clone board).

  2. Installed Raspbian Jessie Lite on the Pi 3 and configured it for ssh remote access over wifi.

  3. Installed openag_brain (links: github repo, wiki documentation) which includes ROS (Robot Operating System) and CouchDB.

  4. Installed openag_python (links: github repo, readthedocs documentation) which provides command line tools for flashing firmware to the Arduino and working with the database.

  5. Loaded the default fixture (links: fixture JSON on github, fixture wiki docs).

  6. Started ROS in a terminal window with rosrun openag_brain main. Because you did this after loading a new fixture, it automatically flashed new firmware to your Arduino Mega (look at the console output above).

  7. Used CouchDB's Futon administration interface to browse the database entries created by loading the default fixture.

  8. Used the OpenAg UI (links: github repo, architecture description, wiki docs, hosted app) to connect to CouchDB's API running on your Pi 3. There wasn't much to see in the UI because this guide didn't include connecting sensors or loading a climate recipe.

Next steps

Assuming you want to learn how the Personal Food Computer v2 works and perhaps contribute improvements, here are some ideas you could try:

  1. Study the system architecture diagram on the OpenAg wiki to understand how the different software components of a PFC2 are related.

  2. Read the PFC2 build instructions in the OpenAgInitiative/openag_pfc2 repo on github. The build instructions include lots of high resolution photos to help you understand how the PFC2 hardware works.

  3. Use the GrabCAD online CAD file viewer to view interactive 3D models of all the PFC2 components and assemblies. The first screen just looks like a big list--to view models, click the links for the .SLDASM and .SLDPRT files.

  4. Read all the links mentioned on the main Personal Food Computer 2 wiki page.

Shutting down gracefully

When you do rosrun openag_brain main, it will keep running and logging status messages to the console indefinitely. You can stop it with ctrl-C. To shut down the Pi 3, the normal sudo shutdown -h now works well.

@wsnook
Owner
wsnook commented Jan 29, 2017

This is what the CouchDB database browser looks like at http://raspberrypi.local:5984/_utils/:

couchdb-utils

couchdb-firmware_module_type

couchdb-software_module

couchdb-software_module_type

@wsnook
Owner
wsnook commented Jan 29, 2017

This is what the UI looks like at https://openaginitiative.github.io/openag_ui/

ui-setup

ui-main

ui-add_recipes

ui-import_recipe

ui-charts

ui-export_csv-1

ui-export_csv-2

@ayburton

The files "ssh" and "wpa_supplicant.conf" disappear from the SD card when I unplugged the Pi after trying to identify its connection with my wifi router (I didn't see it connect so I pulled the power from the Pi and thus checked the micro SD card). Any reason for this? Your method seems helpful but I think I need to start from the very beginning of what OpenAg has provided.

@ayburton

Nevermind, I got past that issue. When trying to install openag_brain main to the Arduino, I get this error in the terminal window: [ERROR] [WallTime: 1486969465.923487] Unable to sync with device; possible link problem or link software version mismatch such as hydro rosserial_python with groovy Arduino

Thoughts on what could have went wrong? I followed the steps exactly up to that point except when "rosrun openag_brain load_fixture default" failed to run, I typed "openag db init" like the program suggested I do before doing rosrun.

@lio-p
lio-p commented Feb 25, 2017

I ran into this same issue: Unable to sync with device; possible link problem or link software version mismatch such as hydro rosserial_python with groovy Arduino.

I fixed it by changing a value in the default fixture (~/catkin_ws/src/openag_brain/fixtures/default.json):

  • Look for the entry openag_brain:handle_arduino.py at the line 581
  • In this entry, modify the parameter should_flash from false to true.

Run the command: rosrun openag_brain main -f default

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment