Quick hack to auto play the dino game with PMod ALS, PYNQ-Z2 and MS speech recognition

pynq_dino_game.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# PYNQ Dino game \n",
    "\n",
    "Use a PYNQ-Z2 board with a Pmod ALS sensor and audio out from the board, combined with a Windows 10 Speech recognition to automatically play the Dino game. \n",
    "\n",
    "For this example I played the output from the board through a speaker and used a mic from my laptop placed beside the speark. \n",
    "Ideally the output from the board would go straight to a mic input to the laptop but I didn't have an appropriate cable. Surprisingly (to me at least!) this \"sort of\" worked!! I would like to get an audio cable with a line for the mic to see if this improves the quality. Sometimes the speech recognition didn't work well. I also wanted to keep the speech command (\"space bar\") short to increase the responsiveness of the system. I'm sure this makes it harder for the Speech recognition. \n",
    "\n",
    "Ref: chrome://dino/ - open this link in Chrome browser to play the game. \n",
    "        \n",
    "##  Setup\n",
    "Tape the ALS sensor to the screen to detect the \"trees\" in the dino game. It may take some trial and error to find the right position, and to calibrate the ALS sesor value. \n",
    "\n",
    "I used \"masking\" tape which didn't seem to leave a residue on my screen ;-)\n",
    "\n",
    "This project is a quick hack. The MS speech recognition may not be responsive. I disabled and didn't try the \"cloud option\" as I assume this woudl add delay. \n",
    "The sensor sample reading rate may not be high enough, but this worked OK for me to score a few points in the game!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Download the PYNQ overlay"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from pynq.overlays.base import BaseOverlay\n",
    "base = BaseOverlay(\"base.bit\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "pAudio = base.audio"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Run once to record \"space bar\" speech and get audio to calibrate the windows speech recognition. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#pAudio.select_microphone()\n",
    "#pAudio.record(0.5)\n",
    "#pAudio.save(\"space_bar1.wav\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Run once to record MS speech recognition paragraph \"peter prefers ...\"\n",
    "#pAudio.select_microphone()\n",
    "#pAudio.record(8)\n",
    "#pAudio.save(\"peter.wav\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Test recording in browser\n",
    "#from IPython.display import Audio as IPAudio\n",
    "#IPAudio(\"peter.wav\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Test recording on output (line out)\n",
    "I used this to configure the Win 10 speech recognition"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#pAudio.load(\"peter.wav\")\n",
    "#pAudio.set_volume(50)\n",
    "#pAudio.play()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Test \"space bar\" "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "pAudio.load(\"space_bar.wav\")\n",
    "pAudio.set_volume(60)\n",
    "pAudio.play()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Set up ALS sensor on PMod A"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from pynq.lib import Pmod_ALS\n",
    "my_als = Pmod_ALS(base.PMODA)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "ALS_THRESHOLD may need ot be calibrated"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Run the game\n",
    "ALS_THRESHOLD = 50\n",
    "\n",
    "pAudio.load(\"space_bar1.wav\")\n",
    "while(1): \n",
    "    light_value = my_als.read()\n",
    "    if(my_als.read() <ALS_THRESHOLD):\n",
    "        pAudio.play()\n",
    "        # print(light_value) # Debug"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}