GSoC 2016 Final Work Product Report
| Name | Role | Contact |
|---|---|---|
| Ananya Bahadur | GSoC Intern | ananya95@gmail.com |
| Jonathan P. Newman | Mentor | jpnewman@mit.edu |
The Open-Ephys project provides powerful extensible platforms to neuroscientists who have diverse requirements for conducting their research. One of the most popular tools, the Open Ephys (Plugin) GUI is the central product which connects and controls various hardware and software solutions.
The primary purpose of the GUI is to allow the user to design a data flow for neural activity signals (acquired with the Open Ephys Aquisition Board). Recently, the GUI was redesigned to use a plugin architecture. Since then, a lot of community developed plugins have been made that allow for event detection, visualisation to be baked right into the data flow.
The Cyclops LED Driver is a "Precision, wide-bandwidth current source with optional optical feedback mode for driving high-power LEDs" and is designed for the Optogenetic Stimulation use case. It can be controlled digitally, using the Arduino IDE and (now, even the) Open-Ephys GUI, or be used as a fast analog device to amplify an input signal and deliver an optical signal.
The optical signal is used to stimulate neural circuits, and with response analysis and feedback loops, scientists gain insight on their function and characteristics.
For more general information on the Open Ephys project, visit the Open-Ephys Wiki, project on GitHub, and the Public Mailing List. There's also a slack channel for devs.
Recently, more scientists are exploring the idea of closed-loop feedback neurons and neural circuits. In this scenario, neurons are not only be monitored but also stimulated in a way that is contingent on their past activity.
This is analogous to the feedback loops used in control systems. Fairly recently, opengenetic techniques have allowed scientists to stimulate genetically defined sets of cell using different colors of light.
This project focused on creation of a plugin for the OE GUI that can talk to Cyclops during an experiment, and forward "actions" based on events detected in the GUI.
See my abstract for more details.
This project has two major goals:
- Port the Cyclops project to the Teensy Family of micro-controllers which are based on ARM Cortex M4 architecture. (Jump)
- Implement a plugin and RPC to program and communicate with the Cyclops (Teensy) (Jump)
Hardware Programming on Teensy 3.2
Cyclops Wiki for users. The project is well documented for developers using Doxygen.
- All commits to
jonnew/cyclops::gsoc-dev(descriptive commit messages!) - All PRs (have been merged to
gsoc-devbranch) - All issues
- My clone
This work will be merged into jonnew/cyclops::master when Cyclpos Rev3.6 [which uses Teensy 3.2] is into beta (it is currently in alpha).
A chronological account of the work done follows.
Cyclops boards can be stacked (up to 4), and only a single Arduino / Teensy must be able to control up to 4 boards. See a very detailed issue in jonnew/cyclops.
The scheduler uses the look-ahead strategy to schedule timer interrupts for future, but performs the scheduling when not servicing interrupt.
- See related bug which is simple to resolve.
- Full design docs (architecture) (will move this to wiki)
- State: working, tested @
c8351d1
Source objects provide the "signal" data which is driven on the LED as an optical signal.
A potential "source" of confusion The equivalent of
Sourceobjects on the OE GUI is called "Signalobject". This differentiation is intentional.
STORED💯% support- The signal points are stored completely in memory.
- Also suited for representing PWM signals.
SQUARE💯% support- Compact representation for a Square wave, with supporting convenience methods.
GENERATEDlimited support- Most compact representation. Instead of saving Signal points, the generating function is stored.
- Signal is evaluated at runtime. Caution
The Arduino Leonardo (or equivalent) powered boards can be used to control up to 4 boards, but at a severe blow to the system's resolution, when the device is controlled digitally. They can deliver a max resolution of just 200 μsec (5kHz), and with each additional connected board the resolution will drop further.
Note By
resolutionwe mean the update-rate of the LED voltage WHEN Cyclops is being driven by a digital device, like an Arduino.
There is virtually no loss (see this) when an input signal is to be amplified and drive the LED.
- See docs for full explanation.
This step implemented a Remote Procedure Call interface on the Teensy and Arduino serial ports.
Thanks to @PaulStoffregen awesome work on the Teensy USB, it was easy to setup a Task Queue.
Built a small python GUI using tkinter to send and receive RPC packets from a connected Teensy.
Completed @fd4b15d
![teensy-gui-screenshot-here][teensy-gui-img]
Documentation coverage for Teensy is 💯, but is missing in many places for Arduino. Usage guide is WIP on Open-Ephys wiki.
If you are not familiar with the Open Ephys GUI and its workflow, I suggest you read this [excellent article][oe-wiki-guide] to form a better context for the following section.
You can also skip "Introduction" and "Usage".
This was the most challenging part of the project. The Cyclops Plugin is very different from other plugins, and its design pushes the plugin-architecture of the OE-GUI project to its limits. The architecture,
- does not allow instances of plugins to (easily) hold shared resources/references,
- does not provide a general interface for inter (and intra) plugin communication, and
- is very strict about ownership of core objects.
Significant time was devoted on developing workarounds for these during the project. But, I must also note that such features are (and will be) required only by few plugins and that investing time in a general interface for all these would be overkill.
Another cool feature is the concept of Cyclops sub-plugins. So, the dynamically loaded Cyclops Plugin can dynamically load more (user) sub-plugins. I took a lot of inspiration (and code! credit: @aacuevas (Aarón Cuevas López) and @yapatel (Yogi Patel) from the OE GUI's Plugin Manager to make the Cyclops Plugin Manager
Succinctly, the Cyclops Plugin provides a way to grab events from the Data Graph and transform them into actions for a Cyclops device.
Other plugins process the data channels (aka data streams) during "aquisition", whereas the Cyclops Plugin does not do any kind of processing -- this apparently dumb behaviour is an intended feature! During acquisition (ie, when the experiment is running), the plugin just passes
- (event) data from
GUI->sub-pluginand - actions (RPC) from
sub-plugin->Cyclops device(via USB).
How is being dumb an intended feature?
[Wiki][plug-wiki] for users. The project is well documented for developers using Doxygen.
- All commits to
arrow-/plugin-GUI::cyclops(descriptive commit messages!) - All PRs (some have been merged to
developmentbranch) - All issues
- My clone
This work will be merged into development pending review and completion. It will eventually be merged into master when the next version of the plugin-GUI is released.
A chronological account of the work done follows.
In OE GUI jargon,
Editorsrepresent plugin instances and you see them on the bottom (Editor Viewport). If the plugin needs to plot something on the screen, it plots on aVisualizerwhich appears on the top right.
See motivation for a shared canvas.
The shared canvas has better UI interactions, that should be moved to the base Visualizer class.
- Changes, older and recent, were made to
VisualizerEditorandVisualizerto make "canvas sharing" possible and to refactor it (see forum). - Also see related PR. My thanks to @sept-en (Kirill Abramov) for suggesting the use of Observer Pattern (
Listenerinterface).
The API depends on openFrameworks and uses the existing OE code for access to serial ports. The API exposes Cyclops RPC commands.
This was based on code from OE GUI's PluginManager, greatly simplified though.
- Completed @16cc0c6
Users extend this base class to implement their "custom" plugins. This base class clearly defines the capabilities and responsibilities of any Cyclops sub-plugin.
Please see project documentation and the User Guide.
- The CyclopsPlugin has an built in Timer, which can be used to perform open-loop control using periodic tasks (
betafeature: see 660332) - There is no need to hardcode the signal data, which is to deployed via the Cyclops device, into the
sub-plugincode. - The Cyclops
sub-pluginsare compiled using a separateMakefile.cyclops. They are not built with the OE GUI plugins.
As I kept adding more and more functionality, the code for these two classes started to develop a smell and become increasingly coupled. Once again, a solution inspired by Observer Pattern was implemented.
Relevant commit.
This was needed to support experiments which might require more than 4 Cyclops Boards. The UI interaction helps to:
- Quickly to migrate a Cyclops
Editor(aka hook) from one Canvas to another, with all of its configuration intact. [2 clicks] - Quickly Migrate all hooks if canvas is "closed". [between 2 to N clicks]
Even though it is not complete, the current implementation is built with soon-to-be-added features in mind.
This happened throughout the project. Especially interesting are the drag-and-drop SignalViews and the Migration UI.
During acquisition, migration and Cyclops-Testing, all UI widgets become "inactive", this is because these three operations can execute only in mutual exclusion (as they change state/core settings).
The Cyclops device is used to drive different signals on the LED.
Currently only
STOREDandSQUAREwave signals can be deployed from the GUI. See why?
Users will spend considerable time to define signals. This nifty package that leverages "ease-of-use" from IPython, allows users to edit, view and save signals quickly using popular scientific python tools like matplotlib, numpy and scipy.
- Save in
YAMLfor OE GUI 💯 - Excellent documentation available from inside
IPython. 💯 - Future Work:
- Save in plain-text for
gnuplot,matlab,json - Launch from within OE GUI, and update OE GUI's signal database "live". This will also help others to make plugin utilities in python.
- Save in plain-text for
These items are required for the completion of the project (as per me). None of these could be delivered by the GSoC 2016 deadline date (Aug 23 UTC) :sad:
- Canvas Upgrades:
- Connect the
HookViewwith the LED by drag-n-drop.
- Connect the
- Core:
- Automatically generate code for Teensy 3.2, Arduino Due (and equivalent).
- Flash the generated code to Teensy, without using Arduino IDE or the Teensy Loader (can be done using simple Makefiles that are available in upstream repos).
- Launch and communicate with any subprocess using redirection.
- Use the subprocess communications to launch the SignalEditor from the OE GUI.
EditorUpgrades:- Make channel-mapper like object. Very challenging given the size constraints.
Here you'll find long-ish explanations for some design decisions.
- Each canvas represents a single Cyclops device. Almost all configuration for the device and
sub-pluginis done here. - Each
Editorrepresents a hook in the data flow, that fishes out (grabs) events from that point in the data flow and forwards them to a Cyclopssub-plugin. - Each Cyclops
sub-plugincan control only 1 Cyclops Board.
Since each Cyclops device can control up to 4 Cyclops Boards, a single canvas must be responsible for more than 1 Editor.
It is impossible to anticipate how events will or can be transformed into actions -- the answer to that will be found when more research -- which this system (hopefully 😉) enables -- is done!
This is especially true when designing experiments around an object as complex as the mammalian brain. Therefore, it was very important to provide the user with extreme flexibility in determining how stimulus patterns would be generated based upon incoming neural events.
Because this system allows unprecedented performance and customizability in the fairly new domain of Optogenetic Stimulation, all while being fairly easy to use, it may enable research that narrows down exactly what stimulus patterns are most effective at evoking neural activity and, consequently, what features of neural activity are most important for circuit function.
The transformation from neural events to optical output happens inside the cyclops sub-plugins. These are normal C++ programs which implement function(s) that, well... perform the transformation and invoke Remote Procedure Calls on the Cyclops device (via an API over USB).
By passing the buck to the user, to make the sub-plugin program intelligent, the system is flexible enough to suit literally any kind of research in this domain. Not to mention the expressive power, popularity and infinite capabilities of the C++ language, which makes this sub-plugin interface an ideal and elegant solution.
Also see future work.
Computation of arbitrary expressions is a costly operation. Even seemingly innocent floating point arithmetic can take many CPU cycles on a Teensy 3.2, because it lacks a FPU.
Teensy 3.6 will have an FPU. The CPU clock is also up to
2xfaster. Additionally, teensy 3.6 is nearly pin-for-pin compatible with the 3.2, which was targeted by this project. Therefore this upgrade will likely come for free 😄
By beta we mean that it cannot be used from the OE GUI, GENERATED can be used now if you are programming the Cyclops directly (which is entirely possible, btw!).
GENERATED will stay in beta till we can confirm that it does not make a significant affect to the resolution in usual scenarios.
####Google OSPO for this wonderful programme and Open-Ephys and Jonathan for this exciting project!