Instantly share code, notes, and snippets.

Embed
What would you like to do?
InterMine Tool Integration (Proposal)

IM InterMine tool integration framework (proposal)


Mission Statement

To make available to the InterMine developer community, a simple framework for building modular Javascript-based tools that make use of the InterMine REST API to communicate with one or more publicly accessible InterMine instances, and perform data integration, analysis or visualization on the client-side. These tools can be configured to run as standalone entities or embeded within the scope of the InterMine UI, BlueGenes.

(The facets described below correspond to the various topics covered during the round-table discussion at the recent 2017 InterMine Developer Workshop)

Facets
Tool scaffold

Creating an InterMine tool scaffold (layout for code organization) and making it available would greatly benefit the developers of InterMine integrations, ensuring that tools produced by different development groups are modular, re-usable units. Such a scaffold can have the following properties:

  • Adhering to a standardized app layout. More complex applications are free to define a custom layout according to their own requirements. For example:

     .
     +-- myapp/
     |   +-- index.html
     |   +-- scripts/
     |       +-- myapp.js
     |   +-- styles/
     |       +-- myapp.css
     |   +-- assets/
     |       + logo.png
    
  • Specifying third-party dependencies via npm package.json config. This will allow the InterMine environment to determine dependencies at runtime, and share resources across multiple applications loaded within the same scope. For example:

     $ cd myapp
     ### install and save desired dependcies from npm
     $ npm i biojs-vis-box-plot --save
     $ cat package.json
     {
     	"name": "myapp",
     	"description": "My cool InterMine app which generates fancy boxplot visualizations"
     	"version": "0.1.0",
     	"dependencies": { 
     		"biojs-vis-box-plot" : "0.1.9"
     	},
     	"devDependencies": { },
     	"engines": { }
     }
    
  • Including API helpers:

    • Inclusion of the InterMine JS Client Library (imjs) as dev/test dependencies enables the developer to make use of the library during the course of the tool development (on their personal machines). When the tool is deployed to the InterMine framework, imjs is automatically injected into the global scope and made available commonly to all tools.

    • For developers who do not wish to make use of imjs, inclusion of an interactive Swagger API console (in-conjuction with plans to refactor the InterMine web-service description to follow the Swagger spec) would be beneficial to developers, enabling to the testing of API calls prior to including them in the tool.

Tool entry point

Standardizing the entry point for third-party InterMine tools ensures that tools can expect to recieve a consistent set of inputs from InterMine. Tools can be of several types and operate in different scopes/contexts. For example:

  • Report page context
    • Accepts: current InterMine Object; its ID, attributes, references and collections
  • List analysis context
    • Accepts: current List Name
  • Search Results context
    • Accepts: one or more InterMine Objects IDs
  • Custom context
    • Accepts: free-form input (An example would be a BLAST tool)

Facilitating InterMine-to-tool (and vice-versa) communication is another aspect that should be standardized within the InterMine core framework:

  • Exposing a publish/subscribe messaging system that will allow developers to hook in their tools to listen for events dispatches from InterMine in order to perform specific tasks.
  • Listening via specific channels for incoming messages, allowing tools to pass objects back into the scope of InterMine.
Tool tagging

Encouraging the developers of InterMine tools to add tags/keywords about the tool, ensures the availability of human and machine readable metadata, increasing their discoverability by others in the community.

  • Making use of the keywords property in the npm package.json file allows for the addition of plain-text tags used to describe key features of the tool. Tags can be data model related, consisting of InterMine Class names pertaining to queries used or entities being operated upon in the tool. (Please refer to discussion on NPM github regarding caveats of using this property).
Tool approval

Ensuring that all InterMine tools go through some form of informal approval and feedback process can guarantee a certain level of consistency across the wide gamut of tools that are being developed by community members.

Involving the community in the approval/feedback process ensures that members from different areas of the genomics research community can participate in an informal peer-review process of sorts, collectively helping establish a high quality, standardize set of published tools that can be utilized by current and future adopters of the InterMine framework.

  • Proposing a set of standards that is expected to be satisfied by third-party contributed tools, some of which are listed below:
    • Require
      • Adhere to basic styling standards, consistent with InterMine UI framework
      • Publish code in open-source code repositories (GitHub, GitLab, Bitbucket, etc.)
    • Strongly advice
      • Include tests (using a test suite that has been standardized by the InterMine core team)
      • Write well-documented code
  • Making use of online open code review workflows to facilitate the tool approval process:
    • One potential option: GitHub code review
    • Each submission could be assigned 2 or more reviewers, 1 from the core InterMine team and 1 or more from the community (based on their area of expertise)
    • Example: JBrowse-Registry is an active, community-contributed repository of JBrowse-compatible plugins.
  • Announcing the public availability of tools (for review or for general use) via InterMine social media channels (Dev mailing list, Twitter, Discord, etc.)

These standards could be drafted and proposed via an RFC to the community, soliciting their feedback.


@justinccdev

This comment has been minimized.

justinccdev commented Apr 26, 2017

Sounds good to me, though I think any formal approval/feedback process will be too heavyweight to maintain. I'm not a frontend/viz/web developer so discount my opinions accordingly.

@vivekkrish

This comment has been minimized.

Owner

vivekkrish commented Apr 26, 2017

@justinccdev, Thanks for your comment!

I agree that a very formal feedback process can definitely be cumbersome. I was thinking of something more informal, similar to the recent efforts by GMOD/JBrowse community to build a comprehensive registry of available plugins. I've updated the text above with some more clarifications to the Tool approval section.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment