Efficient Annotation APIs for Go.Orchestrating Annotated Data.Collaborative, Scalable Annotations.Powerful Tools for Annotation.Building Annotation Ecosystems.Unified Platform for Annotation.Elevate Data with Smart Annotations.Manage Annotations at Scale.Streamline Annotated Content.Next-Gen Solutions for Annotation.Drive Data Insights Through Annotations.Simplifying Complex Annotations.Expertly Engineered for Annotation.
Table of Contents
The Modware Annotation project is an open-source, containerized solution that enables processing and management of OboJSON annotations using gRPC APIs and an Arangodb repository. Its primary features include OboJSON file handling via CRUD operations, messaging for interprocess communication, and scalable Kubernetes deployments with automatic updates, linting, testing, and containerization. By following best practices for CI/CD and providing a clear GitHub workflow structure, Modware Annotation ensures efficient development and deployment of annotation-related tools and services.
└── modware-annotation/
├── .github
│ ├── dependabot.yml
│ └── workflows
├── LICENSE
├── README.md
├── build
│ ├── ci
│ └── package
├── cmd
│ └── modware-annotation
├── deployments
│ └── charts
├── go.mod
├── go.sum
└── internal
├── app
├── collection
├── message
├── model
└── repository
.
| File |
Summary |
| go.mod |
Manages project dependencies via go.mod file in the modware-annotation repository, importing essential modules for ArangoDB interaction, gRPC development, logging, testing, and CLI utilities. |
| go.sum |
Creating new annotation definitions with custom keys, values, and associated descriptions2. Attaching annotations to entities or data points for easy retrieval and reuse3. Implementing a flexible schema for annotations that allows them to be extended as needed over time4. Supporting querying, filtering, and sorting based on annotation information5. Providing an intuitive API or user interface for working with annotations, making it convenient for developers and users alike. |
cmd.modware-annotation
| File |
Summary |
| main.go |
Launches Modware-annotation microservice with GRPC backends and ontology loading functionality. It handles command-line arguments for server settings, ArangoDB collections, and logs formats. The main file initializes the CLI app and runs specified commands using imported modules. |
internal.repository
| File |
Summary |
| error.go |
Create custom error types for various repository operations, enabling clear identification of issues. This file includes definitions for AnnoNotFoundError, GroupNotFoundError, AnnoListNotFoundError, and AnnoGroupListNotFoundError, with their corresponding Error() methods. Additionally, IsAnnotationNotFound(), IsGroupNotFound(), IsAnnotationListNotFound(), and IsAnnotationGroupListNotFound() functions check if an error is of a specific type. |
| repository.go |
Interface repository.go defines TaggedAnnotationRepository, enabling access to annotation data via methods like GetAnnotationByID, AddAnnotation, and ListAnnotations for managing and retrieving tagged annotations with associated groups and tags using Arangomanager, go-genproto, and internal model structures. |
internal.repository.testdata
| File |
Summary |
| dicty_phenotypes.json |
The file internal/repository/testdata/dicty_phenotypes.json in the modware-annotation repository serves as test data for handling and processing phenotype graph information. The main purpose of this JSON file is to store a predefined, expected set of phenotype node data conforming to the OBO (Open Biological and Biomedical Ontologies) format. This enables testing the functionalities related to reading, parsing, and processing ontology data within the application.This test data is crucial for validating the correctness and reliability of the repository package when dealing with real-world phenotype data. It enables unit testing various scenarios such as creating and updating nodes, querying nodes, and maintaining consistent relationships between related nodes. By using standardized OBO formats like the one in this file, tests can be designed to ensure robust handling and proper integration of biological ontologies within the modware-annotation architecture. |
| dicty_annotation.json |
The internal/repository/testdata/dicty_annotation.json file is a configuration or test data file used within the modware-annotation repositorys architecture. Specifically, its located under the repository/testdata directory which implies its purpose is related to testing and data preparation for the repository's functional components. This JSON file stores graph data, possibly representing various nodes, likely related to the annotation domain within the Bio-Ontology or OBO (Open Biological and Biomedical Ontologies) context. Given the location and structure of this codebase, it appears the dicty_annotation.json file could serve as a mock or test dataset for testing and validating various aspects of the modware-annotation library's repository functionalities. |
internal.repository.arangodb
| File |
Summary |
| arangodb.go |
Verify database connection. Set up collections for annotations and ontologies using ArangoDB. Ensure persistent indexes for faster querying. Create graphs for storing version history and annotation group relationships. |
| annotation_write_test.go |
Tests annotation repositorys functions for adding and editing tagged annotations as well as managing groups. Ensures proper versioning and data consistency in Arangodb repository. |
| field.go |
Maps filter attributes to corresponding database fields for ArangoDB interaction in modware-annotations repository structure. Provides a translation layer between application filters and database schema, enhancing data consistency. |
| statement.go |
Create an ArangoDB query that fetches the most recent version of annotations for a given entry ID, ranked by version number, and returns only those with a specified tag from a specific ontology. |
| annotation_delete.go |
The internal/repository/arangodb/annotation_delete.go file implements the RemoveAnnotation function for deleting annotations from the repository. This includes checking if the annotation exists, marking it as obsolete when not purging, and removing it from the database upon request. Additionally, it provides a method to remove annotations from existing annotation groups with RemoveFromAnnotationGroup. |
| annotation_read_test.go |
The file internal/repository/arangodb/annotation_read_test.go is a part of the modware-annotation repository, specifically located within the package arangodb in the internal/repository folder. This package deals with interacting with the ArangoDB database in the context of the modware-annotation project.The primary purpose of this file is to contain test functions for reading annotations from an ArangoDB instance, ensuring data correctness and testing various querying capabilities against the database. It utilizes Gos built-in testing package and imports required dependencies, including custom models used by the modware-annotation project and the go-genproto/dictybaseapis/annotation package that defines data structures for storing and exchanging annotation data. By providing comprehensive test coverage, the code in this file plays a crucial role in the reliability of the larger repository's architecture dealing with reading and handling annotations stored in ArangoDB instances. |
| parameters.go |
Create parameters for Arangodb database configurations are defined in this parameters.go file. It contains structures for necessary collections, such as Annotation, AnnoGroup, AnnoTerm, and others. These configurations ensure proper functionality when dealing with annotation data within the repository architecture. |
| annotation_read.go |
Retrieve annotations for ontology ontology and tag tag. If tag exists, create new annotation with rank rank and update ontology term association accordingly. |
| ontology.go |
Loads OBJJSON files into an Arangodb database using the go-obograph library. This package function handles data importation and creation of necessary collections within the Arangodb repository. It processes OBO format JSON files, ensuring their successful upload into the specified ontology database. |
| pairwise.go |
Create iterator for pairwise processing of strings and model structures in Arangodb repository. The internal/repository/arangodb/pairwise.go file defines two iterators, StringPairWiseIterator and ModelAnnoDocPairWiseIterator, for efficiently fetching pairs from slices of strings and model.AnnoDoc types respectively, with proper error checking and iteration control. |
| annotation_delete_test.go |
Deletes annotations from a specified annotation group in ArangoDB repository. The test file verifies the correct functionality of removing annotations from an existing group using given ids. If an attempt is made to delete an already deleted group, it should return an error. Similarly, the function tests removal of obsolete annotations. |
| annotation_write.go |
Call AddAnnotation with required parameters and ontology to store the new annotation document. This method handles transactional writes to the database while maintaining group association. |
| arangodb_test.go |
Verify test data is loaded successfully.Create annotated model pairs and sort them by creation date.Check groups have correct number of annotations with desired tags and creators.Use map function to extract IDs from annotation documents. |
internal.collection
| File |
Summary |
| collection.go |
Determines string inclusion with IncludeString and removes specified strings from slice using RemoveStringItems. These functions enhance data manipulation capabilities within the modware-annotation project's collection package. |
internal.app.validate
| File |
Summary |
| validate.go |
Validates server arguments passed to the application during startup. Checks for mandatory parameters such as Arangodb password, database name, user, NATS host, and port. Returns an error if any argument is missing, ensuring correct configuration before proceeding. (internal/app/validate/validate.go) |
internal.app.server
| File |
Summary |
| server.go |
Create an ArangoDB connection and configure a NATS messaging publisher for communication. The ArangoDB repository ensures data persistence, while the NATS publisher facilitates interprocess communication. |
internal.app.service
| File |
Summary |
| service.go |
The service.go file within the modware-annotation project implements an AnnotationService that manages OboJSON file uploads and processing. It includes functions to handle stream writing, create new AnnotationService instances, and define the services GetGroupResourceName method. The service interacts with a repository to load OboJSON files, and it communicates upload status and messages back to clients through gRPC. |
| delete_service.go |
Delete service in internal/app/service/delete_service.go facilitates removing annotation groups and annotations from the repository through API calls. It validates input, interacts with repository functions, and returns error responses accordingly. |
| read_service.go |
ListAnnotations)2. Update existing annotations using PATCH with modified data in the request body. (Reference: ListAnnotations)3. Delete annotations by sending DELETE requests with desired annotation ID. (Reference: ListAnnotations)4. Filter annotations with the given parameters and retrieve the matching results. (Reference: ListAnnotations, ListGroups)5. Access annotation details like name, ontology, and other attributes using GET request on annotag service. (Reference: GetAnnotationTag) |
| write_service.go |
UpdateAnnotation and CreateAnnotation. Interacts with the repository layer via dependency injection for editing and adding data, and publishes updates using a publisher. |
internal.message
| File |
Summary |
| message.go |
Manages publishing functionality for message annotations in this Go package. Implements an interface with methods to publish annotation objects using specified subjects and close underlying messaging server connection. Part of internal message module within the modware-annotation repository. |
internal.message.nats
| File |
Summary |
| nats.go |
The nats.go file inside modware-annotation/internal/message/ directory creates a NATS publisher using the nats-io library. It implements a Publish method for sending tagged annotations as messages, and handles connection and encoding. |
internal.model
| File |
Summary |
| model.go |
Model package initializes data structures for handling annotated documents, groups, and database transactions using Arangodb driver in the modware-annotation repository. Contains functions for converting interface to AnnoDoc type and creating unique models. |
.github
| File |
Summary |
| dependabot.yml |
Schedules automatic updates for dependencies in the Go modfile and GitHub Actions workflows on a weekly basis, ensuring the projects develop branch stays up-to-date with the latest package versions. |
.github.workflows
| File |
Summary |
| create-pull-request.yml |
The create-pull-request.yml workflow file in this repository initiates automated pull requests, streamlining the merge process and ensuring continuous integration for the modware-annotation project. |
| testcov.yml |
In this GitHubActions workflow file, tests are configured for the modware-annotation project. It triggers on pushes to the main branch and runs tests to ensure code coverage, enhancing the repository's reliability and ensuring high-quality deliverables. |
| lint.yml |
In the modware-annotation repository, the.github/workflows/lint.yml file initiates automated lint checks to ensure code quality and consistency across the project's Go packages (located in the internal directory) as part of its continuous integration workflow. |
| staging-build.yaml |
Orchestrates automated staging builds for the modware-annotation repository using GitHub Actions, ensuring seamless integration of new features and bug fixes before deployment. Critical elements include dependabot updates and package building in the designated environment. |
| ci.yml |
The ci.yml file in the modware-annotation repository initiates Continuous Integration workflows. It automates building and testing new code commits for quick feedback and ensuring consistent software quality. |
| chatops.yml |
In this GitHub Workflows file (chatops.yml), automation is defined for handling various development tasks in the modware-annotation repository, including dependencies updates, tests, and deployments, ensuring an efficient software development process and reducing manual intervention. |
build.ci
| File |
Summary |
| skaffold.yaml |
The build/ci/skaffold.yaml file configures Skaffold for constructing container images of the modware-annotation project, employing a custom Dockerfile and managing tagging. It's crucial to our CI workflow within this repository structure, ensuring efficient and consistent image builds. |
build.package
| File |
Summary |
| Dockerfile |
Builds Go binary for modware-annotation using Golang image with dependencies installed, compresses it with Upx, and creates a container with distroless static image to run the binary. |
deployments.charts.annotation-api-server
| File |
Summary |
| Chart.yaml |
Deploy annotation API service with this Helm chart, versioned at 1.1.0, integrating seamlessly into the parent repositorys Kubernetes deployments. Description and appVersion set to 1.0.0" in the v1 architecture. |
| .helmignore |
Ignoring specific file patterns during Helm chart building is essential for maintaining clean package deployments. The provided.helmignore file, situated within the annotation-api-server chart of the modware-annotation repository, ensures excluded files arent unintentionally included during this process. |
| values.yaml |
Configures and deploys the annotation-api service using the latest modware-annotation image from Dictybase's repository. Defines connection details for an Arangodb database and sets up log level, ports, and resources. Adjusts values in the dictybaseConfiguration, service, logLevel, resources, and tolerations sections as needed. |
deployments.charts.annotation-api-server.templates
| File |
Summary |
| deployment.yaml |
Deploys annotation API server as a Kubernetes Deployment with customizable settings like replica count, image version, ports, and environment variables using templates in the charts directory. |
| NOTES.txt |
The NOTES.txt file in the annotation-api-server chart directory outlines crucial deployment instructions for the Modware Annotation project. It includes essential environment variables and database configuration details to ensure a successful Helm installation and seamless integration with the parent repositorys architecture. |
| service.yaml |
Create and configure Kubernetes services for the annotation-api server within the chart directory, defining the API version, kind as Service, providing necessary metadata, and setting the specification with the desired service type and port assignments. |
| _helpers.tpl |
Define custom helper functions for charts name expansion and creation in annotation-api-server" templates, ensuring full names comply with Kubernetes naming constraints. |
System Requirements:
- Clone the modware-annotation repository:
$ git clone https://github.com/dictyBase/modware-annotation
- Change to the project directory:
- Install the dependencies:
Run modware-annotation using the command below:
Run the test suite using the command below:
Contributions are welcome! Here are several ways you can contribute:
Contributing Guidelines
- Fork the Repository: Start by forking the project repository to your github account.
- Clone Locally: Clone the forked repository to your local machine using a git client.
git clone https://github.com/dictyBase/modware-annotation
- Create a New Branch: Always work on a new branch, giving it a descriptive name.
git checkout -b new-feature-x
- Make Your Changes: Develop and test your changes locally.
- Commit Your Changes: Commit with a clear message describing your updates.
git commit -m 'Implemented new feature x.'
- Push to github: Push the changes to your forked repository.
git push origin new-feature-x
- Submit a Pull Request: Create a PR against the original project repository. Clearly describe the changes and their motivations.
- Review: Once your PR is reviewed and approved, it will be merged into the main branch. Congratulations on your contribution!
Contributor Graph
This project is protected under the SELECT-A-LICENSE License. For more details, refer to the LICENSE file.
- List any resources, contributors, inspiration, etc. here.
Return
