Created
October 9, 2022 06:43
-
-
Save geetikabatra/cd715cb836100783d858e2710537c921 to your computer and use it in GitHub Desktop.
Pridhi Outreachy
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# Pridhi Outreachy | |
Send this to Máirín Duffy | |
"Máirín Duffy" <duffy@fedoraproject.org> | |
Hi Máirín | |
I am Pridhi, from India. Currently an Outreachy candidate. I am writing this email to express interest in one of the projects that you have listed. I am particulary interested in the project **Improve how plugin instance logs are stored in the backend**. I have done a little bit of my part on the research side. This project requires storing logs in object storage. By object storage, right now Amazon s3 and it's equivalent Minio comes to my mind. Though, I would be needing some time to get a grasp on both of these technologies, but I am willing to make my best effort. What do you think about the technologies I have mentioned? | |
Also, please let me know how can I start my contribution? I have started listening to the talks that are mentioned on the project details. Also, I have started setting up ChRIS. I have some knowledge of python, docker and kubernetes. If there is something specific you would like me to take up as a part of my contribution; I would highly appreciate it as it will provide me a proper path to start my work and prove my dedication. | |
Thanks | |
Regards | |
Pridhi Arora | |
pylint command
pylint **/*.py
autopep8 command
autopep8 --in-place --aggressive --aggressive **/*.py
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Store ChRIS Logs in Object Storage
Table of Contents
Glossary
Summary
ChRIS project currently stores log messages as strings in Postgres DB, and this causes Postgres DB to be populated with relatively large strings. The large size causes the backend to slow down.
One quick fix is to truncate the log string that is already implemented and use only a limited string. Right now, a substring until 3000 words is populated into the database, and the rest is truncated. But this devoids us of a lot of information. The decoration of log strings is done to make messages aesthetically pleasing. Removing unnecessary characters, such as emojis and Chinese characters, reduces the string size from 4000B to 2000B. Both the above fixes are still not enough as log size increases daily, and it is essential to find a better solution, and this is where object storage comes into the picture.
Motivation
The absence of a proper storage system for logs is slowing the backend, causing many users to be frustrated. The idea is to have an appropriate design so that ChRIS system is seamless.
Goals
Proposal
User Stories
Story - Decreasing the latency in ChRIS backend.
Stacy works at an IT department of an organization where ChRIS backend is used; she intends to use it regularly and needs logs to refer to. Storing logs in object storage would give her more readable logs.
Requirements
Functional Requirements
Non-Functional Requirements
Implementation Details/Notes/Constraints](#implementation-detailsnotesconstraints
Proposed Changes
Currently, the summary gets saved in Postgres.
When object storage will be implemented, the following lines would need to be changed.
Following is a code snippet taken from the official Minio Docs that can be used to implement the functionality.
Test Plan
Graduation Criteria