- About Me
- Abstract
- Implementation
- Timeline
- Why I am interested in working on this project
- Commitments
- Open Source Contributions
Name : Sarthak Negi
Github : sarthaksarthak9
Email : sarthaknegi908@gmail.com
Location : Delhi,India
Timezone : IST(UTC +5:30)
Portfolio/blog : https://sarthak007.hashnode.dev/
Currently, Armada is limited to using Postgres as its database, which has presented numerous challenges, particularly in terms of adoption and testing. Therefore, the primary objective of this project is to develop interfaces around Armada's Postgres access. These interfaces will allow users to integrate their preferred SQL databases into Armada seamlessly. This provides flexibility for developers to use these interfaces in their own applications. Furthermore, the project will enhance testing capabilities by enabling the addition of SQL call mocks and allow to run unit tests without an active database.The project will initially focus on supporting Postgres, but it will be design in such a way so that it supports other relational databases such as MSSQL, SQLLite, and MySQL. The project will not directly support these databases until there is a request to do so, but the code will be prepared so that it can be easily added in the future.
Related Issue : armadaproject/armada#2121
1. Approach 1 Using pq package to implement the database/sql interface. Therefore, we can use the pq package to connect and interact with Postgres databases from Go.
2. Approach 2 Use the GORM ORM library to connect and interact with a Postgres database. It makes it easy to interact with databases from Go.
Using pq package to implement the database/sql interface. Therefore, we can use the pq package to connect and interact with Postgres databases from Go.
go get github.com/lib/pqimport (
"database/sql"
"fmt"
"log"
_ "github.com/lib/pq"
)
func ConnectDB() (*sql.DB, error) {
db, err := sql.Open("postgres", "postgres://user:password@localhost/dbname?sslmode=disable")
if err != nil {
return nil, err
}
// Test the connection
if err := db.Ping(); err != nil {
db.Close()
return nil, err
}
return db, nil
}- The ConnectDB() opens a connection to a PostgreSQL database.
- If the connection fails, the function returns an error.
- If the connection succeeds, the function then tests the connection by calling the db.Ping() method.
- If the Ping() method fails, the function closes the connection and returns an error.
- If the Ping() method succeeds, the function returns the sql.DB object and no error.
func GetUsers(db *sql.DB) ([]User, error) {
rows, err := db.Query("SELECT id, name, email FROM users")
if err != nil {
return nil, err
}
defer rows.Close()
var users []User
for rows.Next() {
var user User
if err := rows.Scan(&user.ID, &user.Name, &user.Email); err != nil {
return nil, err
}
users = append(users, user)
}
return users, nil
}- The GetUsers() fetch all users from Postgre database. It takes a pointer to a sql.DB object as a parameter and returns an array of User.
- If the query fails, the it returns an error.
- If the query succeeds, the it iterates over the results of the query and creates a User object for each row. The User object is then added to an array.
type UserRepository interface {
GetUsers() ([]User, error)
}
type PostgresUserRepository struct {
db *sql.DB
}
func NewPostgresUserRepository(db *sql.DB) *PostgresUserRepository {
return &PostgresUserRepository{db: db}
}
func (r *PostgresUserRepository) GetUsers() ([]User, error) {
return GetUsers(r.db)
}
- It defines an interface UserRepository that represents a repository for users.
- It includes a struct PostgresUserRepository that implements the UserRepository interface specifically for Postgres databases.
- The NewPostgresUserRepository function is used to create a new instance of PostgresUserRepository, and the GetUsers method is implemented to fetch users using the PostgresUserRepository.
func TestGetUsers(t *testing.T) {
db, err := sql.Open("postgres", "postgres://user:password@localhost/dbname?sslmode=disable")
if err != nil {
t.Errorf("Error opening database: %v", err)
}
defer db.Close()
users, err := GetUsers(db)
if err != nil {
t.Errorf("Error getting users: %v", err)
}
if len(users) == 0 {
t.Errorf("Expected at least one user, got none")
}
for _, user := range users {
t.Logf("User: %d, %s, %s", user.ID, user.Name, user.Email)
}
}- The TestGetUsers() takes a testing.T object as a parameter.
- It ensures that the GetUsers() returns an array of User objects with at least one user.
- If the connection fails, the function fails the test.
- If the connection succeeds, the it then calls the GetUsers().
func CloseDB(db *sql.DB) {
db.Close()
}Use the GORM ORM library to connect and interact with a Postgres database. It makes it easy to interact with databases from Go.
go get -u gorm.io/gormgo get -u gorm.io/driver/postgresimport (
"gorm.io/driver/postgres"
"gorm.io/gorm"
)
func ConnectDB() (*gorm.DB, error) {
dsn := "host=localhost user=user password=password dbname=dbname port=5432 sslmode=disable"
db, err := gorm.Open(postgres.Open(dsn), &gorm.Config{})
if err != nil {
return nil, err
}
return db, nil
}- The ConnectDB() opens a connection to a PostgreSQL database.
- The gorm.DB object can be used to execute SQL queries and to interact with the database.
type User struct {
gorm.Model
Name string
Email string
}
func (u *User) TableName() string {
return "users"
}
func CreateUser(db *gorm.DB, user *User) error {
return db.Create(user).Error
}
func GetUserByID(db *gorm.DB, id uint) (*User, error) {
var user User
if err := db.First(&user, id).Error; err != nil {
return nil, err
}
return &user, nil
}- The User struct represents a user in a PostgreSQL database.
- It implements the gorm.Model interface, which allows it to be used with the GORM ORM library.
func TestConnectDB(t *testing.T) {
db, err := ConnectDB()
if err != nil {
t.Errorf("Error connecting to database: %v", err)
}
}
func TestCreateUser(t *testing.T) {
db, err := ConnectDB()
if err != nil {
t.Errorf("Error connecting to database: %v", err)
}
user := &User{
Name: "Sarthak",
Email: "sarthaknegi@example.com",
}
err = CreateUser(db, user)
if err != nil {
t.Errorf("Error creating user: %v", err)
}
userFromDB, err := GetUserByID(db, user.ID)
if err != nil {
t.Errorf("Error getting user from database: %v", err)
}
if userFromDB.Name != user.Name {
t.Errorf("Expected user name to be %s, got %s", user.Name, userFromDB.Name)
}
if userFromDB.Email != user.Email {
t.Errorf("Expected user email to be %s, got %s", user.Email, userFromDB.Email)
}
}- It takes a testing.T object as a parameter and ensures that the ConnectDB() returns a gorm.DB object without an error.
- If the connection fails, the it fails the test.
- If the connection succeeds, then it ensures that the ConnectDB() returns a gorm.DB object.
func CloseDB(db *gorm.DB) {
db.Close()
}- Week 1:
Familiarising with codebase + Discussion of ideas with Mentors. - Week 2:
Identify the different SQL databases that Armada will support. - Week 3 - Week 5:
Design the interfaces for interacting with the different SQL databases. - Week 6 - Week 7:
Implement the interfaces for interacting with the different SQL databases. - Week 8:
Mid-Term Evaluations - Week 9 - Week 10:
Test the interfaces for interacting with the different SQL databases - Week 11:
Deploy the interfaces for interacting with the different SQL databases - Week 12:
Final Evaluations + Blog
I have only listed down the potential priority and focus tasks for each week in the above timeline, In addition to that my work in each week would be a blend of the following :
- Breaking down issues into smaller modular tasks
- Interacting and getting feedback from mentors
- Team meetings
- Writing blog post sharing my progress and LFX experience along the way
I am interested in working on this project and Armada because it presents an exciting opportunity to apply my skills in developing a interface using SQL and golang to increase flexibility and extensibility of Armada, while getting mentored by experienced devs.I believe that by developing a well-defined interface for interacting with different database, we are improving code quality and maintainability. Additionally, I am interested in working with Armada because it is a well-established project with a strong community and this mentorship offers me a great opportunity to make a meaningful contribution to the Armada ecosystem.
There is no conflict of interest in the coding phase as of now and I don't plan to have any other commitment during the period and I will dedicate my summer to this project. If there will be any sudden changes in the university schedule, I will discuss with the mentors.
- Write a guide for deploying Armada to GKE
- Write a guide for deploying Armada to AKS
- Airflow - If you mark a dag as failed jobservice will still keep listening on the jobset.