Skip to content

Instantly share code, notes, and snippets.

@jessedearing
Created August 26, 2017 15:35
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save jessedearing/5b7f8ce09d49be9b4c518a44d4351442 to your computer and use it in GitHub Desktop.
Save jessedearing/5b7f8ce09d49be9b4c518a44d4351442 to your computer and use it in GitHub Desktop.

This is a call to the RDS API to create new instances of MySQL. One of the things I've found really helpful is that as a staticly typed language, Go gives me reliable parameter autocomplete (via https://github.com/nsf/gocode) so I can pretty easily look up parameters to the *Input and *Output structs in all editors that have the integration In my case vim-go.

Amazon's Go API has a few quriks (like the pointers for everything needing the aws.String and aws.Bool convenience functions), but once you get into it flows easily.

Since the AWS functions are all in interfaces, I can mock github.com/aws/aws-sdk-go/service/rds/rdsiface using https://github.com/maxbrunsfeld/counterfeiter and write expectations in my unit tests.

package provision
import (
"context"
"net/url"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/aws/session"
"github.com/aws/aws-sdk-go/service/rds"
"github.com/aws/aws-sdk-go/service/rds/rdsiface"
)
// CreateMysql is the command responsible for creating new mysql instances
type CreateMysql struct {
//Identifier will be the identifier for the database
Identifier string
// ResultURL is the url of the created resource with host name, password, and username
ResultURL url.URL
Cluster string
RDSClient rdsiface.RDSAPI
}
//NewCreateMysql creates a new CreateMysql Command
func NewCreateMysql(cluster, name string) *CreateMysql {
sess := session.Must(session.NewSessionWithOptions(session.Options{
Config: aws.Config{Region: aws.String("us-east-1")},
}))
svc := rds.New(sess)
return &CreateMysql{
Cluster: cluster,
Identifier: name,
RDSClient: svc,
}
}
// Run creates the specified database instance
func (c *CreateMysql) Run(ctx context.Context) error {
paramGrpCreateInput := &rds.CreateDBParameterGroupInput{
DBParameterGroupFamily: aws.String("mysql5.6"),
DBParameterGroupName: aws.String(c.Identifier + "-group"),
Description: aws.String(c.Identifier + " parameter group"),
}
_, err := c.RDSClient.CreateDBParameterGroup(paramGrpCreateInput)
if err != nil {
log.Println("ERROR", err)
}
_, err = c.RDSClient.ModifyDBParameterGroup(&rds.ModifyDBParameterGroupInput{
DBParameterGroupName: paramGrpCreateInput.DBParameterGroupName,
Parameters: []*rds.Parameter{
&rds.Parameter{
ParameterName: aws.String("performance_schema"),
ParameterValue: aws.String("1"),
ApplyMethod: aws.String("pending-reboot"),
},
&rds.Parameter{
ParameterName: aws.String("innodb_adaptive_hash_index"),
ParameterValue: aws.String("0"),
ApplyMethod: aws.String("immediate"),
},
}})
if err != nil {
log.Warn(err)
}
pass := GenerateRandomPassword()
createdInstance, err := c.RDSClient.CreateDBInstance(&rds.CreateDBInstanceInput{
DBInstanceIdentifier: aws.String(c.Identifier),
DBParameterGroupName: paramGrpCreateInput.DBParameterGroupName,
Engine: aws.String("mysql"),
AllocatedStorage: aws.Int64(100),
DBInstanceClass: aws.String("db.t2.medium"),
MasterUsername: aws.String("admin"),
MasterUserPassword: aws.String(pass),
VpcSecurityGroupIds: aws.StringSlice([]string{securityGroups[c.Cluster]}),
DBSubnetGroupName: aws.String(subnets[c.Cluster]),
MultiAZ: aws.Bool(true),
PubliclyAccessible: aws.Bool(false),
StorageType: aws.String("gp2"),
StorageEncrypted: aws.Bool(true),
})
if err != nil {
log.Error(err)
return err
}
c.ResultURL.Scheme = "mysql"
c.ResultURL.User = url.UserPassword("admin", pass)
c.ResultURL.Host = *createdInstance.DBInstance.DBInstanceIdentifier + accountRDSSuffix[c.Cluster]
return nil
}
func (c *CreateMysql) URL() url.URL {
return c.ResultURL
}
package provision_test
import (
"context"
. "provision"
"provision/fakes"
"github.com/aws/aws-sdk-go/service/rds"
. "github.com/onsi/ginkgo"
. "github.com/onsi/gomega"
)
var _ = Describe("CreateMysql", func() {
var (
createMysql CreateMysql
mockRDS *fakes.FakeRDSAPI
)
BeforeEach(func() {
mockRDS = &fakes.FakeRDSAPI{}
createMysql = CreateMysql{
RDSClient: mockRDS,
Cluster: "unit-test-cluster",
Identifier: "unit-test-db",
}
})
Context("Run", func() {
It("should create the RDS instance", func() {
mockRDS.CreateDBInstanceStub = func(in *rds.CreateDBInstanceInput) (*rds.CreateDBInstanceOutput, error) {
return &rds.CreateDBInstanceOutput{
DBInstance: &rds.DBInstance{
DBInstanceIdentifier: &createMysql.Identifier,
},
}, nil
}
Expect(createMysql.Run(context.Background())).To(BeNil())
Expect(mockRDS.CreateDBInstanceCallCount() > 0).To(BeTrue())
})
})
})
Copy link

ghost commented Aug 27, 2017

Cool! I really like the idea of having some richer tooling (with good autocomplete) and being able to write tests in the same language as the deployment scripting language. This looks like it lends itself to deployment on things with good API's. What's your current thinking on using this to update existing things, and the required tracking or discovering or overwriting of existing state? Or does this assume creating things from zero each time?

On the Ansible side of things, the promise of idempotence and being able to rerun things and apply only the necessary changes sounds awesome, but my experience so far is that it feels like a lot of extra work to make that happen for not so much reward (vs just burning something to the ground and rebuilding it from zero). Re:Ansible testing, a colleague has also been experimenting with test-kitchen and inspec to test the results of executing an ansible script against locally spun up vagrant instances. Re:autocomplete for ansible, the best I have found is the VS code plugin (based on an atom plugin I think), which, basically just dumps a subset of ansible documentation inline for a given core ansible module/command.

For Amazon provisioning, right now, we are looking at Terraform. It has the state tracking built in to avoid applying the same changes twice, but the lack-of tooling around editing and [automated] testing does not seem great. I had started to wonder what a well architected bash script that just uses the Amazon CLI would like, but looking at what you're doing in GO with the Amazon API seems like a stronger direction to encourage more rigor around testing, and gain the advantages of better tooling support.

Lastly, for Go, I did find this on github after seeing your initial comment on twitter:

https://github.com/tj/stack

It is focused more on the commands you run, locally, to configure an instance that has been provisioned already, and tracks state by storing a log of hashes of every command that has run against that local machine to avoid re-running the same thing twice. Kind of reminds me of rails migrations or Flyway, but for for system config.

Go seems like it is on the ascent language of choice for tools in this space. A colleague recently pointed out to me that all the Hashicorp stuff is written in Go.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment