Skip to content

Instantly share code, notes, and snippets.

@loama
Last active March 31, 2023 16:19
Show Gist options
  • Save loama/e6ad9d65f8ffc1d7d752da0525aa2a55 to your computer and use it in GitHub Desktop.
Save loama/e6ad9d65f8ffc1d7d752da0525aa2a55 to your computer and use it in GitHub Desktop.
@loama
Copy link
Author

loama commented Mar 31, 2023

Option Pros Cons
Keeping everything the same for now - No additional setup or infrastructure required.
- Simplest option and no need to do anything.
- May not be scalable for larger data sets or more complex analytics needs.
- May cause performance issues if the database is being used for both transactional and analytical queries.
- May require additional development effort to optimize queries.
Adding a read replica to the db for analytics and big reads - Can offload analytical queries to a separate replica instance, reducing the load on the main database.
- Enables more complex analytics to be performed without affecting the performance of transactional queries.
- Can improve query performance for analytical queries.
- Requires additional infrastructure setup and maintenance.
- Synchronization delays between the replica and the main database can cause discrepancies in data.
Creating a BigQuery database with this info - Enables complex analytics to be performed at scale.
- No need to manage infrastructure or perform maintenance tasks like backups, patching, etc.
- Integrates with other Google Cloud services, like Data Studio, for easy visualization of data.
- Data needs to be migrated to BigQuery, which can be time-consuming and complex.
- Data may need to be transformed or modeled to work with BigQuery, which requires additional development effort.

Overall and TLDR; I would suggest

  • Keeping everything the same if we aren't adding too many new analytical queries.
  • Otherwise adding a read replica for them.
  • And leaving BigQuery out of the picture, for now.
    • it is a database designed for really big datasets with millions or rows and the associated complexity to set it up and maintain it isn't likely to pay off at our scale, and as I've said before, I also personally think our database will go through a big reorganization before we arrive to millions of records.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment