Semant1ka/simple_data_solution_comparison.md

## simple_data_solution_comparison.md

      
    Raw
  

              simple_data_solution_comparison.md
            
          
Stack
Performance
Cost
Usage
Maintenance
Scalability


Elasticsearch
Kibana dashboard
Depends on amount or RAM for cluster, if full index is RAM will have close to zero latency, also supports storing similar information in one cluster to enhance speed
Elastic Only - $265.63/month
Elastic + Kibana by ElasticCloud on Google Cloud -$1464.16/month
Free trial available
Needs a special query language. Has beta SQL intefrace. Will be realized this year, beta is available now for usage.
Easy
Scalable - each node store some indexes, scaled by adding nodes.


MongoDb
BI connector
Visuablization

Free for Atlas users. Build to support SQL.
Easy
Not That Scalable, still needs SQL like schema, can’t do joins between collections."Scalability - if your queries will bring back millions of rows the BI Connector (an ODBC driver) will fall over. You'll need to do something on the software or hardware side to chunk the queries"


MongoDb
ETL
AWS redshift
Visualization

Pricing
No charge for data transfer between S3 and Redshift
Build on top of PostgreSQL, supports PostgreSQL syntax
Somewhere in the middle.
Scalable - scales into clusters, can set up clusters for different team, though there is a downtime when scaling occurred.


Hadoop
Spark
SQL database
Impala
Visualization
This is the most popular big data setup, query to hadoop itself can be slowed done when growing in size,so the best approach is to process data and store it in relational database.

SQL query can be performed through Impala.
Hard
Scalable - can have as many nodes in HDFS as you want.


AWS Athena
AWS S3

$5.00 per TB of data scanned.
SQL
This solution looks like cutting edge no-ETL.
AWS S3 is scalable noSQL storage
Stack	Performance	Cost	Usage	Maintenance	Scalability
Elasticsearch Kibana dashboard	Depends on amount or RAM for cluster, if full index is RAM will have close to zero latency, also supports storing similar information in one cluster to enhance speed	Elastic Only - $265.63/month Elastic + Kibana by ElasticCloud on Google Cloud -$1464.16/month Free trial available	Needs a special query language. Has beta SQL intefrace. Will be realized this year, beta is available now for usage.	Easy	Scalable - each node store some indexes, scaled by adding nodes.
MongoDb BI connector Visuablization		Free for Atlas users. Build to support SQL.	Easy	Not That Scalable, still needs SQL like schema, can’t do joins between collections."Scalability - if your queries will bring back millions of rows the BI Connector (an ODBC driver) will fall over. You'll need to do something on the software or hardware side to chunk the queries"
MongoDb ETL AWS redshift Visualization		Pricing No charge for data transfer between S3 and Redshift	Build on top of PostgreSQL, supports PostgreSQL syntax	Somewhere in the middle.	Scalable - scales into clusters, can set up clusters for different team, though there is a downtime when scaling occurred.
Hadoop Spark SQL database Impala Visualization	This is the most popular big data setup, query to hadoop itself can be slowed done when growing in size,so the best approach is to process data and store it in relational database.		SQL query can be performed through Impala.	Hard	Scalable - can have as many nodes in HDFS as you want.
AWS Athena AWS S3		$5.00 per TB of data scanned.	SQL	This solution looks like cutting edge no-ETL.	AWS S3 is scalable noSQL storage