Deep dive in system design with detailed knowledge on some important components and approaches of system design

advance-system-design.md

Dynamo: How to Design a Key-value Store?

1. Amazon’s Dynamo : https://www.allthingsdistributed.com/2007/10/amazons_dynamo.html
2. Eventually Consistent : https://www.allthingsdistributed.com/2007/12/eventually_consistent.html
3. Bigtable : https://research.google/pubs/pub27898/
4. DynamoDB : https://www.allthingsdistributed.com/2012/01/amazon-dynamodb.html
5. CRDT : https://en.wikipedia.org/wiki/Conflict-free_replicated_data_type
6. A Decade of Dynamo : https://www.allthingsdistributed.com/2017/10/a-decade-of-dynamo.html
7. Riak : https://docs.riak.com/riak/kv/2.2.0/learn/dynamo/
8. Dynamo Architecture : https://www.youtube.com/watch?v=w96lLsbI1q8
9. Dynamo: A flawed architecture : https://news.ycombinator.com/item?id=915212

Cassandra: How to Design a Wide-column NoSQL Database?

1. Bigtable : https://research.google/pubs/pub27898/
2. Dynamo : http://www.read.seas.harvard.edu/~kohler/class/cs239-w08/decandia07dynamo.pdf
3. Datastax docs : https://docs.datastax.com/en/dse/6.8/dse-arch/datastax_enterprise/dbArch/archTOC.html
4. Tombstones common problems : https://opencredo.com/blogs/cassandra-tombstones-common-issues/
5. The Phi Accrual Failure Detector : http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.80.7427&rep=rep1&type=pdf
6. Cassandra introduction video : https://www.coursera.org/lecture/cloud-applications-part2/2-3-1-cassandra-introduction-olmpu

Kafka: How to Design a Distributed Messaging System?

1. Confluent Docs : https://docs.confluent.io/current/kafka/design.html#ak-design
2. New York Times use case : https://www.confluent.io/blog/publishing-apache-kafka-new-york-times/
3. Kafka Summit : https://www.confluent.io/resources/kafka-summit-san-francisco-2019/
4. Kafka Acks Explained : https://medium.com/better-programming/kafka-acks-explained-c0515b3b707e
5. Kafka as distributed log : https://www.youtube.com/watch?v=ElilYxUOjOQ
6. Minimizing Kafka Latency : https://www.confluent.io/blog/configure-kafka-to-minimize-latency/
7. Kafka internal storage : https://thehoard.blog/how-kafkas-storage-internals-work-3a29b02e026
8. Exactly-Once semantics : https://www.confluent.io/blog/exactly-once-semantics-are-possible-heres-how-apache-kafka-does-it/
9. Split-brain : https://techthoughts.typepad.com/managing_computers/2007/10/split-brain-quo.html

Chubby: How to Design a Distributed Locking Service?

1. Chubby paper : https://research.google/pubs/pub27897/
2. Chubby architecture video : https://www.youtube.com/watch?v=PqItueBaiRg
3. Chubby vs. ZooKeeper : https://www.youtube.com/watch?v=zokwJeukDrI
4. Hierarchical Chubby : https://www.scs.stanford.edu/17au-cs244b/labs/projects/bohn_dauterman.pdf
5. Bigtable : https://research.google/pubs/pub27898/
6. Google File System : https://research.google/pubs/pub51/

GFS: How to Design a Distributed File Storage System?

1. GFS paper : https://research.google/pubs/pub51/
2. Bigtable : https://research.google/pubs/pub27898/
3. GFS: Evolution on Fast-forward : https://queue.acm.org/detail.cfm?id=1594206

HDFS: How to Design a Distributed File Storage System?

1. HDFS paper : https://storageconference.us/2010/Papers/MSST/Shvachko.pdf
2. HDFS High Availability (HA)architecture : https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/HDFSHighAvailabilityWithNFS.html
3. Apache HDFS Architecture : https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/HdfsDesign.html
4. Distributed File Systems: A Survey : http://ijcsit.com/docs/Volume%205/vol5issue03/ijcsit20140503234.pdf

BigTable: How to Design a Wide-column Storage System?

1. BigTable : https://research.google/pubs/pub27898/
2. SSTable : https://medium.com/databasss/on-disk-io-part-3-lsm-trees-8b2da218496f
3. Dynamo : https://www.allthingsdistributed.com/2007/10/amazons_dynamo.html
4. Cassandra : https://cassandra.apache.org/
5. HBase : https://hbase.apache.org/