Skip to content

Instantly share code, notes, and snippets.

@dadarom
Last active November 30, 2017 06:00
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save dadarom/910ce7f7d77c96dca64d887e2f606155 to your computer and use it in GitHub Desktop.
Save dadarom/910ce7f7d77c96dca64d887e2f606155 to your computer and use it in GitHub Desktop.
Info Collection
分布式配置中心,监控报警,flume管控,进程管控【分布式管控】,binlog,vue,common frontend,H5(app),知识储备(牛博),
@dadarom
Copy link
Author

dadarom commented Jan 7, 2017

@dadarom
Copy link
Author

dadarom commented Jan 7, 2017

@dadarom
Copy link
Author

dadarom commented Jan 7, 2017

Tech之外

市值风云
百家

@dadarom
Copy link
Author

dadarom commented Jan 12, 2017

@dadarom
Copy link
Author

dadarom commented Jan 12, 2017

@dadarom
Copy link
Author

dadarom commented Jan 13, 2017

Spring

spring-kafka https://projects.spring.io/spring-kafka/#quick-start
spring-reactive
spring-plugin
spring-flo HTML5 visual,used as the basis of the stream builder in Spring Cloud Data Flow
spring-retry
spring-hadoop
spring-shell
spring-ldap
spring-xd solve common big data problems such as data ingestion and export, real-time analytics, and batch workflow orchestration
spring-batch-extensions
spring-sync
spring-bus

@dadarom
Copy link
Author

dadarom commented Jan 15, 2017

@dadarom
Copy link
Author

dadarom commented Jan 17, 2017

大数据

sequence-file http://dongxicheng.org/mapreduce/hdfs-small-files-solution/
浅析Hadoop文件格式

fastdfs最佳实践
HDFS主要解决并行计算中数据的分布式(分块)存储,其单个数据文件通常很大;
FastDFS主要用于大中网站,为文件上传和下载提供在线服务。所以在负载均衡、动态扩容等方面都支持得比较好,FastDFS不会对文件进行分快(切分)存储。

列式存储与行式存储
为什么列存储数据库读取速度会比传统的行数据库快?
列存储的数据库更适合OLAP,行存储的数据库更适合OLTP

HDFS分级存储
Hadoop小文件问题及解决方案

Parquet VS ORC
数据库好像汽车,引擎就是发动机

ODS、数据集市、数据仓库的异同点

example

用Hive分析CSDN泄露的用户数据

spark

spark 原理
Spark累加器
Spark共享变量
Spark广播变量与累加器
var counter = 0 var rdd = sc.parallelize(data) // Wrong: Don't do this!! rdd.foreach(x => counter += x) // reduceByKey

Spark性能优化(1)——序列化、内存、并行度、数据存储格式、Shuffle
Spark性能优化(2)——广播变量、本地缓存目录、RDD操作、数据倾斜
Tuning Spark

spark2.0源码解析系列

Spark Network 模块分析(基于Netty的实现)

深入研究 spark 运行原理
Spark 2.0分布式集群环境搭建

盘点大数据开源软件Google Trends指数
linkedin Hadoop任务调度

Apache Kylin在唯品会大数据的应用

大数据盘点之Spark篇

@dadarom
Copy link
Author

dadarom commented Jan 17, 2017

@dadarom
Copy link
Author

dadarom commented Jan 17, 2017

web framework

58 Argo

@dadarom
Copy link
Author

dadarom commented Feb 4, 2017

@dadarom
Copy link
Author

dadarom commented Feb 4, 2017

@dadarom
Copy link
Author

dadarom commented Feb 4, 2017

@dadarom
Copy link
Author

dadarom commented Feb 17, 2017

@dadarom
Copy link
Author

dadarom commented Feb 18, 2017

@dadarom
Copy link
Author

dadarom commented Feb 24, 2017

点滴杂记

SQL的JOIN语法解析

@dadarom
Copy link
Author

dadarom commented Apr 11, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment