分布式系统相关技术

作者：rainoftime
链接：https://www.zhihu.com/question/30026369/answer/46528616
来源：知乎
著作权归作者所有。商业转载请联系作者获得授权，非商业转载请注明出处。

1. 背景知识
体系结构

系统和网络
通信：RPC、RMI、MOM。。

进程和线程：
用户态、内核态；轻量级进程；协程；Actor。。

分布式相关问题
同步和互斥：保证相互冲突的并发进程可以共享资源
Double checked Locking、Immutable Value、Future 。。

事件分离和分发：Reactor、Proactor。。。

选举：从进程集中选出一个进程执行特别的任务

2. 分布式理论
数据结构
B树
log merge tree
merkle tree
一致性hash
DHT
vector clock
lock-free data structure
....

CAP、BASE
CAP: Brewer’s Conjecture and the Feasibility of Consistent, Available, Partition-Tolerant Web
BASE an Acid Alternative

状态、时序

Time Clocks and the Ordering of Events in a Distributed System

Virtual Time and Global States of Distributed Systems

Distributed Snapshots: Determining Global States of a Distributed System

2PC、3PC 、Paxos ...
A brief history of Consensus- 2PC and Transaction Commit
Paxos Made Simple.
Paxos Made Practical
Paxos made live . An engineering perspective.

一致性、事务
Life beyond Distributed Transactions: an Apostate’s Opinion
Impossibility of distributed consensus with one faulty process.
Consensus on Transaction Commit.
Uniform consensus is harder than consensus

3. 分布式系统
分布式基础设施
消息队列
RabbitMQ 、ZeroMQ...

分布式锁服务、协调
The Chubby lock service for loosely-coupled distributed systems
Zookeeper

集群Monitoring

The ganglia distributed monitoring system:design, implementation, and

experience

Chukwa: A large-scale monitoring system

分布式存储系统
分布式文件系统
The Google file system.
Lustre
Cepth
Panasas

分布式块存储
Sheepdob
Parallax
Petal

分布式k-v存储系统
Dynamo: Amazon’s highly available key-value store

分布式表格系统
Amazon DynamoDB
Bigtable: A Distributed Storage System for Structured Data.

分布式数据库
Spanner: Google's Globally-Distributed Database

分布式计算
Map-Reduce
MapReduce: Simplified Data Processing on Large Clusters

内存计算
Resilient Distributed Datasets: A Fault-Tolerant Abstraction for In-Memory Cluster Computing

流式计算
S4: Distributed Stream Computing Platform
Twitter Storm

图计算
GraphLab: A New Framework for Parallel Machine Learning
Pregel: a system for large-scale graph processing


4. 分布式应用
图片、视频等
Finding a Needle in Haystack: Facebook's Photo Storage

搜索
Web search for a planet: The Google cluster architecture

IM