dynamo-cassandra

• get()
– the coordinator requests all existing versions of data for that key from the N highest-ranked reachable nodes in the preference list for that key, a – waits for R responses before returning the result to the client. – If the coordinator ends up gathering multiple versions of the data, it returns all the versions it deems to be causally unrelated. The divergent versions are then reconciled and the reconciled version superseding the current versions is written back.
12
Replication(3/4)
• Cassandra system elects a leader amongst its nodes using a system called Zookeeper • All nodes on joining the cluster contact the leader who tells them for what ranges they are replicas for • The leader makes a concerted effort to maintain the invariant that no node is responsible for more than N-1 ranges in the ring. • The metadata about the ranges a node is responsible is cached locally at each node and in a fault-tolerant manner inside Zookeeper • This way a node that crashes and comes back up knows what ranges it was responsible for.
Middle Layer
Commit log Memtable
Top Layer
Tombstones Hinted handoff Read repair
7
Partition(1/3)
• Consistent Hash Table
8
Partition(2/3)
• Problems:
– the random position assignment of each node on the ring leads to non-uniform data and load distribution – the basic algorithm is oblivious to the heterogeneity in the performance of nodes.
Handling permanent failures
• Replica synchronization: anti-entropy
– To detect the inconsistencies between replicas faster and to minimize the amount of transferred data
10
Replication(1/4)
• high availability and durability • replication_factor:N
11
Replication(2/4)
• Strategy
– Rack Unaware – Rack Aware – Datacenter Aware –…
• Bigtable-like features
– Column family – SSTable disk storage
• Commit log • Memtable • Immutable Sstable files
3
Data Model(1/2)
• A table is a distributed multi dimensional map indexed by a key
Cassandra
chengxiaojun
1
Backend
• Amazon Dynamo • Facebook Cassandra（Dynama 2.0）
– Inbox search
• Apache
2
Cassandra
• Dynamo-like features
– – – – Symmetric, P2P architecture Gossip-based cluster management DHT Eventual consistency
13
Replication(4/4)
• Cassandra provides durability guarantees in the presence of node failures and network partitions by relaxing the quorum requirements
14
Data Versioning
• Vector clocks
15
Consistency
W+R>N
16
Consistency
• put() :
– the coordinator generates the vector clock for the new version and writes the new version locally. – The coordinator then sends the new version (along with the new vector clock) to the N highest-ranked reachable nodes. – If at least W-1 nodes respond then the write is considered successful.
• Wiki:
– /cassandra/API
6
Architecture Layers
Core Layer
Messaging service Gossip Failure detection SSTable Cluster state Partitioner Replication Indexes Compaction Bootstrap Monitoring Admin tools
are converted to this range by MD5 hashing for comparison with Tokens.
– NetworkTopologyStrategy:calculate the tokens the nodes in
each DC independently. Tokens still needed to be unique, so you can add 1 to the tokens in the 2nd DC, add 2 in the 3rd, and so on.
Cassandra Consistency For Read
Level ANY Behavior Not supported. You probably want ONE instead. Will return the record returned by the first replica to respond. A consistency check is always done in a background thread to fix any consistency issues when ConsistencyLevel.ONE is used. This means subsequent calls will have correct data even if the initial read gets an older value. (This is called ReadRepair) Will query all replicas and return the record with the most recent timestamp once it has at least a majority of replicas (N / 2 + 1) reported. Again, the remaining replicas will be checked in the background. Returns the record with the most recent timestamp once a majority of replicas within the local datacenter have replied. Returns the record with the most recent timestamp once a majority of replicas within each datacenter have replied. Will query all replicas and return the record with the most recent timestamp once all replicas have replied. Any unresponsive replicas will fail the operation.
9