Managing distributed file system with consistent distributed store & leader based algorithm

Atin Mukherjee (~atin)


7

Votes

Description:

In a typical distributed system managing n number of nodes in an effective way is always a challenge. Distributed systems always demands to meet CAP theorem which is really hard to meet with a good performance numbers. All the nodes participating in the cluster should have the consistent data which is one of the criteria of the CAP theorem. This could be achieved by keeping the configuration details across all the nodes in the cluster, however this algorithm doesn’t scale (considering n X n number of exchange of information) and can end up having split brain situations. This could be avoided having distributed consistent store across m number of nodes (considered) as leaders where m < n. There are few technologies like consul/etcd by which this can be achieved.

Prerequisites:

People should be aware of what a distributed system means and its good to have background on it.

Content URLs:

http://www.slideshare.net/AtinMukherjee/consensus-algo-withdistributedkeyvaluestoreindistributedsystem

Speaker Info:

Atin is a Glusterfs hacker and currently working as a Senior S/W Engineer in Redhat India. Apart from storage he has also worked on different other domains which include telecom, BFS.

Section: Others
Type: Talks
Target Audience: Intermediate
Last Updated:

The CAP theorem is an impossibility result according to wikipedia. So could you explain what you mean by "Distributed systems always demands to meet CAP theorem which is really hard to meet with a good performance numbers"?

Aditya Manthramurthy (~donatello)

CAP theorem happens to gurantee any two of consistency, availability & partition tolerance. If you fail to meet even two out of three then you are in danger. IMO, meeting all three is impossible, but not two.

Atin Mukherjee (~atin)

Login to add a new comment.