Properly, you could hear quite a bit about “distributed system” or “raft” itself. However, you could surprise use it.
On this tutorial, we’ll be discussing deal with kv database operating in cluster mode utilizing Golang and Raft library. This tutorial is extra targeted on the code and utilizing raft clusters quite than discussing the raft algorithms.
Raft is a protocol with which a cluster of nodes can keep a replicated state machine.
The state machine is stored in sync by using a replicated log. Nevertheless, the small print of the Raft protocol are exterior the scope of this tutorial, For extra particulars on Raft, see In Search of an Understandable Consensus Algorithm
Golang implementation of the raft
Raft algorithm comes in quest of an comprehensible consensus algorithm. Sadly, many of the go libraries on the market required a deep data of their implementation and APIs.
The raft library we’ll be utilizing on this tutorial was born to align with the understandability raft precept and its sole function is to offer consensus with the minimalistic, easy, clear, and idiomatic API.
Etcd Raft is probably the most broadly used Raft library in manufacturing. However, it follows a minimalistic design philosophy by solely implementing the core raft algorithm which leaves gaps and ambiguities.
So, as a substitute of reinventing the wheel,
shaj13/raft library makes use of
etcd raft as its core.
That’s how one can profit from the facility and stability of
etcd raft, with an comprehensible API. Certainly, it retains your deal with constructing superior software program.
We’re going to begin by creating our venture.
mkdir raft && cd raft && go mod init raft && contact raft.go
This may create a brand new listing referred to as
raft and initialize the venture with
Earlier than we write any code we have to write some obligatory code to make this system run.
package deal predominant
We’re going to take away the road that prints out
Raft!!, add the flag package deal and initialize it.
package deal predominant
addr := flag.String("raft", "", "raft server handle")
be a part of := flag.String("be a part of", "", "be a part of cluster handle")
api := flag.String("api", "", "api server handle")
state := flag.String("state_dir", "", "raft state listing (WAL, Snapshots)")
We’re going to implement a struct named
kv database that reads and applies a key worth alongside taking
database snapshot and restoring it.
We’re going to add the gorilla mux package deal and initialize the router inside the principle perform.
router := mux.NewRouter()
Now we’re going to set up the endpoints of our API, the best way we’ll set this up is to create all of our endpoints in the principle perform, each endpoint wants a perform to deal with the request and we’ll outline these beneath the principle perform.
router.HandleFunc("/", http.HandlerFunc(save)).Strategies("PUT", "POST")
Now we simply must outline the capabilities that can deal with the requests.
Earlier than we begin we have to declare two variables to permit routes entry information.
- Node represents raft course of
- FSM represents raft course of state machine
In the principle perform and beneath the router we have to declare our raft node and
gRPC server to permit present raft node to speak with different raft nodes.
Your file ought to now look one thing like this:
Constructing raft cluster
go mod tidy && go construct raft.go
Working single node raft
First, begin a single-member cluster of raft:
./raft -state_dir=$TMPDIR/1 -raft :8080 -api :9090
Every raft course of maintains a single raft occasion and a key-value server.
raft server handle (
-raft), state dir (
-state_dir), and http key-value server handle (
-api) are handed by the command line.
Subsequent, retailer a worth (“
medium”) to a key (“
curl -L http://127.0.0.1:9090/ -X PUT -d '"Key":"howdy", "Worth":"medium"'
Lastly, retrieve the saved key:
curl -L http://127.0.0.1:9090/hello
Working a neighborhood cluster
Let’s deliver two extra raft situations.
./raft -state_dir $TMPDIR/2 -raft :8081 -api :9091 -join :8080
./raft -state_dir $TMPDIR/3 -raft :8082 -api :9092 -join :8080
Now it’s potential to jot down a key-value pair to any member of the cluster and likewise retrieve it from any member.
To check cluster restoration, write a worth “
foo” to key “
curl -L http://127.0.0.1:9090/ -X PUT -d '"Key":"foo", "Worth":"foo"'
Subsequent, cease a node (9092) and change the worth with “bar” to verify cluster availability:
curl -L http://127.0.0.1:9090/ -X PUT -d '"Key":"foo", "Worth":"bar"'
curl -L http://127.0.0.1:9090/foo
Lastly, deliver the node again up and confirm it recovers with the up to date worth “
curl -L http://127.0.0.1:9092/foo
Nodes may be added, eliminated, or up to date from a operating cluster. Let’s take away node utilizing requests to the REST API.
First, listing all raft cluster nodes, and get node id.
curl -L http://127.0.0.1:9090/mgmt/nodes
Then take away a node utilizing a DELETE request:
curl -L http://127.0.0.1:9090/<ID> -X DELETE
The node will shut itself down as soon as the cluster has processed this request.