Changes to the server
To change our server to be distributed, we need to do a few things:
integrate the raft library into our code
store our data in a way that can be replicated
add a way for the replicas to tell each other about themselves
We will go over this last part in more detail later, but this is a nice reminder that libraries are not magic. Replicas do not just automatically know about each other. This surprised us when we learned it - we had really hoped ZooKeeper (the first distributed lock server we ever used) would just know about other ZooKeeper entities automatically, and instead you had to know the IP addresses of every node. It was a real pain.
The first step in dealing with our rewrite is move functions to a separate package. This makes documentation and testing easier, and means our code can be depended on by multiple libraries.
Set up the store folder#
To do this, we will create a new folder called
store. All Go files in this folder will have
package store as one of their first lines.
Note that it doesn't declare the entire package path - that is determined by the parent
go.mod file, which we're not creating here as it causes all sorts of issues (if we updated
store, we would need to update the
go.mod of the root package, which we can't do until we've deployed and waited for all of the caches to clear). It works fine, but it tends to cause confusion around what code you are actually calling.
Go modules are usually best tied to their version control. That is the
go.mod should be at the root of your version control. So if your Git repository contains multiple packages, that's fine, but try to keep just one Go module (ie one
go.mod file) to keep things sane. The downside is then all of your packages share a dependency tree, because dependencies are defined in
go.mod. This won't affect your built binaries though, just how many dependencies you download when you build your package.
Inside of this folder, we can create Go files with any name. We will call our first file
store.go but it could be literally anything and still work.
Set up the
First things first, we will create a Config struct for whomever uses our server to initialize. We will add all of the data access functions to this struct.
We mention this in other chapters as well, but using a struct pointer allows for us to do a few things easily:
share configuration without globals
do testing using mocks
make sure the server is initialized before calling
We are making the fields of the struct
private because we do not want anyone outside of this package to be able to modify the values, but we still want to be able to access and mutate them in the
store package. Note here that:
lowercase fields are the equivalent of
capitalized functions and types are
We will implement our methods shortly, but we need two more very important functions, one that creates the config and one that returns a middleware for the server to use. We will walk through the middleware in the "Connecting the servers" section, but first let us build our config factory (a factory is a common name for a function that generates an instance of a type).