Meet Go-zero — A Way to Define and Load Configuration From Files | by Kevin Wan | May, 2022

How we simplified the parsing of configuration information

Once we write purposes, we mainly use configuration information, from varied shells to nginx, and so forth., all have their very own configuration information. Though this isn’t too troublesome, the configuration gadgets are comparatively cumbersome, and parsing and verifying them could be troublesome.

On this article, I’ll inform how we simplify the definition and parsing of configuration information.

If we need to write a Restful API service, the configuration gadgets will most likely have the next contents.

  • Host — the listening IP, if not crammed, the default is
  • Port— the port to hearken to, required, can solely be a quantity, larger than or equal to 80, lower than 65535
  • LogMode— logging mode, solely file or console could be chosen
  • Verbose — see whether or not to output detailed logs, non-compulsory, default is false
  • MaxConns — the utmost variety of concurrent connections allowed, default 10000
  • Timeout — timeout setting, default 3s
  • CpuThreshold — units the brink for CPU utilization to set off load shedding, default 900, 1000m means 100%

Beforehand we used json for the configuration file, however there was an issue with json that we could not add feedback, so we switched to yaml format.

Now let’s see how simple it’s to outline and parse such a configuration file with go-zero

First, we have to outline the above configuration necessities right into a Go construction, as follows.

RestfulConf struct 
Host string `json:",default="`
Port int `json:",vary=[80,65535)"`
LogMode string `json:",options=[file,console]"`
Verbose bool `json:",non-compulsory"`
MaxConns int `json:",default=10000"`
Timeout time.Period `json:",default=3s"`
CpuThreshold int64 `json:",default=900,vary=[0:1000]"`

As you possibly can see, we’ve got sure definitions and restrictions for every configuration merchandise, a few of that are outlined as follows.

  • default, the default worth is used if the configuration just isn’t crammed. You’ll be able to see that 3s in it is going to be robotically resolved into time.Period kind
  • non-compulsory — you possibly can go away the merchandise clean. If not, use the zero worth of its kind
  • vary — restricts the quantity kind to the given vary
  • choices — restricts the configured worth to solely one of many given values

And, some attributes can be utilized collectively, equivalent to.

  • default and vary can be utilized collectively to each add a variety restriction and supply a default worth
  • default and choices can be utilized collectively so as to add choice restrictions and supply a default worth

As a result of we give a whole lot of default values when defining the configuration, in addition to utilizing non-compulsory to specify as non-compulsory, so our configuration file has comparatively few configuration gadgets that may use the default values with out writing explicitly, as follows.

# As a result of lots of them have default values, you solely want to jot down those
# that have to be specified and those that do not have default values
Port: 8080
LogMode: console
# You'll be able to learn the values of atmosphere variables
MaxBytes: $MAX_BYTES

Here’s a notice, if the worth of the configuration chars are all numbers, and also you outline the configuration kind is string, for instance, some individuals typically use 123456 to check the password, however the password will usually be outlined as string, the configuration must be written as follows (simply an instance, passwords are usually not really helpful to be written within the configuration file)

Password: "123456"

Right here the double quotes cannot be missed, in any other case kind mismatch might be reported, as a result of yaml parser will parse 123456 as int.

We’ve the configuration definition (config.go) and the configuration file (config.yaml), the subsequent step is to load the configuration file, which could be executed in 3 ways.

  • should be loaded efficiently, in any other case this system exits, we usually use so, if the configuration just isn’t appropriate, this system won’t be able to proceed
// If error, exit this system straight
var config RestfulConf
conf.MustLoad("config.yaml", &config)

The default code generated by go-zero‘s goctl instrument additionally makes use of MustLoad to load the configuration file

  • Load the configuration and deal with the error by yourself
// deal with errors by yourself
var config RestfulConf
// For simplicity, LoadConfig might be modified to Load later,
// LoadConfig has been marked as Deprecated.
if err := conf.LoadConfig("config.yaml", &config); err ! = nil
  • Load the configuration and skim the atmosphere variables
// Robotically learn atmosphere variables
var config RestfulConf
conf.MustLoad(configFile, &config, conf.UseEnv())

Right here is why we have to explicitly specify conf.UseEnv() — as a result of if default to make use of, it’s possible you’ll must escape when writing particular characters within the configuration, so the default doesn’t learn the atmosphere variables.

We often use encoding/json or the corresponding yaml library when implementing yaml/json parsing. However for go-zero, we have to have extra exact management over unmarshal, which leads us to customise yaml/json parsing ourselves. The whole code is right here:

That is additionally an excellent instance of how mirror can be utilized and the way unit exams can be utilized to make sure correctness in advanced eventualities.

I all the time advocate the thought of Fail Quick.

We do the identical on loading configuration information. As soon as there’s an error, instantly exit, in order that ops will discover the issue in time when deploying providers, as a result of the method cannot stand up and operating.

All of the configuration gadgets of go-zero providers are loaded and robotically verified on this manner, together with the configuration of lots of the instruments I wrote are additionally primarily based on this, hope it helps!

Welcome to make use of go-zero and star to help us!

More Posts