WP LarryD's Blog

Working with JSON in Go

WP LarryD's photo
WP LarryD
·

3 min read

Table of contents

Go JSON decoding is bit different than others again. It does not accept a string and return an object like JS or Python. Actually it does not return anything, we need to pass the pointer of a struct, and decode into it. This allows user to control the struct type before decoding.

What is the behaviour when the shape of json string is not exactly same as struct type definition?

Default decode behaviour

// call Unmarshal to decode
var in = `{"keywords": "json"}`
json.Unmarshal([]byte(in), &out)

For missing fields which not appear in json text, it's still decoded into struct out with Go zero values (e.g int is 0, bool is false). For unknown fields which not defined in out but appeared in json text input, they will be ignored.

// if we want to error out when seeing unknown
var in = `{"keywords": "json"}`
json.NewDecoder(bytes.NewReader(in)).
 DisallowUnknownFields().
 Decode(&out)

Control default values

If we want default values different than Go "zero" values.

Use simple wrapper

Let's say if we want decode request payload, set a default size if it's empty.

type Options struct {
  Keyword string `json:"terms"`
  Size    int `json:"size"`
}

const defaultSize int = 10
func(t *Options) FromRequest(body io.Reader) {
  json.NewDecoder(body).Decode(t)
  if t.Size == 0 {
    t.Size = defaultSize
  }
}

// from the handler
p := NewPayload()
p.FromRequest(req.Body)

The issue is we cannot differentiate if a field is missing or it's passed in with a real empty value? e.g a bool false or a int 0 . Also the default value is built in the parsing logic, user cannot control.

Implement Unmarshaler interface with type alias

func(t *Options) UnmarshalJSON(r []byte) error {
    // avoid infinite loop
    type alias Options
    // default value
    p := alias{Size: 10}
    if err := json.Unmarshal(r, &p); err != nil {
      // if request body passed in size, it will overide the default
    }
    *t = Options(p)
    return nil
}

It looks clean, user just need to call json.Unmarshal with default value when it's missing. But will keep whatever value passed in, including "empty" one, e.g size: 0 will override the default 10.

But user still could not control, default is built into parsing logic.

Let user control: all pointer fields

A nil pointer means non-exist, so user can use it to identify if a field is missing.

type Options struct {
  Keyword  *string `json:"kw"`
  Size   *int   `json:"size"`
}

Now setting default value can happen after decoding, and the control is in user's hands.

  if err := json.Unmarshal(text, &p); err != nil {

  }
  if p.Size == nil {
    p.Size = "whatever"
  }

There are practical use cases, like official protobuf package and AWS Go SDK, a lot of result structs from service call use nil pointer to represent each field.

Further Reads

https://www.orsolabs.com/post/go-json-default-values/

https://eli.thegreenplace.net/2019/go-json-cookbook/
https://eli.thegreenplace.net/2020/representing-json-structures-in-go/
https://eli.thegreenplace.net/2020/optional-json-fields-in-go/

Go Languagejson