Nullable JSON Field Mapping in GORM

When using GORM to deal with mapping between DB table’s fields to their corresponding struct’s properties, most of the time we can rely on its built-in conventions to just define a simple model struct.

For these nullable fields with common types, int, string, time etc, we can use go sql package sql.NullString, NullInt to handle reading from and writing into these fields.

What if we want to store nullable DB JSON field? This is one of Go interface’s flexibility coming to play: we can implement Scanner (database/sql) and Valuer (database/sql/driver) interfaces to let db driver know how to load from and save into these fields and properly handle nil → null.

Let’s say we want to add a nullable JSON field:

{
  "type": "",
  "ids": [1,2,3]
}

We could have the Go struct defined below:

type CustomData struct {
    Type  string  `json:"type"`
    Ids  []int64 `json:"ids"`
}
type NullableCustomData struct {
    // Data  map[string]interface{}
    Data  CustomData
    Valid bool
}

Implement Scan method for reading from the db:

// from sql package
    // Scan assigns a value from a database driver.
    //
    // The src value will be of one of the following types:
    //
    //    int64
    //    float64
    //    bool
    //    []byte
    //    string
    //    time.Time
    //    nil - for NULL values
func (d *NullableCustomData) Scan(value interface{}) error {
    if value == nil {
        d.Valid, d.Data = false, CustomData{}
        return nil
    }
    var valueBytes []byte
    switch v := value.(type) {
    case []byte:
        valueBytes = v
    case string:
        valueBytes = []byte(v)
    default:
        return fmt.Errorf("unsupported data type of value: %v", value)
    }

    if err := json.Unmarshal(valueBytes, &d.Data); err != nil {
        return err
    }
    d.Valid = true
    return nil
}

Implement Value method for writing into the db field, insert null if no data in it.

func (d NullableCustomData) Value() (driver.Value, error) {
    // insert sql null
    if !d.Valid {
        return nil, nil
    }
    bytes, err := json.Marshal(d.Data)
    return string(bytes), err
}

When defining GORM model, we can use this custom type

type model struct {
  CustomField  NullableCustomData `gorm:"type:nullable_nudgectx"`
}

NOTE: If we define CustomData with map instead of specific struct map[string]interface{}

When db tries to read into this field by Unmarshalling from db json field, Go encoding/json package interprets the number as a float64, not int64 or int, as the data inside is defined as map[string]interface{}. if we have the json structure {"type": "", id: 123}, we need type assertion with float64 instead of int, otherwise you will never get the int number back from the json.

    v, present := n.Context.Data["ref_id"]
    if !present {
        return 0
    }
    id, ok := v.(float64)
    if !ok {
        return 0
    }

Nullable JSON Field Mapping in GORM

Further Reading