[Awesome Go] Use mapstructure to decode generic map values to structure in Golang (Go)

Posted on Edited on In , , Views: Word count in article: 6.9k Reading time ≈ 6 mins.

mapstructure

mapstructure is a Go library for decoding generic map values to structures and vice versa, while providing helpful error handling.

This library is most useful when decoding values from some data stream (JSON, Gob, etc.) where you don’t quite know the structure of the underlying data until you read a part of it. You can therefore read a map[string]interface{} and use this library to decode it into the proper underlying native Go structure.

Go offers fantastic standard libraries for decoding formats such as JSON. The standard method is to have a struct pre-created, and populate that struct from the bytes of the encoded format. This is great, but the problem is if you have configuration or an encoding that changes slightly depending on specific fields. For example, consider this JSON:

1
2
3
4
{
  "type": "person",
  "name": "Mitchell"
}

Perhaps we can’t populate a specific structure without first reading the “type” field from the JSON. We could always do two passes over the decoding of the JSON (reading the “type” first, and the rest later). However, it is much simpler to just decode this into a map[string]interface{} structure, read the “type” key, then use something like this library to decode it into the proper structure.

Installation

Standard go get:

1
$ go get -u github.com/mitchellh/mapstructure

Usage & Example

func Decode

1
func Decode(input interface{}, output interface{}) error

Decode takes an input structure and uses reflection to translate it to the output structure. output must be a pointer to a map or struct.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
type Person struct {
	Name   string
	Age    int
	Emails []string
	Extra  map[string]string
}

// This input can come from anywhere, but typically comes from
// something like decoding JSON where we're not quite sure of the
// struct initially.
input := map[string]interface{}{
	"name":   "Mitchell",
	"age":    91,
	"emails": []string{"one", "two", "three"},
	"extra": map[string]string{
		"twitter": "mitchellh",
	},
}

var result Person
err := Decode(input, &result)
if err != nil {
	panic(err)
}

fmt.Printf("%#v", result)
// Output:
// mapstructure.Person{Name:"Mitchell", Age:91, Emails:[]string{"one", "two", "three"}, Extra:map[string]string{"twitter":"mitchellh"}}

EmbeddedStruct

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
// Squashing multiple embedded structs is allowed using the squash tag.
// This is demonstrated by creating a composite struct of multiple types
// and decoding into it. In this case, a person can carry with it both
// a Family and a Location, as well as their own FirstName.
type Family struct {
	LastName string
}
type Location struct {
	City string
}
type Person struct {
	Family    `mapstructure:",squash"`
	Location  `mapstructure:",squash"`
	FirstName string
}

input := map[string]interface{}{
	"FirstName": "Mitchell",
	"LastName":  "Hashimoto",
	"City":      "San Francisco",
}

var result Person
err := Decode(input, &result)
if err != nil {
	panic(err)
}

fmt.Printf("%s %s, %s", result.FirstName, result.LastName, result.City)
// Output:
// Mitchell Hashimoto, San Francisco

Metadata

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
type Person struct {
	Name string
	Age  int
}

// This input can come from anywhere, but typically comes from
// something like decoding JSON where we're not quite sure of the
// struct initially.
input := map[string]interface{}{
	"name":  "Mitchell",
	"age":   91,
	"email": "[email protected]",
}

// For metadata, we make a more advanced DecoderConfig so we can
// more finely configure the decoder that is used. In this case, we
// just tell the decoder we want to track metadata.
var md Metadata
var result Person
config := &DecoderConfig{
	Metadata: &md,
	Result:   &result,
}

decoder, err := NewDecoder(config)
if err != nil {
	panic(err)
}

if err := decoder.Decode(input); err != nil {
	panic(err)
}

fmt.Printf("Unused keys: %#v", md.Unused)
// Output:
// Unused keys: []string{"email"}

Omitempty

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
// Add omitempty annotation to avoid map keys for empty values
type Family struct {
	LastName string
}
type Location struct {
	City string
}
type Person struct {
	*Family   `mapstructure:",omitempty"`
	*Location `mapstructure:",omitempty"`
	Age       int
	FirstName string
}

result := &map[string]interface{}{}
input := Person{FirstName: "Somebody"}
err := Decode(input, &result)
if err != nil {
	panic(err)
}

fmt.Printf("%+v", result)
// Output:
// &map[Age:0 FirstName:Somebody]

RemainingData

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
// Note that the mapstructure tags defined in the struct type
// can indicate which fields the values are mapped to.
type Person struct {
	Name  string
	Age   int
	Other map[string]interface{} `mapstructure:",remain"`
}

input := map[string]interface{}{
	"name":  "Mitchell",
	"age":   91,
	"email": "[email protected]",
}

var result Person
err := Decode(input, &result)
if err != nil {
	panic(err)
}

fmt.Printf("%#v", result)
// Output:
// mapstructure.Person{Name:"Mitchell", Age:91, Other:map[string]interface {}{"email":"[email protected]"}}

Tags

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
// Note that the mapstructure tags defined in the struct type
// can indicate which fields the values are mapped to.
type Person struct {
	Name string `mapstructure:"person_name"`
	Age  int    `mapstructure:"person_age"`
}

input := map[string]interface{}{
	"person_name": "Mitchell",
	"person_age":  91,
}

var result Person
err := Decode(input, &result)
if err != nil {
	panic(err)
}

fmt.Printf("%#v", result)
// Output:
// mapstructure.Person{Name:"Mitchell", Age:91}

WeaklyTypedInput

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
type Person struct {
	Name   string
	Age    int
	Emails []string
}

// This input can come from anywhere, but typically comes from
// something like decoding JSON, generated by a weakly typed language
// such as PHP.
input := map[string]interface{}{
	"name":   123,                      // number => string
	"age":    "42",                     // string => number
	"emails": map[string]interface{}{}, // empty map => empty array
}

var result Person
config := &DecoderConfig{
	WeaklyTypedInput: true,
	Result:           &result,
}

decoder, err := NewDecoder(config)
if err != nil {
	panic(err)
}

err = decoder.Decode(input)
if err != nil {
	panic(err)
}

fmt.Printf("%#v", result)
// Output:
// mapstructure.Person{Name:"123", Age:42, Emails:[]string{}}

References

[1] GitHub - mitchellh/mapstructure: Go library for decoding generic map values into native Go structures and vice versa. - https://github.com/mitchellh/mapstructure

[2] mapstructure · pkg.go.dev - https://pkg.go.dev/github.com/mitchellh/mapstructure#example-Decode