Accessing Dynamic JSON Objects in Golang

When working with JSON data in Go, you often encounter scenarios where the structure of the JSON object is dynamic or unknown at compile time. This can make it challenging to access nested fields within the JSON object. In this article, we'll explore a utility function that allows you to easily retrieve values from dynamic JSON objects.

The Problem

Consider a dynamic JSON object where the structure is not fixed, and you need to access nested fields dynamically. For example:

{
    "name": "John",
    "age": 30,
    "address": {
        "city": "New York",
        "zip": "10001"
    },
    "education": {
        "highschool": {
            "name": "Central High",
            "year": 2005
        },
        "university": {
            "name": "State University",
            "year": 2010,
            "degree": "Computer Science"
        }
    },
    "work_experience": [
        {
            "company": "Tech Corp",
            "position": "Software Engineer",
            "years": 5
        },
        {
            "company": "Innovate Ltd",
            "position": "Senior Developer",
            "years": 3
        }
    ],
    "certifications": [
        {
            "name": "AWS Certified Solutions Architect",
            "year": 2018
        },
        {
            "name": "Certified Kubernetes Administrator",
            "year": 2020
        }
    ],
    "hobbies": ["reading", "traveling"]
}

To access the certifications - name field, you would typically need to know the exact structure and write code to navigate through each level. This can become cumbersome when dealing with deeply nested or dynamic structures.

The Solution

We can create a utility function in Go that allows us to access nested fields dynamically.

Get Key Value or Element of Array

First we need a function that can get either key value or element of array:

func GetField(data interface{}, field interface{}) interface{} {
    switch v := data.(type) {
    case map[string]interface{}:
        return v[field.(string)]
    case []interface{}:
        index, ok := field.(int)
        if !ok {
            return errors.New("field is not an integer")
        }
        if index < 0 || index >= len(v) {
            return errors.New("index out of range")
        }
        return v[index]
    default:
        return errors.New("unsupported type")
    }
}

Recursive Function to get the Fields

Second, we need a recursive function to get the fields:

func GetFields(data interface{}, fields ...interface{}) interface{} {
    firstField, restFields := fields[0], fields[1:]
    if len(restFields) == 0 {
        return GetField(data, firstField)
    } else {
        return GetFields(GetField(data, firstField), restFields...)
    }
}

Main Functions

Last, is we call the function:

func main() {
    // Sample dynamic JSON

    jsonData := ...

    var data map[string]interface{}
    err := json.Unmarshal([]byte(jsonData), &data)
    if err != nil {
        fmt.Println("Error parsing JSON:", err)
        return
    }

    fmt.Println(GetFields(data, "address", "city"))
    fmt.Println(GetFields(data, "education", "highschool", "name"))

    for _, experience := range GetFields(data, "work_experience").([]interface{}) {
        fmt.Println(GetFields(experience, "company"), GetFields(experience, "position"))
    }

    for _, certification := range GetFields(data, "certifications").([]interface{}) {
        fmt.Println(GetFields(certification, "name"), GetFields(certification, "year"))
    }
}

Full main.go

Here is sample of full main.go:

package main

import (
    "encoding/json"
    "errors"
    "fmt"
)

func GetFields(data interface{}, fields ...interface{}) interface{} {
    firstField, restFields := fields[0], fields[1:]
    if len(restFields) == 0 {
        return GetField(data, firstField)
    } else {
        return GetFields(GetField(data, firstField), restFields...)
    }
}

func GetField(data interface{}, field interface{}) interface{} {
    switch v := data.(type) {
    case map[string]interface{}:
        return v[field.(string)]
    case []interface{}:
        index, ok := field.(int)
        if !ok {
            return errors.New("field is not an integer")
        }
        if index < 0 || index >= len(v) {
            return errors.New("index out of range")
        }
        return v[index]
    default:
        return errors.New("unsupported type")
    }
}

func main() {
    // Sample dynamic JSON
    jsonData := `{
        "name": "John",
        "age": 30,
        "address": {
            "city": "New York",
            "zip": "10001"
        },
        "education": {
            "highschool": {
                "name": "Central High",
                "year": 2005
            },
            "university": {
                "name": "State University",
                "year": 2010,
                "degree": "Computer Science"
            }
        },
        "work_experience": [
            {
                "company": "Tech Corp",
                "position": "Software Engineer",
                "years": 5
            },
            {
                "company": "Innovate Ltd",
                "position": "Senior Developer",
                "years": 3
            }
        ],
        "certifications": [
            {
                "name": "AWS Certified Solutions Architect",
                "year": 2018
            },
            {
                "name": "Certified Kubernetes Administrator",
                "year": 2020
            }
        ],
        "hobbies": ["reading", "traveling"]
    }`

    var data map[string]interface{}
    err := json.Unmarshal([]byte(jsonData), &data)
    if err != nil {
        fmt.Println("Error parsing JSON:", err)
        return
    }

    fmt.Println(GetFields(data, "address", "city"))
    fmt.Println(GetFields(data, "education", "highschool", "name"))
    for _, experience := range GetFields(data, "work_experience").([]interface{}) {
        fmt.Println(GetFields(experience, "company"), GetFields(experience, "position"))
    }
    for _, certification := range GetFields(data, "certifications").([]interface{}) {
        fmt.Println(GetFields(certification, "name"), GetFields(certification, "year"))
    }

Explanation

GetFields Function: This function takes a JSON object (data) and a variable number of fields (fields). It recursively retrieves the value of the nested fields. If there are no more fields to process, it calls GetField to get the value of the current field.
GetField Function: This function retrieves the value of a single field from the JSON object. It uses a type switch to handle different types of JSON objects. In this example, we handle map[string]interface{} and also []interface{}.
Main Function: In the main function, we provide an example JSON object, unmarshal it into a map, and use the GetFields function to retrieve the value based what the fields parameter inputted.

Conclusion

By using the GetFields and GetField functions, you can easily access nested fields in dynamic JSON objects without knowing the exact structure at compile time. This approach is particularly useful when dealing with APIs or data sources where the JSON structure may vary.

Bonus

There is also case, when you work with mongodb (like i do). Then you need to update the function GetField little bit:

package main

import (
    "errors"
    "go.mongodb.org/mongo-driver/bson"
)

func GetField(data interface{}, field interface{}) interface{} {
    switch v := data.(type) {
    case bson.M:
        return v[field.(string)]
    case bson.A:
        index, ok := field.(int)
        if !ok {
            return errors.New("field is not an integer")
        }
        if index < 0 || index >= len(v) {
            return errors.New("index out of range")
        }
        return v[index]
    default:
        return errors.New("unsupported type")
    }
}