How to create a generic JSON request function, over HTTP, in Go?

Introduction

Since Golang 1.18 version release, we, gophers, got the power of generics, assuming that we will use it responsible in the codebases. And I remembered that a lot of times, the code used for integrating with third party REST HTTP services, often times turned into repeatable set of operations, either some insecure boilerplate code.

Fortunately, generics can solve that problem, and there could be an efficient way to reuse HTTP requests.

What is wrong with previous approach?

In order to get what are the main problems with it, first let's do an implementation of a function that executes this kind of requests

and this function can be used the following way:

func main() {
	resp, err := doJSONRequest(
    	"POST", // Although, it's recommended to use net/http package constants 
        "https://api.example.com/data", 
        RequestData{
            Key:   "foo",
            Value: "bar",
        },
        Header{"User-Agent", "xxx-service-api-client"},
        Header{"Authorization", "Bearer xxxx"},
     )
	if err != nil {
		fmt.Println("Error:", err)
		return
	}
	defer resp.Body.Close()

	fmt.Println("Response Status Code:", resp.Status)

	// Handle the response body
	responseBody, _ := ioutil.ReadAll(resp.Body)
	fmt.Println("Response Body:", string(responseBody))
    
    var &resp ResponseData
    err = json.Unmarshal(responseBody, &resp)
    if err != nil {
      panic(err)
    }
    
    fmt.Println("Response: ", resp)
}

The main problem with this kind of approach, is that it might end up with a lot of boilerplate code as well, since the handling of response body is done outside the `doJSONRequest` function. And wrapping it into other function, will strong couple it with specific response body, and therefore it losses it's reusability property.

How this can be fixed? Short answer: Generics

Since Go 1.18, Golang have introduced the concept of Generics. It is a powerful tool, if it's been used wisely. Thankfully, Go has specific limitations on Generics usage, making them a great tool to create generic, yet efficient code.

Let's try to re-implement the function above:

func doJSONRequest[R any](method string, url string, requestData interface{}, headers ...Header) (R, error) {
	var resp R

	// Serialize the request data to JSON
	jsonData, err := json.Marshal(requestData)
	if err != nil {
		return resp, err
	}

	// Create an HTTP request
	req, err := http.NewRequest(method, url, bytes.NewBuffer(jsonData))
	if err != nil {
		return resp, err
	}
	req.Header.Set("Content-Type", "application/json")
    for header, headerValue := range

	// Send the request using the default HTTP client
	client := &http.Client{}
	resp, err := client.Do(requestData)
	if err != nil {
		return resp, err
	}
    defer resp.Body.Close()

	_, err = rs.Body.Read(body)
	if err != nil {
		return resp, err
	}

	err = json.Unmarshal(body, &resp)
	return resp, err
}

and, the usage of this simplified tremendously:

func main() {
	resp, err := doJSONRequest[ResponseData](
    	http.MethodPost,
        "https://api.example.com/data", 
        RequestData{
            Key:   "foo",
            Value: "bar",
        },
        Header{"User-Agent", "xxx-service-api-client"},
        Header{"Authorization", "Bearer xxxx"},
     )
	if err != nil {
		fmt.Println("Error:", err)
		return
	}
    fmt.Println("Response: ", resp)
}

Ok, now we solved the reusability problem, but now we have a flexibility problem.

What does that mean?

For instance, we might want to serialize and deserialize with different methods, and in that case we might need to change the entire function, that is not very ok with `Open Closed Principle`.

To fix that, we will create a struct, that will have the role of the argument for generic `doJSONRequest` function. It will store all the important information about requests, along with serialization/deserialization functions, body bytes reader function, and http client initialization, and timeout:

type SerializationFunc func(any) (byte, error)
type DeserializationFunc func(rq *http.Request) (bool, error)

type JSONRequestConfig struct {
	Method       string
	URL          string
	Headers      map[string]string
	Client       *http.Client
	Timeout      *time.Duration
    Serialize    SerializationFunc
    Deserialize  DeserializationFunc
}

func doJSONRequest[R any](conf JSONRequestConfig) (R, error) {
	if conf.Client == nil {
		conf.Client = &http.Client{
			Timeout: *conf.Timeout,
		}
	}

	var resp RP
	var body []byte
	var err error

	if conf.Serialize != nil {
		body, err = conf.Serialize(conf.Body)
	} else {
		body, err = json.Marshal(conf.Body)

	}
	if err != nil {
		return resp, err
	}

	// Serialize the request data to JSON
	jsonData, err := json.Marshal(body)
	if err != nil {
		return resp, err
	}

	// Create an HTTP request
	req, err := http.NewRequest(conf.Method, conf.URL, bytes.NewBuffer(jsonData))
	if err != nil {
		return resp, err
	}
	req.Header.Set("Content-Type", "application/json")
    for header, headerValue := range conf.Headers {
    	req.Header.Set(header, headerValue)
    }

	// Send the request using the default HTTP client
	client := &http.Client{}
	resp, err := client.Do(req)
	if err != nil {
		return resp, err
	}
    defer resp.Body.Close()

	_, err = rs.Body.Read(body)
	if err != nil {
		return resp, err
	}

	if conf.Deserialize != nil {
		err = conf.Deserialize(body, &resp)
	} else {
		err = json.Unmarshal(body, &resp)
	}

	return resp, err
}

The function now can be used, only by passing an instance of `JSONRequestConfig` struct:

func main() {
	resp, err := doJSONRequest[ResponseData](JSONRequestConfig{
    		Method: http.MethodPost, 
			URL: "https://api.example.com/data",
            Headers: map[string]string{
            	"User-Agent": "xxx-service-api-client",
                "Authorization": "Bearer xxxx"
            }
            Body: RequestData{
                Key:   "foo",
                Value: "bar",
            },
            Serialize: Serialize: func(a any) ([]byte, error) {
                return json.Marshal(a) // it can be any other library function, that handles payload efficiently for this specific use case
            },
    	},
    )
	if err != nil {
		fmt.Println("Error:", err)
		return
	}
    fmt.Println("Response: ", resp)
}

This adds a lot of flexibility, since there can be used serializers and deseralizers for specific use case, that handles efficiently large and/or small payloads.

Conclusion

There is still space for improvements for this function, but the main idea that having a generic JSON request function over HTTP, it simplifies the HTTP client code, and there is possibility to create reusable and flexible solutions, that can also leverage efficiency on the most critical phases of JSON request handling.