Traffic Shadowing of a Golang service with Gor

In this project I’m going to give an example of how to use Gor in order to perform a HTTP Traffic Shadowing.

For this example, I’m going to build HTTP services with Golang, using Echo framework. After that I’m going to make one of the services as a shadow for the other service traffic. Therefore every HTTP request that I send to the first service will also be sent to the second service.

First let’s create our Golang HTTP services. We are going to do this by creating a Golang project named traffic-shadowing and installing Echo framework.

mkdir traffic-shadowing
cd traffic-shadowing
export GO111MODULE=on
go mod init github.com/YOUR-GITHUB-USERNAME/traffic-shadowing

Now that we have our project setup, let’s download Echo framework.

go get github.com/labstack/echo/v4

Ok, now that we have everything we need, let’s create two files, one for implementing HTTP handler and one for starting the service.

mkdir internal && cd internal
mkdir cmd && touch cmd.go
mkdir http && touch handler.go

The project structure should be like this:

|_ internal/
    |_ cmd/
        |_ cmd.go
    |_ http/
        |_ handler.go

So now we are going to create a HTTP handler. Open http/handler.go file and add the following codes.

The above struct, creates a Handler with one method for handling GET requests. Now we are going to use this handler in initializing our service.

Go into cmd/cmd.go file and add the following codes.

In this code, we are creating a HTTP handler and we also create a new Echo app, then we define the root route of our service. After that we are starting our server on localhost:8080 .

So let’s add a feature to get service port from command line arguments. In this case, we don’t have to set the port number hard-coded.

Ok, so now we are getting the port number from command line arguments by using Golang os package and Args field.

And for the final part, we are going to export our logs into a file like logPORT.txt , in this case we want our logs to be written into log8080.txt .

Everything is set now. You can start Golang HTTP service with following command:

go run internal/cmd/cmd.go 8080

On the next step, we are going to work with Gor tool in order to shadow our traffic between Golang services.

What is Gor?

Gor (short term of GoReplay) is an open-source tool for capturing and replaying live HTTP traffic into a test environment in order to continuously test your system with real data.

It can be used to increase confidence in code deployments, configuration changes and infrastructure changes.

GoReplay offers unique approach for shadowing. Instead of being a proxy, GoReplay in background listen for traffic on your network interface, requiring no changes in your production infrastructure, rather then running GoReplay daemon on the same machine as your service.

You can visit Gor website for more details. And also you can read the documentation for installing it on your system in here.

Ok, now let’s create two HTTP services.

go run internal/cmd/cmd.go 8080
go run internal/cmd/cmd.go 8081

Now we have two services. Make a curl request to check each service.

curl localhost:8080

If you open log8080.txt you have to see something like this:

... user [::1]::4987

Now that you have two servers running, enter the following command to start the traffic shadowing.

sudo gor --input-raw :8080 --output-http http://localhost:8081

First of all, if you are executing the Gor command as a non-root user, you need to use the sudo, otherwise you will get errors. The input-raw flag tells GoReplay to receive requests from an HTTP service over port 8080. The output-http flag tells GoReplay to send the requests to a HTTP server with the following address (which in this project runs on localhost:8081 ).

At this moment, if you send requests to localhost:8080 , you can see that the request is also being sent to localhost:8081 . Open log8080.txt and log8081.txt to see the logs.

That’s it. We are now shadowing traffic of a service to the other one.

You can visit the project codes at my Github repository. It would be a pleasurer for me to give the project an star if you liked it.

Thank you
Amirhossein Najafizade