Distributed tracing with Opencensus and Jaeger in Java

4 min readJul 2, 2018

With microservices architectures distributed tracing is increasingly seen as an essential component for observing distributed systems and applications. In order to provide a meaningful view of a request all of the trace data must be coordinated and collated.

OpenCensus is a toolkit for collecting application performance and behavior data. It is a single distribution of libraries that automatically collects traces and metrics from your app, displays them locally, and sends them to any analysis tool.

This working example we will show you how to export traces and propagate context between 2 services.

Exporter

OpenCensus includes exporters for storage and analysis tools. Right now the list includes Zipkin, Prometheus, Jaeger, Stackdriver, and SignalFx.
In this example will use the Jaeger exporter using the all-in-one docker image.

Application Example

In this example we will run 2 services:

hello-service
greetings-service

Hello service
This service will run on port 8888 and exposes the endpoint /hello.
This endpoint returns the String “Hello from Service”

Greetings service
This service will run on port 8080 and exposes the endpoint /greetings/hello.
When calling this endpoint, this service calls the hello-service and returns the String received in the response.

When a request is processed, traces are sent to Jaeger tool where you will get a detailed view of the operation.

Application stack:

Java 8
Spring boot
Maven
Docker compose

Add the dependencies to your project

<dependencies>
  <dependency>
    <groupId>io.opencensus</groupId>
    <artifactId>opencensus-api</artifactId>
    <version>0.14.0</version>
  </dependency>  <dependency>
    <groupId>io.opencensus</groupId>
    <artifactId>opencensus-exporter-trace-jaeger</artifactId>
    <version>0.14.0</version>
  </dependency>  <dependency>
    <groupId>io.opencensus</groupId>
    <artifactId>opencensus-impl</artifactId>
    <version>0.14.0</version>
    <scope>runtime</scope>
  </dependency>
</dependencies>

OpenCensus trace and span

In OpenCensus a span represents a single operation within a trace. Spans can be nested to form a trace tree.

We will use an helper class in order to create new spans in order to always set the recording and sampling policy.

In order to use the Jaeger exporter we will need to configure it in both services. E.g.:

Jaeger exporter configuration

Hello Service implementation

The following snippet shows the RestController implementation for the hello-service. A span is built whenever the/hello endpoint is called.

Hello service controller

Using a try-with-resource block we are defining that any span opened within the helloService.printHello()method will be nested inside the one we just created.

As previously shown in the application architecture, Hello Service will receive requests from the Greetings Service.

In order to monitor the complete flow of the request coming from the client, passing to the Greetings Service, then to the Hello Service and then back again, we need to maintain and pass the span context between the services.

This is done in a filter which will process every request:

Span Context between services

First we need to define a getter which is responsible of reading the headers of the request to check if any OpenCensus header was added.

private static final TextFormat.Getter<HttpServletRequest> getter = new TextFormat.Getter<HttpServletRequest>() {
    @Override
    public String get(HttpServletRequest httpRequest, String s) {
        return httpRequest.getHeader(s);
    }
};

If a Span Context header is found we will create a span with remote parent. In this case, any other span opened when processing the request will be added as inner spans.

spanBuilder = tracer.spanBuilderWithRemoteParent(spanName, spanContext);

Greetings Service implementation

The logic of creating Span within the service is the same as explained before. The difference we have here is that when the Greeting Service calls the Hello Service endpoint we need to add the Span Context header to the request in order to propagate the scope.

In order to do that, we will use a wrapper which sends the HTTP request after adding the headers:

A setter needs to be defined:

private static final TextFormat.Setter setter = new TextFormat.Setter<HttpURLConnection>() {
    public void put(HttpURLConnection carrier, String key, String value) {
        carrier.setRequestProperty(key, value);
    }
};

This will be used to inject the headers that will propagate the span context into the request:

textFormat.inject(span.getContext(), conn, setter);

Build and run the example

Example can be run with docker-compose.
Build:

docker-compose build

Run:

docker-compose up

Call the endpoint to test it: http://localhost:8080/greetings/hello

You see the trace by accessing to Jaeger at http://localhost:16686/

Source code

The whole code is available at:

https://github.com/maurocanuto/distributed-tracing-opencensus