Register Microservices To Consul Out Of The Box Using Scala Macro Annotations

TL;DR: https://github.com/iamsmkr/consul-scala-macro-annotations

Off lately at my work which currently involves moving our platform code from monolithic to microservices architecture, I have come to realise that there is a lot of boilerplate code that not only needs to be implemented in every other service but also involves maintaining pertaining tribal knowledge within the team. This ofcourse means that there isn't any standard way of implementing a service, raising the possibility of a lot of code duplication thereby resulting into maintainence nightmare. Also implementing a new service every time requires code from other services to be referred to in order to figure out the implementation of the basic infrastructure layout.

Pondering over possible solutions to fix this issue it occured to me that for a particular domain the implementation (the code design) is very much coupled to the solution (the architectural design) for the domain itself which is why every other service seems to share a lot of boilerplate infrastucture code. This however needs to be standardized. Guess what? Turns out that it is the similar problem that most of the frameworks try to solve by abstracting a lot of potential boilerplate code using metaprogramming capabilities of the langauge they provide support for. To me this seems like a reasonable approach to get rid of potentail boilerplate code in platform microservices at work too and bring some order in the way services are built.

Our platform microservices needs to register themselves to Consul for service discovery. To me this seems like a potential boilerplate that could possibly be abstrated with Scala macro annotations, one of Scala's metaprogramming capabilities. Macro annotations is however an experimental feature as of now and may probably be included in future Scala releases. Since macro annotations are only available as part of macro paradise compiler plugin, services would need to define scala compiler and macro paradise plugin as their dependencies in order to use them.

I propose to implement a macro annotation called @EnableServiceDiscovery which would abstract service registration to a Consul agent, allowing services to register themselves out of the box by annotating their main object like so,

Let's see how would we go about implementing it.

@EnableServiceDiscovery macro annotation

Before we learn the implementation details, it is important to understand that Scala macro annotations are compile time metaprogramming capability i.e., they are intended to modify the AST of a program to which they are added to at compile time. In the context of our problem what it would mean is that the service registration to Consul agent logic will unfold into our service's main object definition that implements @EnableServiceDiscovery macro annotation, at compile time. Also, macro annotations needs to be compiled as a separate module/project before they could be used in any other module/project.

In order to implement service registration to Consul, I have chosen to use a Java Consul client by rickfast for having not found a better alternative lib in Scala. This library to me seemed to be well maintained and popular enough to be used safely in production code.

Here is what it would look like:

Macro annotation is created by extending the StaticAnnotation trait and providing the macro implementation for macroTransform(annottees: Any*): Any method. The implementation iterates over a Seq of annotated definitions and extracts object declaration using quasiqouted extracter pattern as the only supported definition that could be annotated with @EnableServiceDiscovery macro annotation. Further, the Abstract Syntax Tree for service registration to Consul logic is created and returned by wrapping it around quasiqoute string interpolator. Quasiquotes is yet another Scala's cool metaprogramming capability that lets you manipulate Scala syntax trees with ease.

As part of service registration to Consul, we have to first build service definition before submitting it to Consul for registration. The service name is created by pulling title and version info from the META-INF/MANIFEST.MF of the service's jar artifact. Service port, tags and Consul info are pulled from application.conf. Health check is however comissioned via status.sh script. (Note that we would need to start Consul agent with enable-script-checks set to true for health check scripts to work.) Besides, every service is given a unique serivce id as a function of md5 of service name and host name.

myservice

Let's see now how we could use @EnableServiceDiscovery macro annotation in our services.

Here, we have defined a simple service called MyService that hosts a trivial rest API endpoint. For this service to register itself to Consul upon startup, we have annotated the object definition with the @EnableServiceDiscovery annotation. To see it in practise, let's first build our project.

Clone Repo

Build Package

Start Service

Before starting service make sure consul agent is up and running with script checks enabled. You can find details on how to start consul agent here.

Direct Browser @localhost:8080

Accessing root at @localhost:8080 should return following message.

Microservices can use @EnableServiceDiscovery macro-annotation to register themselves to Consul out of the box.

Verify Registration

We can use HTTP endpoints to talk to Consul.

We can even get details of a registered service, like so:

Health Check

Last but not least. Health of your service gets monitored by Consul out of the box. Look for "Status": "passing" for a healthy service.

Let's turn our service down to see updated health check status. Look for "Status": "warning" or "Status": "critical" for unhealthy service.