The article exposes some interesting patterns, we have come across in Pragmatic Coders when developing products in the cloud in microservice‑oriented architecture. Besides static information, you will find here some descriptions of encountered issues, tips, and loose thoughts as well. In the next version, we will refine/enrich topics we already covered. Also, we will add other interesting parts from the Cloud/Microservices world.
Authors: Dima Iabluchanskyi (Senior Java Developer) , Jakub Noga (Java Developer)
This article differs a little bit from the others. This is a living article! Let us explain what it means.
We are developers and we decided to develop this document like we do with code – the structure (architecture) will be driven itself by doing refinement cycles. So do some tests by reading and share your output with us.
As our current project is based on AWS/Spring/JVM you will also encounter information related to that stack.
Mantra: Think of everything as a service – including session (Redis as a service), files (S3 as a service) and database (RDS as a service).
Service registration and Discovery patterns
This concept allows us to automatically determine and discover addresses of microservices instances, which can be dynamically changed (scaled, descaled, failed).
One of the key points that have to be considered in order to build a reliable distributed system is how our system is going to be immune to topology changes. In other words, how do we want to ensure that our system will still be working as expected once one of our services has started to live at different IP? Certainly, we want our service to be discovered regardless of IP and the port changes. There are two main patterns for service discovery and both of them based on a Service Registry (database of services) which contains service instances along with its addresses.
In this scenario, the client makes a request to the router (load balancer/reverse proxy) and the router contacts the service registry and forwards a request to the backend services. This approach is useful if we don’t want clients to have a knowledge of things like routing or failure handling. For example, the client is external to our system so it would be nice to have a centralized failure logic which resides inside of our reverse proxy. It helps in reducing the overall system complexity, on the other side, it may not be a good solution to have a single failure pattern either for internal and external services. Another downside is that this approach introduces additional network hop as you can see on the attached screen.
AWS Elastic Load Balancer (ELB)
When it comes to client-side discovery, the client itself has a greater control as he is responsible for the service discovery and load balancing. This logic (if no shared lib) may be localized in each client so it is possible to implement a logic, such as a retry of a failure, on a case-by-case basis. Client application does not have to route through a router or a load balancer and therefore can avoid that extra hop.
Netflix Eureka (service registry) and Netflix Ribbon (client)
Each service instance has to be registered/unregistered with the Service Registry. There are several cases which we should take into account. Registration on startup, unregistering on shutdown and unregistering in case of a crash or other case in which our service is unable to handle requests. So the question is how to handle this? What is the best approach in order to let our clients know that our services are available?
There are two existing usable approaches.
Self Registration pattern
The Service Registry that we chose obviously exposes some API for registration. So once our service instance starts-up it registers itself, periodically sending some heart-beat which indicates that the service is healthy and available. In case the Registry Service stops receiving these heart-beats from any service instance, the registration of that service instance will timeout and our service will be automatically unregistered. Naturally, the unregistering happens when our service shuts down.
As you see, this pattern implies that the service takes responsibility for registering/unregistering, which also means that it’s coupled with the Registry. There are some cases in which it may be a big disadvantage. For example, when you have written your services in different languages and you need to provide different implementations in order to make registration possible.
3rd Party Registration pattern
With this pattern, as the name implies, you are going to setup and manage yet another component which is called the Registrar, which indeed takes responsibility for registering our service with the Registry. As you may expect all these heartbeats and other functions that in the previous example were in charge of the Service Instance itself, are now on the Registrar side.
In this case, our service is totally decoupled from the Registry, it does not have to know anything about the registration process. Hence in case of the multi-language based system, this may be an advantage.
Some platforms/environments provide the Service Registration implicitly and there is no need to care about this concern. This gives us decoupling and no need to maintain/operate with another component. One of such platform is AWS which gives us Application Load Balancer described later in this article.
Most popular Service Registries:
- Netflix Eureka
- open source
- has web UI
- include the health check
- configuration storage
- can be a repository for Vault
- has web UI
- Apache Zookeeper
- it’s just Registry service
- used by a lot of big companies
- open source
- from the different age, comparing to the modern solutions
Worth to mention that Spring Cloud supports all of the above.
Some systems such as Kubernetes, Marathon and AWS ELB/ALB have an implicit service registry.
AWS Application Load Balancer – the remedy
This component has been recently introduced by AWS and is rapidly gaining popularity. It is mainly due to the fact that it provides server-side discovery, server-side registration and server-side load balancing out of the box along with many other important features like support for gently working with containerized microservices! No more hassles, less complexity, and lower costs.
In the next version of this article, we will try to put more light on this topic and describe our issues and impediments we had to fight with before the ALB era.
Let’s now move to another quite important pattern called API-Gateway.
As the name implies an edge service is a service that seats at the edge of the architecture. At the edge of the system. It’s the first entry point for requests incoming into our system. A big facade between client side (web application, mobile or some native application, etc) and backend side. Hence it is the perfect place to adjust and expose a gate specific to the client. Such a gate would handle important concerns like security, provisioning, migration process, testing and of course client-server contract. So for example, if a client has a specific kind of payload that he needs, or he adheres to the different type of contract/protocol this is the place to adopt the requirements of the client to our downstream services which usually provide more fine-grained API.
Rate-limiting, proxying, data transformation should also be handled and processed inside of an edge service, usually called API Gateway. Typically, you will deploy one API gateway per each client type. Can you imagine how tedious it would be to accommodate all these cases into every single one of our micro-services? – Neverending story.
API Gateway may also serve as a great utility in order to help with transitioning from a monolithic application to microservice-oriented one. It will abstract from the outside world all the changes related to the transition.
Encountered issues with AWS API Gateway:
- Mutual authentication not supported! Hence we had to provide another Load Balancer which gives as such possibility.
- Amazon implementation – Gateway API
- + out of the box product
- + has decent integration with AWS stack
- +/- still has some missing parts, but AWS update it quite often
- 2nd version includes the recently non-blocking approach
- config file could be really tiny
- has integration with Spring Cloud
Add resilience with Circuit breakers
The circuit breaker is a pattern which gives us control over delays and errors during the calls through the network. The main idea is to stop cascading failure in a distributed system, which has a lot of different components. It allows us to return an error as fast as we can.
The main parameters of a Circuit Breaker pattern are the number of failed requests and the time to wait after a Circuit Breaker turned on.
Probably the main and the most popular implementation of given pattern is Hystrix. It has a really good integration with Spring Cloud. Hystrix gives us interface for defining a fallback method – called during the unsuccessful calls. It can help us to send default answer.
Hystrix generates a list of metrics (result, speed, etc), which give us the ability to perform analysis of the system.
And so the first version of the article comes to an end. Do not hesitate to put your comments or questions!