Handling Distributed Transactions in a microservices environment

 In a distributed microservices environment, we do not have complex 2-phase commit transaction managers. 

We need a simpler approach and we have design patterns to address this issue. A good article explaining this strategy is available here - https://docs.microsoft.com/en-us/azure/architecture/reference-architectures/saga/saga

The crux of the idea is to issue compensating transactions whenever there is a failure in one step of the flow. A good sample usecase for a banking scenario (with source code) is available here - https://github.com/Azure-Samples/saga-orchestration-serverless

Ruminating on Netflix Conductor Orchestration

We have been evaluating the Netflix open source Conductor project and are intrigued by the design decisions made in it.

Netflix Conductor can be used for orchestration of microservices. Microservices are typically loosely coupled using pub/sub semantics and leveraging a resilient message broker such as Kafka. But this simple and proven event driven architecture did not suffice the needs for Netflix.

A good article on the challenges faced by the Netflix team and the genesis of Conductor is available here. Snippets from the article:

"Pub/sub model worked for simplest of the flows, but quickly highlighted some of the issues associated with the approach:

• Process flows are “embedded” within the code of multiple applications - e.g. If you have a pipeline of publishers and subscribers, it becomes difficult to understand the big picture without proper design documentation. 
• There is tight coupling and assumptions around input/output, SLAs etc, making it harder to adapt to changing needs  - i.e. How will you monitor that the response for a particular task is completed within a 3 second SLA? If not done within this timeframe, we need to mark this task as a failure. Doing this is very difficult with pub/sub unless we code this ourselves.
• No easy way to monitor the progress - When did the process complete? At what step did the transaction fail?

To address all the above issues, the Netflix team created Conductor. Conductor servers are also stateless and can be deployed on multiple servers to handle scale and availability needs.

Ruminating on idempotency of REST APIs

Most REST services are stateless - i.e. they do not store state in memory and can be scaled out horizontally.

There is another concept called a 'idempotent' operation. An operation is considered to be idempotent if making the same call with the same input parameters repeatedly gives the same results. In other words, the service does not differentiate between one call or a million calls.

Typically all GET, HEAD, OPTIONS calls are safe and idempotent. PUT and DELETE are also considered idempotent, but they can return different responses, but with no state change. A good video describing this is here  - http://www.restapitutorial.com/lessons/idempotency.html

POST calls are NOT idempotent, as every call creates a new record. 

Ruminating on the single-threaded model of NodeJS and Node-RED

Many developers and architects have asked me questions on the single-threaded nature of NodeJS and whether we can use it effectively on multi-core machines. Since Node-RED is based on NodeJS, whatever is valid for NodeJS is also valid for Node-RED.

NodeJS has a single thread per process model. When you start NodeJS, it would start a single process with one thread in it. Due to it's non-blocking IO paradigm, it can easily handle multiple client requests concurrently, as there is no thread that is blocking for any IO operation.

Now the next question is around the optimal usage of multi-core machines. If you have 8 core or 16 core machines on your cloud and just run one NodeJS process on it, then you are obviously under-utilizing the resources you have paid for. In order to use all the cores effectively, we have the following options in NodeJS:

1) For compute-intensive tasks: NodeJS can fork out child processes for heavy-duty stuff - e.g. if you are processing images or video files, you can fork out child NodeJS processes from the parent process. Communication between parent and child processes can happen over IPC.

2) For scaling REST APIs: You can start multiple NodeJS processes on a single server - e.g. if you have 8 cores, start 8 NodeJS processes. Put a load balancer (e.g. nginx) in front of these processes. You would anyways have some kind of load-balancing setup in your production environment and the same can be leveraged.

3) Cluster support in newer versions on NodeJS: In the latest versions of NodeJS, you have support for Clusters. Cluster enables us to start multiple NodeJS processes that all share the same server port - e.g. 8 NodeJS processes all listening to port 80.  This is the best OOTB option available today in NodeJS.

Hence it is indeed possible to effectively utilize NodeJS on multi-core machines. A good discussion on the same is available on StackOverflow here.

Another interesting question that is often asked is whether Java developers should jump ship and start development in NodeJS because of its speed and simplicity. Also, NodeJS evangelists keep harping about the fact that the single-threaded nature of Node removes the complexity of multi-threading, etc.

Here are my thoughts on this:

1) Today Java has first class support for non-blocking IO, similar to NodeJ. Take a look at the superb open-source  Netty library that powers all the cloud services at Apple.

2) Most of the complexity of multithreading is abstracted away by popular frameworks such as Spring - e.g. Spring Cloud enables developers to write highly scalable and modular distributed applications that are cloud-native without dealing with the complexities of multi-threading.

3) The Servlet 3.0 specification introduced async requests and the Spring REST MVC framework also supports non-blocking REST services as described here.  

Thus today, all the goodies of NodeJS are available on the Java platform. Also plenty of  Java OSS to kick start your development with all the necessary plumbing code.

Ruminating on non-blocking REST services

In a typical REST service environment, a thread is allocated to each incoming HTTP request for processing. So if you have configured your container to start 50 threads, then you can handle 50 concurrent HTTP requests and any additional HTTP request would be queued till a thread is free.

Today, using principles of non-blocking IO and reactive programming, we can break the tight coupling between a thread and a web request. The Servlet-3 specification also supports async requests as explained in this article - http://www.journaldev.com/2008/async-servlet-feature-of-servlet-3. The core idea is to  delegate the long-running or asynchronous processing to another background thread (Task Executor), so that the HTTP handler threads are not starved.

One might argue that we are just moving the 'blocking thread' bottleneck from the HTTP threads to the backend thread pool (Task Executors). But this does result in better performance, as we can serve more HTTP clients.

The Spring MVC documentation on Async REST MVC is worth a perusal to understand the main concepts - http://docs.spring.io/spring/docs/current/spring-framework-reference/htmlsingle/#mvc-ann-async

A good article that demonstrates how Sprint MVC can be used to build non-blocking REST services that call a backend service exposed via JMS - https://dzone.com/articles/non-blocking-rest-services-with-spring. All the source code for this example is available here. One must understand the basics of Spring Integration framework to work with the above example.

The following discussion thread on StackOverFlow would also give a good idea on how to implement this - http://stackoverflow.com/questions/14134086/request-queue-in-front-of-a-rest-service

The following blog-post shows another example using Spring Boot on Docker - http://spiral-architect.pl/2016/04/25/asynchronous-processing-of-restful-web-services-requests-in-spring-and-java-ee-worlds-wrapped-in-docker-part-1-spring/

Concurrency and scaling strategies for MDPs and MDBs

Message Driven Beans offer a lot of advantages over a standalone JMS consumer as listed in this blog-post. The Spring framework provides us with another lightweight alternative called MDP (Message Driven POJOs) that has all the goodies of MDB, but without any heavy JEE server baggage.

The strategy for implementing a scalable and load-balanced message consumption solution is a bit different in MDBs vs. MDPs.

An MDB is managed by the JEE container and is 'thread-safe' by default. The JEE container maintains a pool of MDBs and allows only one thread to execute an MDB at one time. Thus if you configure your JEE container to spawn 10 MDBs, you can have 10 JMS consumers processing messages concurrently.

Spring MDPs are typically managed by the DMLC (DefaultMessageListenerContainer). Each MDP is typically a singleton, but can have multiple threads running through it. Hence an MDP is NOT 'thread-safe' by default and we have to make sure that our MDPs are stateless - e.g. do not have instance variables, etc. The DMLC can be configured for min/max concurrent consumers to dynamically scale the number of consumers. All connection, session and consumer objects are cached by Spring to improve performance. Jotting down some important stuff to remember regarding DMLC from the Spring Java Docs.

"On startup, DMLC obtains a fixed number of JMS Sessions to invoke the listener, and optionally allows for dynamic adaptation at runtime (up to a maximum number). Actual MessageListener execution happens in asynchronous work units which are created through Spring's TaskExecutor abstraction. By default, the specified number of invoker tasks will be created on startup, according to the "concurrentConsumers" setting."

It is also possible in Spring to create a pool of MDPs and give one to each TaskExecutor, but we have never tested this and never felt the need for this. Making your MDPs stateless and hence 'thread-safe' would suffice for almost all business needs. Nevertheless, if you want to create a pool of MDP objects, then this link would help - http://forum.spring.io/forum/spring-projects/integration/jms/25794-mdb-vs-mdp-concurrency

Another good article on Spring MDP scaling that is worth perusing is here - http://bsnyderblog.blogspot.in/2010/05/tuning-jms-message-consumption-in.html

Design Patterns for Legacy Migration and Digital Modernization

While designing the approach for any legacy migration, the following design patterns crafted by Martin Fowler can be very helpful.

• Strangler Pattern - http://www.martinfowler.com/bliki/StranglerApplication.html
• Event Interception Pattern - http://www.martinfowler.com/bliki/EventInterception.html
• Asset Capture Pattern - http://www.martinfowler.com/bliki/AssetCapture.html

Instead of a rip-n-replace approach to legacy modernization, the gist of the above patterns is to slowly build a new system around the edges of the old system. To do this, we leverage event driven architecture paradigms to capture inbound events to the old system and route these events to the new system. This is done incrementally till we can kill the old system. 

Having been in the architecture field for over a decade, I have realized that 'current state' and 'future state' architectures are just temporal states of reality!

It's impossible to predict the future; we can only be prepared for the future by designing our systems to be modular and highly flexible to change. Build an architecture that can evolve with time and be future-ready and not try to be future-proof. 

Another humble realization is that - the code we are writing today; is nothing but the legacy code of tomorrow :)
And in today's fast-paced world, systems become 'legacy' within a short period of time. Legacy need not just mean a 50-year-old mainframe program. Even a monolithic Java application can be considered legacy. Gartner now defines legacy as any system that is not sufficiently flexible to meet the changing demands of the business. 

Orchestrating Microservices using Event Driven Architecture (EDA) paradigms

If you follow the microservices architecture style, you would have a bunch of services running in their own independent process space.

But how do you orchestrate services for a given business workflow? For e.g. a business transaction that spans multiple calls to microservices.
Point to point integrations would result in 'Dependency Hell'. If each microservice calls the other microservice directly over HTTP API calls, then very soon we have a spaghetti of API dependencies.

One simple design pattern to resolve this is by using the EDA (Event Driven Architecture) paradigm. Each microservices does its job and then publishes an event. Other microservices subscribe to the event and act on it as necessary.

This pub/sub model results in loose coupling between the services and makes the system much more maintainable. A good blog-post covering this paradigm in more details is present here. 

List of Microservices resources

Paweł Pacana has compiled a long list of 72 useful resources to learn about Microservices  :)

The link to the list of resources is here - http://blog.arkency.com/2014/07/microservices-72-resources/

It's important to understand that microservices is an architecture style and follows many of the best practices and principles of SOA. In fact, IMHO microservices is nothing but SOA done right! :)

Microservices is an architectural paradigm of building systems as a suite of independent services, each running in its own process space and communicating with each other using lightweight REST calls.

I found Martin Fowler's talk on Microservices an excellent source of information for beginners to learn about microservices.
The YouTube video is available here - https://www.youtube.com/watch?v=wgdBVIX9ifA

Service Discovery Mechanisms in Microservices

In a microservices based architecture, we would not know the number of instances of a server or their IP addresses beforehand. This is because microservices typically run in VMs or Docker containers that are dynamically spawned based on usage load.

So consumers would need some kind of service discovery mechanism to communicate with microservices. There are two options to design this -

a) Server-side Service Discovery - Here the consumers make a request to a load-balancer/service registry and then the request is routed to the actual service end-point. This paradigm is clearly explained on this blog here. Examples of this design pattern is the AWS Elastic Load Balancer.


b) Client-side Service Discovery - Here the consumers use a small library for making service calls. This library makes calls to the service registry and obtains the load-balanced actual service end-point. Netflix uses this approach and its service registry is called Eureka and its client library is called Ribbon.

Handling failures and improving resilience in microservices

In a microservices architecture, one has to build services that can handle failures. For e.g. If a microservice calls another dependent microservice that is down, then we need to handle this using timeouts and implement the Circuit Breaker pattern.

Netflix has open-sourced an incredibly useful library called as Hystrix to solve such problems. Anyone building large scale distributed architectures on the Java platform would find Hystrix a boon. When you make a remote service call through Hystrix libraries, it does the following:

1. If the remote service call does not return within a specified threshold, Hystrix times-out the call.
2. If a service is throwing errors and the number of errors exceed a threshold, then Hystrix would trip the circuit-breaker and all requests would fail-fast for a specified amount of time (recovery period)
3. Hystrix enables developers to implement a fall-back action when a request fails, for e,g returning a default value or a null value or from cache. 

The full operating model of Hystrix is explained in great details on Github wiki 

It was also interesting to learn that the tech guys at Flipkart have taken Hystrix and implemented a service proxy on top of it called 'Phantom'. Looks like the advantage of using Phantom is that your consumers do not have to code against the Hystrix libraries.