Lets discuss this in very high-level manner and try to see how these technologies work together. Also lets try to understand what all these cryptic names in the Hadoop ecosystem really mean and what everything is for at a very high level.

Major Components

Lets briefly touch on all these different technologies. We can group them in to three major areas,

  1. Core Hadoop Ecosystem

Core Hadoop Ecosystem

PINK colored things are part of Hadoop itself. Everything else is soft of add on projects that have come out over time that integrate with it so we can solve specific problems.


There is a process we need to follow when designing Microservices architecture. We must not forget that designing Microservices architecture should be methodical. The worst thing that can happen when designing Microservices architecture is to rush into development. It should be like “Plan more, Code less”.

The more you plan, the more you think about the overall system mapping, the more you know what you are going into, the less you will code later.

And of course, the less we code, the fewer bugs we create and the less maintenance we have.

Following this process is critical to the project’s…


I will be listing down my articles about K8s best practices here, so we will have one stop place to access all as I write more in future.

More to follow :)…


Transitioning to the cloud is one of the most important things happening in IT today.

Lets talk about a an organization without cloud computing.

It is a typical organization with a headquarters and some number of branch offices with users in them. Users rely on applications that run directly on the hardware that belongs to their organization.

Probably running in some on premise data center, those applications could run on bare metal servers or run inside virtual machines. …


Over Architecting is BAD

Following best practices is very important to make great software. The question is, should we blindly follow them in every project and make things complicated?

Things are changing really fast and even if we spend years to build something, there will be changes to that. The bottom line is we can not really predict the future, so why we need to take care of every possible scenarios and build a software with features that we do not want RIGHT NOW.

The best is to keep things simple (KISS) until “simple” is no longer satisfactory. KISS is not about writing bad…


One of the most common topics when talking about Microservices is breaking monolith to Microservices. A lot of organizations trying to go this path of converting their existing legacy monolith to modern Microservices system. There are various reasons for such migrations and we have to be prepared for such requests.

Many times organization wants to improve the current system and it believes that migrating to Microservices will do just that. This is not always the case as we have discussed earlier.

If you decided to go with the change, then you must make sure its is thoroughly planned and must…


Photo by Fabio Ballasina on Unsplash

If you read my previous articles, we have discussed what needed to design a robust, reliable and easy to maintain Microservices architecture. In reality, things are not going in correct way always. Microservices requires thorough design and a lot of the work is happening before writing the first line of code. They are not “Fire and Forget”. You can not just say to the developers that we are going to do Microservices and good luck with that ;).

In-fact, its quite easy to make mistakes that will cause the project to fail. …


We discussed lot about Microservices and why they are so good. Sometimes you should not use the Microservcies architecture. Microservices are not one-size fit-all solution. Designing every system as a Microservices system is a mistake which can lead to a lot of problems. There are cases where they should not be used. Even if we used, they might even cause damage and lead to failures.

Therefore use of Microservices should be evaluated on a case by case basis.

Small Systems

Small systems in general should not use Microservices. Small systems with low complexity should usually be a monolith. We should always remember…


We will discuss one of the most important aspects of the Microservices system, which is the logging and monitoring. Without proper logging and monitoring, we are going to have a lot of problems with our system. It might lead to its failure too.

As I said earlier, this is extremely important in Microservices projects even more than monolith. The reason for that is that in Microservices, the flow goes through multiple processes. In this case its difficult to get holistic view of the system or entire flow. Most of the time in traditional monolith, we can examine logs and see…


We will discuss one of the newest concept of Microservices architecture, which is “Service Mesh”. This is one of the hottest topics in Microservices architecture world and there are growing number of products, implementation and articles. It is also one of the most misunderstood concepts.

In order to take real advantage, we should know what it is and when we should use it or when not to use it.

What is Service Mesh?

In single sentence, it is managing all service-to-service communication aspect in a Microservices architecture. In addition, it provides some additional services that we will explore later.

Problems Solved by Service Mesh

As you know, Microservices communicates…

Ishan Liyanage

Passionate Technical Lead, Senior Software Developer and free and open source software advocate. Based in Singapore.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store