.Net + Kubernetes Lessons

I’ve been developing a new cloud scale project over the last few months. It’s certainly not finished yet, but there have been a number of learnings during the development phase which continues into 2019. I’m sure there will be a second post to follow this one, but the timing was right for a first round as a companion post to Microservices: .Net, Linux, Kubernetes and Istio make a powerful combination.

Should you design for microservices right from the start?

Martin Fowler, a Software Architect of some renown who helped create the Manifesto for Agile Software Development has both written and spoken on the Monolith First approach to development. Certainly in his 2015 post, Martin refers to seeing a number of failures with projects platformed as microservices from the start but also felt it was too early to say what the overall trend was going to be. Roll forward to 2018 and the tooling available to developers has significantly improved in addition to the accumulated lessons (good and bad).

The industry trend to Infrastructure as Code is a significant help. When you divide your solution into in various business or delivery capabilities, you introduce boundary complexity. Open-sourced solutions such as Docker, Kubernetes and Istio focus on reducing the TCO of microservice deployments, whether it’s via automatic scaling and balancing, traffic flow analysis, routing and compartmentalisation. If you were a town planner, it’s as if all you had to worry about was building the school, the town hall, the office and making sure they functioned as efficiently as possible. Someone else has kindly provided a management and deployment framework that takes care of all the roads, water, sewage, electricity. As a bonus, they’ve also sorted out the traffic management, garbage collection and border control for you.

Having all these tools doesn’t stop you from spiraling out of control. To continue with the town analogy, there are many cities that have grown so large that they bring a whole different scale problem. The industry as a whole is still in a learning phase. The big pools of resources commandeered by Facebook and Google et al that design for hyper-scale from the concrete foundations up have been giving back through their commitments to open-source. However, the global pool of talent to take advantage of these tools is still nascent. I’ve posted questions to a number of forums that have gatherered electronic tumbleweed, so it’s head down to figure this stuff out.

In short, two takeaways for me thus far in the development journey:

  • Know your domain - Microservices really aren’t a good place to be learning a business problem in addition to a software problem. As an Enterprise Architect, capability modelling is a excellent tool to deploy to understand what both you and your solution need to be good at and also highlightling where cross-functional team boundaries lie (and the influence on software design). This translates into effective partitioning of your microservices and becoming comfortable with concepts such as eventual consistency.

  • Be prepared to tear things down - Microservices don’t remove the need for refactoring, they just make it less likely to break everything in the process. Even with a good design, refactoring will be with you. Remember when building your magical new microservice city, YAGNI.

I believe the benefits of a microservice based approach is worthwhile, even for the most traditional of monolithic systems (I’m looking at the ERP’s and accounting packages cowering in the corner).

There are some useful architectural patterns out there

Microsoft (and many others) have put a number of resources out there that provide good direction for your microservice architecture.

Patterns such as Command and Query Responsibility Segregation really come into their own in a microservice architecture. Some of your containers will only have Read methods and some of your containers will only have Create/Update/Delete methods. When configuring your Kubernetes Deployments, you construct 75/25 splits in your Pod levels (or higher if the read demand requires it). With caching models, those read-only containers will have extremely high-performance read-only persistence layers.

I’ll point out that some of the design patterns here are able to be facilitated with the new platforms. A good example is the Strangler Pattern which you can now construct using Istio policy rather than rolling your own.

Some ‘old’ architectural patterns are still relevant for microservices

Design Patterns: Elements of Reusable Object-Oriented Software (1994) written by the GoF has been influential in object-oriented programming through establishing a series of reusable patterns. Those good practices at a class level, apply equally well at a container level such as the Abstract Factory and Facade.

Why did I include a webpage in a Backend for Frontends service container?

In a word, debugging. You could consider it baggage in the container and it’s certainly not a page I would expose outside of the cluster. However, having a landing page that shows the heath of the container instance (I included a basic status report on connections established etc.) has been useful during the early phases of development. It doesn’t obviate the need for creating your restful 404 and 503 responses and a centralised monitoring (via Prometheus or otherwise) system.

Don’t pick one language per container type just because your team feels like it!

I’ll approach this from two angles. The first is in regards to Componentisation via Services. The traditional approach was to create shared libraries which would get compiled into your code but libraries wouldn’t necessarily make your solutions independently deployable. If you changed a library, you’d still end up compiling and redeploying the entire solution. Microservices enforce component boundaries (well designed ones do), and documentation tools e.g. Swagger assist with informing users of API usage.

There are dangers of strictly making everything a service call. Service calls have latency and overhead, regardless of how well you tune your systems. There are techniques that use netmap and VALE that can achieve 1.5bpps for userspace to userspace messaging in a virtualised environment, but it is still an Inter Process Communication. Watch Avoiding Microservice Megadisasters - Jimmy Bogard as an entertaining example of where this went horribly wrong.

Standardising on a minimal set of languages let’s you take advantage of other packaging tools that provide close to the same benefit in compartmentalisation. In .Net, I’d use a private NuGet feed to share code that makes sense to be used in multiple services, but doesn’t justify creating a seperate microservice container.

  • I still get separation of concerns - Different teams can work on shared libraries, and IoC design patterns can minimise solution dependancies.

  • I still get pinning and labeling - I can specify in my NuGet config that x version of service depends on y version of NuGet package

  • I still get CI/CD - All the benefits ascribed to a devops approach still apply

  • I now get higher performance - Because I’m performing an in-memory, in-container transaction

An example might be a shared type library, or utility classes. To maximise the benefit of this, you need to minimise the number of languages used in your microservices architecture. You may group by design pattern and have all Backends for Frontends using .Net for example. You may have a seperate group using a private Python package repository for their Python based services, or Maven and Java. Your initial capability analysis will still be very important!

Secondly, by having 10’s of different languages you lose the ability for cross-pollination of teams and knowledge. I’d argue that most developers are really good in a couple of languages, pretty good in maybe 2 or 3 more and then it falls away pretty quickly after that. It also makes hiring more challenging as well. In this respect, homogeneity is not a bad thing.

Wrap up

For me microservices, is like Service Oriented Architecture (SOA) growing up. Unfortuately the SOA term has become tainted through unrealised benefit. Here’s hoping designing a microservice based platform has taken those lessons to heart.

Tony Sinclair