Microservices – best practices to retrieve the related data to specific user from the other micro services with the minimal memory/time loss

by

Let’s say you have services: MS1, MS2, MS3, MS4. The web app / mobile app hits MS1 for information. Now MS1 needs to return a response containing data that are managed by MS2, MS3 and MS4.

Poor Solution – MS1 calls MS2, MS3 and MS4 to retrieve information, aggregates them and returns the final aggregated data

Recommended Solution

  1. Use log-based change data capture (CDC) to generate events from databases of MS2, MS3 and MS4 as and when the DBs are updated by their respective services

  2. Post the events to one or more topics of a streaming platform (e.g. Kafka)

  3. Using stream processing, process the events and create the aggregated data for each user in cache and DB of MS1

  4. Serve the requests to MS1 from the cache and / or DB of MS1

Note, with this approach, the cache or DB will have pre-aggregated data which will be kept up-to-date by the event and stream processing. The updates may lag a little resulting in serving stale data. But the delay shouldn’t be more than a few seconds in normal circumstances.

If all the user data can be stored in cache, you can keep the entire data set in cache. Otherwise, you can keep a subset of data in cache with a TTL. The least recently used data can be evicted to make space for new entries. The service will retrieve data from the DB unless itbis not already available in cache.

Advantages:

  1. The latency will be less improving user experience as the response is pre-computed
  2. The tight coupling with other micro services is eliminated. Thus, even if MS2, MS3 or MS4 go down temporarily, your users will still get to see the data, albeit a bit stale, but that’s better in most cases than a delayed response or an error message.

More Related Contents:

  1. Logout: GET or POST?
  2. What’s the difference between “Layers” and “Tiers”?
  3. Component based game engine design [closed]
  4. Domain Driven Design: Domain Service, Application Service [closed]
  5. Why do stacks typically grow downwards?
  6. What is N-Tier architecture?
  7. Servicestack – architecture & reusing POCOs for everything
  8. what is a store buffer?
  9. When are you truly forced to use UUID as part of the design?
  10. Microservices with shared database? using multiple ORM’s? [closed]
  11. Transactions across REST microservices?
  12. On 32-bit CPUs, is an ‘integer’ type more efficient than a ‘short’ type?
  13. What am I not understanding about REST?
  14. Best way to store Badge criteria?
  15. Why shared libraries between microservices are bad? [closed]
  16. How do you organize your small reusable cffunctions?
  17. dma vs interrupt-driven i/o
  18. Difference between a “coroutine” and a “thread”?
  19. How to list supported target architectures in clang?
  20. What is opinionated software?
  21. Microservice Authentication strategy
  22. What is a “feature flag”?
  23. How to detect target architecture using CMake?
  24. CMake Multiarchitecture Compilation
  25. DDD, Anti Corruption layer, how-to?
  26. Where should I store photos? File system or the database? [duplicate]
  27. What’s the difference between REST & RESTful
  28. When to Redis? When to MongoDB? [closed]
  29. NOT using repository pattern, use the ORM as is (EF)
  30. How exactly does the Spring BeanPostProcessor work?

Leave a Comment