Scalable solution with caching: a study

This architecture is platform agnostic. Although, I did have a handful of AWS services in mind while brainstorming on it.

This was part of a technical discussion, and got praised by the interviewers. Which acted as a confident booster to turn it into a quick blog.

This is a common scenario where low-latency is desired, and data consistency might come in second. Hence, there’s some room for stale data to provide a response to the consumer.

Let’s begin from the basic & consider the following:

We have two datasources, one is a DB that our system owns (or we have more control over it in terms of scaling), and another is a 3rd party API. The task is to merge data from these two sources and send a unified response to the consumer of our API (that begins with the white box on the left).

Let’s assume that a 3rd party API provides the biometric data of users. The target is to send out a merged response of the data of a user. There can be certain liitation on the 3rd party API. For instance, it might not have scaling capabilities like my system, or it may fail randomly.

The highlight of this challenge was to implement caching & justifying the why’s of it.

I envisioned the following caching layers:

C0: A CDN edge server that will cache the data (ideally with a short TTL)

C1: A cache where the data is already merged and ready to be served (with longer TTL & a well thought cache eviction policy)

C2: A cache where the third party API data is cached (to be used in case of a failure from that side)

If a merge is unsuccessful due to any reason, the user data form the database should be returned. For this specific case, we instruct the browser not to cache the data.

I’ll let you go through the design below and have a glance at the bigger picture from my condensed description-