Q. Why should you know about these design patterns?
A. These patterns are visible in everyday coding, refactoring and design. Put simply, design patterns help you solve new problems with old solutions.
1. Request Collapsing
Consider the simple scenario of multiple concurrent requests vying for a cache entry. If the cache does not contain the entry, it is likely to make multiple requests to DB.
Requests sent in parallel with a cache miss request the same resource
That's a lot of redundant work leading to a very cold start!
The idea of request collapsing is simple: if you have multiple requests for the same resource, allow only one to pass through but use its result for all responses.
This pattern reduces the load on your database and bandwidth consumption. It also avoids redundant transformations of data from DB to objects on the server.
We use this pattern at InterviewReady to limit the number of database queries for (almost) static information.
Every visit to our learn page results in the entire course contents being loaded from DB. This data is ideal for caching on the server and is likely to be queried by multiple clients on server start.
Requests asking for the same cache load are made to wait on the same response
Patterns like these help keep our AWS cloud costs low, and allow us to serve customers on busy days (customers usually prepare for interviews during weekends).
In conclusion, you are likely to find the request collapsing pattern relevant when:
- You have a shared piece of information relevant to multiple parties.
- These parties can make requests in parallel/concurrently.
- This information is likely to be cached.
- Loading the data from DB to cache is a significantly heavy operation.
- You are alright with (some) head of line blocking, in case the request times out.
The implementation of the collapsed request is likely to be as shown in this cache implementation.
Look at line-76. That line runs ONLY when the cache future is empty. The data structure of a future or promise, with eventual completion and callbacks, allow an efficient implementation of request collapsing.
This is part 1/4 of the series on "Concurrency Patterns for Senior Engineers". See you next week with a new blog!
P.S. As homework, checkout the idea of "Request Hedging". It helps mitigate the risk of Point 5 mentioned above :D