In a recent system design interview, I was asked to design a system that used data from around 100 zones. The system was to include an API that, when given a zone ID as input, would return a list of all the restaurants in that zone. To avoid delays, the zone data was stored in memory, so that the API response time was 50ms.
If the zone data is approximately 25GB, then if the service is scaled to say 5 instances, it would need 125GB ram.
The demand is now to run 5 instances but allocate only 25 GB RAM , with the data being divided among the instances.
I understand that we would need a separate application to act as a configuration manager that would assign which instance is in charge of which zone data. This would enable the instances to know which zones they should be tracking when they start up. I am uncertain, however, how to ensure that requests for a zone are sent to the correct instance, especially when using Kubernetes. Furthermore, if an instance holding partial zone data restarts, how can we track which zone data it was in charge of?
If the zone data is approximately 25GB, then if the service is scaled to say 5 instances, it would need 125GB ram.
The demand is now to run 5 instances but allocate only 25 GB RAM , with the data being divided among the instances.
I understand that we would need a separate application to act as a configuration manager that would assign which instance is in charge of which zone data. This would enable the instances to know which zones they should be tracking when they start up. I am uncertain, however, how to ensure that requests for a zone are sent to the correct instance, especially when using Kubernetes. Furthermore, if an instance holding partial zone data restarts, how can we track which zone data it was in charge of?