Prefetching using Markov Chain

Abstract

This presentation talks about U.S. patent (12,061,553B1), "Microservices Prefecthing using Markov chain", assignee CVS Pharmacy, Inc., and myself as a solo inventor. The present disclosure relates to systems and methods for prefetching data for use in servicing data requests. In particular, this relates to using a Markov Chain (MC) model for microservices prefetching. In a microservices architecture, services are fine-grained and the protocols are lightweight. Many different services are provided independent of others especially in domain driven design. This loose coupling provides modularity, scalability, and easy integration. However, service calls over a network have a higher cost in terms of network latency and message processing time. These problems can be solved with memory cached databases, but the data stored in cache is typically stored based on last usage (aftermath event). Prefetching is a technique for speeding up fetch operations by beginning a fetch operation whose result is expected to be needed soon (time ahead event). Usually, this is before it is known to be needed. A Markov predictor can identify the most probable next action based on historical usage patterns from various users/personas, thus reducing time-consuming round trips and improving response times. An ergodic MC converges to a stationary distribution, and this property is used to prefetch the next N services based on system parameters and capacity.

Bio

Image

Joydeep Bhattacharjee is the AVP of Platform Engineering at The Hartford, a leader in P&C insurance and Group Benefits; leveraging over 25 years of IT experience, he drives the GenAI/LLMOps platform development there. Previously, he led the Technology Research team at CVS Health for AI & Quantum Computing track and also spearheaded App, Cloud, and Data modernization there. Early in his career, he worked as an IT consultant in the healthcare, P&C insurance, and airline sectors.