Software Engineer 2 · Microsoft Office AI

Building the infrastructure behind agentic AI.

I design model-agnostic agent workflows, LLM serving systems, and high-scale backend platforms that make AI products faster, safer, and easier to operate.

Agentic AI infrastructure Model routing
Agent tool calls
Memory
Reliable inference.
72M+LLM requests per day routed at AWS Bedrock
97–98%prompt cache hit rate across 100+ clusters
1 week → 2–3hmodel switching effort reduced at Office AI
$4.4M/yrestimated loss prevented at PhonePe

About

I am a software engineer who is still a little too excited about algorithms, cloud systems, and making AI infrastructure behave nicely in production.

My journey started in Bengaluru, where writing code went from being a coursework requirement to something that genuinely hooked me. During my Master’s in Computer Science at Arizona State University, I leaned deeper into artificial intelligence, machine learning, cloud computing, algorithms, and the kind of systems work that keeps you thinking even after you close the laptop.

At PhonePe, I worked on backend systems for payments and merchant experiences at large scale. That is where I learned that a small backend bug is never really small when millions of users are involved, and that good engineering can save time, money, and a few people from panic-refreshing dashboards.

At AWS Bedrock, I moved into large-scale AI infrastructure, working on LLM inference systems, Kubernetes, capacity orchestration, sticky routing, prompt caching, trace export, and observability. There is something very satisfying about shaving seconds off latency or making a rollout safer when the system is handling millions of AI requests.

Now at Microsoft Office AI, I work on agent workflows and model-agnostic infrastructure that helps teams move across LLMs, tools, and response formats without getting locked into one model family. I enjoy the messy middle where backend systems, distributed infrastructure, and AI products meet, because that is usually where the most interesting problems hide.

Outside work, I like mentoring engineers, exploring LLM and agent infrastructure ideas, playing badminton, and hiking. I am still chasing the same thing that got me into software in the first place: taking a complicated problem, finding the clean design inside it, and getting that quiet little eureka moment when everything finally clicks.

Bhaskar Kumar
AI Infrastructure · Backend Systems · Agents

Experience

Production AI systems, from agents to inference.

A career focused on platform engineering, reliability, latency, and developer velocity across Microsoft, AWS, Amazon, and PhonePe.

Microsoft · Office AI

Software Engineer 2 · Agent Workflow · MCP · Tools · LLM · Claude SDK · Copilot SDK · TypeScript
Mar 2026 – Present
Model-agnostic agent layer. Decoupled core agent logic from model-specific implementations so teams can switch across Claude, GPT, and newer LLMs without rewriting workflows.
Faster model adoption. Designed abstraction boundaries for model invocation, tool handling, and response parsing, reducing model-switching effort from one week to 2–3 hours.

AWS · Bedrock

Software Development Engineer · LLM Inference · Kubernetes · Java · DynamoDB · S3 · CDK
Jul 2024 – Mar 2026
Capacity orchestration. Redesigned rebalance workflows using reservations and optimizer-backed planning, improving success rates from 60% to 96%.
Prompt caching. Built weighted sticky routing using consistent hashing, achieving 97–98% hit rates across 100+ clusters and cutting prefill latency from 20s to 6s.
Cluster productionization. Automated capacity ingestion, validation, and shadow-to-live rollout, reducing launch time from 24 hours to 4 hours.
Observability and safety. Created trace export for 14B+ LLM inference events with configurable redaction and enforcement checks.

Amazon · Fire TV Ads

Software Development Engineer Intern · React · Java · Spring Boot · DynamoDB · S3 · CDK
May 2023 – Aug 2023
Campaign verification. Owned design and delivery of a web application that removed external dependencies and reduced integration time by 80%.
Real-time previews. Removed a 24-hour display delay from the Fire TV campaign preview workflow, accelerating decisions and reducing manual coordination.

PhonePe

Software Engineer - Backend · Java · Dropwizard · Spring Boot · Microservices · Distributed Systems
Jul 2019 – Jul 2022
Merchant systems. Built MyStores using transaction, favorite, and lending signals, reducing QR scanning by 50% and improving payment flow speed.
Search relevance. Launched weekly business-hours search serving 22K daily hits for stores open at query time.
Regulatory enforcement. Delivered NPCI MCC enforcement for merchant incentives, preventing an estimated $4.4M/year loss.
API efficiency. Fixed an inadvertent DoS pattern, reducing invalid API calls from 75K/hour to 0.

Selected work

Systems that compound impact.

A curated view of the themes I want recruiters and hiring managers to remember.

Agent workflow infrastructure

Model-agnostic abstractions for invocation, tools, parsing, and evaluation so agent workflows can evolve with the best available models.

MCPToolsClaude SDKCopilot SDK

LLM inference reliability

Routing, capacity orchestration, productionization, and observability for inference systems operating across Kubernetes clusters at high scale.

KubernetesBedrockDynamoDBStep Functions

Backend platforms at scale

Distributed backend services, caching, search workflows, regulatory systems, and API efficiency improvements for payments and commerce platforms.

JavaSpring BootKafkaHBase

Skills

Depth where production systems fail or scale.

Languages

Java, Python, TypeScript, JavaScript, Go, C/C++, Scala, SQL

AI and distributed systems

Agent workflows, LLM inference, model serving, prompt caching, consistent hashing, capacity orchestration, high-throughput systems

Backend and data

Spring Boot, Dropwizard, REST, microservices, DynamoDB, HBase, Cassandra, MongoDB, Redis, Kafka, RabbitMQ, Elasticsearch

Cloud and DevOps

AWS, Kubernetes, Docker, CDK, CloudFormation, S3, SQS, CloudWatch, Jenkins, CI/CD, Linux

Education

Research-minded foundations in AI, systems, and algorithms.

A stronger academic section that shows the technical base behind my production AI infrastructure work.

Arizona State University

M.S. in Computer Science · Aug 2022 – May 2024 · Tempe, Arizona

Graduate Fellow Scholarship · Coursework focus: Artificial Intelligence, Machine Learning, Cloud Computing, Algorithms, Data Mining, Data Visualization.

Artificial IntelligenceMachine LearningCloud ComputingAlgorithmsData MiningData Visualization

PES Institute of Technology

B.E. in Information Science and Engineering · Aug 2015 – Jun 2019 · Bengaluru, India

Foundational coursework in Operating Systems, Data Structures and Algorithms, Database Management Systems, and Computer Networks.

Operating SystemsDSADBMSComputer NetworksACM ICPC

Academic portfolios

A live archive of selected ASU academic work, projects, and technical artifacts.

Research & Projects

Applied research taste, shipped through systems.

Projects and academic work that connect AI ideas with cloud infrastructure, search algorithms, and usable products.

Enterprise chat platform using text summarization

Research publication around summarization for enterprise chat workflows, connecting NLP techniques with productivity and information retrieval use cases.

Research publication · NLP · Summarization

Cloud-based image classifier

Built an on-demand image classification service using AWS EC2, S3, SQS, and Lambda with autoscaling to serve 100 concurrent requests within 5 minutes.

Academic MS project · Python · AWS · Keras · Portfolio

Bi-directional search in AI

Implemented bidirectional search with Pacman and food endpoints, then compared custom heuristics against DFS, BFS, and A* baselines.

AI algorithms · Python · Heuristics · GitHub

Vocabs Thrive

Designed an offline vocabulary learning Android app with search, section-based daily goals, and color-coded word levels for structured learning.

Personal project · Flutter · Android · Play Store

Algorithmic leadership

Represented PES at ACM ICPC, led academic AI project planning, and mentored 50+ juniors in Data Structures and Algorithms.

ACM ICPC · Mentorship · DSA

Backend systems experiments

Built projects across job scheduling, API rate limiting, and meet-in-the-middle search, showing practical depth beyond workplace systems.

Systems projects · Java · Redis · Algorithms · GitHub

Recommendations

Trusted by engineers across AI infrastructure teams.

Full LinkedIn recommendations shown as a horizontal review wall, ordered with the most detailed recommendations first so visitors see the strongest context immediately. View LinkedIn profile.

KO
Kevin Orellana
Building Code Interpreter and Browser Tools for AI agents @ AWS
Kevin was senior to Bhaskar but did not manage Bhaskar directly

“I have worked with Bhaskar for the past several months, and he has proven to be a consistently strong contributor across technical delivery, operational excellence, and team collaboration. He built our operations dashboard from the ground up, led several significant technical initiatives, and delivered meaningful performance improvements to existing systems. What I appreciate most is his pragmatic approach — he does not just implement features, he takes time to understand the operational context and often identifies improvements to surrounding systems, documentation, and processes along the way. Bhaskar is someone you can count on when things get difficult. He has stepped up repeatedly for production issues and critical escalations, bringing solid debugging skills and a level-headed approach that helps the team stay focused. He is also invested real effort in mentoring interns and junior engineers, writing thorough code reviews that actually teach rather than just critique. His documentation tends to be clear and actionable, which has made onboarding and knowledge sharing noticeably smoother for the team. What makes Bhaskar valuable is the combination: he delivers technical work reliably, improves team processes without being asked, and maintains a collaborative attitude that makes him easy to work with even during stressful situations. He would be a solid addition to any engineering team looking for someone who can both execute independently and contribute to overall team effectiveness.”

PT
Prasoon Tandon
SDE @ Amazon Industrial Robotics
Prasoon worked with Bhaskar on the same team

“I had the pleasure of working closely with Bhaskar for over a year, and in that time he consistently stood out for his exceptional work ethic and technical rigor. Bhaskar approaches complex engineering problems with a level of curiosity and discipline that is rare, he does not settle for surface-level answers, but instead dives deep into systems to understand the root cause and build durable solutions. During our time working together on large-scale infrastructure and model hosting systems, Bhaskar demonstrated strong expertise in distributed systems and Kubernetes-based environments. One example of his impact was his work on a cache aware routing client that optimized LLM latency and throughput. Bhaskar played a key role in building and refining the system to intelligently route requests using cached signals, helping improve the overall responsiveness of model serving at scale. Beyond his technical strengths, Bhaskar is a thoughtful and dependable teammate who contributes positively to team discussions and collaborates effectively across engineers. He takes real ownership of his work and consistently raises the bar for the team through his attention to quality and operational excellence. Any organization working on high-scale infrastructure or ML systems would benefit from having him on their team. I would gladly welcome the opportunity to work with Bhaskar again in the future and look forward to seeing the impact he will continue to make.”

WH
WanGyu Han
Machine Learning Engineer @ Amazon
WanGyu was senior to Bhaskar but did not manage Bhaskar directly

“I had the opportunity to work closely with Bhaskar on the core implementation of the Sticky Routing project on the AWS Bedrock team. While tackling complex traffic routing challenges, Bhaskar consistently proved to be an engineer I could rely on and collaborate with confidently. Bhaskar has a strong technical foundation and an excellent ability to quickly organize discussed ideas and translate them into stable, production-ready implementations. He approaches feedback on design and implementation with an open mindset, and goes beyond simply meeting requirements by thoughtfully considering better solutions and continuously improving his work. Beyond his technical strengths, Bhaskar is genuinely a pleasure to work with. He communicates with a friendly and calm demeanor, and even during deep technical discussions, he helps keep the team environment productive and positive. This made our collaboration smooth and efficient throughout the project. Bhaskar combines solid technical skills with a mature, collaborative attitude. I am confident he would quickly become a trusted and valuable teammate on any team, and I highly recommend him.”

RB
Rohit Bankar
SDE @ Amazon · Agentic AI
Rohit reported to Bhaskar directly

“Bhaskar was my mentor during my internship at AWS Bedrock, and he was easily one of the strongest engineers I have worked with. His depth of knowledge across ML infrastructure, distributed systems, and software engineering design patterns is exceptional, but what stood out even more was the way he translated that expertise into clear, practical guidance. He has a rare ability to break down complex technical problems and help you reason through them in a way that genuinely makes you a better engineer. His debugging skills were exceptional; watching him diagnose distributed failures and performance bottlenecks was genuinely inspiring. He was always approachable, incredibly responsible, and consistently pushed me to grow while giving me the space to explore and learn. I am genuinely grateful for his mentorship. Anyone who gets the chance to work with Bhaskar will benefit from not just his technical strength but also his calm, thoughtful, and collaborative approach. Highly recommended for any Software, Data, or ML role.”

JY
Je Yeong Soh
SWE @ DoorDash · ex-AWS AI/ML
Je Yeong worked with Bhaskar on the same team

“Bhaskar is one of best early-level engineers I have met in my time at AWS. He is extremely strong technically. In every project he touches, Bhaskar delivers well-architected code at remarkable speed — turning challenging requirements into robust, maintainable solutions faster than most people I have worked with. He is a very strong engineer. Beyond his technical prowess, Bhaskar stands out for his genuine kindness and generosity. He is always the first to offer guidance — whether pairing on a tricky bug, reviewing pull requests, or sharing best practices in our team channel. His positive attitude and collaborative spirit lift everyone around him, fostering an environment where engineers feel supported and empowered to do their best work. Any team would be lucky to have Bhaskar: his combination of very skilled coding and his unwavering commitment to helping others makes him both a huge asset to projects and a truly wonderful colleague. I recommend him without reservation.”

KX
Kayla Xu
Incoming @ Stripe · CS @ Princeton
Kayla worked with Bhaskar but on different teams

“During my internship with the AWS Bedrock team, Bhaskar was an invaluable mentor and one of the most technically sharp engineers I had the pleasure of working with. He has a rare combination of deep technical expertise and a strong ability to communicate it clearly, whether spotting subtle readability issues in a code review or walking me through complex problem-solving approaches with patience and care. In our 1:1s, Bhaskar guided me toward better thinking, offering thoughtful critiques that challenged me to write cleaner, more readable, and more reliable code. Those conversations had a lasting impact on how I approach software engineering. Beyond his technical ability, Bhaskar consistently made time to support me, offered encouragement when I needed it, and treated my growth as something worth investing in. I strongly recommend Bhaskar to any team looking for an engineer who is not only skilled, but also the kind of colleague who makes everyone around him better.”

RV
Ravi Vignesh
Prev. SDE Intern @ AWS Bedrock
Bhaskar was senior to Ravi but did not manage Ravi directly

“During my time as an SDE intern with the AWS Bedrock team, Bhaskar was someone I could always count on when I was stuck on tough technical problems. Whenever I reached out, he consistently took the time to offer thoughtful suggestions and practical ways to approach challenges. He has strong technical depth and a very methodical approach to problem solving, which really showed when he helped me debug complex issues in a structured and logical way. Conversations with him helped me become more systematic in how I break down problems, debug issues, and think about building reliable systems. I really appreciate the support he extended during my internship, and I am glad I got the chance to learn from him. Any team would be lucky to have an engineer like Bhaskar, and I strongly recommend him to any team looking for a skilled and dependable software engineer.”

JL
Jay Liu
Machine Learning Engineer / SDE @ Amazon
Jay was senior to Bhaskar but did not manage Bhaskar directly

“I had the privilege of working with Bhaskar at Bedrock, where he demonstrated exceptional expertise in AI infrastructure engineering. His standout achievement was designing an innovative application load balancer pattern for our inference layer, significantly optimizing traffic partitioning across our LLM systems. He also led crucial initiatives in regional hosting expansion, showcasing his deep understanding of distributed systems. Bhaskar combines technical excellence with clear communication, making complex LLM infrastructure challenges seem straightforward. His contributions have had a lasting impact on our systems' performance and reliability. Any team would be fortunate to have him as a technical leader in AI infrastructure.”

VG
Varun Gunda
Senior Software Engineer @ AWS Bedrock · Agentic AI
Varun was senior to Bhaskar but did not manage Bhaskar directly

“I have worked with Bhaskar closely across multiple projects and he consistently punches above his level. He owns problems end-to-end, moves fast, and proactively communicates and resolves blockers — often coming to the table with solutions already in hand. His work on LLM model hosting platforms and Kubernetes infrastructure demonstrates strong technical depth in some of today's most in-demand areas. Beyond feature work, he proactively identifies and drives operational improvements — a trait that sets him apart. He brings the ownership, technical instincts, and communication skills you would expect from a far more seasoned engineer and would be a strong addition to any team.”

BT
Balaji Tummala
Software Engineer at Amazon
Balaji worked with Bhaskar on the same team

“I worked closely with Bhaskar on infrastructure automation and scaling initiatives, where he demonstrated strong ownership and reliability. He led key efforts in scaling workflows and GPU/Trainium capacity ingestion, ensuring systems were production-ready and stable under real workloads. He played an important role in improving automation around scaling and releases, using metrics-driven approaches to reduce risk and increase confidence during deployments. Bhaskar combines solid technical depth with clear execution, collaborates well across teams, and approaches complex challenges with a structured and composed mindset.”

RX
Renhao Xie
Coinbase · ex-AWS
Renhao was senior to Bhaskar but did not manage Bhaskar directly

“I had the pleasure of working with Bhaskar on the Bedrock model hosting team, and I was consistently impressed by his technical rigor. He does not just scratch the surface; he is someone who truly dives deep into complex infrastructure challenges to find the root cause of any issue. His work ethic and commitment to high-quality engineering made a tangible impact on our team's stability. He would be an asset to any high-scale engineering org.”

AK
Avnish Kumar
Senior Software Engineer @ AWS Bedrock
Avnish was senior to Bhaskar but did not manage Bhaskar directly

“I have had the privilege of working directly with Bhaskar for more than a year at AWS, and I can confidently say he is an exceptional engineer. Bhaskar is not only hardworking and reliable, but he brings deep expertise in hosting Large Language Models and Kubernetes — skills that are increasingly valuable in today’s tech landscape. Bhaskar would be a strong asset to any team.”

Scroll sideways to read more recommendations →

Let us build.

Open to senior backend, AI infrastructure, agent platform, distributed systems, and cloud platform engineering conversations.

Profile focus

AI infrastructure · Agent workflows · LLM inference · Backend platforms · Distributed systems · Production reliability.