what's new in mlops & production ml

Yipeng Liu, Chang Liu, Si Shen +2

this paper introduces ubep, a re-architected communication library for expert parallelism in production superpods. it aims to improve the efficiency of large-scale model training and inference by optimizing communication for distributed systems.

Sadra Saremi

this paper presents adaptivesd, a framework for optimizing large language model (llm) inference on cpu-constrained systems. it helps practitioners improve the efficiency and stability of llm deployments by dynamically adapting decoding strategies.

Roopam W. Sure

this paper proposes agl-1, an enterprise ai governance layer designed to act as a control plane for trusted enterprise intelligence. it's relevant for practitioners aiming to implement robust governance, compliance, and trustworthiness across their ai initiatives, especially within large organizations.

Luan Pham

this paper focuses on anomaly detection and root cause analysis for general microservice systems, which are common in mlops deployments. it offers methods to quickly identify and diagnose issues in complex distributed systems, helping practitioners maintain system reliability and performance.

Soham Bhattacharjee, Karun Sharma, Vinay Kumar Sankarapu +1

this paper introduces curatorkit, a framework for data curation and synthetic data generation, specifically for post-training large language models. it offers practitioners tools to improve the quality and quantity of training data, which is crucial for fine-tuning and adapting llms effectively.

Abd Elghani Meliani, Arora Sagar, Adlen Ksentini +1

this paper introduces zero touch predictive orchestration for automating the deployment and management of time-series models across cloud and edge environments. it offers practitioners a way to streamline the operationalization of predictive analytics in distributed systems, reducing manual effort and improving scalability.

Faezeh Amou Najafabadi, K. Rajenthiram, J. Bogner +2

this review compiles significant mlops guidelines for integrating and deploying machine learning models. it offers practitioners a consolidated view of best practices and architectural considerations to streamline their ml model production pipelines.

Khalid Adnan Alsayed

proposes a governance framework for deploying high-stakes ai systems that orchestrates deployment decisions based on fairness, transparency, and risk thresholds. teams building models for regulated domains or critical applications need this framework to ensure deployments meet governance requirements before going live.

Abhijit Chakraborty, Suddhasvatta Das, Yash Shah +2

introduces a policy layer that manages retraining budgets—compute, annotation, energy—for continuously adapting ml systems, preventing runaway costs in production. practitioners operating models that drift over time can use this approach to make principled decisions about when and how often to retrain.

MW Matthias Wagner, PR Per Runeson, STSule Tekkesinoglu

this review paper examines how explainability (xai) can be integrated across the entire mlops lifecycle. for practitioners, it highlights how xai can improve model understanding, debugging, and compliance, leading to more transparent and trustworthy ml systems in production.

Ruoyu Qin, Weiran He, Yaoyu Wang +5

this paper proposes "prefill-as-a-service" to optimize large language model (llm) serving by disaggregating prefill and decode operations and enabling cross-datacenter kv cache sharing. practitioners deploying llms at scale can use these insights to improve inference efficiency and reduce latency, especially in distributed environments.

Jiang Liu, John Martabano Landy, Yao Xuan +6

this paper introduces a template-driven approach for developing and deploying machine learning models at scale within large organizations. it helps practitioners standardize ml workflows, accelerate model deployment, and manage complex model ecosystems more efficiently.