ARTICLE: Closing the Gap: Ensuring Seamless ML Model Deployment with API-Driven Testing

Machine learning (ML) models are developed and validated, but they may fail when deployed in production environments due to various reasons. These include different data, different environments, and features incompatibility. Research uses a data lake and query engines for real-time inference, while production systems use Java and C. Deploying the model as a web endpoint can be a solution, but performance requirements may be too strict.

Features incompatibility is another challenge, as researchers can use a feature store to extract real-time data and save it to a database. However, this solution is inflexible and increases the overhead of changing or adding new features. In some projects, the preferred delivery method is a specification document with pseudo code or proof of concept (POC) code as a reference. Skilled developers can implement code based on pseudo code and specs, but the end product may not be exactly the same.

Development and Research must go through many rounds and resolve bugs to get desired results. Developers can’t test their code, as they can only compare the final results and verify the specifications and pseudo code. This leads to differences between the Research and Development models.

To address these challenges, researchers can deliver tests using an API endpoint, which allows them to validate the model created by the development team. This increases Research’s ownership and allows for faster development. Test-driven development (TDD) methods make it easy to test code while writing it, gaining confidence that the code is ready once it passes testing.

Testing can solve common issues such as different data, different environments, and features in the tests. By using an API, tests act as an intermediate language, avoiding the need for constant backtrack and rechecking.