Welcome to this issue of the WeAreDevelopers Live Talk series. This article recaps an interesting talk by Nico Axtmann who introduced us to MLOps

About the speaker:
Nico Axtmann is a seasoned machine learning veteran. Starting back in 2014 he observed how the field changed from purely model development to building and ship AI for real world use cases. He has consulted companies across Germany and Europe on machine learning and cloud. Furthermore, he transformed cutting edge research into products and built a speech recognition engine for doctors and scaled it up to process more than 10.000 hours on a daily basis. In his previous role as the Head of AI at certainly.io, he developed conversational AI to drive ecommerce sales and built the infrastructure to run thousands of models concurrently. These days, he leads the MLOps activities at Oxolo, building large scale AI applications and pioneering Artificial Intelligence.

AI observed by an Engineer‍

When looking into the AI field it’s basically impossible to summarize all the great breakthroughs that happened in the last couple of years. So, here is just a quick attempt by Nico:
Back in 2012, a major breakthrough happened in image classification - the first deep neural network was introduced. Sidenote: this was also the year when data scientist was called the “sexiest job to work in the 21st century”.

Later, in 2017, the text area followed up with natural language processing. This year also saw the birth of the transformer models and a lot of papers were named after characters of the Muppet show.

And in 2022 a lot of advances in generative AI occured. With just inserting some information you can now generate a picture. In this example, the words “a cute corgi lives in a house made out of sushi” were used:

   
Effects of the AI research push‍

It’s important to note that AI is a research-driven field. This circumstance comes with some consequences. First, it’s basically impossible to keep up with the studies of this environment as there are many papers published every day. Because of the amount of information, it’s also very hard to spot the specific ones most relevant to the area you are working on. Another effect would be that a lot of the research is centered around beating isolated benchmarks which means that people are working on perfectly crafted and trained data sets. As a result, the progress made in one particular area is glorified as it happened in a perfect environment.

And finally, a lot of the education in this field is focused on the theory and understanding the algorithms rather than on real-life projects. This lack of practical knowledge is essentially un favorable in a market with lots of movements like this.‍

‍‍

Effects on the industry‍

AI has also created some wrong expectations for companies. A lot of them think that adding some AI to the data they have collected over the past years, gives them some sort of superpower, and revolutionizes their business. But the honest truth is that building something with AI is a long and painful process as there are many traps involved.

These days there is a huge gap between AI research and industry adoption: “90% of corporate AI initiatives are struggling to move beyond test stages.” (IBM Blog – AI at Scale 2021). Or in other words: the industry is unable to build lots of products in this area.

Although there are some big Ai-first companies that are the major contributors and are able to adapt their research to real-life projects there are also a lot of small and mid-sized ones that are simply not able to adjust their products to the current market. On the other side, you can also find a couple of corporations that claim to do AI but only as a way to seem innovative and contemporary.

So, all in all, there is a big underestimation in the industry of the efforts that are required to build products that contain any kind of AI.

‍

‍Applied AI – challenges of building real-world systems‍

In order to give a better overview of the requirements of applied AI, let's have a look at an image taken from a paper (“Hidden technical debt in machine learning”) that was published by Google in 2015.

Here you can see different boxes and the size indicates the effort that is needed for each of those challenges. Only the small black one in the middle is the actual ML code. The rest of the work is dedicated to other tasks like “Data Collection” and “Serving Infrastructure”.

Nico recommends to everyone interested in this field to reading this paper as it's easy to understand and he considers it the birth of MLOps because it talks about the challenges you need to deal with when working with applied AI. But what is MLOps? Here are two definitions:

‍“MLOps or ML Ops is a set of practices that aims to deploy and maintain machine learning models in production reliably and efficiently.”

(https://en.wikipedia.org/wiki/MLOps)‍

“MLOps […] focuses on four key areas within model training, tuning and deployment (inference): machine learning must be reproducible, it must be collaborative, it must be scalable, and it must be continuous.”

(https://cio-wiki.org/wiki/MLOps)

‍

Nico also has a personal take on what MLOps is. According to him it solves the AI engineering challenges around AI development and deployment to ship reliable and robust products. But when building applications with AI there are a lot of side effects: Data you need to be aware of, code that is pre- or postprocessing data, models that are bound to a specific piece of the set, and many more. So with all these in mind, you have to build a bridge between product management, project management, engineering teams and AI teams to deliver a true AI solution.

‍

To overcome some of these challenges and make AI – life easier for you and your company, Nico shares three of his key learnings with the audience:

‍

-   Invest in data, model and experiment management

Also, enforce reproducibility to establish trust in the model results.

By investing you will gain fast and easy access to your data which will accelerate experimentations

-   Use open source and standardize

There are a lot of awesome open-source projects out there, so don’t be afraid to use them as it will make your workflow faster and easier.

-   Software/Data/ML engineering is the key to succeed

Adapting to AI advances is the usual bottleneck in the development process. The better the software engineering in your company is, the faster you can push a model from the research stage to a final product.

‍

Thank you for reading this article. If you are interested in hearing Nico Axtmann himself you can do so by following this link and joining our platform: https://events.hubilo.com/wearedevelopers---world-congress-2022/session/146472

‍