OctoML CEO: DevOps must take precedence over MLOps

MLOps has emerged as a means of gaining control over the complexity of industrial AI applications.

Luis Ceze, co-founder and CEO of OctoML, a firm developing tools to automate machine learning, argues that endeavour has so far failed.

At this point, Ceze told ZDNet via Zoom, “It’s still fairly early to transform ML into a regular practise”.

So why am I critical of MLOps? Because we’re naming something that’s not very well defined, and there’s something that is very well defined, called DevOps, which is a very well defined process of bringing software to production, and I believe that we should be using it..”

As Ceze put it, “I personally believe that we won’t need ML Ops if we execute this well.”

For DevOps, you need to be able to treat the machine learning model like any other piece of software: It has to be portable, it has to be performant, and doing all of that is something that’s very hard in machine learning because of the tight dependence between the model, and the hardware, and a framework, and the libraries,” he said.

According to Ceze, the machine learning stack’s extremely fragmented structure necessitates resolving dependencies.

OctoML promotes the idea of “models-as-functions,” which is a term used to describe machine learning models. Machine learning model construction and traditional software development are both said to be better served by this technique because of the smoother cross-platform interoperability.

Ceze and his co-founders created the Apache TVM compiler, which OctoML now offers as a commercial service.

According to a press release issued by the company on Wednesday, the company’s technology now supports a broader range of public cloud instances, such as AWS, GCP, and Azure, as well as CPUs, GPUs, and NPUs of various architectures from a variety of vendors, including those from Intel and AMD.

“We want to get a much larger range of software developers to be able to deploy models on standard hardware without any specific expertise of machine learning systems,” said Ceze. Ceze.

Because model creation maturity has improved significantly in recent years, Ceze said that the code is intended to solve an industry-wide problem: “a tremendous difficulty in the business.” What do I do now that I have a model?”

For every new machine learning model that is developed, only half of those models are really used in the workplace, according to Ceze.

“We aim to reduce it to hours,” Ceze said.

OctoML describes “Intelligent Applications” as “apps that have an ML model incorporated into their functioning” if the technology is implemented correctly, according to Ceze.

Ceze cited instances like as Zoom’s background effects and a word processor’s “continuous NLP,” or natural language processing, as examples of how this new class of applications is “becoming most of the apps.”

As Ceze put it: “ML is moving everywhere, it’s becoming an important component of what we use, therefore we started out on a mission to fix that issue.”

To make a human engineer understand the hardware platform, choose the proper libraries, work with the Nvidia library, say, the right Nvidia compiler primitives, and arrive with something they can execute is the state of the art in MLOps, according to Ceze.

The OctoML system “automates everything,” he claimed. According to him, the new reality should be “Get a model, convert it into a function, and call it.” “You can download a Hugging Face model from a URL.”

With the latest release, Nvidia’s Triton inference server software is better integrated than ever before.

Triton’s “portability, versatility, and adaptability” make it an excellent partner for the OctoML platform, according to Nvidia’s prepared comments.

Ceze cited “the convergence of DevOps with AI and ML infrastructure” as a potential market for OctoML as a company. Just over 100 billion is spent on DevOps each year, while AI and ML infrastructure is in the billions.