Scaling MLOps for the enterprise with multi-tenant systems

In the context of MLOps, the benefits of using a multi-tenant system are many. Machine learning engineers, data scientists, analysts, modelers, and other professionals who contribute to MLOps processes often need to perform similar tasks with similar software stacks. It is extremely beneficial for a business to just maintain one instance of the stack or its capabilities: this reduces costs, saves time and improves collaboration. In essence, MLOps teams on multi-tenant systems can be exponentially more efficient because they don’t waste time switching between stacks or systems.

Growing demand for multi-tenancy

The adoption of multi-tenant systems is on the rise, and for good reason. These systems help unify computing environments, discouraging scenarios where individual groups configure their own custom systems. Fragmented computing environments like these are highly duplicative and exacerbate the cost of ownership because each group likely needs a dedicated team to keep their local system running. This also leads to inconsistency. In a large company, you may have some groups running software that is version 7 and others running version 8. You may have groups using certain technologies but not others. The list goes on. These inconsistencies create a lack of common understanding of what is happening in the system, which then exposes the potential for risk.

Ultimately, multi-tenancy is not a characteristic of a platform: it is a basic security feature. It’s not enough to simply plaster security as an afterthought. It must be part of the fundamental architecture of a system. One of the biggest benefits for teams striving to build multi-tenant systems is the architecture’s implicit commitment to security, because security is inherent in multi-tenant systems.

Challenges and good practices

While the benefits of implementing multi-tenant systems are not without their challenges. One major obstacle for these systems, regardless of discipline, is scale. Every time a scaling operation is started, patterns emerge that probably weren’t apparent before.

As you start to scale, you gather more diverse user experiences and expectations. All of a sudden, you find yourself in a world where users start interacting with everything that gets scaled and using the tool in ways you didn’t expect. The biggest and most fundamental challenge is that you need to be able to handle more complexity.

When you build something multi-tenant, you’re probably building a common operating platform that will be used by multiple users. This is an important consideration. Something that is multi-tenant is also likely to become a core part of your business because it’s such a significant investment.

To successfully build multi-tenant systems, robust product management is critical, especially if the system is built by and for machine learning experts. It is important that the people who design and build a domain-specific system have a strong grasp of the field, enabling them to work backwards to end-user requirements and capabilities, while being able to anticipate future business and technology trends. This need is only underlined in evolving domains such as machine learning, as demonstrated by the proliferation and growth of MLOps systems.

Aside from these best practices, be sure to obsessively test every component of the system and the interactions and workflows they enable (we’re talking hundreds of times), and encourage users to test every emergent feature element and property. Sometimes, you’ll find that you have to implement things a particular way because of the business or technology. But you really want to be true to your users and how they use the system to solve a problem. You never want to misinterpret a user’s needs. A user might come up to you and say, « Hey, I need a faster horse. » You could then spend all your time training a faster horse, when what they actually needed was a more reliable and speedy means of transportation that wasn’t necessarily powered by hay.