In June of 2023, we started to look into how to scale our backend and API infrastructure to meet the growing demands of enterprise clients. We could see the market opportunity was there, but wanted to be able to move quickly to establish an offering that was ready for scale.
To tackle this, we partnered with No Moss, a tech consultancy specialising in agile development and strategic technology solutions.
The Challenge
As our platform’s user base was growing, the need for a robust and scalable API became an increasing priority for us.
Our existing infrastructure was sufficient for individual users, but enterprise customers required a more powerful solution to handle high volumes of image generation requests efficiently. We has to tackle challenges on multiple fronts, including:
- Managing High Concurrency: The existing system needed to accommodate thousands of concurrent requests from enterprise users without compromising performance.
- Ensuring Stability During Scaling: With millions of users already on the platform, any broad changes to the backend needed to be implemented with high confidence to avoid disruptions.
- Expanding Front-End Capabilities: To support the new enterprise API offering, the user interface and self-service functions required enhancements.
The Solution
The No Moss team applied their expertise in agile development and lean thinking to address these challenges, delivering tailored solutions, including:
- Round-Robin Load Balancing: To manage the high concurrency needs of enterprise customers, No Moss implemented a round-robin style load balancing system. This solution allowed the Leonardo platform to configure specific concurrency limits, ensuring that image generation jobs were dispatched efficiently. The system was tested rigorously and deployed within a few weeks, initially supporting specific partners before rolling out to the broader user base.
- Enhanced Testing for Stability: Understanding the risks associated with scaling, No Moss prioritised building a comprehensive testing framework. The team added approximately 200 new tests during the initial phase, targeting core functionalities and critical scenarios. This robust testing environment provided the confidence needed to implement changes without disrupting existing services.
- Scaling the Front-End: As the backend scaled, No Moss also focused on enhancing the front-end experience for enterprise users. The team developed a new self-service portal, allowing users to manage API keys, top-up credits, and handle billing autonomously. A notable innovation was the creation of a Slackbot that allowed No Moss staff to generate images directly within Slack, providing valuable insights into the user experience and further refining the API’s capabilities.
The Results
No Moss’s collaboration with Leonardo led to significant advancements in the platform’s scalability and user experience.
- Seamless Enterprise Integration: The new load-balancing system enabled our platform to handle thousands of concurrent requests from enterprise clients without performance issues.
- Increased Stability and Confidence: The expanded testing framework allowed the Leonardo team to implement changes quickly and confidently, minimising the risk of disruptions as the platform scaled.
- Improved User Experience: The new self-service features and Slackbot integration provided a more intuitive and efficient experience for enterprise users, contributing to a smooth and scalable API rollout.
Embracing the Future of Gen AI for Enterprise
No Moss played a key role in enabling the Leonardo team to meet the demands of its growing enterprise client base.
“Our No Moss team is proud to have contributed to the success of a cutting-edge Australian Generative AI platform.”
– James O’Toole, Co-founder and Tech Director, No Moss
By leveraging agile methodologies, rigorous testing, and innovative front-end solutions, No Moss ensured that our API could scale effectively while maintaining stability and delivering a seamless user experience.