In a significant development for artificial intelligence infrastructure, DigitalOcean has revealed that Workato's AI Research Lab is utilizing its cloud platform to bolster the creation of advanced enterprise AI agents at production scale. This collaboration has allowed Workato to transition its AI Labs workloads to DigitalOcean's specialized inference-optimized cloud, which is built upon NVIDIA Hopper GPUs. This strategic move aims to optimize AI development and deployment for enhanced efficiency and performance.
Following this transition, Workato observed immediate and substantial improvements in the performance of its leading-edge models, including Llama-3.3-70B. DigitalOcean reported that Workato achieved a remarkable 67% reduction in inference costs, bringing them down to a mere $0.77 per 1 million tokens. Concurrently, throughput saw an impressive 67% increase, reaching 13,561 tokens per second per GPU. Furthermore, the time-to-first-token, a critical metric for responsiveness, improved by 77% to 1,455 milliseconds under high load. This initiative also dramatically shortened the time-to-value from several weeks to just a few days, signifying a more than twofold acceleration in deployment and operational readiness.
DigitalOcean collaborated closely with Workato to devise and fine-tune a distributed inference architecture hosted on DigitalOcean Kubernetes. This setup incorporated NVIDIA Dynamo to effectively coordinate workloads across interconnected GPU clusters. This innovative configuration led to a reduction in redundant computations, improved system responsiveness during peak demand, and ultimately provided Workato with a 33% advantage in hardware price-performance, underscoring the efficiency and power of DigitalOcean's tailored cloud solutions.
The successful partnership between DigitalOcean and Workato exemplifies how strategic infrastructure choices can revolutionize AI development and deployment. By prioritizing efficiency, cost-effectiveness, and high performance, businesses can unlock new possibilities in artificial intelligence, driving innovation and achieving remarkable technological advancements. This collaboration highlights the critical role of robust and optimized cloud platforms in accelerating the progress of AI technologies, ultimately benefiting a wide range of industries and applications.