Enterprises Overpaying for Simple AI Queries: Snowflake's Solution
Enterprises are overpaying for simple AI queries — Snowflake's gateway now auto-routes to cut costs up to 3x | VentureBeat
Orchestration, Infrastructure, Data, Security
Enterprise teams running AI agents at scale face a challenge: a single model often performs poorly across diverse tasks. Model routing, which automatically selects the most suitable model for each task, is emerging as a solution.
Snowflake’s Cortex AI Gateway introduces dynamic model routing, allowing enterprises to choose “auto” instead of fixed models. This results in significant cost savings, reducing token expenses by up to 3x, especially for simpler queries. Previously, simple questions were often handled by the most capable (and expensive) model, leading to slower and more costly responses.
This move aligns with a broader industry trend towards automated model routing, with major players like Databricks, AWS, Google Cloud, and Nvidia also introducing similar technologies. Snowflake distinguishes itself by focusing on context, trust, and model choice within its Cortex AI Gateway.
How Dynamic Routing Works
Dynamic routing leverages two mechanisms:
- Small Model First Approach: Similar to an advisor pattern, a smaller model attempts the task initially. If it fails, it calls upon a larger model for assistance.
- Classifier-based Sorting: A separate classifier, trained on historical queries, intelligently routes straightforward questions to simpler models.
Customization Options: Customers retain control; auto-routing is optional, and they can restrict routing to specific models or groups of models.
Cost Efficiency and Security
Snowflake prices AI solely based on token usage, ensuring that cheaper models translate into lower bills without additional fees for routing decisions. Furthermore, access controls follow the task rather than data, providing granular permissions for agents and models while maintaining a secure environment for all inference activities, regardless of origin.
Recent Acquisitions Strengthening Snowflake's Position
Snowflake’s recent acquisition of Natoma brings over 100 MCP connectors with scoped, governed access, enabling agents to have restricted access to connected tools like email. This enhances the platform’s capabilities in data governance and security.