Orbis cuts analytics cost ~30% migrating Amazon Redshift to Amazon Athena
Amazon Athena
Amazon S3
Amazon RedshiftShared anonymously — the customer’s name is held by VeUP and available on request.
VeUP advised the team behind Orbis, an alternative-asset market-data platform, on moving its analytics layer from Amazon Redshift to serverless Amazon Athena over S3-resident data — cutting analytics cost by roughly 30% on a cleaned, structured dataset.
The challenge
Orbis collects alternative-asset market data by scraping unstructured auction sources and standardizing it into a structured dataset. Its analytics layer ran on Amazon Redshift, where provisioned-warehouse cost grew faster than query value for a spiky, R&D-heavy workload. The team needed to reduce analytics cost without losing the ability to run ad-hoc queries over a growing, S3-resident data lake, and to clean noisy scraped inputs into an analysis-ready dataset.
The solution
VeUP advised migrating the analytics layer from a provisioned Amazon Redshift warehouse to serverless query with Amazon Athena over S3-resident data — paying per query rather than for idle warehouse capacity. The scraping-and-standardization pipeline produces a clean, structured alternative-asset dataset on Amazon S3 that Athena queries directly, with the stated roadmap of deploying AI models on top of the curated dataset.
Production outcomes
| KPI | Result |
|---|---|
| Production outcomes | The move from Amazon Redshift to Amazon Athena cut analytics cost by roughly 30% (customer-stated), and the scrape-and-standardize pipeline turned noisy unstructured inputs into a clean, structured alternative-asset dataset. The customer relationship has since ended; the engagement stands as a historical example. |
| Engagement window | A short, sharp engagement delivered in April 2024. The customer relationship has since ended. |
| Cost / TCO posture | The migration was cost-driven: moving from a provisioned Redshift warehouse (paying for idle capacity) to serverless Athena (pay-per-query over S3) removed the fixed warehouse cost floor for a spiky analytics workload — the source of the ~30% analytics-cost reduction. |
| Lessons & continuation | For spiky, ad-hoc analytics over an S3-resident lake, serverless Athena beats a provisioned warehouse on unit economics; data standardization/cleaning of noisy scraped inputs is the precondition for both reliable analytics and any downstream AI build. |
Architecture
The shape of the move: the scrape-and-standardize pipeline lands its structured alternative-asset dataset in Amazon S3, an AWS Glue Data Catalog holds the schema, and serverless Amazon Athena queries it directly — paying per query where the provisioned Redshift warehouse once billed for idle capacity.
Where it started
Baseline · historical exampleAlternative-asset market-data platform · Provisioned-warehouse analyticsUnstructured auction market-data scraped and standardized on customer-managed compute, loading a provisioned Amazon Redshift warehouse in a private analytics subnet.
An always-on Redshift cluster billed between ad hoc query bursts — cost grew faster than query value for a spiky, R&D-heavy workload.
Unstructured scraped auction inputs required standardization before any analytics could run reliably.
Serverless, pay-per-query analytics stayed out of reach while the analytics layer remained a provisioned warehouse.
The starting point of the analytics layer before its move to serverless — S3, Glue, and Athena.