Hardening a ~3-billion-events/month client-side threat-detection pipeline on AWS
Amazon Kinesis
AWS Lambda
Amazon OpenSearch
Amazon API GatewayVeUP mapped Feroot's ~3-billion-events/month client-side threat-detection pipeline end to end (API Gateway → Lambda → Kinesis → OpenSearch, with Firehose → S3) and delivered a costed modernization plan that pairs every reliability and cost driver with a specific AWS-native fix.
The challenge
The platform detects client-side threats by ingesting browser-side telemetry at scale — up to roughly 3 billion events per month, with a single customer project able to spike to half a billion on its own. That real-time detection plane has to stay reliable and fresh under bursty, highly uneven traffic, and its cost has to scale sub-linearly with event volume. When VeUP looked under the hood, the streaming-ingest pipeline showed the classic high-volume failure modes: hot Kinesis shards driven by an uneven partition key, OpenSearch write-amplification from frequent in-place updates, and overage traffic rejected only after it had already been ingested and paid for. The company needed a partner with deep AWS streaming and FinOps expertise to harden the pipeline for peak reliability and bend its cost curve.
The solution
VeUP ran a discovery of the live pipeline and turned it into an engineering Art-of-the-Possible proposal with a scoped proof-of-concept plan. Discovery mapped the pipeline end to end: Amazon API Gateway → AWS Lambda → Amazon Kinesis Data Streams (an ingest stream plus an enriched stream) → ~15–20 AWS Lambda consumers → Amazon OpenSearch Service for hourly aggregates, with Amazon Kinesis Data Firehose → Amazon S3 for raw-event backup and replay. Each failure mode got a specific fix: (1) de-skew the Kinesis partition key so no single project can create a hot shard; (2) push entitlements and throttling to the edge so the platform stops paying to ingest traffic it then rejects; (3) move OpenSearch to an append-only write pattern with materialized aggregates and index lifecycle management, removing the update-churn write-amplification; (4) instrument FinOps per ARN across Kinesis, Lambda, and OpenSearch. The plan also scoped an optional real-time OLAP pilot alongside OpenSearch and set clear success criteria: near-zero throttled or dropped events at peak, sub-60–90-second freshness plus hourly rollups, and a lower cost per ingested event.
Production outcomes
| KPI | Result |
|---|---|
| Production outcomes | The ~3-billion-events/month detection pipeline is now fully mapped, and Feroot holds a costed modernization plan that pairs every reliability and cost driver with a specific AWS-native fix — de-skewed Kinesis sharding, edge-enforced entitlements, an append-only OpenSearch pattern with index lifecycle management, and per-resource FinOps instrumentation. Along the way, VeUP advanced Feroot’s AWS Marketplace listing and settled into an ongoing solutions-architect-on-demand advisory role, most recently aligning on the roadmap in a Q1 review. |
| Engagement window | The relationship began in late 2024. VeUP delivered the pipeline deep-dive and modernization plan in September 2025, and the solutions-architect-on-demand advisory continues today. |
| Cost / TCO posture | FinOps was load-bearing: discovery showed Kinesis and Lambda (reads and writes) as the largest cost drivers, with OpenSearch cost climbing from update churn. The plan bends the cost curve three ways — edge-enforced entitlements so the platform stops paying to ingest overage it then rejects, append-only OpenSearch with index lifecycle management, and per-ARN cost attribution so the dominant drivers stay visible as volume grows. |
| Lessons & continuation | On high-volume streaming ingest, the dominant reliability and cost risks are structural — an uneven partition key creating hot shards, update-churn write-amplification, and overage rejected only after it has been paid for. Attributing cost per ARN across Kinesis, Lambda, and OpenSearch is what keeps the savings durable. The savings themselves stay unpublished until the proof of concept proves them in production. |
Architecture
One pipeline, four fixes. Telemetry flows Amazon API Gateway → AWS Lambda → Amazon Kinesis Data Streams → Amazon OpenSearch Service, with Kinesis Data Firehose → Amazon S3 for raw-event backup — and the modernization plan reshapes it with a de-skewed Kinesis partition key, edge-enforced entitlements, an append-only OpenSearch pattern with index lifecycle management, and per-ARN FinOps instrumentation.
Amazon Kinesis Data Streams
Amazon Kinesis Data Firehose
Amazon S3