Lead data diagnostics at Vibe
We are estimating our product to be aggregating around 2 billion data points in the next year. Each data point will be a combination of 15 to 20 information items (text) about entities. We use Python/Django for the application and APIs and Elasticsearch as the data store.
Will this stack scale to 2 billion records and processing? If no, what should I be considering. Would appreciate your suggestions.