Pocket: Ephemeral Storage For Serverless Analytics

Authors:
Ana Klimovic Stanford University
Yawen Wang Stanford University
Christos Kozyrakis Stanford University
Patrick Stuedi IBM Research
Animesh Trivedi IBM Research
Jonas Pfefferle IBM Research
Christos Kozyrakis stanford university

Introduction:

Serverless computing is becoming increasingly popular.However exchanging intermediate data between execution stages in an analytics job is a key challenge as direct communication between serverless tasks is difficult.the authors present Pocket, an elastic, distributed data store that automatically scales to provide applications with desired performance at low cost. We present Pocket, an elastic, distributed data store that automatically scales to provide applications with desired performance at low cost.

Abstract:

Serverless computing is becoming increasingly popular, enabling users to quickly launch thousands of shortlived tasks in the cloud with high elasticity and fine-grain billing. These properties make serverless computing appealing for interactive data analytics. However exchanging intermediate data between execution stages in an analytics job is a key challenge as direct communication between serverless tasks is difficult. The natural approach is to store such ephemeral data in a remote data store. However, existing storage systems are not designed to meet the demands of serverless applications in terms of elasticity, performance, and cost. We present Pocket, an elastic, distributed data store that automatically scales to provide applications with desired performance at low cost. Pocket dynamically rightsizes resources across multiple dimensions (CPU cores, network bandwidth, storage capacity) and leverages multiple storage technologies to minimize cost while ensuring applications are not bottlenecked on I/O. We show that Pocket achieves similar performance to ElastiCache Redis for serverless analytics applications while reducing cost by almost 60%.

You may want to know: