BG: A scalable benchmark for interactive social networking actions

Abstract BG is a benchmark that rates a data store for processing interactive social networking actions such as view a member profile and extend a friend invitation to a member. It elevates the amount of stale, inconsistent, and erroneous (termed unpredictable ) data produced by a data store to a first class metric, quantifying it as a part of the benchmarking phase. It summarizes the performance of a data store in one metric, Social Action Rating (SoAR). SoAR is defined as the highest throughput provided by a data store while satisfying a pre-specified service level agreement, SLA. To rate the fastest data stores, BG scales both vertically and horizontally, generating a higher number of requests per second as a function of additional CPU cores and nodes. This is realized using a shared-nothing architecture in combination with two multi-node execution paradigms named Integrated DataBase (IDB) and Disjoint DataBase (D 2 B). An evaluation of these paradigms shows the following tradeoffs. While the D 2 B scales superlinearly as a function of nodes, it may not evaluate data stores that employ client-side caching objectively. IDB provides two alternative execution paradigms, Retain and Delegate, that might be more appropriate. However, they fail to scale as effectively as D 2 B. We show elements of these two paradigms can be combined to realize a hybrid framework that scales almost as effectively as D 2 B while exercising the capabilities of certain classes of data stores as objectively as IDB.