Scaleplus: Towards Fast Scaling of Distributed Streaming Dataflows

2020 
Streaming dataflows are usually deployed as longterm services and suffer from the fluctuation in arrival rates. Many scaling controllers provide the elasticity to scale up or down the dataflow to meet the target throughput. However, for most controllers with specific target Service Level Objective (SLO), it is difficult to use a single decision to accurately complete a scaling action. Reaching the target iteratively will increase the completion time of the scaling, which leads to the performance degeneration or the wasting of resources. In this paper, we present Scaleplus, which provides pluggable services for helping iterative scaling controllers make more accurate decisions to complete the scaling rapidly. Scaleplus builds decision models incrementally without the off-line sampling procedure, which can be used out of the box, then a recommendation strategy is leveraged to recommend accurate decisions. Besides, Scaleplus can be flexibly integrated with only 3 HTTP APIs. We evaluate Scaleplus with 3 different scaling controllers on Apache Flink and Apache Heron. For the simulated 9-day traces of Twitter, Scaleplus reduces the scaling duration by 36.5%, 51.1% and 54% respectively compared with DS2, mRB and Dhalion.
    • Correction
    • Source
    • Cite
    • Save
    • Machine Reading By IdeaReader
    17
    References
    0
    Citations
    NaN
    KQI
    []