Container Orchestration on HPC Clusters

Use of software containers and services in science is a rising trend that is not satisfied by the HPC computing resources often available in research contexts. We propose a method to grow Kubernetes clusters onto transient nodes allocated through the Grid Engine batch workload manager. The method is being used to run a mix of data-intensive service applications and bursty HPC-style workflows on an OpenStack-based Kubernetes deployment, while keeping a homogeneous job management, logging, monitoring, and storage infrastructure. Moreover, it is relatively straightforward to convert the implementation to be compatible with other workload managers.