CloudAR: A Cloud-based Framework for Mobile Augmented Reality

Computation capabilities of recent mobile devices enable natural feature processing for augmented reality (AR), but the scalability issues are still faced by mobile AR applications. In this paper, we propose CloudAR, a mobile AR framework utilizing the advantages of cloud and edge computing through task offloading. We design an innovative tracking system for mobile devices which provides lightweight tracking with 6 degree of freedom (6DoF) and hides the offloading latency from user»s perception. We also design a multi-object image retrieval pipeline that executes fast and accurate image recognition tasks on servers. Experiments are carried out to evaluate the performance of CloudAR. The mobile AR App built with CloudAR framework runs at 30 frames per second (FPS) on average with precise tracking of only 1~2 pixel errors and accurate image recognition of at least 97% accuracy.