Comparing Apples and Oranges in IoT context: A deep dive into methods for comparing IoT platforms

Many researchers try to make a comparison between various Internet-of-Things (IoT) platforms based on specific requirements. However, none of the reviewed studies proposed a thorough analysis of the variety of comparative methods. Since there is a lack of comparison frameworks for IoT platforms, individuals or companies have difficulties when selecting a suitable IoT platform matching their associated business requirements. In order to support this selection process, a set of functional and nonfunctional requirements is identified. A framework containing methods in selecting an IoT platform is presented. The methodology is based on statistical and visualization techniques to recommend a suitable IoT platform. Five IoT platforms: 1) Azure; 2) AWS; 3) SaS; 4) ThingWorx; and 5) Kaa IoT are studied to evaluate the performance of the framework. Different comparison methods are proposed and a multicriteria decision analysis method was applied by using an analytical hierarchical process (AHP). One of the methods clusters the functional requirements and compares the IoT platforms based on their ability in supporting a specific requirement or not. The $K$ -means clustering was applied to determine the clusters of functional requirements. The comparison was made based on the hierarchical level of requirements per main requirement. The other methods use the following statistical tests: error bar test, one-way Anova test, and Tukey’s honest significant difference test. Based on the selected requirements, an approach is suggested for which IoT platform can be used.