MVFFNet: Multi-view feature fusion network for imbalanced ship classification

Abstract The accurate classification of moving ships is of fundamental importance to maritime authorities for ensuring the safety and security of shipping operations. With the wide use of automatic identification systems (AISs), which allow ships to receive identification/location information from nearby ships, it is feasible to classify the types of ships by analysing ship behaviour from AIS-based trajectories. However, the imbalanced features of AIS data make it difficult to achieve satisfactory classification results in the presence of several different types of ships. To overcome these potential limitations, we propose a multi-view feature fusion network (MVFFNet) to achieve accurate ship classification with imbalanced data. To guarantee the powerful representation and generalization abilities of MVFFNet, we first extract several multi-view features (i.e., motion features and morphological features) from AIS-based ship trajectories. Several kinematic variables related to ship behaviour are empirically adopted as motion features. The morphological features are automatically extracted via convolutional auto-encoder (CAE) networks. CAE networks are capable of optimally learning the features from informative trajectory images, which are strictly related to the original ship trajectories. The bidirectional gated recurrent unit (BiGRU) network is then proposed to combine multi-view features to generate the ship classification results. In addition, a hybrid loss function is presented to handle the imbalance problem of ship types, potentially leading to enhanced robustness and accuracy of ship classification. Comprehensive experiments on two realistic datasets have demonstrated that our proposed MVFFNet consistently outperforms other competing methods in terms of classification accuracy and robustness.