Multimodal Segmentation Neural Network to Determine the Cause of Damage to Grasslands

Agricultural fields suffer from damage due to changing climate conditions and wildlife foraging. Damage incurred by wild boar is classified as the main contributing factor to grasslands damage. Such damage results in losses for farmers due to reduced yield potential and repair costs. The extent of wild boar damage is typically estimated manually by using ground based approaches (e.g. GPS measurements), which is a time consuming task with questionable accuracy. Building upon our previous work, we present an autonomous approach to detect and classify the cause of damage to grasslands (wild boar, mole etc.). The approach entails utilizing convolutional neural networks for semantic segmentation of grasslands. An RGB baseline was established, in addition to evaluating multimodal architectures that incorporate different surface model feature representations, leading to a joint representation of spectral and elevation information. Testing and experimentation was performed in real-world grasslands around Bonn, Germany. The results show that incorporating elevation features with late fusion enhances the overall performance of the network over the RGB baselines.