The authors from the Remote Sensing and Image Analysis group (Institute of Geodesy) just published Is Your Training Data Really Ground Truth? A Quality Assessment of Manual Annotation for Individual Tree Crown Delineation in the special issue AI Remote Sensing of the MDPI Remote Sensing journal.

To accurately and automatically map forest stands, we need to detect trees in satellite and aerial imagery. Supervised deep learning models, also known as AI, are used for this task. To train a reliable model, you need an accurate tree crown dataset. These training datasets are still created manually. For example, a person circles each tree crown individually on a satellite or aerial image. Tree crowns are complex, forests are dense, and images are not detailed enough to make manual annotations reliable. Manually annotated crowns probably don't represent the true conditions. When these annotations are used as training data for deep learning models, which is often the case, the mapping results may be inaccurate. This study validates the accuracy of such manual tree crown annotations at two sites: a forest-like plantation in a cemetery and a natural urban forest. The validation is based on (1) tree data from the official tree register of Frankfurt am Main and (2) drone laser scanning point clouds to assess the quality of the training dataset. The manual annotations correctly identified only 37% of the tree crowns in the forest-like plantation and only 10% in the natural forest. Also, in both locations, multiple trees were often marked as one. This paper demonstrates the poor quality of commonly used reference data in AI research on individual tree crown delineation.

Paper can be read here.