WP6 - Accuracy assessment and cross-validation

Lead: DLR

Classifications based on remotely or in-situ sensed data sets are basically abstract approximations of the true physical environment being monitored. Spatial resolutions, multi-temporal changes, sampling schemes or uncertainties in the specific processing or measuring methods are responsible, among many other effects, for errors and inconsistencies within the resulting mapping products. The value of satellite derived land products, as well as in-situ sensed data for science applications, research or user-oriented product development, is  dependent upon the known accuracy of the data. Beyond this, the value of the tools is dependent on the validity and robustness of the algorithms. A clear definition of the information products derived and produced in WP2 and WP3, as well as knowledge of the accuracy and thus of the value of the information is the highest ranking goal of WP6.

Thus, WP6 is based on three main pillars: First, systematic cross-validation aims to statistically validate the methodologies used. This means that, for instance, the influence of the sampling scheme of in-situ measurements and its impact on the mapping result will be analysed and validated. Second, systematic accuracy assessment is accomplished. Therefore, the data from the field campaigns (WP4,WP5) in the test areas will be used. Beyond this reference, data sets derived from high resolution satellite data as well as existing spatial layers from the local stakeholders will be gathered to assess the accuracy of the mapping products as well as the in-situ sensed data. A framework and concept for accuracy assessment to relate spatial as well as punctual ground truth or reference data sets to the automatically derived spatial classifications in WP 2 and 3 will be developed. Third, the remote sensing products developed in WP2 and WP3 will be compared to existing data sets coming from Copernicus(formerly GMES) projects or other relevant spatial products such as those from the OpenStreetMap project. With it, WP6 aims at systematically incorporating its products into the existing operational map products and to evaluate and document the capabilities and limitations of the new base maps derived in SENSUM. Based on these three pillars, WP6 aims to clearly define the remote sensing products derived with respect to thematic quality and accuracy, in order to evaluate the potential integration of the proposed approach and methodologies in the Copernicus service offer. In particular, this will consider a vision of enlarging the Copernicus offer to non-European, data-poor countries.

The results will be provided to WP2 and WP3 for back coupling, to WP7 for communication to the stakeholders, to the scientific community at large, as well as to the end-users from the disaster management community.

The key objectives of this WP are:

  • Gathering and preparing an extensive multi-source spatial reference data set for a data rich and a data poor area.
  • Accuracy assessment of remotely sensed results in SENSUM (outputs of WP4 and WP5), comparison to other available spatial products, and cross-validation of the methodologies used.
  • Definition of the products produced.