Fifth National Climate Assessment
APPENDIX 2. Information Quality

The National Climate Assessment (NCA) is required by law to meet the highest information quality standards set by the Federal Government. Specific information quality requirements are set out by the Information Quality Act (IQA),¹ the Office of Management and Budget (OMB) Final Information Quality Bulletin for Peer Review for highly influential scientific assessments (HISA),² and the Foundations for Evidence-Based Policymaking Act (EBPA, or Evidence Act 2018).³ Because NOAA is the administrative agency for NCA5, its agency guidelines under these federal standards serve as the guiding principles for the Assessment. Information quality compliance is affirmed by the NCA Information Quality Officer, furnished by the administrative lead agency.

The IQA ensures that the Assessment data and information are of a sufficient quality by meeting standards of objectivity, utility, and integrity. This includes the requirement for information in the Assessment to demonstrate a capability of being reasonably reproduced. For analytic data, reproducibility means that independent analysis of the original or supporting data using identical methods would generate similar analytic results to an acceptable degree of precision.

The OMB Final Information Quality Bulletin for Peer Review establishes guidance to enhance the practice of peer review of government science documents in order to increase the quality and credibility of scientific information generated across the Federal Government. The bulletin sets the minimum peer-review requirements for documents with the HISA designation, including the National Climate Assessment.

The Evidence Act expands on prior open government policy initiatives and public access to agency data assets. It requires that all federal data assets and supporting information be open by default, distributed under an open license, and published as machine-readable—subject to legal exemptions for privacy sensitivities and/or intellectual property rights.

Sources (or source materials) broadly include all information used in the Assessment. Authors must consider information quality when deciding whether to use a given source in a chapter. Information quality standards must be met any time a source is associated with the Assessment, including incorporating the source into the chapter, citing the source, or linking to the source.

Authors compiled information sources from peer-reviewed literature, other literature, Indigenous Knowledge, and other expert and local knowledge. Authors were provided with additional Federal guidance on use of Indigenous Knowledge.⁴ A fifth type of information that may be used in the NCA is climate data processed and prepared for authors by NOAA’s Technical Support Unit (TSU; App. 3). Authors were advised to evaluate each source based on the following attributes:

• Applicability and Utility: The source is important, relevant, and useful for its intended audience (not only from the author’s perspective but also from the public’s).

• Transparency and Traceability: The source material, the methods supporting conclusions, and the evaluation of the source are documented and clear.

• Objectivity: The purpose, methods used to create the source material, presentation, substance, and interpretation of conclusions are clear, accurate, reliable, and unbiased.

• Integrity and Security: The source material will remain reasonably protected and intact over time, and both the information and the owners of the information are respected.

• Reproducibility: Procedures surrounding source materials are documented such that they can be reproduced, with checks for robustness on non-reproducible data.

While development of an NCA chapter often involves a comprehensive literature review, not every source evaluated in the report development process will be cited in the NCA chapters. An important responsibility of the authors is to determine which sources are most useful to include in the Assessment. References cited in the Assessment are those that are highly relevant and critical to the understanding of the intended users or audience of the Assessment.

To assist in source evaluation, authors were provided Information Quality Decision Pathways to determine whether a source met these standards and could be cited in the NCA. Additional source information came from previous Assessments and technical input reports; feedback from 34 public engagement workshops (App. 1) and other engagement events; expert awareness of the literature from authors; chapter-specific submissions of technical resources and relevant literature to author teams; and a public request for scientific and technical inputs. Information submissions were requested via a Federal Register Notice published by NASA on behalf of the US Global Change Research Program in 2020.⁵ This notice called for the public to submit “relevant scientific and/or technical research studies—including observed, modeled, and/or projected global change and climate science information, as well as societal drivers, vulnerability, impacts, and responses.” The public call specifically encouraged submissions of regional information and information for cross-cutting or new topics since NCA4. This public call for information was open from October 15, 2020, through January 31, 2023.

The final report cites approximately 8,300 sources meeting the IQA requirements.

In addition to being evaluated on the attributes listed in the Source Materials section above, information based on data (including figures) must adhere to the standards of reproducibility and openness. To meet these requirements, NCA5 collected figure and data sources; dataset, analysis, and visualization methods; and hardware and software metadata (details about the underlying data and information), adhering to NOAA’s implementation of ISO-19115 documentation standards.⁶ All metadata is made openly accessible as a component of the final report and demonstrates NCA5 compliance with information quality standards.