AHRQ Updated Public Access Plan:

This document is the Agency for Healthcare Research and Quality (AHRQ) plan for establishing an updated policy for public access to scientific publications and scientific data in digital format resulting from AHRQ funding.

1. Background & Purpose

2. Scope

3. Requirements

4. Definitions

5. Applicability

6. Legal Authorities

7. Roles and Responsibilities

8. Implementation

9. Metrics, Compliance, and Evaluation

10. Timeline

11. Resources

12. Public Consultation

13. Interagency Coordination

14. Public Notice

15. Update and Re-Evaluation of the Policy

16. Additional Information

17. Material Superseded

1. Background & Purpose

On February 22, 2013, the White House Office of Science and Technology Policy (OSTP) released the memorandum entitled "Increasing Access to the Results of Federally Funded Scientific Research." This memorandum requires federal agencies to make the results of federally funded scientific research available to and useful for the public, industry, and the scientific community. In response, in February 2015, AHRQ published a plan for establishing a policy for public access to scientific publications and scientific data in digital format resulting from AHRQ funding, in February 2016 published a Policy for Public Access to AHRQ funded Scientific Publications (NOT-HS-16-008), and in May 2020, published a Data Management Plan Policy (NOT-HS-20-011). 

On August 25, 2022, the White House Office of Science and Technology Policy (OSTP) released the memorandum entitled “Ensuring Free, Immediate, and Equitable Access to Federally Funded Research” (2022 Memo).  The 2022 Memo provides policy guidance to federal agencies with research and development expenditures on updating their public access policies. In accordance with this memorandum, OSTP recommends that federal agencies, to the extent consistent with applicable law:

1. Update their public access policies as soon as possible, and no later than December 31, 2025, to make publications and their supporting data resulting from federally funded research publicly accessible without an embargo on their free and public release;

2. Establish transparent procedures that ensure scientific and research integrity is maintained in public access policies; and,

3. Coordinate with OSTP to ensure equitable delivery of federally funded research results and data.

This document is the AHRQ updated plan for updating policies for public access to scientific publications and scientific data in digital format resulting from AHRQ funding pursuant to recommendations from the 2022 Memo. AHRQ intends, to the fullest extent possible and contingent on the availability of funds, to make available to the public all scientific publications and data arising from unclassified research and programs funded wholly or in part by AHRQ.

The AHRQ mission is to produce evidence that makes health care safer, higher quality, more accessible, equitable and affordable, and to work within the U.S. Department of Health and Human Services (HHS) and with other partners to make sure the evidence is understood and used.

The AHRQ Public Access Policy promotes the following objectives:

• Establish AHRQ's commitment to public access to scientific research results.
• Ensure that the public can read, download, and analyze in digital format final published documents.
• Facilitate easy public search, analysis of and access to peer-reviewed scholarly publications directly arising from research funded by AHRQ.
• Ensure full public access to publications' metadata without charge.
• Ensure the attributes to authors, journals, and original publishers are maintained.
• Ensure that publications and metadata are in an archival solution.
• Ensure that all extramural researchers receiving AHRQ grants, cooperative agreements, and contracts for scientific research, and intramural researchers develop data management plans, as appropriate, describing how they will provide for long-term preservation of and access to scientific data in digital format.
• Ensure the expedited translation of research results into knowledge, products, and procedures to improve health.
• Facilitate the education of new researchers and enable the exploration of topics not envisioned by the initial investigators.
• Permit the creation of new datasets when data from multiple sources are combined.
• Establish transparent procedures that ensure scientific and research integrity is maintained in public access policies; and
• Coordinate with OSTP to ensure equitable delivery of federally funded research results and data.

2. Scope

The updated AHRQ Public Access Policy will be fully effective by December 31, 2025, with the policy implementing Scientific and Research Integrity in Agency Public Access Policies by December 31, 2026. The Public Access Policy will apply to all scientific publications and data in digital format. Implementation will be prospective and will not apply to any publication or digital data set arising from an AHRQ-sponsored grant, cooperative agreement, contract, or intramural research project funded prior to publication of the final updated AHRQ Public Access Policy.

The updated AHRQ Public Access Policy will apply to all research funded by AHRQ, except where such research is administered or performed by a partner Agency with a comparable Public Access Policy, in which case AHRQ will defer to the partner Agency's policies on the management of scholarly publications and digital data sets.

Digital scientific data that are covered by this policy include:

• Field data.
• Lab data.
• Other data (e.g., quality control samples, sample ID data, and instrument calibration data).
• Digital scientific, clinical, or institutional data that can be in scope at the discretion of the  program or in an appropriate context:
  • Models and model-related content, including parameters and outputs, including models of public health emergencies.
  • Command files, such as SAS or STATA files created from secondary sources (typically referred to as secondary or outside data).
  • De-identified Electronic Health Record (EHR) patient treatment records.
• Non-proprietary records and data collected as part of the National Hospital Preparedness Program.
• Digital scientific data that are not in scope for this policy include:
  • Personally identifiable data, however, AHRQ will attempt to make all research data available to the public by creating de-identified public use data files.
  • Proprietary trade data.
  • Data related to protecting critical infrastructure.
  • Other data whose release is limited by law, regulation, security requirements, or policy.

Per OMB Circular A-110, the following are not research data and therefore not subject to this policy: preliminary analyses, drafts of scientific papers, plans for future research, peer reviews, communications with colleagues, or physical objects, such as laboratory specimens. However, investigators are encouraged to include relevant content from laboratory or field notebooks in their published data sets if that information provides documentation that would help future users successfully re-use the data. (That is, lab and field notebooks are not themselves scientific data, but may contain relevant metadata to be included in the published data set.).

3. Requirements

Publications

For scholarly publications, the AHRQ Public Access Policy has required and will continue requiring authors to submit the final peer-reviewed accepted journal manuscripts to PubMed Central (PMC). In lieu of the final peer-reviewed manuscript, AHRQ has and will continue accepting the final published article, provided the authors can ensure AHRQ has the rights to make the published version public. AHRQ's Public Access Policy is subject to law; Agency mission; resource constraints; U.S. national, homeland, and economic security; and the objectives listed in the OSTP directive.

Scientific data

Scientific data underlying peer-reviewed scholarly publications resulting from federally funded research should be made freely available and publicly accessible by default at the time of publication, unless subject to limitations as described in Section 3(c)(i) of the August 25, 2022 OSTP memo and should be subject to federal agency guidelines for researcher responsibilities regarding data management and sharing plans, consistent with Section 3(c) of the memorandum.

Scientific Data include the recorded factual material commonly accepted in the scientific community as of sufficient quality to validate and replicate research findings. Such scientific data do not include laboratory notebooks, preliminary analyses, case report forms, drafts of scientific papers, plans for future research, peer-reviews, communications with colleagues, or physical objects and materials, such as laboratory specimens, artifacts, or field notes. The definition of “scientific data” is similar to but broader than the term “research data” defined by 2 CFR 200.315 (e) and 45 CFR 75.322 (e).

AHRQ funded researchers must follow federal laws and OMB policies that govern federal agencies’ information management practices and protect certain types of data, to the extent that the scientific data created by, collected by, under the control or direction of, or maintained by the federal researchers is subject to those laws and policies.

To the extent feasible and consistent with applicable law and policy; agency mission; resource constraints; U.S. national, homeland, and economic security; and the objectives listed below, digitally formatted scientific data resulting from unclassified research supported wholly or in part by federal funding should be stored and publicly accessible to search, retrieve, and analyze. For sharing of data in digital format, all AHRQ-funded researchers have been and will continue to be required to include a data management plan for sharing final research data in digital format, or state why data sharing is not possible. Data management plans will be evaluated by AHRQ staff and peer review committees for merit while considering the values of long-term preservation, access, and the associated cost, and administrative burden. AHRQ expects the timely release and sharing of data to be no later than the acceptance for publication of the main findings from the final dataset. The specific time will be influenced by the nature of the data collected. Researchers will be directed to work with AHRQ staff and commercial repositories to deposit data upon publication. It may not be feasible to share all data; the costs and benefits of sharing data should be considered in data management planning. AHRQ intramural research is conducted using public use data. These data are also available to the public. AHRQ data that include personally identifiable data are available to the public in the AHRQ data center. AHRQ will attempt to make all intramural research data available to the public by creating de-identified public use data files.

If an AHRQ-funded grant, cooperative agreement, contract or intramural research project transforms or links datasets, rather than producing a new set of data, the researchers will be required to submit a data management plan. If there are limitations associated with the data sharing agreements for the original data that preclude subsequent sharing, the research applicants must explain these limitations.

Data from small studies can be analyzed and submitted for publication relatively quickly. If data from large epidemiologic or longitudinal studies are collected over several discrete time periods or waves, it is reasonable to expect that the data would be released in waves as data become available or main findings from waves of the data are published. AHRQ-funded extramural and intramural researchers will be expected to include updates on public access to publications and data in digital format resulting from their research in quarterly reports, where applicable, annual progress reports, final reports, and via NLM's Bibliography tool, My NCBI. All new grant and cooperative agreement applications, and contract proposals will include a section detailing the applicants' progress to-date with sharing of publication and data in digital format arising from previous AHRQ funding.

Scientific and Research Integrity in Agency Public Access Policies

AHRQ will take actions to ensure that scientific and research integrity policies are in place in order to strengthen public trust in federally funded science. By December 31, 2024, AHRQ plans to submit to OSTP and OMB a second update to their public access plans specifying approaches taken to implement the provisions in this Section 4 of the 2022 memo. AHRQ plans to complete and publish full policy development for plans implementing these provisions by December 31, 2026, with an effective date no later than one year after the publication of the agency plan.

In the plan to be submitted to OSTP and OMB by December 31, 2024, AHRQ will discuss and explore how AHRQ plans to act to achieve the goals specified and approaches recommended in the OSTP memo.

White House Office of Management and Budget Open Data Policy Memorandum (M-13-13)

AHRQ will meet the requirements of M-13-13 as specified in the accompanying memorandum, titled HHS Guiding Principles and Common Approach for Enhancing Public Access. AHRQ will develop and maintain an AHRQ Enterprise Data Inventory for the data sets related to their major Information Technology investments. AHRQ will develop and maintain an AHRQ Enterprise Data Inventory, cataloguing the various sets of data acquired or managed. The inventory will catalog both data sets that are made publicly available and data sets that are private to the Agency.

4. Definitions

Scientific Data: “scientific data” include the recorded factual material commonly accepted in the scientific community as of sufficient quality to validate and replicate research findings. Such scientific data do not include laboratory notebooks, preliminary analyses, case report forms, drafts of scientific papers, plans for future research, peer-reviews, communications with colleagues, or physical objects and materials, such as laboratory specimens, artifacts, or field notes. The definition of “scientific data” is similar to but broader than the term “research data” defined by 2 CFR 200.315 (e) and 45 CFR 75.322 (e).

Research Data: Research data is defined in OMB Circular A-110 as the recorded factual material commonly accepted in the scientific community as necessary to validate research findings, but not any of the following: preliminary analyses, drafts of scientific papers, plans for future research, peer reviews, or communications with colleagues. This 'recorded' material excludes physical objects (e.g., laboratory samples).

Research data do not include:

• Trade secrets, commercial information, materials necessary to be held confidential by a researcher until they are published, or similar information which is protected under law.
• Personnel and medical information and similar information, the disclosure of which would constitute a clearly unwarranted invasion of personal privacy, such as information that could be used to identify a particular person in a research study.

Digital Scientific Data: For the purpose of this plan and consistent with OMB Circular A-110, digital scientific data is defined as "the digital recorded factual material commonly accepted in the scientific community as necessary to validate research findings including data sets used to support scholarly publications, but does not include laboratory notebooks, preliminary analyses, drafts of scientific papers, plans for future research, peer reviews, communications with colleagues, or physical objects, such as laboratory specimens."

For the purpose of this plan, the definition of digital scientific data does not include software. However, AHRQ recognizes that in some cases, software and other tools such as interview protocols, measures, coding guides, or manuals may be necessary to interpret data. In such cases, the data management plan will be expected to include a description of these tools.

A published data set consists of at least one formal metadata document, the digital scientific data described by that metadata, and supplemental information provided to assist the data user. The metadata for scientific data will include, at a minimum, the common core metadata schema in use by the Federal Government, found at https://project-open-data.cio.gov/.

A published data set can be cited in the scientific literature and, following the practice used by most publishers of scholarly journals, has a persistent and unique identifier associated with it. A published data set is expected to persist over time, just like the scholarly articles that are based on the digital scientific data.

Data Sharing Plan: A data sharing plan outlines whether and how data will be made available to others. The plan must also explicitly describe how the data that underlies scientific publications will be available for discovery, retrieval, and analysis. It may include the expected timeline for when the data will be available, the format of the final dataset, the documentation and any analytic tools that will be provided, and the mode of data sharing (e.g., under their own auspices by mailing a disk or posting data on their institutional or personal website, through a data archive or enclave). A plan might also specify whether or not a data sharing agreement will be required and, if so, a brief description of such an agreement (including the criteria for deciding who can receive the data and whether or not any conditions will be placed on their use). If data sharing is not possible, the plan would provide an explanation.

Data Management Plan: Data management plans are more comprehensive than data sharing plans in that they include additional elements such as descriptions of the data to be produced in the proposed study, any standards to be used for collected data and metadata, mechanisms for providing access to and sharing of the data (including provisions for protection of privacy, confidentiality, security, intellectual property, or other rights), provisions for reuse and redistribution, and plans for archiving and long-term preservation of the data, or explaining why long-term preservation and access cannot be justified.

Final peer-reviewed manuscript: A final peer-reviewed manuscript is defined as an author's final manuscript of a peer-reviewed paper accepted for journal publication, including all modifications from the peer-review process.

Final published article: A final published article is defined as a publisher's authoritative copy of the paper, including all modifications from the publishing peer-review process, copy editing, stylistic edits, and formatting changes.

5. Applicability

The AHRQ Public Access Policy will apply to the