Undertaking Evaluation
Rigorous evaluation is a central tenet of the Foundation’s approach to reducing the harmful use of alcohol around the world. The Foundation supports researchers to measure, monitor, and evaluate progress toward reducing the harmful use of alcohol. The Foundation implements a transparent approach to data sharing in order to maximize the utility of the data.
Evaluation Approach
The Foundation has contracted with HBSA, a supporting organization of the Pacific Institute for Research and Evaluation (PIRE), to oversee the measurement and evaluation of the Global Smart Drinking Goals (GSDG) City Pilot program. HBSA is responsible for:
- Developing the measurement approach and evaluation design for the City Pilot program,
- Measuring progress toward outcomes of reducing the harmful use of alcohol in the adult populations, and alcohol use per se in the youth populations, of each of the six City Pilots,
- Managing the execution of the Data Sharing Policy as it applies to scientific partners,
- Managing the Data Warehouse.
HBSA has developed an overarching evaluation design that it is applying across City Pilots and their respective interventions. Other researchers, including HBSA, are also conducting focused evaluations designed to assess the effects of promising prevention and early intervention strategies included in the City Pilots program. HBSA urges interested researchers to read a manuscript that describes the design of its evaluation, which may be downloaded here.
Evaluation metrics
The primary outcome measure for the evaluation of the City Pilot program is years of healthy life (YHL). YHL is a composite measure that represents the number of years that a person is expected to continue to live in a healthy condition1.
The secondary outcome measure is per capita consumption. The World Health Organization (WHO) calculates per capita consumption by dividing a city’s total consumption by its population. This is the primary manner in which alcohol harm is quantified across the globe, and is the most widely used metric for estimating progress towards meeting Sustainable Development Goal 3.5.2, which is to reduce harmful use of alcohol by 10 percent by 2025.
Ensuring Information and Data Access
The Foundation has developed a set of policies and guidelines to ensure that the scientific data and information shared as part of this program are appropriately protected, shared, and used.
The Foundation’s Data Sharing Policy specifies that the data gathered for its projects be stored in a data warehouse that is managed by the evaluation contractor, HBSA. Qualified researchers can access these data sets upon written requests submitted to HBSA with the expectation that the data sets requested will be used to contribute to the knowledge base concerning strategies designed to reduce the harmful use of alcohol. Data sets may also be requested so that researchers can seek to replicate the findings reported by HBSA and other scientists working on the program. Survey data, code books, and basic table sets for the period 2016 to 2018 are now available upon request.
With its emphasis on protecting the confidentiality of research participants and the integrity of program data, the Data Sharing Policy provides important information for users of these data, including specifications for data sharing and transfer, guidance on data protection and management best practices, considerations when engaging third parties, and requirements for post-program data retention.
The Foundation’s Publication Guidelines provide a detailed discussion of its expectations of standards that authors uphold and the principles to which the Foundation is committed in disseminating data from the program. The purpose of the Guidelines is to ensure a transparent process of data sharing and communication of the results derived from the analyses of program data to advance scientific knowledge and to inform efforts to reduce the harmful use of alcohol. Further supporting high standards in communicating information about the program are the values adopted in the academic peer-review process, including timely publication, as well as full disclosure of any support and funding provided by the Foundation and the role Foundation staff play in the process of publication development, if any. In addition, the Foundation is committed to the highest ethical standards both in the conduct of program research involving human subjects and in the publication process.
HBSA will be launching an external data warehouse website in early 2020. Until then, please reach out to paschall@prev.org for access to data.
1. Murray, A. Lopez. The Global Burden of Disease Harvard University Press, Cambridge, MA (1996) Google Scholar View in article