Evaluation and Quality Improvement
This page provides tools and resources from ASTDD and CDC, followed by materials from many other sources:
- 2023 ASTDD Evaluation and Technical Assistance Annual Report
- General Evaluation Methods and Tools, including logic models, program evaluation and quality improvement
- Evaluation Hot Topics, such as partnerships and coalitions, communication strategies and plans, collaboration efforts, oral health surveillance systems, oral health state plans, sealant programs, mobile/portable programs, quality improvement, and policy and advocacy efforts
- Using evaluation information to communicate and demonstrate impact and outcomes.
ASTDD Program Review and Technical Assistance on Evaluation Issues
ASTDD offers State Oral Health Program Reviews and many types of technical assistance to state oral health programs. Technical Assistance Request form. ASTDD uses consultants from the Emory Centers for Public Health Training and Technical Assistance Center for Program Evaluation and Quality Improvement to provide evaluation technical assistance to state/territorial oral health programs (S/TOHP.) Their short bios follow. If S/TOHP would like TA from any of these consultants, contact Bev Isman at bev.isman@comcast.net
Dr. Mary Davis MSPH, DrPH is a Lead Evaluation Specialist at Emory Centers where she guides teams to plan, implement, and report on program evaluation and quality improvement initiatives. Dr. Davis has extensive experience with nearly all evaluation designs and data collection approaches, including mixed-method designs. She has trained public health students and practitioners in evaluation methods through workshops, formal courses, technical assistance webinars, and employment opportunities. Further, she has provided evaluation capacity building assistance to organizations, such as coalitions, public health departments, and foundations. In her nearly 30 year career, she has worked in academia at the University of North Carolina, Gillings School of Global Public Health as well a national member association and a for-profit consulting firm. Her areas of expertise include preparedness, health department accreditation, quality improvement, and partnerships.
Maggie Pustinger, MPH, CHES is an Evaluator at Emory Centers and provides evaluation planning and implementation support to S/TOHPs and to the Association. Her professional interests include logic model development, survey design, evaluation capacity building, and process improvement. Maggie is a Certified Health Educator (CHES) and holds an MPH from Emory University.
Kristin-Giordano, MPH is an Evaluation Specialist at Emory Centers where she supports evaluation and quality improvement for multiple initiatives. For over four years, she has managed and contributed to mixed methods evaluation and learning projects for multi-sector partnerships, including work in community health, education, and economic opportunity. She has alos provided evaluation support and evaluation technical assistance for non-profits, foundations, and healthcare organizations. Her professional interests include equity-focused evaluation, effective data visualization, mixed methods approaches, and evaluation capacity building.
ASTDD Evaluation and Quality Improvement Materials
Evaluation of Training Programs: In this webinar, ASTDD evaluation consultants Mary Davis, Rebecca Fils-Aime, and JoAnna Hillman provide an overview of how to create a simple evaluation strategy for a training program. This session will help you understand how to apply assessment and evaluation during various stages of training development and implementation to increase the effectiveness of learning; outcome targets and indicators for training program evaluations; and models for evaluating training programs. (December 2021)
Qualitative Methods Webinar Series (1 of 2): "Getting to the How and the Why." In this first webinar of a two-part series, ASTDD evaluation consultants Mary Davis and JoAnna Hillman provide a complete overview of planning and designing qualitative evaluations. This session will help you understand when would be most appropriate to use a qualitative method for a program or activity's evaluation and offers helpful tips for setting your qualitative evaluation up for success. (February 2021)
Qualitative Methods Webinar Series (2 of 2): "Analysis, Reporting, and Wrap-up". In this second webinar of a two-part series, ASTDD evaluation consultants Mary Davis and JoAnna Hillman provide an overview of analysis and reporting for qualitative evaluations. This session will help you understand the approaches to qualitative analysis, the tools available to support qualitative analysis and how to approach reporting. (March 2021)
Survey Methods: From Design to Implementation: In this webinar, ASTDD evaluation consultants JoAnna Hillman and Mary Davis, and Emory Centers evaluator, Christiana Reene, provide a complete overview of surveys from planning to wrap up. It addresses good survey practices including proper question construction, developing questions that provide data you can use, and how to use logic (branching, skip, display). The session will help you identify common issues and help you to avoid pitfalls. (December 2020)
Evaluating Oral Health Surveillance Systems: In this webinar, ASTDD consultants Kathy Phipps, Mary Davis, and JoAnna Hillman provide guidance on how to evaluate oral health surveillance programs using the CDC evaluation framework. The webinar includes resources and examples from state oral health programs. (September 2020)
ASTDD Performance Management Toolkit for State Oral Health Programs: The purpose of this toolkit is to provide state oral health programs with an orientation to performance management (PM) as well as resources to further learning and competency related to PM. The toolkit grew out of an identified need among Association of State and Territorial Dental Director (ASTDD) members and requests for information and technical assistance. (August 2017)
Adopting Performance Management Strategies to Improve Oral Health in Your State: Performance Management and Quality Improvement are more than just buzz words. Performance Management and Quality Improvement approaches and tools can help state Offices of Oral Health improve their efficiency and effectiveness. This webinar provides an introduction to Performance Management and Quality Improvement approaches and tools as well as examples of how these are used in state Oral Health Offices. (April 2017)
Toolkit for Planning, Evaluating, and Improving Interagency Collaboration for Oral Health Programs: The purpose of the Toolkit is to provide a framework and specific strategies for planning, improving, and evaluating inter-agency collaboration to more effectively and efficiently address our most complicated and complex national oral health issues through strategic alliances and organizational partnerships. The toolkit contains a Handbook, a Workbook, and worksheets.
ASTDD Communications Plan Templates and Resources
Tools for planning and evaluating a single project or document and a year-at-a-glance template for multiple communication strategies.
CDC Evaluation Materials
CDC Program Performance and Evaluation Office-webpage includes CDC Framework for Program Evaluation and multiple resources and background materials on the framework.
CDC Sealant Efficiency Assessment for Locals and States (SEALS): SEALS software helps states and communities evaluate the effectiveness and efficiency of their school dental sealant programs.
Developing an Effective Evaluation Plan: The purpose of this workbook is to help public health program managers, administrators, and evaluators develop an effective evaluation plan in the context of the planning process. It is intended to assist in developing an evaluation plan but is not intended to serve as a complete resource on how to implement program evaluation. Rather, it is intended to be used along with other evaluation resources (2011)
Impact and Value: Telling your Program's Story: This CDC workbook describes how to develop a system for collecting and creating stories that will allow you to best communicate the accomplishments of your programs. (2007)
Practical Strategies for Culturally Competent Evaluation: This guide from the CDC's Division for Heart Disease and Stroke Prevention and the National Asthma Control Program highlights the prominent role of culture in evaluation and provides important strategies for approaching an evaluation with a critical cultural lens to ensure that evaluation efforts have cultural relevance and generate meaningful findings that stakeholders ultimately will value and use.
Success Stories & Writing Resources: Writing resources to create impactful success stories, plus a searchable repository of stories from National Center for Chronic Disease Prevention and Health Promotion (NCCDPHP) grantees.
ASTDD State based Oral Health Surveillance System Conceptual Framework and Operational Definition
CDC Updated Guidelines for Evaluating Public Health Surveillance Systems
Tip Sheet: Types of Evaluation: This tip sheet from CDC's Division of STD Prevention briefly summarizes the most common types of evaluation, differences between process and outcome evaluation, as well as situations in which one would choose the different types of evaluation.
Updated Guidelines for Evaluating Public Health Surveillance Systems: (Article from MMWR, July 27, 2001;50(RR13):1-35.
General Evaluation Methods and Tools
Reversible Decay: Oral Health is a Public Health Problem We Can Solve: This brief report shows how assessment results can be presented in a user-friendly formate to non-technical audiences. More information on the assessment can be found at http://dentaquest.com/oral-health-resources/reversible-decay/
WK Kellogg Foundation Evaluation Handbook: An online handbook that covers all aspects of planning for evaluation, implementing your plan and analyzing the data, as well as communicating findings and using the results.
Reversible Decay: Oral Health Is a Public Health Problem We Can Solve:
This brief report shows how assessment results can be presented in a user-friendly formate to non-technical audiences. More information on the assessment can be found at
Logic Model Development Guide: This guide from the WK Kellogg Foundation provides a step-by-step process to develop and use different types of logic models; it includes many templates, checklists and exercises to develop a model that fits your program.
The Community Tool Box: Chapter 36. Introduction to Evaluation: An introduction to key terms and frameworks in evaluation from the University of Kansas. Includes sections on how to engage community stakeholders in evaluation and working with a professional evaluator.
American Evaluation Association: Find evaluation focused conferences and other professional development opportunities or identify AEA members available for evaluation consulting or to serve on evaluation teams due to specific expertise in particular methodologies.
Western Michigan University, The Evaluation Center, Evaluation Checklists: Checklists for multiple types of evaluation.
Making Data Talk Workbook: The National Cancer Institute's workbook explains how to present scientific and health data to decision-makers and the public in engaging and effective ways. It is based on the book "Making Data Talk: Communicating Public Health Data to the Public, Policy Makers, and the Press."
Evaluation Hot Topics
Equity and Evaluation
Better Evaluation an international resource guide to evaluation processes, management, communications and other resources
Equitable Evaluation Initiative: Resources and framework to conceptualize, implement, and utilize evaluation to promote equity.
Expanding the Bench: Evaluation network that promotes diverse perspectives and experiences that inform evaluation methods to benefit communities.
Partnerships and Coalitions
Evaluating Community Partnerships Resource List
Evaluating Partnerships to Prevent and Manage Chronic Disease
Policy Evaluation
Performance Measurement for Public Health Policy: Joint publication of the American Public Health Association (APHA) and the Public Health Foundation (PHF) from 2012.
Seal America. The Prevention Invention, Step 10, Program Evaluation: Guide for determining school-based dental sealant quality, determining program cost, program participation and other measures.
Quality Improvement
Evaluation, Performance Management, and Quality Improvement: Understanding the Role They Play to Improve Public Health: Presentation that provides a nice overview of how Program Evaluation aligns with Quality Improvement and Performance Management in public health.
Embracing Quality in Public Health: A Practitioner's Quality Improvement Guidebook: This second edition of Embracing Quality in Public Health reflects five years of study, action, and learning among the authors and many other generous, thoughtful individuals who contributed their time and talent to informing this resource. It is a comprehensive resource on planning and implementing QI activities in a health department.
Population Health Improvement Partners. Step by Step QI Guide
NACCHO QI Road Map: The Quality Improvement Roadmap provides health departments with guidance on progressing through six phases of QI maturity. This resource also describes six foundational elements of a QI culture that health departments should cultivate over time.
IHI Public Health Page: The Institute for Healthcare Improvement, in collaboration with the Public Health Foundation, hosts a Public Health Resources page addressing quality improvement, including changes, measures, and tools for improvement that can be used in a public health setting.
Public Health Foundation: The Public Health Foundation provides QI tools, training, resources, and technical assistance to state and local health departments.
Applying Quality Improvement Measurement to Population Health Initiatives: Article in AMCHP PULSE July/August 2016.
Population Health Improvement Partners. Step by Step QI Guide: Walks you through each phase of a QI project and provides examples, tools and templates.
Moving from Goal to Impact: A Quality Improvement Approach to Advancing Children's Oral Health in Medicaid: This CMS Technical Assistance Brief (2015) describes a step-by-step strategic planning approach to help states design effective oral health action plans. It highlights approaches that represent the interests of a diverse set of key stakeholders, are feasible to implement, are measurable, and are likely to achieve the desired impact on children's oral health.
QI Resources for Epidemiologists. AMCHP 2014: Material for this publication originated from a roundtable discussion with epidemiologists about their needs to successfully take part in QI projects.
EQIPP Oral Health Course: This course from the American Academy of Pediatrics delivers all the tools and resources you need to identify and close practice gaps in key clinical oral health activities! This quality improvement course helps you address: performing oral health risk assessment, providing oral health anticipatory guidance and education, ensuring fluoride varnish application for high-risk patients, and establishing a dental home. The course may be taken by pediatricians, nurse practitioners, and allied health care professionals. The cost is $199 for non AAP members.
