Association‌ of‌ State‌ &‌ Territorial‌ Dental‌ Directors
3858‌ Cashill Blvd.,‌ Reno,‌ NV‌ ‌89509
Phone‌ 775-626-5008‌‌

Evaluation and Quality Improvement

This page provides tools and resources from ASTDD and CDC, followed by materials from many other sources:

  • General Evaluation Methods and Tools, including logic models, program evaluation and quality improvement
  • Evaluation Hot Topics, such as partnerships and coalitions, communication strategies and plans, collaboration efforts, oral health surveillance systems, oral health state plans, sealant programs, mobile/portable programs, quality improvement, and policy and advocacy efforts
  • Using evaluation information to communicate and demonstrate impact and outcomes.

ASTDD Program Review and Technical Assistance On Evaluation Issues

ASTDD offers State Oral Health Program Reviews and many types of technical assistance to state oral health programs. Technical Assistance Request form. ASTDD uses two consultants to provide evaluation technical assistance to state oral health programs (SOHP.) Their short bios follow. If SOHP would like TA from either of these consultants, contact Bev Isman at bev.isman@comcast.net

Carissa Beatty, MPH, CHES. In her role as a Senior Program Associate at the Emory Centers for Training and Technical Assistance, Carissa contributes to the development, implementation, and evaluation of training programs and products for the public health workforce. She is the instructor for the Emory Centers' Logic Model Essentials and Fundamentals of Evaluation courses. Other professional interests include data visualization, strategic use of data/information, social innovation, health communication, and collaboration throughout the realm of chronic disease prevention. Carissa is a Certified Health Education Specialist (CHES), and holds an MPH from Emory University. Ask her about: Logic model development, evaluation planning, survey design, data collection, qualitative and quantitative methods / analysis, and reporting to contribute to ongoing data-driven program development and implementation.

Dr. Mary Davis is principal and owner of Project Y Evaluation Services, LLC specializing in public health evaluation and applied research consulting, training, and technical assistance services. Dr. Davis has extensive experience with most evaluation designs and data collection approaches, including mixed-method designs. She has trained public health students and practitioners in evaluation methods through workshops, formal courses, technical assistance webinars, and employment opportunities. Further, she has provided evaluation capacity building assistance to organizations, such as coalitions, public health departments, and foundations. In her more than 20 year career, she has worked in academia at the University of North Carolina, Gillings School of Global Public Health as well a national member association and a for-profit consulting firm. Her areas of expertise include preparedness, health department accreditation, quality improvement, and partnerships.

  • ASTDD Evaluation Materials

ASTDD Performance Management Toolkit for State Oral Health Programs - The purpose of this toolkit is to provide state oral health programs with an orientation to performance management (PM) as well as resources to further learning and competency related to PM. The toolkit grew out of an identified need among Association of State and Territorial Dental Director (ASTDD) members and requests for information and technical assistance. (August 2017)

Adopting Performance Management Strategies to Improve Oral Health in Your State - Performance Management and Quality Improvement are more than just buzz words. Performance Management and Quality Improvement approaches and tools can help state Offices of Oral Health improve their efficiency and effectiveness. This webinar provides an introduction to Performance Management and Quality Improvement approaches and tools as well as examples of how these are used in state Oral Health Offices. (April 2017)
PowerPoint

Partnership Evaluation Webinar This webcast enhances knowledge of key concepts related to partnership evaluation, evaluation tools, and things to consider when undertaking partnership evaluations. B.J. Tatro, PhD addressed topics including when to do partnership evaluation, how to match evaluation questions and evaluation tools to the partnership's stage of development, methods to use, stakeholders to involve, and how to utilize results to strengthen partnerships. Examples from states that are planning or conducting internal and external partnership evaluations were used and resources were shared. (February 2015)
Partnership Evaluation Webinar PowerPoint

Speaker Biographies
PowerPoint presentation
PowerPoint presentation with Speaker Notes
ASTDD Guidelines Matrix
Oral Health Evaluator Competency Chart
References

Toolkit for Planning, Evaluating, and Improving Interagency Collaboration for Oral Health Programs

The purpose of the Toolkit is to provide a framework and specific strategies for planning, improving, and evaluating inter-agency collaboration to more effectively and efficiently address our most complicated and complex national oral health issues through strategic alliances and organizational partnerships. The toolkit contains a Handbook, a Workbook, and worksheets.

ASTDD Communications Plan Templates and Resources

Tools for planning and evaluating a single project or document and a year-at-a-glance template for multiple communication strategies.

Mobile-Portable Dental Manual
Chapter 5 in this online manual provides an overview of evaluation strategies to measure effectiveness and outcomes for mobile and portable dental programs, with links to a number of resource materials.

CDC Evaluation Materials

CDC Framework for Program Evaluation
Article from MMWR 1999;48(No. RR-11).

Logic Model for a State Oral Health Plan
Conceptual Model of Comprehensive Oral Health State Plan Process

CDC Introduction to Program Evaluation for Public Health Programs: A Self-Study Guide This "how to" guide for planning and implementing evaluation activities is based on CDC's Framework for Program Evaluation in Public Health. It uses federal, state and local examples.

Updated Guidelines for Evaluating Public Health Surveillance Systems
(Article from MMWR, July 27, 2001;50(RR13):1-35.

CDC Webcast on Evaluation of State Plans
This PowerPoint was presented by Paul W. Mattessich, PhD during a 2005 CDC webcast.

Impact and Value: Telling your Program's Story
This CDC workbook describes how to develop a system for collecting and creating stories that will allow you to best communicate the accomplishments of your programs. (2007)

Tip Sheet: Types of Evaluation
This tip sheet from CDC's Division of STD Prevention briefly summarizes the most common types of evaluation, differences between process and outcome evaluation, as well as situations in which one would choose the different types of evaluation.

Success Stories & Writing Resources
Writing resources to create impactful success stories, plus a searchable repository of stories from National Center for Chronic Disease Prevention and Health Promotion (NCCDPHP) grantees.

CDC Oral Health State Plan Review Index
A tool to help evaluate an oral health state plan.

CDC Sealant Efficiency Assessment for Locals and States (SEALS)
SEALS software helps states and communities evaluate the effectiveness and efficiency of their school dental sealant programs.

Practical Strategies for Culturally Competent Evaluation
This guide from the CDC's Division for Heart Disease and Stroke Prevention and the National Asthma Control Program highlights the prominent role of culture in evaluation and provides important strategies for approaching an evaluation with a critical cultural lens to ensure that evaluation efforts have cultural relevance and generate meaningful findings that stakeholders ultimately will value and use.

General Evaluation Methods and Tools

WK Kellogg Foundation Evaluation Handbook
An online handbook that covers all aspects of planning for evaluation, implementing your plan and analyzing the data, as well as communicating findings and using the results.

Community-Based Oral Health Programs: A Need and Plan for Evaluation
A 20-page document from the WK Kellogg Foundation that provides a plan and parameters for developing an Oral Health Program evaluation.

Logic Model Development Guide
This guide from the WK Kellogg Foundation provides a step-by-step process to develop and use different types of logic models; it includes many templates, checklists and exercises to develop a model that fits your program.

The Community Tool Box:  Chapter 36. Introduction to Evaluation
An introduction to key terms and frameworks in evaluation from the University of Kansas. Includes sections on how to engage community stakeholders in evaluation and working with a professional evaluator.

American Evaluation Association
Find evaluation focused conferences and other professional development opportunities, or identify AEA members available for evaluation consulting or to serve on evaluation teams due to specific expertise in particular methodologies.

Western Michigan University, The Evaluation Center, Evaluation Checklists
Checklists for multiple types of evaluation.

Making Data Talk Workbook
The National Cancer Institute's workbook explains how to present scientific and health data to decision-makers and the public in engaging and effective ways. It is based on the book "Making Data Talk: Communicating Public Health Data to the Public, Policy Makers, and the Press."

Evaluation Hot Topics

Partnerships and Coalitions

Evaluating Community Partnerships Resource List

Resource list compiled by Public Health Ontario, Partners for Health.

Butterfoss FD. Evaluating partnerships to prevent and manage chronic disease. Prev Chronic Dis 2009;6(2):A64

Policy Evaluation

Evaluating Advocacy and Policy Change Efforts
Articles from the Evaluation Exchange, 2007.

Performance Measurement for Public Health Policy
Joint publication of the American Public Health Association (APHA) and the Public Health Foundation (PHF) from 2012.

Chriqui, J., O'Connor, J., and Chaloupka, F. (Spring 2011). What Gets Measured, Gets Changed: Evaluating Law and Policy for Maximum Impact. In Using Law, Policy, and Research to Improve the Public's Health

Seal America. The Prevention Invention, Step 10, Program Evaluation
Good guide for determining school-based dental sealant quality, determining program cost, program participation and other measures.

Quality Improvement

Adopting Performance Management Strategies to Improve Oral Health in Your State - Performance Management and Quality Improvement are more than just buzz words. Performance Management and Quality Improvement approaches and tools can help state Offices of Oral Health improve their efficiency and effectiveness. This webinar provides an introduction to Performance Management and Quality Improvement approaches and tools as well as examples of how these are used in state Oral Health Offices. (April 2017)
PowerPoint

Evaluation, Performance Management, and Quality Improvement: Understanding the Role They Play to Improve Public Health; Craig Thomas, PhD Liza Corso, MPA Harald Pietz
Presentation that provides a nice overview of how Program Evaluation aligns with Quality Improvement and Performance Management in public health.

Embracing Quality in Public Health: A Practitioner's Quality Improvement Guidebook
This second edition of Embracing Quality in Public Health reflects five years of study, action, and learning among the authors and many other generous, thoughtful individuals who contributed their time and talent to informing this resource. It is a comprehensive resource on planning and implementing QI activities in a health department.

NACCHO QI Road Map
The Quality Improvement Roadmap provides health departments with guidance on progressing through six phases of QI maturity. This resource also describes six foundational elements of a QI culture that health departments should cultivate over time.

Public Health Quality Improvement Exchange
The Public Health Quality Improvement Exchange (PHQIX) is an online community designed to be a communication hub for public health professionals interested in learning and sharing information about quality improvement in public health. It includes an online, searchable database of QI projects in public health as well and provides opportunity for dialogue and learning among users.

IHI Public Health Page

The Institute for Healthcare Improvement, in collaboration with the Public Health Foundation, hosts a Public Health Resources page addressing quality improvement, including changes, measures, and tools for improvement that can be used in a public health setting.

Public Health Foundation
The Public Health Foundation provides QI tools, training, resources, and technical assistance to state and local health departments.

Applying Quality Improvement Measurement to Population Health Initiatives
Article in AMCHP PULSE July/August 2016.

Population Health Improvement Partners. Step by Step QI Guide
Walks you through each phase of a QI project and provides examples, tools and templates.

Moving from Goal to Impact: A Quality Improvement Approach to Advancing Children's Oral Health in Medicaid
This CMS Technical Assistance Brief (2015) describes a step-by-step strategic planning approach to help states design effective oral health action plans. It highlights approaches that represent the interests of a diverse set of key stakeholders, are feasible to implement, are measurable, and are likely to achieve the desired impact on children's oral health.

QI Resources for Epidemiologists. AMCHP 2014
Material for this publication originated from a roundtable discussion with epidemiologists about their needs to successfully take part in QI projects.

EQIPP Oral Health Course
This course from the American Academy of Pediatrics delivers all the tools and resources you need to identify and close practice gaps in key clinical oral health activities! This quality improvement course helps you address: performing oral health risk assessment, providing oral health anticipatory guidance and education, ensuring fluoride varnish application for high-risk patients, and establishing a dental home. The course may be taken by pediatricians, nurse practitioners, and allied health care professionals. The cost is $199 for non AAP members.

 border=