Hi @Shashi, @TerryMulligan, At Project ECHO, with over 400 hubs, 850 programs in over 40 countries, we have a fair amount of experience piloting programs, and scaling them across disease areas and geographies in LMIC’s. One of the most important strategies is to engage partners and local experts, so that the cultural context is integrated and the the program is lead by people that are known and trusted.
Impact Evaluation is critical as well, and we use as part of a closed-loop process of continuous improvement. We use a tiered outcome model to assess the impact of telementoring for healthcare and/or other professionals. These levels are common across multiple training/education evaluation frameworks like Moore’s 7-level outcome model and Kirkpatrick’s model.
- First we make sure people participate regularly, engage, and feel like they get something out of the program.
- Then we see if they learned something or are more confident in their knowledge
- Do they intend to use that knowledge to change their behaviors, or ideally, did they actually change some behaviors or practices?
- Ultimately, we want to see if these changes in practice have an impact on patients. Are more patients treated? Do patients spend less time/money and miss less work to get treated? Do patients get better treatments? Do measures of disease severity improve? Are patients more satisfied with their care?
- Finally, we want to see if any of these changes go beyond individual practitioners or patients. For example, do rates of disease burden go down in healthcare facilities or communities where a lot of practitioners participate in ECHO? Is there evidence that best practices are adopted by organizations or systems such as public insurance?
After a specific intervention like ECHO has been shown to be effective, we also use other frameworks to evaluate how well it is being scaled such as the RE-AIM (Reach, Effectiveness, Adoption, Implementation, and Maintenance) framework. This framework helps us think about why certain entities or practitioners do or do not adopt ECHO, and what factors need to be in place for an ECHO program to be sustainable. These frameworks help us assess the public health impact of an intervention. For example, you could have a really effective intervention but it's either difficult to access or very expensive, so it doesn't scale well. The goal is to have the most effective intervention that ALSO can scale at the level of having a public health impact.