Spotlight On: Monitoring

Abt Associates offers its clients around the world deep expertise in monitoring. It’s a key component of our rigorous approach to performance management. It’s how we achieve the best results, time and again. Whether we’re creating a theory of change, measuring program performance or designing a monitoring and evaluation (M&E) system for complex data management, we ensure that high quality data is generated to answer important questions around progress toward intended results. Abt’s approach to monitoring enables decision makers to reflect, learn and adapt over time.

Methods

Our M&E experts employ a wide range of qualitative and quantitative methods to collect and manage actionable information on performance monitoring, context monitoring and complexity awareness monitoring. Using a range of approaches—including, but not limited to, routine monitoring methods, rapid feedback studies, client and community surveys, outcome harvesting and data analytics—we design monitoring efforts so our clients can learn and apply adaptive management. We collect, interpret and use data related to program implementation, reach, relevance, effectiveness, efficiency and sustainability, all of which enables us to enhance our impact.

Relevant Expertise
 

Investing in Women Initiative
Client: Department of Foreign Affairs and Trade (DFAT), Australia

The Investing in Women Initiative is an Australian Government flagship program that seeks to ensure women across Southeast Asia are empowered to make their own economic decisions. As the project manager, Abt has developed a rigorous monitoring, evaluation and learning (MEL) system for adaptive management. Our approach is designed to:

• Assess whether the project’s results are meeting expectations
• Improve project performance by providing contextual insights about the opportunities for—and barriers to—women’s economic empowerment
• Build the evidence base to gain private sector and government acceptance of the importance of women’s economic empowerment.

Our current MEL framework tracks a range of complex and transformative change processes, spanning social awareness and attitudes, policy reforms, formal and informal institutional change within private sector organizations, market responses to the program’s work and improvements in women’s access to resources. We use a variety of methods in addition to routine monitoring tools, such as comparative organizational case studies, rapid feedback studies, media monitoring and secondary context analysis. Abt also uses process tracing and qualitative comparative analyses to determine what factors contribute to higher-level outcomes.

Scaling Up Family Planning II (SUFPII) in Zambia
Client: Department for International Development (DFID)

In 2017, the Abt-led SUFPII program undertook a rapid cycle learning (RCL) activity in the Western and Central provinces of Zambia. The objective was to develop a method to more quickly track progress toward family planning (FP) uptake goals to ensure that findings could be used to improve implementation and national family planning programming. The RCL activity used quantitative and qualitative components to monitor SUFPII’s progress toward FP uptake goals by looking at associated measures that changed.

We found that project interventions—specifically, training for community based distributors in counselling and provision of Depo Provera—and training for facility-based providers in Depo Provera are associated with higher numbers of depo users and revisits. Our methodology allowed us to share the findings in real time, which enabled SUFPII to improve its implementation of the program and the Zambian government to improve its FP activities. The team presented its findings on which approaches are cost-effective at the district and provincial level to guide FP policy and planning.

Surveys To Monitor Influenza Vaccine Coverage Among Pregnant Women and Healthcare Personnel
Client: Centers for Disease Control (CDC)

Annual influenza vaccination has been recommended for pregnant women and health care personnel for decades. Vaccination coverage in both groups has historically been below national targets. To measure coverage, Abt developed and implemented web panel surveys at the midpoint and near the end of the 2014-15 through 2017-18 influenza seasons. Data were cleaned and submitted to CDC in a quick and efficient manner in order to inform CDC presentations of mid- and end-of-season estimates, as well as the next season’s prevention efforts. This project builds on Abt’s Surveys To Monitor Influenza Vaccination Coverage Among Pregnant Women During the 2010-11 Influenza Season and Surveys To Monitor Influenza Vaccination Coverage Among Health-Care Personnel and Pregnant Women During the 2011-12 Influenza Season projects.

Home Health Value-Based Purchasing (HHVBP)
Client: Center for Medicare & Medicaid Services (CMS)

Abt implements the HHVBP implementation and monitoring contract to monitor the performance quality of state-level home health agencies (HHAs). These performance measures are used to determine incentive payments to those that meet performance standards. This is part of Abt’s portfolio of work supporting the CMS home health payment system as a whole. The goals of HHVBP include:

• Encouraging HHAs to provide better quality care with greater efficiency.
• Enhancing current public reporting processes.
• Studying quality and efficiency measures that could be applicable in the home health setting.

Technical Assistance and Support for Global HIV/AIDS Monitoring and Evaluation Activities
Client: Division of Global HIV and TB (DGHT), CDC

Abt is supporting CDC DGHT initiatives with planning and M&E services for related global HIV/AIDS activities and projects. Abt helps build the capacity of in-country CDC staff to better conduct M&E and provide the strategic planning for their programs that are funded by the President’s Emergency Plan for AIDS Relief (PEPFAR). Additionally, Abt responds to requests from DGHT headquarters, regions and country offices for strategic information and technical support in the development, review and analysis of interagency Country Operational Plans, Semi-Annual Program Results (SAPR) reports and Annual Program Results (APR) reports.

Abt supports CDC by:

• Developing multiple country M&E plans to assess national HIV/AIDS strategies.
• Supporting routine PEPFAR reporting requirements, including data collection and data quality assessments by CDC country teams, and providing training on M&E topics such as data triangulation and analysis of HIV/AIDS data.
• Helping HIV clinical- and community-based service providers use data to create more equitable and efficient M&E and health systems that address consumer needs.

Rapid Feedback Monitoring, Evaluation, Research and Learning (RF MERL)
Client: United States Agency for International Development (USAID)

Teaming with Results for Development, Abt is part of a consortium that developed and is now implementing the RF MERL protocol for USAID. RF MERL transforms traditional MEL methods from summative accountability tools into ongoing learning tools that provide rigorous, early project results to identify what is working and what is not, allowing for timely learning and adaptive management. The results are achieved through “feedback experiments,” which are short-duration impact evaluations that compare multiple cutting-edge solutions targeting the same intended outcomes. Through feedback experiments, the RF MERL consortium determines which activities or solutions offer the most value for money early on to allow those activities to be scaled up within program cycles.