The Importance of Strategic Tech Upgrades for EduFinance Program Impact
Quality data collection and analysis is essential for any robust monitoring and evaluation framework. It crucially indicates whether a program intervention is being implemented as intended well as whether implementation is having the intended outcomes.
EduFinance puts data collection at the center of our program design to ensure we are collecting the right information to provide feedback loops for meaningful, iterative model improvements and program implementation. As our program has grown in scale we have adapted by identifying and integrating new data capture processes and tools that are best aligned with our program-needs.
Today there are a growing number of digital data capture solutions available for organizations to consider. It is both an opportunity and a challenge to identify the most well-aligned, user-friendly tools that best fit the context, connectivity and digital literacy levels of the data capture teams. As this is a decision many organizations face across multiple sectors, we wanted to share EduFinance’s journey of digital data capture. Hope Carpenter, Marketing and Communications Associate, sat down with the EduFinance Data and Business Intelligence (BI) team to learn more.
Introduction to the Data and BI Team
Hope: As EduFinance has grown and developed, each of you have played important complimentary roles in developing our data capabilities. Can you describe your roles within the team?
Scott Sheridan: As Director of Operations, I oversee the implementation of all of EduFinance’s data collection, storage and analytical tools. This ranges from data we collect from our 43 financial institution partners to the 25 Education Specialists who work with schools as well as our Monitoring & Evaluation specialist field team.
Guy Winfield: As a Data and Business Intelligence Analyst, I’m known as the “Data Guy” around the office. I focus on two main areas: firstly, connecting data from our field team to our database and reporting on it and, secondly, conducting portfolio analysis for EduFinance financial institution partners.
Briseida Hoxha: In my similar role as a Data and Business Intelligence Analyst, I help improve our data quality through quantitative and qualitative data monitoring. I also assist with database management by maintaining and improving our cloud-based database and reporting process.
Shihabur Rahman: As a Summer Intern, I am using my university experience with mathematic logic to figure out how data can be collected, validated or referred to so that inconsistent data is prevented. Irene and I are currently working on data cleaning for EduFinance’s technological upgrade for data collection. I am also building out dashboards to monitor ranges of our data, so that we can identify idiosyncrasies as they arise.
Irene Sugurova: Similarly, I am using my university experience with statistics, coding and analysis for practical impact. I have had a chance to work in a team as well as my own project. The individual project I was first assigned is focused on analysing the impact of ETAF staff training on loan officer productivity.
HC: What made you want to join the EduFinance team?
GW: In my previous role at Vauxhall Finance, I had moved away from working with data to a more strategic role. However, since data is my eal passion, I started to look for a role with a data-driven challenge and a meaningful impact – 6 months later I found EduFinance!
BH: Before joining EduFinance I was also working for Vauxhall Finance as a Pricing and Reporting analyst. After 4 years in the Automotive industry, I wanted to use my technical skills for a good cause. What better cause than helping children to have a better education in countries where it is needed the most.
IS: I loved my university module “Linear Models and Data Analysis” involving statistical analysis for a client from Procter and Gamble. I felt all my strengths were put to use, so I decided to do a summer internship in data analysis. I have also always believed that one’s future is dependent on the education they receive. When I found out that EduFinance’s main goal is to reduce poverty through quality education, I wanted to be a part of it. Both of my passions rolled into one fantastic internship!
SR: I chose EduFinance specifically because I believed in their work and wanted to try a role that would challenge me and be beneficial to a good cause. In this case: helping more children get a better education. I believe working at EduFinance allows me to contribute to the team’s common goal.
SS: I have always been interested in two things in my professional life: development and finance. From my previous roles in banking I was able to see the potential that financial inclusion has to make the world a better place. I was also raised in a household of teachers so I value education very highly and EduFinance provided me with a great chance to fulfil both objectives. I have been with EduFinance for over four years now, so I am grateful the team has been able to find their roles both challenging and rewarding.
Scott presents at Global Education Finance Conference 2019 Early Data Capture Tools: Challenges as Program Develops
HC: Scott, since you have been with EduFinance from the start, can you describe how the data collection tools and processes for our two main programs - EduFinance Technical Assistance Facility and EduQuality - have progressed over the course of our development?
SS: Initially, prior to establishing the EduQuality program, EduFinance provided technical assistance to our Opportunity International network of financial institution (FI) partners, working closely with their EduFinance managers to help them manage and track their education loan portfolios. We worked to increase productivity and identify potential modifications to their products in ways that could reduce risk. To do this, we needed to collect “real time” data, so we set up data-sharing agreements and a template for banks to provide us data on a monthly basis within Excel.
With the data we collected – including product value outstanding, number of loans, Portfolio at Risk (PAR) and new loan sales - we were able to develop the initial version of the EdPack. This is our 7-page dashboard with charts, data and analysis for each FI, which we email to them monthly to provide feedback and support. For example, if a partner’s PAR > 30 day rate increases during the month, we notify and work with them to identify root causes and solutions.
As we expanded our program to reach more FIs, the data became increasingly difficult to manage simply in Excel. We needed to find a solution to get us where we are today. Our goal has always been to maximise the information we could give to the partner FI whilst minimising the workload caused by our data requests. We therefore ask for enough data to assess the health of the portfolio and trends, but ensure the request can be handled routinely.
HC: Before we move onto the data solutions we currently use, can you briefly outline the history of EduQuality data collection and the challenges we initially faced?
SS: The EduQuality program has been one of the most enjoyable and rewarding projects we’ve done from a data perspective, but it is not without challenges. We launched with only one Education Specialist in each country who coordinated cluster meetings with schools. We first tried reports in Word and email formats to capture information, but they were time-consuming for Education Specialists as well as difficult to aggregate to track and identify trends and themes from the meetings, especially as we continued hiring and building our team.
GW: Recognizing we needed a data capture tool for EduQuality, we implemented a first version. This was an open source tool which was a great fit for exploring and learning about what types of data collection were most effective. However, it was not efficient enough when connecting to our database and updating surveys and it wasn’t particularly user-friendly. We also received data of lower quality because there were not many restrictions on answers.
For example, with the previous tool, it would have been possible for field staff to enter ‘200 students enrolled’ and ‘250 student drop-outs’ for the same school, which is clearly impossible. This required us to catch these red flags through manual review which was time-consuming.
SS: As our EduQuality program evolved and our learning about the data we needed collect from schools grew - the surveys on our previous tool needed to evolve too. We had to ensure all of our field staff could smoothly navigate these survey updates.
Taking a Strategic Approach to Tech Solution Evaluation
HC: How does the team approach the evaluation of new tools and tech solutions, to ultimately provide the best possible services for our partner financial institutions and schools with minimal disruption?
SS: We are cautious about investing large amounts of money in technology solutions that are yet to be proven in our context. We have a ‘due diligence’ template which we use in all major decisions about digital platform changes. We include several options for an update and run a weighted cost/benefit analysis by marking the options against a list of key requirements, features and costs. We definitely feel that this step-by-step approach has helped ensure we remain lean and efficient, avoiding unsustainable or costly upgrades.
Due Dilligence Template Example
New Data Collection Tools: Piloting & Rollout
HC: This sounds like a very strategic approach to ensuring EduFinance upgrades to the most appropriate, cost-effective technologies for data collection and management. Could you describe our newest data collection and analysis tools and how they met your evaluation criteria?
SS: For the EduFinance technical assistance side of our work with partner financial institutions, we upgraded our system in 2017. We coded our EdPack calculations into SQL, migrating the data to our cloud database and automating the monthly partner portfolio analysis as much as we could. With time series data over almost 7 years, we are able to conduct robust analyses such as lending seasonality to spot any trends, forecast and troubleshoot. This allows us to add more value to our partners. Being able to reliably deliver monthly analyses for our partners builds trust, strengthening our dialogue with their senior management and our overall relationship.
GW: To increase our EduQuality data collection capabilities, this year we partnered with Social Cops1 , an organisation focused on data collection for NGOs, to use their Collect application. This is a data collection app usable on tablets and smartphones allows our EduQuality team in the field to digitally record the details of their school cluster meetings and school visits in a user-friendly way. They can also upload videos, photos and recordings via the app. We can now apply logical data entry restrictions which reduce errors and improve data quality.
The field teams simply download the software once, and then when we make any changes to the back-end their surveys are revised as soon as they connect to the internet.
SS: Before we finalize our selection, we always pilot the tools we intend to upgrade to. First, we test it on ourselves and our Global team. Then our EduQuality Manager, Renée McAlpin, coordinates a pilot with one to two of the EduSpecialist teams. These two processes are conducted in parallel, helping to ensure that if testing does not work well, we can troubleshoot the issue without affecting a large amount of our field team. Once we are sure the tool is working correctly, we then roll-out to our entire staff.
HC: As I understand, the Education Specialists are now using the Collect app with ease. How did you ensure a successful rollout?
GW: As general practice, we conduct a launch call to announce our new tool with the team and then aim to conduct in-person training with each field staff member, which we find to be more effective than group training. For Collect, every Education Specialist has had several hours of in-person training for onboarding to the new platform. We use Whatsapp groups and email to notify Education Specialists when any updates have happened.
Guy explains ETAF tools to FI partners at Global Education Finance Conference 2019
Data Quality and Program Impact
HC: Can you expand more on the migration of our previously collected data to these new platforms?
SS: Part of the cost-benefit analysis involved considering the need to merge all of the historic data to the new database whilst ensuring comparable data. This “re-wiring” process was lengthy, but the efficiencies gained in the long run are worth it. We also conducted a thorough data-cleaning process involving identifying outliers, inconsistences and incorrect data. Having really clean and good data enables us to accurately assess our program, identify trends and correlations – the essential building blocks for a strong M&E system. Our two summer interns, Irene and Shihabur, have been a huge asset helping us to ensure the robustness of our data.
HC: We have clearly improved our data capture and analysis capability, which ultimately improves our end products and services. Any notable successes you can share?
GW: In terms of our partner portfolio analysis, we had a great success story last year where we noticed that one of our long-standing partners was disbursing fewer loans than would be expected for the time of year. We reached out to them to inquire about this since it was uncharacteristic. After we discussed the issue and mapped out a solution, the following month they were bettering their historical trends following the renewed focus.
For our EduQuality program, the data our teams collect is being used to improve programming as we look for correlations between the work each regional EduQuality team is documenting from cluster meetings and trainings, and the measurable changes in aspects of education quality and conditions for learning in the schools. This School Profile data - such as number of pupils, drop outs, teachers, WASH facilities, etc. - is collected by our M&E Specialists using Collect. We also use this data to look at how school changes and quality improvements might be linked to our program services over time. This analysis tell us where can we change our approach and improve outcomes.
HC: Now that your team has just completed a major data collection tool rollout successfully, what’s the next priority for the Data & BI Team?
SS: The next really exciting project for us on the Data & BI team is centralising all our data by merging our EduQuality, partner education portfolio, product development and growing market research databases into one. Right now we have 23 market research reports from 17 different countries with a vast amount of data points that could have significant correlations across our other work, but working across large, multiple data sets efficiently is a big challenge. Creating this centralised database will enable us to look at a specific country, for example, and see all the work we have carried out across our program all at once.
HC: Thank you all for sharing these insights!
In line with our values of knowledge sharing, collaboration and transparency, Opportunity EduFinance is committed to sharing our experiences and learnings with the wider community working to tackle the global education crisis. With over 600 million children not learning globally, we recognize that for our collective finite resources to have the maximum impact to improve children’s access to quality education, we have to find the synergies where we can work together for the same goal. We hope that the specific shared experiences from our Data and BI team can be useful to those currently facing decisions or pain points in meaningful data collection.
Social Cops' company name is now Atlan.