Deploying Data for Impact: A PHC data collection project

Sarah Findlay interviewed John Eromosele, the Business Development and Operations Lead at Orodata Science (one of Africa Data Hub’s key partners) to gain insights on a project that was conducted with the aim of improving health care in Nigeria. The project, supported by Africa Data Hub (ADH), has emerged as a beacon of innovation in the field of investigative health reporting. The project delves into the heart of Nigeria’s Primary Health Centers (PHCs), unravelling the hidden truths behind healthcare delivery in the country’s most remote and underserved areas. By empowering local journalists to collect and analyse data firsthand, the project not only illuminates the realities faced by these healthcare facilities, but also sets a new standard for data integrity and community-driven reporting. In this blog post, we explore the project's inception, the challenges overcome, and the valuable lessons learned, offering a comprehensive view of how journalists can leverage data to drive impactful stories and advocate for change.

‍

Sarah (S): How did this project come about? Where did the initial idea come from?

John (J): The idea for the PHC project originated from our fellowship programs focused on gender and health, particularly investigative journalism in the health sector. Unlike traditional approaches, we decided to gather firsthand information directly from both the staff at Primary Health Centers (PHCs) and the patients. This approach involved a deep investigative process supported by ADH, allowing journalists to narrate stories directly from the communities they serve.

In the past, stories about these communities were often told by journalists who travelled from afar, lacking the intimate perspective of local storytellers. This time, we capacited community journalists to tell their stories more authentically, capturing the unique challenges and nuances they faced.

However, this approach wasn’t without its challenges. Accessing remote areas proved difficult, and we were at the mercy of local guides to reach these communities. Despite these hurdles, our first three cohorts of fellows successfully conveyed these narratives. Through their storytelling, a significant pattern emerged: a glaring lack of data on PHCs, especially in rural regions.

We found a disconnect between the National Primary Health Care Development Agency (NPHCDA) and the state agencies, which were predominantly located in urban centres, leaving rural PHCs isolated. It wasn't just about digital data journalism published online; we utilised multimedia storytelling with a strong human angle to bring these stories to life.

Community members were eager to share their stories, motivated by an organisation's willingness to travel great distances to listen to them. We uncovered patterns of infrastructure shortages and neglected welfare of health workers, which had ripple effects on the entire healthcare system.

Our aim was to engage and address these issues, drawing the attention of state governments through our fellowship programs. We discovered that some state officials were unaware of the extent of deterioration in PHCs. For example, in Osun State, our work prompted initiatives to assist pregnant women in accessing PHCs and initiated infrastructure reforms.

Stories like Hospital of Horrors had a profound impact, shocking governments into action. These stories documented the dire conditions of healthcare facilities where mothers sought treatment and attended antenatal classes, prompting governmental response. We realised the importance of making these efforts replicable, not just in Nigeria but across Africa.

To achieve this, we partnered with ADH to support our project. We wanted to ensure our products and models could be replicated continent-wide, helping address the issues of infrastructure neglect and staff welfare, which often led to staff abandoning their posts. Our aim was to report the state of the primary healthcare system across various geopolitical zones effectively.

‍

S: Your approach was to train volunteers to go into the field and collect the data. Firstly, why did you choose to collect data yourself? And why did you choose to go out into the field and not attempt to collect the data remotely?

J: We decided to train volunteers to build an army of civic technologists who would go out into the field for data collection. Our approach began with evaluating what data already existed. We wanted to validate and compare this data, establishing a baseline to understand what was available. Surprisingly, we found some datasets released by the Ministry of Health, which allowed us to not only collect new data but also compare it to existing records to understand how it had been gathered.

A significant challenge we encountered was that many initiatives at the federal level did not trickle down to the granular level of healthcare. To tackle this, we put out a call for volunteers, and many applied. We engaged 30 experts, some of whom were already familiar with data collection. However, to ensure a smooth process, we provided rigorous training, particularly infusing journalists into this team to teach them data collection strategies.

We chose not to notify the government about our data collection efforts to avoid any preemptive preparations. We wanted to capture the genuine state of affairs, without any artificial enhancements.

Data integrity was a major concern, especially with post-collection losses often occurring with paper-based methods. To combat this, we deployed technology that could be used in even the most remote and hard-to-reach areas, reducing turnaround time and enhancing data integrity.

We opted to use Kobo Toolbox, blending it with a bit of open data knowledge, to leverage the power of open-source tools for data collection. Some data collection companies reached out, wishing they had been hired, but we prioritised open-source solutions to maintain data integrity and minimise costs, as data collection is often a very expensive exercise that many governments shy away from.

One issue faced by governments and statistics bodies across regions, including other African countries, is the lack of standardised data collection practices. When new administrations or politicians come in, they often bring new ideas and methods for data collection, causing breaks in the continuity and updating of tools and data types.

When building the tool, we involved the data collection team in testing and development. We developed version 0.2, which was then deployed. We needed to validate existing data, as some PHCs described by the federal ministry as "functional" were not functional at all. For instance, the frequency of restocking supplies at PHCs was inconsistent. Patients often had to travel to city centres for basic healthcare, defeating the purpose of having local PHCs - which is to provide locally accessible clinical services.

Finally, we aimed to interface with major stakeholders, including the government, to ensure comprehensive understanding and collaboration.

‍

S: Why was it important that journalists were also the data collectors you trained?

J: We wanted to bring journalists closer to the data, giving them firsthand experience with primary data collection. From our experience with previous cohorts, a recurring challenge was the difficulty in accessing data. By training journalists in primary data collection, we empowered them to gather their own data and tell more compelling stories.

The narratives crafted by those who physically collected the data are inherently different from those based solely on pre-existing datasets. When journalists engage directly in the field, their stories and angles reflect their unique experiences and insights.

For many of them, this was their first encounter with primary data collection, and it was an incredible learning experience. We also encouraged them to write about their experiences outside the main story.

These firsthand experiences often lead to more realistic stories and case studies, providing better insights into new initiatives and uncovering fresh angles for storytelling.

‍

S: How did you choose what data to collect?

J: We began by developing a comprehensive data collection form, aiming to reduce turnaround time by about 70% compared to traditional paper-based methods. Our approach included visiting several PHCs within a short period, based on our predictions and planning.

When creating the data template and standards, we brainstormed to identify the key challenges faced by healthcare workers. I also drew from personal conversations, including insights from my sister, who has over 25 years of experience as a community health extension worker. These discussions helped us understand the issues prevalent in these communities.

We mapped out the primary challenges, which included:

• Infrastructure: The “Hospital of Horrors” stories about women giving birth in dire conditions inspired us to focus on infrastructure-related questions.

• Service delivery: We examined the quality of service delivery at the community level, recognizing that good infrastructure, welfare, and service delivery are interdependent. Since quality should never be compromised, we integrated quality management auditing into our surveys to understand the standards at these PHCs.

• Welfare/Personnel: We explored why healthcare workers might abandon their posts or take on additional jobs, such as farming, in rural areas. A root cause analysis pointed to poor welfare as a potential reason. To understand this better, we knew we had to conduct field surveys and listen to the workers themselves.

From these insights, we formulated questions based on our hypotheses in these categories. By engaging directly with the community, we aimed to capture accurate data reflecting their lived experiences.

‍

S: Why did you choose to use ODK over other types of tools available?

J: We opted for ODK (Open Data Kit) because of its flexibility and robust features, which were critical for our project’s needs. ODK, combined with Kobo Toolbox, allowed us to customize data collection processes effectively while maintaining high data integrity. Its open-source nature and ability to be adapted for various scenarios were key factors in our decision. ODK's capacity to handle complex data collection tasks and synchronize data across remote locations made it an ideal choice for our project.

‍

S: Your process included data collection, cleaning, and verification. How much time did this whole process take, and how did you ensure accuracy?

J: Having a clear vision for the project from the beginning was crucial. The initial proposal and documentation for ADH provided us with a detailed roadmap of the entire pipeline, helping us anticipate and mitigate potential challenges.

Data collection was done using structured formats, and while the raw data we gathered was extensive and overwhelming, our structured approach helped manage it effectively. We processed and cleaned the data in granular forms, sorting it by PHCs, local governments, and states. This approach significantly reduced turnaround time and allowed us to produce high-quality, granular data.

Data accuracy was remarkably high (99.5%) thanks to the standards we set and the technology used. Our data collection tool, which I personally led the development of, was customised to prevent data entry errors. For example, fields meant for numbers would reject text and vice versa. We also conducted rigorous training with the data collection team and iterated on multiple versions of the tool to ensure it met our needs.

One challenge we encountered was with the accuracy of geo-coordinates. We aimed for a navigation layer that would allow users to locate PHCs accurately. We set an accuracy level of no more than 5 metres, and in most cases, we achieved this. For areas where accuracy exceeded 5 metres, we excluded the data to maintain reliability.

Our technology allowed us to collect data even in remote locations without mobile network or internet access. We could monitor live surveys and provide technical and administrative support in real-time. For instance, by removing the SIM card from the phone, we ensured that data collection relied solely on GPS, bypassing local mobile networks that could affect accuracy. This method maintained an accuracy level of less than 5 metres, ensuring reliable navigation data.

‍

S: You’re also planning on making a tool. What is the purpose of it, and why do you think this is an important part of the project instead of simply making the data available?

J: Creating this tool is essential because we have different target audiences in mind. While making data available is important, it caters to a different category of people than the tool does.

When addressing those in power, it’s crucial to provide evidence-based arguments and present information in compelling formats that relate to real-life scenarios. We want to showcase and support journalists by enabling them to extract and consume data in various formats, all from one place. This tool will help journalists craft their stories more effectively.

Additionally, the tool includes a citizen engagement component, allowing direct interaction with PHCs. This helps bridge the gap between people, PHCs in hard-to-reach areas, and different state and federal agencies. By engaging with the tool, they can see how it benefits the community and themselves.

The tool will present data in granular forms that are understandable to different audience categories. It will be a one-stop shop where users can check and locate PHCs and find out what resources they have or need. Journalists will be able to quickly pull resources from the tool, and citizens will have mobile access to PHC information.

Our goal is to make the overwhelming data accessible and interpretable, even for those with experience. By presenting data in a user-friendly format, we empower both citizens and journalists to engage with and utilise the information effectively.

‍

S: What were some of the biggest unforeseen obstacles you encountered with creating this tool?

J: One of the major unforeseen obstacles was insecurity, particularly in the northern regions. We didn’t have a predefined map of hot zones, so we initially planned our engagement based on an even distribution of PHCs across local governments and wards. However, we had to rely on local community journalists to provide feedback on safety. They informed us about unsafe areas, which led us to reassign our data collection efforts. This situation highlighted the broader issue of security affecting PHC service delivery, and we used it to indirectly encourage the government to focus their security interventions on these vulnerable areas.

Another challenge was official pushback from the government. We anticipated this and made a deliberate choice not to inform them of our arrival in advance, as we expected pushback if they were aware. To address this, we set up situation rooms and provided administrative support to manage these interactions. We also trained our data collectors in emotional intelligence and strategic communication, including role-playing scenarios to prepare them for potential challenges.

‍

S: What are some of the major strengths of this project?

J: This project has several notable strengths:

• Citizen engagement and participatory approach: By involving citizens directly and incorporating their feedback, we’ve ensured that the project is grounded in real experiences and needs. This participatory approach empowers people to have their voices heard and contributes to more meaningful engagement with the data.

• Availability of baseline data: We’re making the data available as open data under a Creative Commons licence. This means that any other project or individual can leverage this data as a baseline for further research, innovations, or smart city development without restrictions. It’s a valuable resource that can spark new initiatives and drive progress.

• Empowering citizens: The project puts power back into the hands of citizens by giving them access to crucial information about their PHCs. Knowing that their input and experiences will be acknowledged helps to reinforce their role in shaping local health systems.

• Support at the grassroots level: By focusing on the grassroots and granular levels of the PHC system, the project supports healthcare in its most fundamental form. Investing in healthcare at this level has a ripple effect, improving overall health outcomes and preparing for emergencies before they arise. As the saying goes, "Prepare for emergencies in the days of good health" or "Prepare for war in the time of peace."

• Using journalism as a tool: The project leverages journalism to communicate the impact of healthcare negligence to those in power. By highlighting these issues through compelling stories, we can effectively advocate for change and hold authorities accountable.

These strengths not only enhance the project’s immediate impact but also contribute to long-term improvements in healthcare delivery and citizen engagement.

‍

S: What impact has this project had thus far? Are there any stories in the pipeline that you could share a bit of information about? What are these stories about and what type of data is being used?

J: The project has had a significant impact in several ways. One of the key outcomes is the revelation of the dire state of primary health care centres (PHCs) in rural areas. For instance, we're currently working on a story that revisits a government initiative called "Save a Million Lives", which aims to improve PHCs in a particular state. We’re using our data to evaluate the effectiveness of this initiative, which was launched three years ago. Despite the intervention, many PHCs are still in a critical condition, and our data shows that the situation remains dire. We’ve visited over 30 PHCs for this story, and the findings are expected to be quite explosive.

We’re also preparing stories that highlight the conditions of doctors working in rural PHCs. These stories will shed light on the significant challenges faced by healthcare professionals in these areas, particularly focusing on their working conditions and the lack of resources.

Another compelling angle involves the unintended consequences of government neglect. In some communities, due to the lack of adequate healthcare facilities, people are turning to herbal remedies, which come with their own set of challenges and risks.

Additionally, we’re exploring the impact of conflict on PHC services in the north. Insecurity has been a major obstacle for data collection and has disrupted service delivery in these regions. Our journalists are actively covering how these conflicts are affecting the availability and quality of healthcare in affected areas.

The feedback from our journalists has been incredibly positive. They’re energised and motivated by the opportunity to work in the field and bring these important issues to light. We’re committed to supporting them by fueling their passion and providing them with the resources they need to tell these impactful stories.

‍

S: With more time and resources, what would you have done differently? Looking back on the process, is there anything you would have approached differently?

J: Reflecting on the project, it’s clear that we’ve only scratched the surface of the data we want to collect on PHCs. Currently, we've managed to gather about 1% of the data we envision. If we had more resources, we would have scaled up significantly. This project was essentially a prototype - a successful model that has proven its viability. With additional resources, our goal would be to expand our data collection efforts, aiming to cover over 90% of the data we need within the next year.

A critical concern is the timeliness of the data. If we delay further data collection, by the time we are ready for the final round, the data we have now might become outdated. We’re keen on ensuring that the data and stories emerging from this project remain relevant and impactful. We also need to guard against any interference that might undermine the stories we are currently developing.

If we had more resources, we would enhance our data collection system to ensure it’s robust and comprehensive. This would enable us to provide a more detailed and accurate picture at both the national and grassroots levels. Additionally, we aim to build a more extensive footprint across all 36 states and six regions. We’re currently present in six regions, but we still have work to do in reaching every state. A larger sample size and more comprehensive coverage would significantly strengthen our ability to provide actionable insights to state development agencies and other stakeholders.

‍

S: What advice would you give to someone looking to carry out a project similar to this?

J: The success of a project like this hinges on several critical factors:

Data structure and standards: Spend ample time developing your data structure, templates, and standards. This foundational work is crucial for ensuring the reliability and usability of your data.

• Training and communication: Invest in thorough training for your team to ensure they fully understand the vision and approach of the project. Clear communication is essential for aligning everyone with the project's goals.

• Strategic support: Establish a situation room to handle challenges effectively. You’ll need a team capable of navigating complex interactions with authorities and government officials. A single influential word from a permanent secretary or government official can significantly impact the project's progress, so having skilled, diplomatic partners is vital.
  
• Community engagement: Build a passionate team committed to reaching even the most remote locations. Our team travelled long distances, sometimes on wooden boats without engines, and stayed overnight in communities. The dedication and inspiration of your team are key to overcoming logistical challenges.

• Continuous feedback: Don’t develop your solution in isolation. Engage with the people who will be using it and incorporate their feedback throughout the process. This approach ensures that the tool or system you create effectively meets the needs of its users.

By focusing on these areas, you’ll enhance the likelihood of success and create a project that makes a meaningful impact.

The fusion of data collection with on-the-ground journalism creates a powerful tool for uncovering and addressing systemic issues in healthcare. The project's success underscores the vital role that data integrity, community engagement, and strategic planning play in crafting compelling narratives that drive change. For journalists embarking on similar endeavours, the insights shared here offer a roadmap to navigating the complexities of data-driven storytelling. By prioritising rigorous data standards, fostering close collaboration with communities, and remaining adaptable to unforeseen challenges, journalists can harness the full potential of data to inform, inspire, and instigate meaningful reform. As we look to the future, the lessons from this project serve as a testament to the transformative power of journalism when paired with meticulous data and a deep commitment to the stories of those who are often unheard.

‍

‍

If you have any questions for John or the ADH team on this project, get in touch with us via info@africadatahub.org.

‍