Safe Water staff write about their work in the field

In early 2011, PATH staff and partners gathered in Cambodia to kick off an evaluation to better understand water-treatment practices there and observe where drinking water was collected and stored. This baseline survey marked the start of a pilot project that tests the theory that low-income consumers in the developing world will pay for a water-treatment product they aspire to own.

3,720 households

Greetings! My name is Carolien (Carly) Swann and, as Liz indicated in an earlier post I am a program assistant with the Safe Water Project. In my role, I don’t get many opportunities to travel, see our work in action, and meet the people that we are focused on helping. So I was privileged to receive a professional-development grant from PATH that supported my travel to the field, allowing me to participate in the baseline survey.

Now, about these 3,270 households…our goal was to supervise as many of the enumerators as possible to ensure that we would be getting consistent and correct data from the surveys. But, as Liz mentioned, language was absolutely a barrier. Without support and collaboration of our local counterparts at the PATH Cambodia office, as well as the expertise and knowledge of the local research firm—Indochina Research, it would have been so much more difficult! Thankfully, Sonita Khun, the program assistant from the PATH Cambodia office, accompanied me and provided translation and guidance through the process. Together, we were able to provide immediate feedback to the enumerators and their supervisors from Indochina Research.

One of the 70 enumerators collects baseline data about water-treatment practices from a survey participant.

From what we learned in the first week, we knew that the flow of the questions would be tricky—there were a lot of skip patterns that needed to be followed and visual aids that needed to be shown at specific intervals. But enumerators also had to ensure that informed consent was properly obtained, that only one person (namely the eligible participant) answered each question, and that each question was properly understood before the participant’s answer was marked down.

Going from house to house, we also took the opportunity to observe and note where drinking water was collected and how the households stored the water. We saw a number of boreholes and other protected water sources; however, many were broken or not functioning well. None of the households in the more rural areas had piped water. Yet, almost all households had huge earthenware pots (often uncovered) used to collect and store rainwater. I can only imagine how much rain must fall during the monsoon season. All the homes were on stilts and our climb up to the front entrance was often narrow and steep!

A community borehole supplies water for the neighboring households. Families collect water from these sources, carry the water home, and store it for household use.

One of the more memorable moments on the trip happened on the last day of supervising the enumerators. We all split up to cover a larger area; Sonita and I went with Mary Hean (project manager for Indochina Research) to supervise a group in a village outside of Kampong Speu. We were dropped off by the car at a crossroads adjacent to a Buddhist temple and told that the village was just down the dirt road not too far. As our car drove away leaving us to make our own way down the road, we learned that the village was actually about 10 km away! We were dressed in neat slacks, button down shirts, and high heels, and it was much too hot and dusty to walk. After 30 minutes of waiting, Mary flagged down a passing motorbike. She hired him and one of his friends to drive us down the dirt road past the rice fields and into the village. I squeezed between the driver—who had my bulky backpack housing my laptop in between his feet—and one of the supervisors from Indochina Research. Miraculously, we made it to the village and found the study teams, no worse for wear but a bit more dusty.

Once in the village, Sonita and I focused on observing as many interviews as possible. Conducting these interviews, being comfortable asking the questions, and making sure that the answers are consistent and correct definitely is no easy task

After only spending 4 days in the field observing data collection, I could see that the enumerators had their work cut out for them. Each survey could take anywhere between 45 to 90 minutes to complete, and walking in between households often took between 10 to 15 minutes. Sampling procedures had to be followed to make sure that we were getting a random sample of the population and houses. In practice this could mean spending quite some time seeking out a house with an eligible participant.

Here are two final examples to illustrate how hard our enumerators must work to survey 3,270 households. One family we visited was made up of two siblings:  a brother and sister. The sister was in charge of collecting and storing the drinking water, however, she was too young to participate in our survey, and so we had to move on to another household. In another household we found an eligible participant, but she would only answer once her husband had provided an answer first! Since both were effectively giving answers, the sampling procedures weren’t being followed, and we had to move on to another household. These experiences in the few days that I participated in the data collection really made me appreciate how hard this team would work to canvas this vast area and produce 3,270 interviews as a final product! Good luck, team!

The team gets ready to enter the field, tackling the arduous task of interviewing 3,270 eligible households about their current water-treatment practices.

Training a team of 100

The baseline survey employed an enormous team of enumerators and supervisors totaling more than 100 team members—all who needed to be trained in the study design.

The first task was to train the data collection team (70 enumerators and 40 supervisors). We partnered with Indochina Research, a local data collection firm, to conduct the training. To give an idea of the enormous size of this team, in my eight years of quantitative data collection conducting similar surveys I had never worked with a team of greater than 30 persons! The challenges in working with a team this large include ensuring data-collection consistency among enumerators, and quality-control checking by supervisors to ensure that your data collection methods are similar enough to have comparable data between enumerators. We all know that asking the same question in just slightly different ways can illicit variable responses, which is exactly what we were trying to avoid.

In the first day of work on the training, the study supervisors and PATH staff discussed the results of a pre-test of the questionnaire (data collection tool). It was during this day-long session—complicated by a very hot room and jet-lag—that Carly and I got our first taste of how complex the process would be. The questionnaire was initially pre-tested by the team leads of Indochina Research. In order to pre-test the questionnaire, they visited households in the study area and conducted an interview each and came back to our session with questions that may have been confusing or caused problems, which ended up being the vast majority of the questionnaire. Ay-yai-yai! In this day-long, small-group, working session we went through each and every concern clarifying the information that a question was intended to capture and ways that the question could be clarified so that it was easily understandable and could be asked consistently.

PATH Cambodia staff member Thunvuth Nop and Indochina Research consultant Mary Hean train the team of 70+ enumerators.

They say a questionnaire is never ready until it has been pre-tested and field tested. This is certainly true! We determined numerous questions did not translate between the English and Khmer languages, and multiple revisions were made in the days that followed.

After that meeting the data collection tool went through a similar process by the team of 40 supervisors and 70 enumerators. The final test for data consistency was two supervisors conducting a mock interview in front of all 70 enumerators via headset and loud speaker. All data collectors filled in their own questionnaire and this data was entered into a database and check for consistency between answers. Amazingly enough, we did it! We felt confident about the questionnaire and enumerators that came out of this process. Now, time for the real work to begin!

Kicking off the baseline survey

My name is Liz Blanton and I am a senior research associate with the Safe Water Project at PATH. Carly Swann, a Safe Water Project program assistant, and I recently traveled to Cambodia to assist in an evaluation for the launch of our pilot project in Cambodia.

PATH staff members Liz Blanton and Carly Swann traveled to Cambodia to kick off a baseline survey. It was the first time to Cambodia for both of them.

We arrived in Cambodia after nearly 20 hours of travel time, having left Seattle two days prior. We headed straight to the PATH country office in Phnom Penh and hit the ground running. Our partner in this effort, Abt Associates, met with us and other representatives from the PATH country office to talk about the monitoring and evaluation (M&E) plan. Our objectives for the trip were three-fold:

• Represent PATH’s perspective in our evaluation efforts.
• Assist in training the data collection team.
• Supervise the first days of data collection in our study areas.

We had a lot to accomplish and had our work cut out for us. This was the first trip to Cambodia for both of us and neither knew a single word of the Khmer language.

Our Cambodia pilot represents one of our most exciting pilot efforts to date. It combines a rigorous evaluation effort with testing the commercial viability of two different ceramic water filters. The ceramic water filter is a household-water-treatment intervention proven to be effective in the reduction of diarrheal diseases. The main goals of the baseline survey were to:

• Describe the population in our pilot area where we will implement the intervention.
• Find out what type of water treatment practices already exist in the area.

The survey took place in two districts of Cambodia: Kampong Speu and Kampong Cham.

This pilot is the culmination of work existing across our three teams in the Safe Water Project: the technology team, the commercialization team, and the research and evaluation team. It is designed to test and measure the impact, scale, and sustainability of two different products, the original Tunsai ceramic water and our cosmetically upgraded Super Tunsai.

We're testing side-by-side sales of two similar water treatment and storage devices. The “Tunsai,”on the left, is a ceramic water purifier that has been in production since 2001. The “Super Tunsai,” on the right, was redesigned by PATH’s product development team to be more aesthetically pleasing.

In this pilot we are testing the sales of these filters through six different mechanisms in these two provinces in Cambodia. In Kampong Speu, a district in the southwest, we will be testing the sales of this filter through a variety of direct sales models:

• Direct sales at full price.
• A try-before-you-buy option.
• Filters sold through microfinance loans in an MFI model.

In Kampong Cham, a district in the southeast, we are testing the sales of the filters through the retail sphere with three different pricing options, using a coupon scheme.

During the baseline survey the study team surveyed 3,720 households across the two provinces; completing a survey of this caliber was no easy task. It took 70 enumerators, 40 supervisors, 5 team leads, and 3 visiting researchers, including Carly and myself, to get this survey off the ground and complete it.

Photos: PATH.