Safe Water staff write about their work in the field

A “feat” of research: a strong research team has skills as diverse as our footwear.

In September 2010, PATH staff and partners gathered in India to field-test our own prototype of a household water treatment filter. The device was designed to meet the unique needs and desires of low-income users in India and is informed by the results of extensive user research among low-income households. PATH staff member Kevin Flick and consultant Michal Lahav report about their experience.

Day 11, Saturday, September 11, 2010

Final day in India…packing luggage, reviewing notes, lounging, and a bit of sightseeing. See you soon, Seattle!

Day 10, Friday, September 10, 2010

Packing up! We spent the day in the PATH Hyderabad office today; cleaning up after ourselves and making sure we’re taking everything we need with us back to Seattle. Each PATH office functions somewhat autonomously, so we can’t take the availability of support for granted. In this case, we got support with a capital “S”! I can’t say enough about how the office graciously absorbed and accommodated eight people, a portable microbiological lab, a mountain of equipment, and our need for logistical support at all hours. Thank you, PATH Hyderabad! And don’t worry, when we come back to bring all the devices from the field, we’ll rent a service apartment.

Day 9, Thursday, September 9, 2010

Today we stayed in the office with the team and did administrative tasks. We finished watching the videos and decided on our conventions for naming photos—all good work to have done before leaving India.

Today was also the farewell for our team. It amazes me how quickly a group of strangers can get to know each other over such a short time. We were all so interdependent, and we spent so much time together that we really gelled. We were fortunate to have such excellent people on our staff and we’re hoping that most of them can stay on the team to help with the middle and final visits in October and November. Huge thanks to all of our research staff!

The best field team you could hope for.

Of all of the conversations that recurred over the week, none was more frequent than talking about why we were asking a particular question and what went into deciding which questions should be asked. Reflecting on this was a valuable exercise for us and we’re changing the way we make moderator’s guides moving forward. In this instance, we spent a lot of time wordsmithing the guide in English; getting the nuance just right, and carefully choosing how to ask things in order to avoid leading or prejudicing the participants. As it turns out, much of that was lost in translation, and had to be recreated rather inefficiently in our iterative review of the guide. From now on, we’re going to focus on the rationale behind the question, rather than the exact wording of it. We’ll spend far less time worrying about the syntax, and more time providing the meaning and nuance of the question so that the team in India can understand why we’re asking what we’re asking. Hopefully, this will let them take better ownership of the moderator’s guide and make a more accurate translation with fewer necessary revisions.

Day 8, Wednesday, September 8, 2010

It probably goes without saying that it has taken a lot of discussion and decision-making to get us to where we are today with our research. One of the of the design guidelines that we established is that the device should only assemble in one correct way. That decision is really paying off. We’re testing the worst case scenario: first assembly with no instructions. Even with that handicap, the families are getting the whole thing together in about 15 minutes. We keep some pictures of the assembled device and a step-by-step pictorial guide of assembly in our back pockets in case the families get stuck somewhere along the way.  About a third of the families needed one of the guides, and they put the device together in short order after getting the nudge. Realistically, we’d probably never ship this product without a pictorial assembly guide, which we’ve observed makes the process practically foolproof and much shorter (about 5 minutes). I know I keep coming back to this, but the value of iteration is immeasurable.

How was the experience for you?…one rock is bad, 4 rocks is good. This is our attempt to quantify some of the responses.

We spent the afternoon watching and translating the videos we shot of the different families assembling the devices. This was a great way to spend the afternoon—a little decompressing and good detail sharing. You’ll recall that we decided to have the research team leave either Kevin or Michal alone with the families to film the assembly process…it was fun to finally understand what they were saying. As we suspected, there were plenty of remarks like “Why did he stay here? He can’t even speak Telugu!” and “Where did Madam (Jhansi, or one of the other researchers) go?!?!”

We touched on the value of piloting research tools on Friday, but I want to come back to it for a moment. In sum, we wrote three versions of the moderator’s guide before sending it to India. We revised it with the team and adjusted translations before going into the field. We piloted it once and then revised again. We piloted it again and revised yet another time.  We took it out and ran it through six families and made small adjustments. Even after all of this, we still made little refinements throughout the remaining families. As a small example (of which there were many), we asked “Does this device fit in your home?”  What we meant was, “Does the device fit, stylistically? In other words, does it look out of place next to your other possessions or does it compliment them?” Since we didn’t provide enough of the rationale behind the question, the word “fitting” was interpreted to be about space, not style. There’s just no substitute for direct observation and continuous improvement and yet, so often, we only have time or money for one or two rounds of review before translation and execution.

Day 7, Tuesday, September 7, 2010

We visited four urban households in Hyderabad today, and our work on the moderator’s guide is really showing. Now we can observe the interactions more, and pay less attention to the execution. One of the nice indicators of this is that our team members are starting to help each other with friendly little pointers. There is pretty much nothing better than that! Since our research is starting to go more smoothly, we’re taking a detour today to share some stories about the families that are participating in our research. We’re intentionally leaving out any interpretation—just focusing on the details to give you a better feel for the lives of our participants.

Courtyard from Tuesday morning.

One of the households we visited was an eight-member family living in a two-room house, with a covered cooking area and small courtyard. The entrance to the space was gated, and the total area was about eight by six meters. When we asked family members if they could take us to the local water source, they refused because there were no men present. As it turns out, the women never leave the dwelling without male accompaniment. In a side conversation, one of the family members told Jhansi that the family had moved to the area from elsewhere and due to cultural or religious differences, they have never really become comfortable in their new home.

Ironing by caged dragon…

One of our afternoon families earned money by ironing clothes. Their home was two stories tall, with the ironing space on the ground floor. The first thing we all noticed was the heavy, smoky, hot air in the house. It felt oppressive to take a deep breath, despite several open windows. Their irons are essentially coal-burning boxes with a massive metal plate on the bottom.

Field notebook. Warning, this notebook will become indecipherable in 48 hours! Review now!

Day 6, Monday, September 6, 2010

We had planned to make today our second big field day, but we modified the schedule to spend a day in the office so that we could review our notes and collect observations into a single data file. Data management isn’t the most fun thing to do for most people, but we had a good time chatting and comparing impressions about the various families. Again, resisting the urge to cram all of the work in as few days as possible is turning out to be a good idea. Our memories of what all of the scribbles in lab notebooks mean are still relatively intact and we’re cutting down on the amount of work we’ll have to do when we get back to Seattle.

Day 5, Sunday, September 5, 2010

Sunday is traditionally not a work day, and our team was giving plenty of not-so-subtle hints that they would like to stay home, so we called it a day off and decompressed a bit ourselves. Being in another country to do research is such an expensive activity that it is tempting to work like mad all the time. Having a day to let things simmer, however, is a luxury worth taking.

We had dinner with our colleague JVG and his lovely family—roti-making lessons, delicious curry, and homemade pickles. We will have full bellies and rested minds going into Monday!

Day 4, Saturday, September 4, 2010

Saturday was our first big field day. We placed devices in six homes, dividing the effort between two teams of four (PATH staff, moderator, translator, and research assistant).  We met at the PATH office at about 8:00 a.m., got our things organized, and departed for the village of Angadi Raichur, 100 km southwest of Hyderabad.

One of our biggest findings from the pilot yesterday was the realization that using the “talk out loud method” was not going to be appropriate in this research context. The talk out loud method is a commonly used approach in which the participant talks through their process, difficulties, and discoveries as they explore a new product. In order for us to use this method, we need a translator, a notetaker, and someone to run the cameras. That many people standing around the participant invites interaction and questions, which defeats the purpose of trying to document the event as if the participant really were assembling the device for the first time. We decided to let the participants assemble the device on their own, with only a single camera person (either Michal or Kevin). That way, even if they asked for our assistance (which they all want to do), we couldn’t help because we don’t speak Telugu! This was a major improvement, and it is enabling us to observe a much more realistic interaction with the prototype.

This is how the “realistic” assembly process looked before making the change in protocol!

Our execution of the moderator’s guide keeps getting better and better. We’re at the point where we’re starting to fine-tune the little details, and that feels good. Our main challenge is to maintain consistent questioning between teams and between households. I think it must feel very stilted for a group of Indians to have a conversation that is by a series of questions on a piece of paper. Generally, conversations here are much more animated and rambling.

Our head-goes-into-wall learning continues…by the time we got to the second and third households, the families were suspiciously very familiar with the assembly of the device. In fact, one of the families asked for a part that they couldn’t find in the box. Looking at the video later in the day, we noted that the husband of the second family watched the first family assemble the device from start to finish. Deep sigh…we’re shifting a couple of families for our next day out so that we have a little more distance between them. As with the health worker on Friday, there is no blame to assess, just a deepening recognition that what we’re asking of our participants is pretty foreign to them.

Day 3, Friday, September 3, 2010

We got to the PATH office at about 8:00 a.m. Our first two field visits would be for the sake of testing the protocol, so we decided to keep it close to Hyderabad. The village where we went was called Wailal, which is about 38 km to the north of Hyderabad. The trip took about 90 minutes each way. Our first family lived in a two-room home on the outskirts of the village. The wife and husband lived there with their two children and his mother and father. They get their water at a nearby pipe when the electricity is on between 3:00 a.m. and 10:00 a.m. She had abundant difficulties with assembling the device, but eventually got it put together after about 40 minutes, with no instructional aids from the research team. As usual, it was a very communal effort, with the whole family joining in to puzzle out the novel device.

A volunteer family puzzles out how to assemble a device for household water treatment and storage without instructions or external help.

We always say that our first field outing is where we want to make all of our mistakes…a protocol isn’t final until you try it a couple of times and fix all of the problems that you didn’t anticipate. Good research comes from assuming that you only partially know what you’re doing and are constantly looking for ways to improve. Testimony to the unassuming nature of our highly trained team (with dozens of years of research experience, cumulatively), we reviewed our initial lackluster performance with gusto. We decided that it would be practically impossible for the families to assemble the device without influence if any Telugu-speaking team member were present, so we decided to see what it would be like if only Michal or I were there to operate the camera.

On to the second household with our freshly modified protocol. To my astonishment and delight, they whipped the device together in less than five minutes! We all celebrated the design of the device and our wisdom in changing the protocol (people in Seattle could hear us patting ourselves on the back). Jhansi had a quick conversation with the local health worker and it turns out that she went ahead of our research team and taught the second family how to assemble the device. Pop, pop…pop…pop…(bubbles bursting). Of course, the health worker is absolutely without blame. I can easily imagine that she wanted to spare the second family the potential embarrassment of fumbling with the device. Our participants always want to please, and sometimes that gets translated into thinking that we want to see them do things “correctly.” Unfortunately, we can only see the hard parts of the assembly process if they don’t already know how to do it. Tomorrow we’ll make sure to communicate to the village liaison that they shouldn’t pass on any tips. Learning, learning, learning! Humility: 1; ability to predict everything: 0.

Day 2, Thursday, September 2, 2010

We arrived at the PATH office at about 10:00 a.m., feeling moderately rested, and spent most of the day getting to know and training our research team:

• Jhansi (research coordinator and moderator): background in agriculture and rural development, prior experience with PATH on a human papillomavirus project and informal user testing for the Safe Water Project.
• Indira (moderator): background in social work with an emphasis on child welfare, prior experience with PATH on an HIV project.
• Madhavi (research assistant): background in social work and business, internships in development (HIV, water, and sanitation).
• Kavitha (research assistant): background in corporate branding and advertising, development experience with UNICEF programs on HIV, vaccines, and the female condom.
• Swapna (translator): master's degree in computer science, experience in software development and with the Rural Development Institute as a translator.
• Rajsekhar (lab technician): hospital lab tech at Apollo hospital, Jubilee Hills.

Michal and I knew it would be important to allow plenty of time for planning and team building since we’re implementing a new protocol with a new team of researchers in a different country. We developed the training material that we used today without knowing our audience very well, and it seemed below their general level of education and experience at first, but they all reflected that it was the first time they’d seen it laid out so simply all in one place. We focused primarily on qualitative research skills and effective moderation and interviewing. While it was mostly review for our team, it was good for us all to get on the same page. After spending a day with our new team, we felt extremely fortunate to have such skilled people and very thankful to our colleague, JVG (our teammate in PATH's Hyderabad office) and the PATH office for doing such an excellent job recruiting.

Our research team takes a break for lunch.

I left the training session at about 3:00 p.m. for a trip to the airport with JVG. The customs office wanted me to sign for a temporarily lost incubator and to prove that it was only for research. I was well prepared with an invoice that verified our intent to use the equipment for research, but it took some convincing. We traded (were it not for the circumstances) comical misunderstandings for a while until they called for JVG to join us. He’d had the brilliant idea to bring some PATH letterhead paper and we wrote a quick letter, signed it, and got our box. Ironically, my business card says nothing about research; changing that might make future similar situations much easier!

When I got back to PATH from the airport, Michal was finishing up the training. A couple more hours of logistical preparation and we were ready for the first day in the field.

Day 1, Wednesday, September 1, 2010

Greetings! As a complement to more formal communications from the PATH Safe Water Project, we’ve decided to experiment with sharing some close to real-time observations from our fieldwork. My name is Kevin Flick—I’m a researcher on the Safe Water Project. Over the next 11 days, my colleague, Michal Lahav, and I will share stories from our work in and around Hyderabad. We’re here to place 15 of our beta prototypes of water filters in rural and urban low income homes, in an effort to refine our product design and the design principles on which it is founded. I hope you enjoy reading some of our experiences!

Unboxing and checking our household water treatment and storage devices for shipping damage.

Michal and I arrived in Hyderabad at about 5:00 a.m. We checked in at the hotel, had a quick breakfast, and decided that we might as well go to the office right away on the theory that it might be easier to stay awake on our feet. We met part of our research team, unpacked the beta units, and got ourselves ready for the week’s work. Not much to report, but it feels like a good beginning.

Photos: PATH.