Save the Children - Mangochi, Malawi

This one of the largest teams that I have had for my training sessions. It is well represented from a broad cross section of the disciplines here at the Mangochi district office of Save the Children USA here in Malawi. Not only are the M&E and IT representatives here, but interested representatives from the various field programs. What a delight!



Our focus for field testing the PDA's is going the Education group, and specifically a Head Teacher PDA survey form that we will test at a couple of schools. The training was conducted in a very large conference room well suited for meetings and presentations. But, I quickly realized I would need to help the participants by dissolving the wide circle of tables in order to bring the participants in closer. This was needful not only to bring them closer together physically in order to hear me speak (I do not lack, I have heard, the ability to project my voice) and to see the training materials, but more importantly to develop the sense of a team. So, I rearranged the room (to the obvious delight to some, and dismay to others desiring to remain on the fringe) to this:




The team quickly picked up on using the PDA's, and after I quickly designed a simple form, I asked for a decision on a form that we would design and take for field testing to the program location. They selected the Head Teacher Interview form, but decided to use it not only for the head teachers (principles), but also the rank and file teachers as well.

What follows is the first few questions of this candidate form. I will focus on the 2nd question of the form: "What is your highest academic qualification?"

Save the Children
Sponsorship Primary Education Program

2007 Standard 4 Data Collection

Headteacher Interview

School:___________________________ Name of Interviewer: _____________________________

Head’s Name: ________________ __________Sex: 0__Male /1 __Female: School Code: 19/__ __

1. Standard(s) teaching _____________
2. What is your highest academic qualification?
A) JCE/ B) MSCE/ C) "A” levels/ D) Other (Specify): ___

The first few questions of this form was simple enough, only requiring brief discussion on the response codes for gender and the source of school identification codes. However, question 2 asking "What is your highest level of academic achievement?" In order to overcome my deficiency on cultural and language, I try to watch for audible and visual cues provided by the group. After asking the group what this question meant, I noticed that several dissenting sub groups emerged. I gave time for their inter- and intra- sub group discussion to occur. I not only wanted them to arrive at a consensus, but I also wanted to observe their decision dynamics, for this team in particular, and possibly cultural, as a whole progressed.

Here was the issue. Question two had, what appeared to be, four different responses to the question, "what is your highest level of academic achievement?" The response could be JCE (2A) OR MSCE (2B) OR "A” levels (2C). There was a 4th response called "Other", which I termed (2D) for discussion purposes. This last response also indicate (but I was not sure), please specify the name of the "other" academic achievement. Actually I annotate all question and survey artifacts with shortened notations. I actually utilize these notations for scripting (a simple form of programming) used by the forms design software to define and operate the logic of the form.

Since a teacher could achieve several levels of academic achievement (e.g. our Bachelors, and then later Masters, or Doctorate degrees), the question was asking for the "highest". So, the response logic was of the "OR" type. You could only respond to one, not two or more. This Other was causing the debate. Was other in addition to a response of 2A, or 2B or 2C, or was it a distinct "highest" achievement of it's own, simply unnamed in the 3 responses?

The debate went on for 1 1/2 hours. I allowed it to proceed since I did not want to influence the decision. But I was also wanting to use this issue and a cause for describing how that paper forms often hide their logic. In addition, I wanted to also point out that the issue is not with the logic of the specific question, but rather goes back to the design of the survey based on the Results Framework. At one point, rather meanly, I even asked the question of how long this questionnaire had been being used? It is amazing how paper forms can perpetuate an illogical or misunderstood indicator.

The team decided that the "Other" was just another unlisted highest academic achievement that should be captured but was not listed in the other three responses. So, 2D was an "OR" response. There was actually a lot of discussion that "Other" actually meant an "AND" response with the intent to capture any additional academic achievement.

This discussion of the minutia of a single question is certainly tedious. But, it is just one example that I began to encounter while endeavoring to have a team build a form in order to field test for value determination of PDA's to replace paper survey methods. In the very beginning I tended to overlook this close examination of the actual meaning of each of the survey questions. But, as I did, I found that our post field test reviews of the data collected tended towards lively debates of what the data meant. I then realized that the issue was neither with the data nor the PDA form, but a lack of consensus of the meaning of the question, or more accurately, what was the presumption of what the answer would provide. I saw that if there was not a clear understanding of the structure, logic and question flow, then I not only could not train a team to produce a usable PDA form, but worse yet, I would possibly perpetuate an instrument of mis-information that might directly affect peoples lives.