Summary

Incorporating evaluation into all phases of curriculum development

From the beginning, a goal was to infuse parent perspectives into the design of the programs in ways that integrated parent and professional or expert knowledge (see Figure 1). We garnered professional knowledge through customary means: we reviewed relevant empirical and theoretical literature and organized an advisory committee of experts who brought years of experience in developing and implementing parenting programs. We attempted to marshal parent knowledge by conducting an interview study with 151 parents in three Indiana communities, two of which were urban. We recruited parents for participation in the study through first- and fourth-grade public school classrooms, and conducted in-depth interviews mostly in the homes of study participants. By design, the sample consisted predominantly of families with limited levels of formal education and income. 85 of the families were two-parent households, and we interviewed both fathers and mothers in all but two of these families. The mothers represented Latino (24%), African American (27%), and Anglo American (47%) backgrounds; fathers represented a similar distribution. Our goal was to recruit for this study a sample of parents similar to those who were likely to be attracted to a parenting education group. The interview with parents focused on a range of topics related to their children's futures.

We learned a good deal from parents. A strong and consistent theme across nearly all interviews was parental worry about their child's future. An overwhelming majority of parents indicated that their child needs to do things differently than they did in order to succeed in today's world. Approximately one third do not expect their child will attain what the parent considers to be an ideal occupation for the child. We learned that parents gave greater emphasis to child characteristics than to environmental influences as contributors to success. That is, more parents emphasized child factors such as academic skills and abilities than environmental factors such as a supportive atmosphere for development and learning. We also learned that the weekday period from 4 to 8 p.m. is a good target time for contemplating enhancements in parent-child interaction within daily family routines. Television surfaced as a dominant organizing force in determining the nature of family interactions.

This type of information proved to be highly valuable in forming decisions about program content. We decided, for instance, that the program needed to address squarely the widespread concern about what parents can do to help their children prepare for unknown and uncertain futures. From these interview data we also envisioned program opportunities to emphasize the idea that child competence is developed through interactions between individual and environment, and we identified challenges in helping parents think about ways of tweaking daily family routines to enhance opportunities for children's learning.

Parent perspectives were tapped through more qualitative means at the second and third stages of curriculum development. We used focus groups frequently to try out various isolated curriculum ideas and activities. We heard stories about parent interactions with schools and with children, for instance, that became the basis of critical incidents incorporated into the curriculum for parent group discussion. We learned that role playing was an enjoyable group activity, and that relations with school personnel were emotionally-charged topics requiring two full sessions. Implementation of the full curriculum in nine pilot sites was chronicled carefully through observations, interviews with group facilitators after each session, and periodic interviews with parents. A main lesson of the pilot work was the need to better support a collaborative team of professional and parent; we were reminded of the deep roots in our culture surrounding the notion that professionals should be in charge. We also learned of the creative accommodations of curriculum ideas made by group facilitators. One facilitator who could not read, for instance, communicated core ideas to her peers through elaborate illustrations on the room walls; one of these illustrations led to an important cartoon in the curriculum about perspective-taking skills as a basis of problem-solving skills.

We have no data to support a claim that the end result -- the final program curriculum -- would have been less responsive to parents if we had pursued a program development path that did not incorporate parent perspectives. Compared to other curriculum development enterprises I have observed and contributed to, however, it seems we had fewer false starts and dead-ends in the Links to Learning program development journey, and that, if nothing else, we were able to approach the final implementation of the full curriculum with a good deal of confidence about its ecological validity.

Model of change mechanisms

Over time we generated a model of how the program might contribute to changes in parent beliefs and behaviors. It has three sequential parts. The first step focuses on parents' beliefs about influences on their child's education, including parents' sense of self-efficacy regarding these influences. This domain builds on a growing literature demonstrating the significance of parents' beliefs about children's development and learning (e.g. Sigel, McGillicuddy-DeLisi, & Goodnow, 1992) and parents' ideas about whether they have an appropriate role in their children's education that can make a genuine difference (e.g. Hoover-Dempsey & Sandler, 1997). The second step or domain is quite simply parental recognition of opportunities for children's learning in family and community settings. Recognizing a teachable moment in the context of riding on a bus, shopping, watching television, or preparing a meal is an illustration of this domain. The third step is maximizing the learning opportunity (the second domain) through parent-child interaction, especially through use of inquiry or question-asking. There is an extensive theoretical and research literature on the value of engaging children in conversations via open questions (e.g. Stemberg & Williams, 1995) that supports this third step. My working metaphor for this model is theatrical in nature: the first step establishes the actors and their roles, the second step provides the stage and the props, and the third step is the enactment through verbal interactions between parent and child.

This conceptual framework was not tested in the Links to Learningcurriculum implementation. Rather, it guided curriculum content and it also helped us make decisions about outcomes and measures and to organize our data analyses. We used an experimental design (wait-list control) to test the efficacy of the program with a sample of 129 parents (71 program, 58 control group parents who subsequently received the program), a majority of whom had annual family incomes of under $40,000 and who represented African American (35%), Latino (26%), and Anglo American (61%) backgrounds in five different communities.

We found positive program impacts on parents' beliefs about the importance of different influences on their child's education, including positive change in parents' beliefs about their ability to have influence on these factors. With regard to maximizing opportunities for learning, we found positive program effects on the extent to which children participate in daily routine tasks including meal preparation, clean-up, and laundry, and on the level of parent participation in school-related activities with the child, but not on family activities with the child or communication with school personnel to support the child's education. In the third domain, we found positive program impact on parents' attempts to understand the child's point of view and to build on what the child says or does in hypothetical parental teaching situations, and a modest program impact on how often parents and children discuss school-related and personal topics. There was no effect, however, on joint decision-making, and on the frequency of daily interactions with the child.

Our strongest program impact, then, occurred in the first domain focused on parental beliefs. Effects were less robust in the behavioral areas of strengthening opportunities for learning and in the domain of parent-child interaction. These types of limited effects may be appropriate for a short-term parenting program.

Differential program effects

Income is defining variable in life that long has been linked to a number of individual beliefs and behaviors, including child-rearing socialization practices (e.g. Kohn, 1969). We organized the sample by three income categories: low income ($19,999 or less), moderate ($20,000 to $39,999), and higher income ($40,000 or above) and found important differences in some outcome areas. Lower-income program parents compared to lower-income control parents showed changes in beliefs about factors that influence their child's education and increases in level of personal influence on these factors, and increases in daily routine activities (involvement in meal preparation, clean-up, laundry). Moderate-income program parents compared to moderate income control parents had more frequent conversations with their child about school-related and personal topics. Higher-income program parents compared to higher-income control parents had greater participation in joint decision-making with their child.

It seems that parents of differing income levels "took" different types of messages from the program. Connecting these findings to the conceptual model of anticipated change mechanism, we see that changes in the first two domains -- beliefs and learning opportunities -- occurred among lower-income parents while changes in parent-child interaction were more likely to occur among moderate and higher-income parents. It appears that parents entered the program with different sets of lenses or frameworks for engaging core messages of the program curriculum. At baseline (pre-program), there were significant differences across many variables by family income, and hence we used baseline scores as a co-variate in analyzing post-program outcomes.

Concluding comment

I view our approach to evaluation in the Links to Learning program as a case study in how a curriculum development effort might strengthen the contributions of evaluation to the responsiveness of program design. As noted at the outset of this paper, certainly much more needs to be done to improve common practices in evaluating parenting programs. Yet evaluation has more to offer than "proof" of a curriculum's efficacy. A pressing issue in the improvement of parenting programs is sensitivity to the needs and characteristics of the population to be served, and the evaluation enterprise has an important role to play in achieving this goal.