1 Acknowledgments

Implementation of the Tips by Text baseline was a joint effort undertaken by members of the Center for Utilizing Behavioral Insights for Children (CUBIC) and the Save the Children Philippines team. The CUBIC team would like to thank Rupasree Srikumar for her initial trial design, Whimcy Sapang for her lead on data collection, Dhwani Yagnaram for her technical guidance and the original creator of the Tips by Text intervention, Dr. Susanna Loeb, for her support and guidance throughout the process.

2 Introduction

2.1 Study Background

With many schools having shut down as a result of the COVID-19 pandemic, it is now more important than ever that parents take an active role their children’s learning and development. However, parents may know little about the best ways to support their children’s learning process or may find themselves frequently pressed for time. We believe that simple behavioral interventions can be impactful in the way parents approach the day-to-day learning of their children, and plan to test this within the context of Metro Manila in the Philippines. Save the Children’s Center for Utilizing Behavioral Insights for Children (CUBIC) will partner with Save the Children Philippines office to roll out a program among parents and children and evaluate its impacts using a randomized controlled trial (RCT). This program and subsequent RCT will leverage both the caregiver well-being mechanism delivered by Save the Children during the current COVID-19 pandemic and results from the Tips by Text study conducted by Dr. Susanna Loeb. The objective of the evaluation is to test whether two simple interventions can encourage parents of preschoolers (3-5 years old) to become more active in their children’s education and development by applying positive parenting techniques. We will also assess whether this increased participation leads to increases in learning outcomes for children.

2.2 Study Objectives and Hypotheses

There are two objectives of the study. The first is to test whether two simple interventions can encourage parents of preschoolers (3-5 years old) to take a more active role in their children’s education and development by applying positive parenting techniques. The second is to assess whether this increased participation in children’s development leads to increases in learning outcomes for children. Our hypotheses are as follows:

  1. Text messages containing tips for parents will positively impact parental engagement with their children and their children’s learning outcomes.
  2. Bi-weekly support calls in addition to text messages containing parenting tips will have a slightly larger positive impact on parental engagement and children’s learning outcomes than just the text message intervention.

3 Methodology

The method of the study is a two-armed randomized control trial. The first intervention (T1) aims to encourage parental involvement by delivering tips and information via text messages. In particular, we will deliver specific instructions to bring about desired behavior change for parent-child interactions. Our second intervention (T2) aims to provide parents not only with information and tips via text messages but also with opportunities to discuss application of the tips with a member of the program-implementation team (an enumerator) to increase motivation, and thereby support and share their reflections and struggles. Those assigned to T2 will receive the information and tips SMS treatment, plus they will be receive a check-in 1-1 call from an enumerator with whom they can discuss the SMS and their experience trying the tips, as well as provide feedback.

Table 1: Study Sample
N % of Total
C 249 14
T1 944 52
T2 611 34
Total 1804 100

3.1 Sampling

While the team originally aimed to enroll a sample size of 2,340 parents of children age 3 - 5, time and budget constraints only allowed for a final sample of 1,804. The original size of 2,340 includes the desired sample size of 1,800 plus an additional 30% to account for the attrition rate experienced by the Philippines team when conducting past RCTs. The desired sample size of 1,800 was suitable for the study’s budget and provided acceptable minimum detectable effect sizes using .8 power to test both of the study’s hypotheses: Control vs. T1 (.209) and T1 vs. T2 (.14). Because this study has been proven in other contexts, the study team decided to maximize the benefit of the intervention by reducing the size of the control group (12.5%), distributing the majority of the sample to the proven intervention group (T1) and allocating the rest of the sample to the unproven intervention group (T2).

3.2 Randomization

Randomization was undertaken at the individual level. Data was stratified by age and gender of the child, and a subsequent stratified random sample of parents was drawn. Random numbers were then assigned to parents within each strata. These numbers were sorted from smallest to largest and calls were made based on this random order. Enumerators aimed to satisfy the simulated stratified table, which represented the optimal breakdown of strata proportionally. All randomization was undertaken using R Statistical Computing Software.

3.3 Data Collection

The baseline instrument was created through the adaptation of several validated surveys, namely Save the Children’s International Development Early Learning Assessment (IDELA) tool, UNICEF’s Multiple Indicator Cluster Survey (MICS), The Washington Group Short Set on Functioning, and the Parenting Sense of Competence Scale (PSOC). Additional demographic and COVID-19 related questions were constructed by the CUBIC and Save the Children Philippine’s teams. The survey included 45 questions.

Because of the COVID-19 pandemic, the data collection and enrollment process was undertaken via phone. Enumerators, working from home, made phone calls to enroll participants and conduct a brief baseline survey (~20 minutes). Data collection took roughly 2 weeks to complete.

It should be noted that the data collected in this baseline pertain to the secondary outcomes of the study, namely: parental well-being, child disability, parental engagement, early childhood development, socio-emotional learning, COVID-19 and positive parenting. Our primary outcome will be measured using the IDELA (International Development Early Learning Assessment) assessment. This assessment will only be conducted at the end of the study. No IDELA questions will be on the baseline, which is conducted for secondary outcomes only.

3.4 Ethics

The study design was submitted to Save the Children’s Ethical Review Committee and certification was granted on November 10th, 2020. Consent to participate in the study was collected from all respondents before the baseline survey was conducted. All data are protected by password on CUBIC’s Microsoft Sharepoint account.

3.5 Limitations

  • While the original target sample of 2,340 included a 30% cushion to account for attrition seen in past RCTs, budget and time restrictions only allowed for the enrollment and baseline data collection of 1804 participants. This may have caused a slight imbalance in gender within the control group, as targets within each balanced strata were not met. Moreover, if the team does see a 30% attrition rate in the study over time, this could also cause the study to lose power at the endline analysis. Finally, the children of respondents are all enrolled in an early learning program that pivoted from center-based to home-based due to the risks of COVID-19. This made data collection much more difficult, and the potential effects of this pivot are still unknown.

4 Findings

4.1 Demographics: Group Characteristics and Balance

Findings from the survey show that a majority of respondents were mothers (83%) and the primary language spoken within the household was overwhelmingly reported as Tagalog. A little over a third of respondents reported that their household monthly income was below Php 9,520, and a little over an additional third reported it was between Php 9,520 to Php 19,040. The other significant categories were Php 19,040 to Php 38,080 (7%) or “don’t know” (17%). Enumerators reported that a number of respondents were uncomfortable sharing their income, which explains the relatively high percentage of those who were categorized as “don’t know”. As can be seen in Figure 1 below, a majority of respondents completed secondary school, though there was a bit of variation in this response.

Tests were also run on several demographic variables to assess whether or not our randomization generated a balance between groups. As can be seen in table 2, randomization has succeeded in generating balance for most variables between groups with the exception of those highlighted in red. The most notable imbalance is that of gender between the control group and treatment one. It should, however, be mentioned that there were no suspected problems with the randomization, suggesting that any imbalances are likely due to chance.

Table 2: Balance Test P-Values
Variable C vs T1 T1 vs T2 C vs T2
Child gender 0 0.15 0.06
Child age 0.12 0.1 0.92
Caregiver type 0.27 0.43 0.4
Primary language 0.09 0.15 0.69
Birth order 0.34 0.36 0.31
Monthly income 0.3 0.05 0.13
Education 0.04 0.49 0.11
Have job/income 0 0 0
Spouse has job/income 0 0 0.48
Participated parent programs 0.12 0.76 0
Note: Values in red mean that a relationship exists between groups for that particular variable; they are not independent of one another and are therefore imbalanced.

4.2 Survey Results

4.2.1 Parental Well-Being

On average, parents sleep about 7.5 hours each night and spend about 2 hours on personal or free time each day. These averages did not vary much across groups, as can be seen in the trends shown in figures 2 and 3. Interestingly, there was a statistically significant relationship between these variables, suggesting that they are not independent of one another.

4.2.2 Child Disability: Washington Group Questions

Four disability identifiers were created Using the Washington Group (WG) on Disability Statistics’ guidelines (WG).These identifiers differ in terms of the threshold used for defining disability prevalence. They are defined as follows.

  • Disability 1: Where the level of inclusion in at least one domain/question is coded as “some difficulty” , “a lot of difficulty” or “cannot do at all”.

  • Disability 2: At least 2 domains/questions are coded as “some difficulty”, or any 1 domain/question is coded “a lot of difficulty” or “cannot do at all”.

  • Disability 3: Any 1 domain/question is coded “a lot of difficulty” or “cannot do at all”. This is the cut-off recommended by the WG.

  • Disability 4: any 1 domain/question is coded as cannot do at all".

The graph below shows the percentage of children in our sample that have a disability according to these four thresholds, separated by group.

As can be seen, the more “strict” the disability threshold, the less percentage of children have disability prevalence. Using the preferred indicator of the WG, we see that around 10 percent of children present disabilities across the treatment groups, and 5 percent in the control group.

While the majority of parents reported that their children had no difficulty with the listed WG criteria, some parents stated that their child had “some difficulty” self-care (such as washing or dressing), concentration (or remembering) and communication (understanding or being understood). It may be beneficial to adjust text messages to address themes surrounding these difficulties.

4.2.3 Parental Engagement

Overall, parents reported being quite engaged with their children. This was consistent across all groups. The exception was when parents were asked if they took children outside of their homes in the past three days, where roughly 40% of parents said no. This percentage was even higher in the control group (51%). This is not particularly surprising, however, given potential COVID-19 restrictions in Metro Manila.

4.2.4 ECCD

A majority of parents reported that their children were able to pick up small objects from the ground, follow simple directions on how to do something correctly, could name and recognize all written numbers from 1 to 10 and could identify or name at least ten letters of the Filipino alphabet. About a third of respondents said there child was sometimes too sick to play and roughly three quarters said their child could not read at least four simple, popular Filipino words.

4.2.5 Socio-Emotional Learning

With regard to children’s socio-emotional learning, a majority of respondents reported that their children get along with other children and are able to do things independently when asked. A little less than half of children get distracted easily, and about a third often kick, bite or hit other children and adults.

4.2.6 COVID-19

A majority of parents reported that their child covers his or her mouth when he or she coughs or sneezes and that their child usually washes his or her hands with soap and water for at least 20 seconds. There was not a very large difference between groups. It is promising that this positive behavior is happening already.

Table 3: Whether or not the child washes his or her hands
Response Control Treatment 1 Treatment 2
Yes 95.2% 96.6% 97.7%
No 4% 3.1% 2.3%
Don’t know 0.8% 0.3% 0.0%
Table 4: Whether or not the child covers his or her coughs or sneezes
Response Control Treatment 1 Treatment 2
Yes 94% 93.4% 88.2%
No 5.6% 6.5% 11.6%
Don’t know 0.4% 0.1% 0.2%

4.2.7 Positive Parenting

With regard to positive parenting, parents were asked to what extent they agreed or disagreed with six statements related to their own parenting beliefs and abilities. Figure X below shows the distribution of responses for each of the questions. Overall, parents tended to either agree or strongly agree they they 1) had the skills necessary to be a good parent/caregiver; 2) met their own personal expectations for expertise in caring for their child; 3) spend enough time talking about their child’s feelings and; 4) had the ability to teach their child at home.

However, the distributions were not concentrated as strongly when it came to their beliefs on punishment and the expression of their own emotions. Though many parents disagreed with the statement that “in order to bring up, raise, or educate a child properly, the child needs to be physically punished”, roughly a third of parents either somewhat agreed or agreed with the statement. This reveals the belief that physical punishment is still somewhat seen to be a practice used in raising children.

5 Conclusions and Recommendations

The results of the baseline survey were quite positive overall, though there is certainly some room for growth with regard to certain variables at endline. Texts may want to be slightly altered based on some of the information gathered. For example, the program team can deliver more targeted messaging around positive parenting, specifically with regard to beliefs on physical punishment. They can do this by replacing COVID-19 messaging, for example, with positive parenting, as the COVID-19 hygiene results were already quite strong.

Recommendations for implementation, further analysis and the endline evaluation are as follows: