UPDATE (28 March 2017)
Question: “Do we need to use an RCT design or will a mixed methods or some other approach be acceptable?”
Response: You may propose any research design. We are looking for the design that will yield the most valid and reliable results given the objectives, time frame and funding ceiling specified in the Call.
PREVIOUS QUESTIONS
Question 1: “… [I]s a specific organization implementing the program/promoting the app, or is it structured in a way that the app will be released publicly and then the research team would be looking into take-up of the app in addition to the other outcomes? In other words, would encouraging people to use the app also potentially be part of the research design?”
Response 1: The digital games will be made available for international download on Google Play and App Store on 20 March 2017. The target group for the intervention is out-of-school Syrian children aged 5 to 10.
The EduApp4Syria partners, through their networks of humanitarian and grass-roots organizations, will encourage households to directly download the games.
The aim of the impact evaluation is to measure the learning and psychosocial well-being effects on Syrian out-of-school children using the intervention. It is not to measure uptake and over-time usage of the games. Encouraging uptake and usage of the digital games could potentially be part of the research design for the impact evaluation. However, the evaluation must address learning and psychosocial effects when children actually use the games regularly over some time (i.e., to control that the treatment is actually received). These findings will be important for encouraging more people to download and use the games.
Question 2: “Given the short timeframe for the investigation (completed draft 31 August) we anticipate small effects from the games; to detect these effects reliably will require relatively large sample sizes. Is there an opportunity to extend the research timeframe?”
Response 2: At this time, the research timeframe is not envisioned to be extended. However, we remain mindful of operational challenges that might require reconsideration of the timeframe.
Question 3: “What measures will the partners or the developers take to ensure that there is use of the apps by refugee children?”
Response 3: The EduApp4Syria partners, through their networks of humanitarian and grass-roots organizations, will encourage households to directly download the games.
Question 4: “Is use of the app envisioned in formal or non-formal environments during the field tests?”
Response 4: The digital games are envisioned as a learning supplement primarily targeted for household use.
Question 5: “What size are the populations into which the apps are distributed? To what ages will the apps/intervention be provided?”
Response 5: The target group for the intervention is out-of-school Syrian children aged 5 to 10. They live primarily in Syria and neighboring countries Turkey, Jordan, Lebanon, Iraq and Egypt. There are approximately 2.5 million Syrian out-of-school children aged 5 to 17. The primary target group is the 5- to 10-year-old segment of this group.
Question 6: “As issues of effect, locale, population affect cost projections, any information that can help frame such issues will be of value.”
Response 6: Applicants can propose a research location that is amenable to the budget allocated at this time.
Question 7: “The investigation of psychosocial impact describes an approach with a treatment group and a control group. We question whether withholding the resource from the control group is ethical if the apps and related programs are effective. Will ACR and DL4D take steps (e.g., provide workshops/ counseling, etc.) to address ethical issues surrounding withholding the app resource and related programs from the control group? What other determinations or provisions have ACR and DL4D made to ensure that the control-group population is treated ethically?”
Response 7: Applicants are expected to discuss in the technical proposal ethical issues in conducting the research and how they propose to address such.
Question 8: “We understand that both apps will be available on Google Play (for free). Are there any structural barriers (e.g., IP address limitations to GP access) that will be instituted by the developers or partners to ensure that control-group members don’t download the apps independently, or to ensure that a treatment-group member doesn’t use both apps?
Response 8: Applicants should propose ways to address the spill-over risk.
Question 9: “Information about distribution of the apps and user support during the proposed data collection periods is very important.”
Response 9: The digital games will be made available for international download on Google Play and App Store on 20 March 2017. The target group for the intervention is out-of-school Syrian children aged 5 to 10.
The EduApp4Syria partners, through their networks of humanitarian and grass-roots organizations, will encourage households to directly download the games.
The research team will have engagement with both digital game operators to support technical issues that arise during data collection.
Question 10: “We would like to inquire if group assessments may be an option the funding consortium would be willing to consider for both the literacy and the psychosocial well-being portions of the work.”
Response 10: There may be an opportunity to consider group assessment. However, it would be preferable to propose a methodology that would require less time per child. For example, a shorter test instrument (e.g., only letter sound, real word and fluency subtests) and more efficient data collection (e.g., two data collectors – one person explains the test procedure and fills out the background questionnaire and the other tests the child). While this Call seeks research that assesses the differential learning gains between the two digital games, our key concern is whether or not the children who play the games improve their reading skills more than children who do not.
