PSYC 502.05 (Spring 2020) Research Methods Chavajay Research Activity #2: Critiquing Science and Media Portrayals of Science REVISED DUE DATE: Saturday, 2/8/2020 – To be uploaded to the course website
Report of critiques (one page, double-spaced) – To be uploaded to course website Goals:
To examine a study and inferences made based on its findings critically
To scrutinize sources and a report of a study in the media judiciously
To understand the limitations of reported findings
This research activity involves critiquing an empirical study (involving two experiments) published in a peer-review journal and a report of its findings in a mainstream news magazine. Nowadays, people in many parts of the world are inundated with information from multiple digital and nondigital sources. Thus, it has become essential for one to develop strong skills in critically evaluating the credibility of sources and information these sources disseminate. Such skills in information literacy and scientific thinking are required in conducting reviews of studies and media accounts of these studies. A review of a published study (or a manuscript submitted to a journal) consists of summarizing and assessing:
a.) the study’s relevance and fit within and contribution to the literature
b.) the logic and correspondence among the study’s research questions, hypotheses/expectations, and methodology (e.g., did the study examine what it proposed to address?; were its measures valid and reliable?)
c.) the soundness of analyses performed on the study’s data (e.g., were the analytical procedures conducted appropriate for the nature of the data collected?)
d.) the coherence among the study’s findings, interpretations and claims made based on these findings, and applicability of these findings in real world practice (e.g., were the study’s expectations confirmed?; were the study’s claims supported by the evidence found?)
These same considerations are applicable in critically analyzing the reports of studies in the media. For this assignment, you must critically examine and compare Mehr et al. (2013) and Sifferlin (2013). Then, for each of these sources, compose responses addressing the points on page two. In total, your report should be limited to one page (double-spaced). Resources you may consult in completing this assignment include:
Heath, W. (2018) – Chapter 2 (especially pages 47-49)
Publication manual of the American Psychological Association: The official guide to APA style. (2020). (7th ed.). American Psychological Association.
PSYC 502.05 (Spring 2020) Research Methods Chavajay Empirical Study (Primary source – an original scholarly study reporting firsthand data) Mehr, S. A., Schachner, A., Katz, R. C., & Spelke, E. S. (2013). Two randomized trials provide no consistent evidence for nonmusical cognitive benefits of brief preschool music enrichment. PLOS ONE, 8(12), e82007. https://doi.org/doi:10.1371/journal.pone.0082007
1.) Read this empirical article carefully. 2.) Identify one problematic aspect of this study (in either or both experiments) and briefly and concisely explain why this aspect is problematic.
3.) Propose a way this problematic aspect could be addressed in the design of a future study investigating the effects of children’s music instruction on their cognitive development. Your proposed idea does not have to be limited to the ways in which the Mehr et al. (2013) study was conducted. Mainstream media outlet (Secondary source) Sifferlin, A. (2013, December 11). Do, re, mi, fa-get the piano lessons: Music may not make you smarter. Time Magazine. https://healthland.time.com/2013/12 /11/ do-re-mi-faget-the-piano-lessons-music-may-not-make-you-smarter/print/
1.) Read this magazine article carefully.
2.) Identify and explain at least one way in which the portrayal of the Mehr et al. (2013) study and its findings were miscommunicated.
3.) Propose a revision for the miscommunication you identified that aims to more accurately portray the Mehr et al. (2013) study, including its findings and implications. Two Randomized Trials Provide No Consistent Evidence for Nonmusical Cognitive Benefits of Brief Preschool Music Enrichment Samuel A. Mehr1,2*, Adena Schachner1¤a, Rachel C. Katz1¤b, Elizabeth S. Spelke1 1 Department of Psychology, Harvard University, Cambridge, Massachusetts, United States of America, 2 Harvard Graduate School of Education, Cambridge, Massachusetts, United States of America Abstract Young children regularly engage in musical activities, but the effects of early music education on children’s cognitive development are unknown. While some studies have found associations between musical training in childhood and later nonmusical cognitive outcomes, few randomized controlled trials (RCTs) have been employed to assess causal effects of music lessons on child cognition and no clear pattern of results has emerged. We conducted two RCTs with preschool children investigating the cognitive effects of a brief series of music classes, as compared to a similar but non-musical form of arts instruction (visual arts classes, Experiment 1) or to a no-treatment control (Experiment 2). Consistent with typical preschool arts enrichment programs, parents attended classes with their children, participating in a variety of developmentally appropriate arts activities. After six weeks of class, we assessed children’s skills in four distinct cognitive areas in which older arts-trained students have been reported to excel: spatial-navigational reasoning, visual form analysis, numerical discrimination, and receptive vocabulary. We initially found that children from the music class showed greater spatial-navigational ability than did children from the visual arts class, while children from the visual arts class showed greater visual form analysis ability than children from the music class (Experiment 1). However, a partial replication attempt comparing music training to a no-treatment control failed to confirm these findings (Experiment 2), and the combined results of the two experiments were negative: overall, children provided with music classes performed no better than those with visual arts or no classes on any assessment. Our findings underscore the need for replication in RCTs, and suggest caution in interpreting the positive findings from past studies of cognitive effects of music instruction. Citation: Mehr SA, Schachner A, Katz RC, Spelke ES (2013) Two Randomized Trials Provide No Consistent Evidence for Nonmusical Cognitive Benefits of Brief Preschool Music Enrichment. PLoS ONE 8(12): e82007. doi:10.1371/journal.pone.0082007 Editor: Marina Pavlova, University of Tuebingen Medical School, Germany Received March 20, 2013; Accepted October 23, 2013; Published December 11, 2013 Copyright: ß 2013 Mehr et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. Funding: This research was supported by a grant from the Dana Foundation awarded to ESS. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. Competing Interests: The authors have declared that no competing interests exist. * E-mail: smehr@wjh.harvard.edu ¤a Current address: Department of Psychology, Boston University, Boston, Massachusetts, United States of America ¤b Current address: Eliot-Pearson Department of Child Development, Tufts University, Medford, Massachusetts, United States of America those taking drama or no lessons. However, two subsequent RCTs failed to find corresponding IQ effects with other types of music training [8–9]. Moreno et al. [8] administered the WISC-III after Kodàly music or painting training and found no evidence for a greater increase in IQ in the music group. In a second study, Moreno et al. [9] administered two subtests of the Wechsler Preschool and Primary Scale of Intelligence—Third Edition (WPPSI-III [11]), an IQ test for younger children that corresponds to the WISC-III, after computer-based music or visual arts training. The music group significantly outperformed the visual arts group on the Vocabulary subtest but not the Block Design subtest; the authors reported no overall effect of music training on general intelligence [9]. Two further RCTs reported similarly mixed results: Costa-Giomi [5] found significant increases in children’s general intelligence (as measured by the Developing Cognitive Abilities Test [12]) after two years of piano lessons, but not after one or three years. Bilhartz, Bruhn, and Olson [6] tested kindergarteners on subtests of the Stanford-Binet Intelligence Scale [13] after group parent-child music classes and found significant increases in performance on one subtest, but no overall Introduction Young children’s lives are saturated with musical activities: parents worldwide sing regularly with their children and most preschool programs incorporate musical activities into their curricula (for review, see [1]). In spite of the pervasiveness of preschool music activities, however, the effects of early music education on children’s cognitive development remain unclear. Many studies have reported associations between music training and improvements in cognitive skills [e.g., 2–3], though the largest correlational study on the topic reported no such effect [4]. Only five published randomized controlled trials (RCTs) have investigated causal effects of music training on areas of cognition seemingly unrelated to music [5–9] – effects that would support the often-repeated claim that ‘‘music makes you smarter’’ – and no clear pattern of results has emerged. Schellenberg [7] reported a significantly greater increase in general intelligence (as measured by the Wechsler Intelligence Scale for Children—Third Edition; WISC-III [10]) in children randomly assigned to keyboard or voice lessons, compared to PLOS ONE | www.plosone.org 1 December 2013 | Volume 8 | Issue 12 | e82007 PLOS ONE | www.plosone.org Weekly Kindermusik [38] group classes (sometimes includes parents) 4.9–6.72 167 Yes No Musical Aptitude Profile [49]; BruininksOseretsky Test of Motor Proficiency [50]; Canadian Achievement Test 2 [51]; Coopersmith Self-Esteem Inventories [52] g via Developing Cognitive Abilities Test ([12], 3 of 3 subtests administered) 52.5 hours over 90 weeks No-treatment control Weekly private piano lessons with ‘‘traditional curriculum’’ 122 67 117 [5] Notes on Main or Ancillary Effects Citation Count1 Correction for Multiple Comparisons? Overall Cognitive Effect? Additional Measures Administered Primary Measure of Child Cognition Training Length Comparison Group Type 2 Music Curriculum Age of Children at Testing (in years) Final Sample Size Initial Sample Size Study [7] 144 132 7.08 (.237) Weekly group ‘‘standard’’ piano lessons or Kodály [53] voice lessons Weekly group drama lessons or no-treatment control 28 hours over 36 weeks g via Wechsler Intelligence Scale for Children ([10], 12 of 12 subtests administered) Kaufman Test of Education Achievement [54]; Behavioral Assessment System for Children [55] Yes No 374 Combined keyboard and voice group shows significantly larger increase in g than combined drama and control group (t(130) = 1.99, p = .049, d = .1843). [8] 37 32 8.82 (.375) Weekly group classroom music in Kodály, Orff, and Wuytack methods [57] Weekly group painting lessons 55 hours over 24 weeks g via Wechsler Intelligence Scale for Children, Portuguese adaptation ([58], 10 of 10 subtests administered) Portuguese European Reading Battery [59]; tests of speech and pitch discrimination [60] with EEG recording No No 183 Music group has significantly fewer errors on 1 of 3 subsets of the reading battery (Tukey test, p,.052) [9] 64 48 5.40 (5.65) Daily computer-based group music listening activities Daily computer-based group visual art activities 15 hours over 4 weeks g via Wechsler Preschool and Primary Scale of Intelligence ([13], 2 of 7 subtests administered) Executive function ‘‘go/no-go’’ task (designed by authors) with EEG recording No No 59 Music group shows significant increase in performance on 1 of 2 subtests (Vocabulary; F(1,62) = 11.37, p = .0013, partial g2 = .33), visual art group does not. Note. Studies are reviewed in chronological order, from 1999 [5] to 2011 [9].
