PSY 5107 Week 7 – Signature Assignment: Assess Between and Within Subjects Experimental Research Designs
PSY 5107 Week 7 – Signature Assignment: Assess Between and Within Subjects Experimental Research Designs
Week 7 – Signature Assignment
The onset of the COVID-19 pandemic brought unprecedented effects on the education landscape for learners of different ages and educational levels. These effects were attributed to COVID-19 mitigation strategies such as lockdowns, stay-at-home orders, and closures of institutions such as schools. To ensure continuity of education and support of students’ needs during the crisis, strategies such as the transformation of traditional physical learning to remote online learning were necessitated (Bozkurt et al., 2022). This shift was inevitable with students and young children who rarely utilized online learning before the pandemic accessing and attending online classes from the comfort of their homes.
This new education delivery modality was associated with various adverse consequences. These include psychological distress stemming from anxiety in coping with remote learning, deepened social gaps from inequalities in access to online learning technologies, loss of organized school activities, diminished social interactions, and heightened unmet needs of vulnerable learners like special learners and those from low socioeconomic status (Hoofman et al., 2021). These issues are pertinent and require sustainable solutions in the event of future pandemics or continued adoption and embrace of remote learning. There is also a need for further research studies to explore the safety and efficacy of remote online learning which will guide policies and strategies for quality improvement.
The purpose of this paper is to explore the effectiveness of an online learning curriculum developed for K-12 students over four years. This will be achieved through a series of studies that seek to address certain goals of a local school administrator. The first goal is intended to evaluate whether the online curriculum offers students adequate gaining from age-appropriate learning materials assessed by the performance or passing of students during the end-of-semester exams. The second objective will assess whether the developed online learning curriculum works equally among students with and without computer experience. The third goal of the study will determine the presence of any improvement during the progression through the semester. The fourth goal will assess whether the developed online learning curriculum equally benefits both male and female students. Any gaps in each study design to meet these goals will be identified and appropriate strategies and potential solutions to these gaps proposed. Two scenarios will be chosen from the series of devised studies. There will also be a determination of whether the studies adopted a posttest-only between-groups design, a pretest-posttest between groups design, a matched pairs design, a block design, a pretest/posttest within-group design, or a longitudinal design.
Scenario A
This experimental study entailed the random allocation of participating students into two distinct groups. One of the assigned groups required a full undertaking of remote learning through an online curriculum. The other group required total engagement in the in-person physical curriculum. The effect of familiarity and computer experience on the adaptation to online learning was taken into consideration. Therefore, students with common traits and abilities in computer experience were identified, paired, and randomly allocated to either the online or in-person curriculum condition. Similarly, those without computer experience were also identified, paired, and randomly assigned to the two study groups. This ensures similarity in attributes of the study participants in the online learning intervention and in-person physical learning control group. The independent variable for this study was the online or physical type of course whereas the dependent variable was the level of learning of age-appropriate materials as per the final examination and assignment performance.
Study Research Design
The study in scenario A best describes a post-test only between groups research study design. This type of design has no pretest entity with measures to establish causality only being taken after an intervention (Krishnan et al., 2019). No measures of student performance through the administration of an examination at the beginning of the term were taken. A post-test after undertaking an online learning curriculum was achieved through an end-of-semester examination. The passing of this examination was then used to determine the extent of learning and effectiveness of the novel remote online learning modality. The comparison of change between the student performance before and after the intervention cannot be determined. An equivalent comparator group with similar attributes as the intervention group was used to identify any causal relationship between the exposure and outcome.
First Administrator Goal
The study design in scenario A met the local school administrator’s first goal. The use of online and remote learning that was extensively adopted during the COVID-19 pandemic has been shown to have consequences on the academic performance and achievements of learners (Ulum et al., 2021). The choice of the dependent variable in this study scenario ensured that the first goal was met. There was an administration of an examination and assignments at the end of the semester after undergoing the online learning course. This enables the educator to determine the extent of learning and understanding of age-appropriate materials offered virtually. The level of performance may have implications on the effectiveness of the remote online curriculum when the results are compared to those of physical learning students. The findings from the study can give meaningful conclusions on whether the distance learning adopted was beneficial with adequate contributions to learning or undermines the academic achievements of involved participants. Gaps in the learning process can also be identified with appropriate measures being undertaken.
Other Met and Unmet Administrator Goals
The second administrator’s goal was also met by this study design. The study was able to establish whether the developed online learning curriculum equally met the needs of students with computer experience capabilities and those without experience. This is because both the intervention and control groups had randomly assigned participants with and without computer experience. The identification and inclusion of this attribute in the study enabled targeted measures to assess any differences in the meeting of academic requirements of these two student groups. The effect of computer experience on the adaptation of distance learning can also be determined.
Click here to ORDER an A++ paper from our Verified MASTERS and DOCTORATE WRITERS PSY 5107 Week 7 – Signature Assignment: Assess Between and Within Subjects Experimental Research Designs :
The third and fourth goals were, however, not achieved. An assessment was only carried out at the end of the semester after the intervention. Any improvement during the course and progression cannot be determined. To address this shortcoming, multiple assessments and follow-ups of the study participants are necessary. The assessments in terms of examinations and assignments of both the online and in-person student groups can be administered at set intervals. Any variations in performance can be appropriately noted. The use of tools such as tests can be summative in evaluating performance and formative in providing meaningful feedback and insights into students’ learning (Yang et al., 2019). There was no allocation of learners into the study groups based on gender. This means the effectiveness of the distance learning course among both males and females cannot be evaluated. The study design should have been restructured to accommodate the gender attribute for this goal to be achieved.
Scenario D
The study in this scenario had a random allocation of study subjects into either an in-person physical classroom learning program or an online learning class group. Before the allocation, the participating learners were identified and chosen depending on various key attributes that could potentially affect their learning. The considered characteristics were abilities in computer experience and gender. Learners with computer experience were identified, paired, and randomly and equally assigned to either the physical or online learning study group. Similarly, those without computer experience capabilities were identified, paired, and randomly allocated to either the online or physical curriculum type. Additionally, students’ gender was considered through the pairing of males and females with subsequent allocation to the study groups. This led to four distinct groups within both sets of study groups. Thus both the remote learning and in-person study groups had a similar composition of male students with computer experience, male students without computer experience, female students with computer experience, and female students without computer experience. The level of learning of age-appropriate materials from the performance of the end-of-semester examination was the measured dependent variable in this study.
Study Research Design
Scenario D adopted a matched pairs study design. This type of study design establishes similarity in baseline key attributes of participants before allocation into the study groups (Xu et al., 2021). The subjects are paired based on matching characteristics that potentially affect the study outcomes, with one subject from the pair being randomly assigned to the intervention group whereas the other subject is assigned to the control group (Xu et al., 2021). The matching of pairs reduces study group variations and heterogeneity increases the comparability of variables and allows for studies with small sample sizes (Shan et al., 2018). The internal validity is improved with the resultant reliability of causal inference. This type of study design can be challenging especially in finding total pairs due to existing subject differences with chances of incomplete pairs that cannot join the study (Ramosaj et al., 2020). This may lead to the unintentional loss of potential study participants who may not find matches. The study considered various key attributes that formed the basis for forming paired matches. Gender and computer experience were the characteristics of focus that were used to pair the learners before being randomly allocated to either the online learning course or the physical classroom curriculum. The random allocation minimized potential selection and allocation bias.
First Administrator Goal Achievement
The first goal of the local school administrator was met. The dependent variable of the study was the extent of learning age-appropriate materials from the online learning intervention. Both sets of students were evaluated at the end of the semester through an examination. The adequacy of learning will be equated to the efficacy of the learning modality. Passing the final examination would indicate the effectiveness of the remote learning modality. The performance of the comparator group is essential in establishing any cause-and-effect relationships between the intervention and expected academic outcomes. This posttest measure may not fully attribute the academic performance to the modality of learning and additional research may be required to longitudinally follow up with the learners.
The Achievement of the Other Administrator Goals
The second and fourth goals of the administrator were also met. The matched pairs were based on the attributes of gender and computer experience. Comparison in the academic performance between learners with computer knowledge and skills and those without these abilities can be determined since these characteristics are shared by the study participants in the intervention group and the control group. The effectiveness of the novel remote online learning curriculum across the different male and female gender can also be established. This is because allocation into the study groups was also based on gender considerations. The gender-matched pairs may be easier and time-saving as compared to pairing based on computer experience which might be tedious to determine with potential for bias. Only one post-test measure was obtained after the intervention. No pretest or evaluation was carried out before the beginning of the semester. Additionally, no frequent evaluation was done during the semester. Any improvement or progress could be elicited thus the third goal was not achieved. This cross-sectional approach may not provide meaningful feedback. The study design should thus be structured to take a longitudinal approach to allow for monitoring of student progress.
Conclusion
The onset of the COVID-19 pandemic was associated with various adverse consequences witnessed at all levels of education. This was attributed to containment measures such as lockdowns and school closures. There was an unprecedented threat to the continuity of learning which necessitated appropriate innovative solutions. A transition from traditional physical classroom learning to remote online learning was one such strategic response. Some of the effects of online learning were psychological distress among online users, widened social disparities in access to the required technology, and loss of social interactions among others. The safety and effectiveness of online learning can be determined through scientific research studies which can adopt varying study designs. The choice of the design is dependent on the desired goals and outcomes of the studies. Restructuring the study designs may be necessary to successfully achieve the study objectives.
PSY 5107 Week 7 – Signature Assignment: Assess Between and Within Subjects Experimental Research Designs References
Bozkurt, A., Karakaya, K., Turk, M., Karakaya, Ö., & Castellanos-Reyes, D. (2022). The impact of covid-19 on education: A meta-narrative review. TechTrends, 66(5), 883–896. https://doi.org/10.1007/s11528-022-00759-0
Hoofman, J., & Secord, E. (2021). The effect of covid-19 on Education. Pediatric Clinics of North America, 68(5), 1071–1079. https://doi.org/10.1016/j.pcl.2021.05.009
Krishnan P. (2019). A review of the non-equivalent control group post-test-only design. Nurse researcher, 26(2), 37–40. https://doi.org/10.7748/nr.2018.e1582
Ramosaj, B., Amro, L., & Pauly, M. (2020). A cautionary tale on using imputation methods for inference in matched-pairs design. Bioinformatics (Oxford, England), 36(10), 3099–3106. https://doi.org/10.1093/bioinformatics/btaa082
Shan, G., Bernick, C., & Banks, S. (2018). Sample size determination for a matched-pairs study with incomplete data using an exact approach. British Journal of Mathematical and Statistical Psychology, 71(1), 60–74. https://doi.org/10.1111/bmsp.12107
Ulum, H. (2021). The effects of online education on academic success: A meta-analysis study. Education and Information Technologies, 27(1), 429–450. https://doi.org/10.1007/s10639-021-10740-8
Xu, X., Zhu, H., & Ahn, C. (2021). Sample size considerations for matched-pair cluster randomization design with incomplete observations of continuous outcomes. Contemporary Clinical Trials, 104, 106336. https://doi.org/10.1016/j.cct.2021.106336
Yang, B. W., Razo, J., & Persky, A. M. (2019). Using testing as a learning tool. American Journal of Pharmaceutical Education, 83(9), 7324. https://doi.org/10.5688/ajpe7324
For your assignment this week, select two of the following scenarios and answer all parts of the questions as prompted.
Although colleges and universities have utilized distance learning (i.e., online classes) for many years, teaching kindergarten through 12th-grade students using an online platform is much rarer. Unfortunately, the COVID-19 (Coronavirus) pandemic of 2020 forced K-12 schools throughout the world to turn to distance learning to teach children under 18. But how effective is an online curriculum for children who are only familiar with physical classrooms? To answer this question, your local school district asks you to conduct a series of studies focusing on the new distance learning curriculum they developed to teach K-12 students. Fortunately, the school district does have a control group option available to you should you need it. That is, some classrooms can use in-person instruction as the school has adequate safety measures in place to protect these students. The school administrator who hired you has four goals (though only the first goal must be addressed in all study designs):
First, the school administrator must know if online students are adequately learning material appropriate for their age group (as measured by an end-of-semester exam that all students must pass).
Second, the administrator would like to know if the curriculum they developed works equally well across students with different computer experience (experienced versus inexperienced).
Third, the administrator would like to know if students are improving as they progress through the semester.
Fourth, the administrator would like to know if the curriculum helps male and female students equally.
The school administrator recognizes that the COVID-19 pandemic is temporary but that other pandemics might occur in the future, or K-12 schools might naturally evolve an online component. Thus, the administrator would like to know how well distance learning students perform over the next four calendar years.
Assignment Instructions:
You have come up with a series of studies to test the administratorâ€s goals
Use the information in the study designs presented below to determine whether the design involves a posttest-only between-groups design, a pretest-posttest between groups design, a matched pairs design, a block design, a within-group design (pretest, posttest), or a longitudinal design.
Determine if the study meets the first goal of the administrator.
Determine whether the study meets at least one other goal, and if not, describe how you would alter the study design to meet at least one other administrator goal.
Scenario A
You design a study where you randomly assign students to one of two conditions. In one condition, students take the curriculum fully online. In the second condition, they take the curriculum fully in person. However, because you think familiarity and experience with computers might impact how students adapt to fully online classes, you first find students who are experienced with computers as well as students who are not experienced with computers. To make sure the two conditions are composed of students who share similar traits and abilities, you pair up experienced computer users and send one to Condition 1 and the other to Condition 2. You do the same for the next pair (and the next). You also pair up students who are not experienced with computers and similarly assign one to Condition 1 and the other to Condition 2. Here, the independent variable is the condition (online versus in-person teaching), and the dependent variable(s) is the extent to which the child learns the age-appropriate material (as measured by passing the final exam and/or assignments).
This study best describes which of the six research designs described in this lesson? Explain your response, including how this study design meets the first administrator goal.
Does this design meet any of the other administrator goals?
If yes, which one and why?
If no, how could you redesign the study to meet at least one other administrator goal?
Scenario B
You design a study where you randomly assign students to one of two conditions. In one condition, students take the curriculum fully online. In the second condition, they take the curriculum fully in person. All students take a pretest at the start of the semester and a post-test at the end of the semester. Here, there are two independent variables. One is the type of course (online versus in person), and the second is timing (pretest at the start of the semester versus post-test at the end of the semester). The dependent variable(s) is the extent to which the child learns the age-appropriate material (as measured by passing the final exam and/or assignments).
This study best describes which of the six research designs described in this lesson? Explain your response, including how this study design meets the first administrator goal.
Does this design meet any of the other administrator goals?
If yes, which one and why?
If no, how could you redesign the study to meet at least one other administrator goal?
Scenario C
You design a study where you randomly assign students to one of two conditions. In one condition, children will have in-person instruction in a physical classroom. In the second condition, the children will have online instruction. Both conditions will use the same curriculum, though the online version is remote. The independent variable will be the condition (in person or online), and the dependent variable(s) will be the extent to which the child learns the age-appropriate material (as measured by passing the final exam and/or assignments).
This study best describes which of the six research designs described in this lesson? Explain your response, including how this study design meets the first administrator goal.
Does this design meet any of the other administrator goals?
If yes, which one and why?
If no, how could you redesign the study to meet at least one other administrator goal?
Scenario D
You design a study where you randomly assign students to one of two conditions. In one condition, children will have in-person instruction in a physical classroom. In the second condition, the children will have online instruction. However, you first match students on multiple key characteristics that you think might impact their learning. In this case, you focus on their computer experience and gender. That is, you pair students with the same computer experience and then randomly assign one member of each pair to the online condition and the other member to the in-person condition.
Similarly, you pair inexperienced students, you pair males, and you pair females, and once again assign one member of each pair to the online condition and the other to the in-person condition. You do this for all students. Thus, male experienced computer users, male inexperienced computer users, female experienced computer users, and female inexperienced computer users are present in both online and in-person classes. The dependent variable(s) is the extent to which the child learns the age-appropriate material (as measured by passing the final exam and/or assignments).
This study best describes which of the six research designs described in this lesson? Explain your response, including how this study design meets the first administrator goal.
Does this design meet any of the other administrator goals?
If yes, which one and why?
If no, how could you redesign the study to meet at least one other administrator goal?
Scenario E
Unfortunately, you find that the school district cannot give you an adequate in-person comparison group; therefore, all students will participate in the new online curriculum. Since they still want your help to assess the program’s effectiveness, you decide to do so by assessing student knowledge at the start of the semester and comparing it to their knowledge at the end of the semester. The dependent variable(s) is the extent to which the child learns the age-appropriate material (as measured by passing the final exam and/or assignments)
This study best describes which of the six research designs described in this lesson? Explain your response, including how this study design meets the first administrator goal.
Does this design meet any of the other administrator goals?
If yes, which one and why?
If no, how could you redesign the study to meet at least one other administrator goal
Scenario F
You design a study where you randomly assign students to one of two conditions. In one condition, children will have in-person instruction in a physical classroom. In the second condition, the children will have online instruction. Fortunately, your study design allows you to follow and assess both sets of students multiple times over four years. Even as the students advance from one grade to the next, your study will determine if taking classes online helps or hurts their ability to pass assignments and final exams (the dependent variables in this study). The independent variables would be (a) the varying amounts of times assessments are taken (quarterly, mid-semester, end of the semester, the following semester, the next school year, etc.) and (b) the two conditions, in-person and online instruction.
This study best describes which of the six research designs described in this lesson? Explain your response, including how this study design meets the first administrator goal.
Does this design meet any of the other administrator goals?
If yes, which one and why?
If no, how could you redesign the study to meet at least one other administrator goal?
Length: 5-7 pages, not including title and reference pages
The completed assignment should address all of the assignment requirements, exhibit evidence of concept knowledge, and demonstrate thoughtful consideration of the content presented in the course. The writing should integrate scholarly resources, reflect academic expectations and current APA standards, and adhere to Northcentral University’s Academic Integrity Policy.
