Internal validity and external validity are the two main categories of accuracy in quantitative research studies. The level or measure to which differences in the dependent variable is caused by the experimental procedure and not a random variable is known as internal validity. The extent whereby the findings can be applied to populations other than the population being studied is known as external validity. This article discusses the threats to experimental validity.
THREATS TO EXPERIMENTAL VALIDITY
This part of article discusses the threats to external and internal validity.
threats to internal experimental validity:
Various threats to external validity have been discussed in this part of the article.
threats to internal experimental validity, HISTORY:
one of the imporatant threats to experimental validity is the history. We are holding a series of training sessions for instructors as a component of our safe-school initiatives with the goal of enhancing their feelings of security and well-being. You conduct a pretest and a posttest to gauge the instructors’ overall sense of wellbeing over the three-month training session as part of the study. All is well thus far. Let’s imagine that your research was carried out during the High School massacres, when two students went on a shooting spree that left both students and instructors dead.
Has this incident had any impact on your research? Certainly! Your posttest results will probably be lower than they may have been if this awful occurrence hadn’t occurred.
Past is an occurrence that takes place beyond the intended study methods and has an impact on the variable being studied. Obviously, history poses a greater risk to your research the longer it is. Histories do not refer to the players’ individual histories.
History is a particular issue in research that omits a control group. Keep in mind that in our scenario, only one group received both the prior test and a follow-up test. The risk of past history was taken into account by keeping it constant between the experimental and control groups, so even if the research included a control group that did not participate in the seminars but was informed about the tragedy, it would still be possible to determine whether the seminar benefited the instructors who attended.
threats to internal experimental validity, MATURATION:
one of the other important threats to experimental validity is maturation.You observe as a kindergarten instructor that many of your pupils struggle to handle colored pencils properly. To help pupils be more prepared to handle a pencil when they enter preschool, you decide to focus on techniques that encourage better methods of using a colored pencil. After conducting a mini-experiment to gauge each student’s fine motor abilities, you spend five minutes every morning “training correct pencil gripping.” After three months, you reassess the pupils’ fine-motor abilities and determine that your intervention was successful.
Consider this before you give yourself a pat on the back. Was the variation caused by your fine-motor “treatment” or was it merely a result of your learners’ organic change? Maybe they got better with time or with maturity achievement! Maturity, then, is a personal transformation brought on by progress, this can happen while one’s body, mind, or emotions are working.
How could you? a maturation check? One approach, once more, is to use a control group; the control organization could be developing as well. Any variations among groups during the therapy could be ascribed to the independent variable because both the treatment and control groups are maturing simultaneously.
threats to internal experimental validity, TESTING:
one of the other important threats to experimental validity is testing. Giving subjects a pretest is a common practice in experimental studies. Let’s imagine that a researcher is investigating the viability of an instruction intended to increase the comprehension of pupils of the Vietnam War. The investigator offers the class a pretest to gauge their level of comprehension before beginning of the study and then conducts a 2-hour session on the conflict (the treatment). Learners do better when the identical test is administered after two days.
Was it the instruction that helped them do better, or was it just the proximity of the exam dates? On the subsequent test, did they merely repeat the information from the first test? Testing danger describes modifications in respondents’ posttest results brought on by what they recall from a pretest. By establishing a control group, investigators can account for an assessment risk; nevertheless, a more straightforward strategy might be to merely lengthen the gap among the pretest and posttest to reduce the likelihood that participants will remember the pretest questions.
INSTRUMENTATION:
Not every test is made equally. Different tests have different levels of difficulty. Many are more valid and trustworthy than other. If the variables in your judgments are not precisely measured, the evaluation of performance will be off. Therefore, even if you show a distinction among groups at the end of your research, it may not be because the independent variable was changed but rather as a result of inadequate evaluations. If instruments with poor reliability and validity are utilized, internal validity is put at risk.
In contrast to the previous examples, the validity threat offered by instrumentation is not taken into account by establishing a control group. Be certain to utilize instruments with high levels of validity and reliability to account for instrumentation. Make certain that the monitors are objective, well-trained, and not worn out by extended periods of observation in studies where monitors are documenting actions to verify the reliability and validity of observations.
STATISTICAL REGRESSION:
Suppose you are conducting a study in which the only participants are the students with the highest and lowest algebra pretest scores in the class. The use of high scores could lead to analytical regression. On an algebra posttest, even if the groups received various treatments, there was a movement of scores to the mean. This implies that pupils who received a score in the top 10% would decline toward the mean, and those with scores in the bottom 10% would move upwards toward the mean.
You wouldn’t know if the research was successful until the conclusion. The statistical analysis you used or the way you were treated explained the difference between the groups.
While it is uncommon for investigators to carry out research employing the individuals with the greatest or smallest score in a group, it is feasible that investigations could look at either the top or smallest individuals. Statistical regression would be applicable in both situations. The tendency for scores to gravitate towards the mean, pushing the higher scores down and the lower scores up, is known as statistical regression.
How can statistical regression be controlled? Control groups come to the rescue, indeed! Regression will occur across both groups, and it will be managed, if people are chosen for the treatment and control groups in a similar way.
DIFFERENTIAL SELECTION OF PARTICIPANTS:
The assumption made by experimental researchers who randomly divide people into groups is that this procedure accounts for any potential variations in inherent ability. Nevertheless, not all assignments are made at random achievable because of moral or useful reasons. When random selection is used not possible, unequal participant selection may be a challenge because the investigator must make use of existing organizations.
Let’s say you have to deal with existing classes given that you have to use a convenience sample or cluster random sample for your research. You will use pre-existing groups that were established before to the study’s begin in either scenario. You have no idea how the groupings were created, and they may not even be comparable to one another.
Consider a scenario in which one class is an honors class and the other is a more diverse group of pupils. Because of this, the groups would be unique before being subjected to the treatment condition. If the students who received honors outperformed the varied persons, what may be the cause of the distinction: the way they were treated or the individual distinctions between the groups (differential selection)?
To eliminate the possibility that volunteers will be chosen at random or will be used. Most researches place people in artificial groups that might be different and serve as a pretest. If the investigation calls for the use of large groups. issues with diverse choices commonly in research on education.
MORTALITY OR ATTRITION
If the students who received honors outperformed the varied persons, what may be the cause of the differentiation: the way they were treated or the particular distinctions between the groups (differential selection)? To eliminate the possibility that volunteers will be chosen at random or will be used.
The majority of studies place people in artificial groupings that might be different and serve as a pretest. If the investigation calls for the use of large groups. issues with diverse choices commonly in research on pedagogy.
Utilizing a pretest is one efficient technique to mitigate the risk of participants leaving and influencing the makeup of the groups. In order to control for death or subject attrition, if a person with a specific score leaves the group being studied, someone who was part of the control group with a similar score on the pretest can be deleted. Investigations that contrast control groups to a treatment group undergoing a therapy course might have issues with mortality.
It is expected that mortality will be higher in the control group if individuals in the control group do not receive any communication or attention from the investigator while those in the group being treated are in constant contact as a result of the intervention.
This poses a major risk to internal validity because it is likely that highly motivated control group participants will be the ones most inclined to seek assistance externally and therefore leave out. It may be possible to manage this danger to validity by keeping in touch with participants in the control group throughout the course of lengthy research.
THREATS TO EXTERNAL VALIDITY:
THREATS TO EXTERNAL VALIDITY, Pretest – Treatment Interaction:
First, only when a pretest is employed in a study is pretest-treatment interaction problematic. In some cases, the participants are affected by the therapy or become more sensitive to it, and the results would have been differently if the individuals hadn’t been pretested. When might this might place?
Think about the following. You are running a study in which an activity created to increase children’s exposure of variety serves as the experimental intervention.
The purpose of the prior test was to gauge students’ awareness of their own views regarding diversity. Students may become more conscious of the problems with diversity and its significance in their life as a result of the pretest itself.
Could your findings be applied to all other groups getting training if the experimental group proves to be more receptive to a variety of concerns at the end of the research, or would they simply apply to groups who took the pretest on attitudes toward diversity? Using a pretest that does not make participants more aware of the behaviors you are trying to change is one technique for regulating for pretest-treatment association.
Some exams mislead you about their genuine intent by utilizing vague inquiry or item names. To control for pretest-treatment interaction, professional researchers frequently employ sophisticated designs that incorporate treatment groups that have not been pretrained.
THREATS TO EXTERNAL VALIDITY, Multiple Treatment Interaction:
Some research studies exposed individuals to a number of therapies as part of a larger program or just to a variety of therapies. When this happens, it might be challenging to pinpoint what therapy caused any observed differences. For instance, a researcher might carry out investigation to find out how attendance at charter schools affects academic performance.
In fact, a charter school may have a variety of behavioral facets, all of which may have an impact on academic performance. A charter school, for instance, might have more compact class sizes and uniformity. After the research’s conclusion, it appears that the pupils in the charter schools are doing better than their peers in the normal public school settings.
Can the findings be applied to all charter schools or simply those that use the same therapy techniques as the institution being inquiry? Limiting the number of treatments administered or administering various therapies at various times helps prevent multiple therapy interactions. Find comparative groups that received various components of a treatment that has multiple parts. One may contrast private institutions with lower enrollments and those with uniforms, for instance.
THREATS TO EXTERNAL VALIDITY, Specificity of Variables:
Every investigation has a particular setting, a specific population, a particular group of participants, variables that are assessed using a specific instrument, and a specific set of conditions. The generalizability of the study is constrained by how particular the circumstances are.
A research with fourth-grade students in an inner-city educational district, using a particular teaching strategy, in the initial period of class, with a particular teacher, and with reading achievement measured using the ABC accomplishment exam, could only be useful in a situation comparable to that.
Another explanation for why investigations frequently have the goal of replicating earlier studies using various groups in various locations and various measurements is due to this. Replicating research can be helpful because there is little generalizability in any one experiment.
However, by randomly choosing individuals and educational institutions that are varied, using tests that have been recognized as trustworthy and reliable, and using interventions that are simple to execute in different contexts without the need for particular assets or conditions, critiques of particularity of factors can be minimized.
Treatment Diffusion:
Every time you do an investigation with an intervention and a control group, there is a chance that both categories will interact, ” dispersing” or obfuscating the medication. With the subsequent learners she employed in her doctoral study on method use, recall, or consciousness, one of us (M.G.L.) encountered this issue.
The control group learnt to employ a straightforward recurrence approach, while the intervention group engaged in game-based meta cognition training. Although every student received individualized instruction, a few learners in the control group used the meta cognition technique (game) that had been given to the experimental group when they took part in the posttest memory task.
The second graders said that their friends (in the treatment group) were telling students regarding the activity when the writer questioned them regarding how they learned to employ this tactic. An intriguing result that the investigator did not foresee! Evaluate how you may structure your research in order to minimize or prevent the interaction between your experimental group and control group.
Be mindful of the chance that your trial team might interact with your control group, giving the control group with knowledge regarding the experimental therapy.
Experimenter Effects:
Good experimental researchers are aware of how their actions may affect the research they are doing. The investigator always plays an autonomous and distinct part in studies that are quantitative, keep it in mind. Certain situations, nevertheless, when the investigator may have an unintended impact on the study’s results. These effects may be brought on by the investigator’s character traits or when the investigator’s goals have an impact on both the conduct and the academic achievement of the people being studied.
Individual traits encompass race, ethnic background, age, and feelings. In order to show a distinction among the control and experimental groups, the investigator may unintentionally give the experimental group any unintended advantage (more testing time, slower instructions, greater focus, encouragement for the right answers, etc.) that affects the conduct of the study contributors.
This phenomenon is referred to as investigator biases. Keep in mind this prejudice can happen accidentally or even unknowingly, particularly if investigators have high hopes for one group over the other. In order to show a distinction among the control and experimental groups, the investigator may unintentionally give the experimental group any unintended benefit (more testing time, slower instructions, greater focus, encouragement for the right answers, etc.) that affects the conduct of the study contributors.
This phenomenon is referred to as investigator biases. Keep in mind this prejudice can happen accidentally or even unknowingly, particularly if investigators have high hopes for one group over the other.
Reactive Effects:
Have you ever participated in a study for research? If yes, you are aware that taking part in a study can have an impact on your emotions, actions, and views.
The Hawthorne effect is one way that reactive organization shows up. The Hawthorne effect happens when a participant’s conduct is influenced by their involvement in the study as a whole rather than specifically by the therapy. This conclusion is supported by a reputable investigation carried out at the Western Electric Company’s Hawthorne Plant in Chicago, thus the plant’s name.
During this investigation, the investigators looked into how to increase productivity among employees. The quantity of light intensity served as the research’s distinct variable, and productivity of employees served as the variable that was dependent on the study.
The efficiency of the workforce grew as light intensity increased. The scientists made the decision to reduce the light intensity and observe what would occur. Guess what, then? Even according to settings with decreasing light intensity, productivity increased. The staff members boosted their output regardless of how they were treated since they believed the adjustments showed that the corporation cared for them, despite the fact they were informed they were taking part in a research project.