Chapter+Five+-+Quantitative+Research

=Chapter Five: Quantitative Research=

The purpose of this chapter is to provide a discussion of the various research designs used in quantitative research studies. Each design should be clearly described, including procedural steps regarding sampling and data collection as well as the type of statistical analysis that would be used to determine the outcome(s) of a study using that design.

1. Overview
Quantitative research involves the collection of numerical data. There are many reasons for conducting quantitative research. Researchers may want to test a theory; so research is designed and carried out to see if the theory holds true in practice. If a large enough sample size can be obtained, quantitative research c an be used to substantiate generalizations about certain groups and to apply the results to many people. For educators, quantitative research is often done to see how a variable effects the outcome in a classroom. For example, the researcher may wish to try a particular teaching method and see how that effects academic achievement, engagement, or social behaviors. Although it may seem impossible, some data that does not appear to be quantitative can be quantified for study purposes. For example, if you ask students the question, “Do you eat breakfast before you come to school?” you could give them answer choices of “Never” “Seldom” “Often” and “Always”. By assigning numbers to those answers you are quantifying the frequency with which they come to school hungry. Then, for example, you can do some statistical analysis to see the relationship between eating breakfast and grades.

1.1 Constructs and Variables
A construct is an aspect of the research that you are studying. It is generally not directly measurable. Some examples are motivation, learning, and intelligence. Your construct could be an independent variable, a dependent variable, or a constant, depending on what you are studying and how you design your research. In a research proposal, it is very important that your construct is well-defined and that you state clearly how it is to be measured.

A variable is a condition or characteristic that can take on different values or categories. A much studied educational variable is intelligence, which varies from low to high for different people. Age is another variable because it varies. Eye color is another variable because it is a property of an object (eye) that can take on different values (blue, green, hazel, and brown). Variables can be:

**discrete**- variables that take on a small set of possible values **continuous**- variables that take on any value between the lowest and highest in the scale (EX scores on a test)
 * quantitative**- varies in degree or amount (EX ages of students)
 * categorical**- varies in type or kind (EX male or female)
 * independent**- causes a change in another variable (EX [amount of studying] affects...)
 * dependent**- influenced by an independent variable (EX amount of studying affects...[test grades])
 * intervening (or mediating)**- a causal chain between two other variables (EX amount of studying [and student motivation] affects...test grades)
 * moderator-** changes the relationship between other variables (EX personality type of students)
 * extraneous**- variables related to the outcome other than an independent variable (EX better reading abilities of students in the study)
 * confounding**- variables not controlled by researcher but may cause result (EX older students in the intervention group)

2.Experimental Designs
True experimental designs are those done using the purest of scientific methodology. The ultimate purpose is to determine cause-and-effect relationships (Johnson & Christensen, pg. 41). Experimental designs contain subjects that are randomly assigned so that even the principal investigator does not know which participants are in the experimental group and which are in the control group. This design model also incorporates a placebo that is decided and controlled by the researcher. The placebo allows for later comparison between the effects of the intervention and addresses the need to reduce bias. Two examples of strong experimental designs are the prettest-posttest control group design and the posttest-only control groups. With randomized pretest-posttest or posttest only design, serious threats to validity are addressed. As such, the experimental design works well in medical research and in laboratory controlled experimentation. However, this design does not work well in educational research; most especially in classroom research. The reason why experimental design doesn't work in the classroom is that a teacher cannot assign a control group and a teacher would always know all his/her students; thus there could be no anonymity. The sample group within a school or classroom is also too small to make any valid inferences from the data gathered through the experiment. For many years, the limited application of experimental design to educational and classroom based research limited the research done by educators, especially at the classroom level. In recent years, action research has become the venue to bring research into the classroom to inform decision making by teachers using modifications to the experimental design.

2.1 Single subject designs
Single-case experimental design: This design is utilized with a single participant to investigate the effect of an experimental treatment. Researchers usually apply this design when they want to focus on one topic regarding something that is not common among other participants. The example used in the book involved investigating the learning strategies of an exceptionally intelligent student (pg. 341). This single-case experimental design would be appropriate because a class might not have more than one exceptionally intelligent student. In this case, the researcher would be studying a non-typical case. Another possible example is a former student, low SES and ELL, who dropped out of high school due to distractions of his job, his car, and his girlfriend. He was, however, able to refocus and reprioritize his life and did go back to school. He's now 21 and will graduate this December with plans to go on to college to become a teacher. The researcher could investigate the factors which caused this student to return to school, newly motivated and college bound. Discovering the factor(s) that helped this student's resilience in the face of many obstacles might assist teachers, administrators, and counselors in helping other such students.

According to the author, a single-case experimental design would also be a time-series design. This means that the design would require repeated investigation of the dependent variable before and after the experimental treatment condition. Even though single-case experimental design is focused on a single participant, you may use this design in a group setting such as when the researcher would like to investigate the efficacy of an independent variable with typical or atypical cases. When this occurs, using the single-case experimental design is appropriate.

Single-Case Experiment Design is divided into three other designs but all are seen as single-case experimental designs, which are the A-B-A and A-B-A-B design, multiple-baseline design, and changing-criterion design.

3. Nonexperimental Designs
Nonexperimental research designs are ones in which the independent variable is not manipulated by the researcher. This happens in some research because there are variables that cannot be controlled by the researcher. For example, the researcher cannot manipulate ethnicity or gender. Also, the researcher should not manipulate variables that would be unethical to manipulate – for example, withholding known successful treatments from people who are ill. In nonexperimental design studies, the researcher looks for patterns that occur naturally. Because this design is nonexperimental, there will not be a strong cause-effect conclusion to be made. In spite of that, much educational research is done this way.  The steps of a nonexperimental design are to decide the problem to be investigated, decide on the variables to include in the study, collect data, analyze the data, and make conclusions based on the results. The researcher needs to be very careful not to assume causation just because the pattern in the study has been observed before. Rather than assuming causality, the researcher should look at his or her discovery as a hypothesis requiring further empirical study.

3.1 Causal-comparative Designs
Causal-comparative designs are nonexperimental research where the researcher studies the relationship between one or more categorical, independent variables and also with quantitative and dependent variables. This type of design tends to look for 3 types of evidence:


 * Differences in effects between groups
 * Differences in causes between groups
 * Differences in outcomes of treatments or interventions**

Causal-comparative research is not considered experimental because the researcher does not have complete control over the independent variable – because it is impractical, impossible, or unethical. Due to the fact that the researcher has no control over one or more variables being studied, the findings are merely suggestive of a causal relationship between the independent and dependent variable. A causal relationship is not considered as strong as in experimental designs. In causal-comparative research, the researcher is studying two or more groups and one independent variable. As sited in the textbook, examining the relationship between gender (IV) and math performance (DV) is considered causal-comparative; gender being the categorical variable and math performance being the quantitative variable (pg. 360). As you can see, “causal” is part of the name of this research type, but you must be careful about reading too much into that. Does gender really “cause” differences in mathematical abilities, or does some environmental difference in the way we treat boys and girls account for the difference?

3.2 Correlational designs
Correlational design is a research approach that tries to find the nature of the relationships between a set of variables. This relationship is not determined by the researcher; it cannot be manipulated. It is naturally present within a group or sample. The relationship between variables can be positive, negative, or there could be no correlation at all. A relationship between variables does not imply causation. This means that the researcher cannot show proof that one variable alters another variable. For example, some correlational studies might suggest that there is a relationship between the amount of time children watch TV and their levels of aggression, but it cannot show that watching TV heightens or diminishes aggression among children. There are other environmental and/or biological variables that must be taken into consideration. Image accessed 11/23/08, courtesy of [|Burke Johnson, University of South Alabama.]

Researchers use a correlation coefficient to determine the correlation strength of the variables under study. This coefficient can range from -1.00 to +1.00. A positive coefficient close to 1.00 determines a strong positive relationship, while a coefficient near to -1.00 indicates a strong negative correlation. In a positive correlation, as one variable increases, the other increases as well (e.g. SAT scores and college achievement). In negative correlations, as one variable increases, the other decreases (e.g. education and years spent in jail). If the coefficient is close to 0 then there is no correlation between the variables.

There are different ways in which correlational studies can be conducted such as naturalistic observations, surveys or questionnaires, and archival research.

3.3 Survey Designs
Survey research is a nonexperimental research method based on questionnaires or interviews. Surveys are used when researchers want to target a large sample size so that generalizations can be made about specific populations. They are also used to research relationships and often use correlational methods. Surveys can also be used to find differences, compare groups to see if there is a correlation, or results can be used to make predictions of outcomes (a current event example includes exit polling attitudes in the 2008 election). There are two types of surveys, cross-sectional and longitudinal surveys. **Cross-sectional surveys:** These surveys are given to a large group of people at a single point in time, and the data is used to compare groups (e.g., race, gender, SES). These surveys are generally given as a “snap shot” of current attitudes or opinions. ** Longitudinal surveys: ** These surveys are given over time so that data can be compared across time. There are three longitudinal designs:
 * __Cohort design:__ This approach takes a sample (a specific population) and assesses them over time. Different individuals can be sampled within the cohort, so it’s not always the same individual being surveyed.
 * __Panel Design:__ This is similar to the cohort design. However, the same individuals are surveyed over time. The most frequent problem with this design is mortality.
 * __Trend design:__ This design takes random samples in a population and asks them the same questions over a period of time. This design is generally used to determine the existence of trends.

4. Statistical Analysis
Statistical analysis can be divided into two areas: descriptive statistics and inferential statistics. Descriptive statistics is used to simply describe sets of data. What does the data say about some phenomenon? Descriptive statistics often overlap inferential statistics. Inferential statistics allow us to draw a sample from a population and infer the results. A population is the entire set of events in which you are interested. This number could be infinite, and therefore, never completely collected. Instead we draw a sample or subset of the population. The sample is the set of actual observations. If we draw conclusions from single or limited observations, we risk a generalizing error, but we cannot make an unlimited number of observations either. We must do something in between, and that is inferential statistics. We can draw a sample of observations from a population and use that sample to infer something about the characteristics of the population. The conclusions we draw are much more accurate if the sample is a truly random sample. A random sample is one in which every member of the population has an equal chance of inclusion.

Image accessed 11/23/08 from [|SouthAlabamaEdu.]

4.1 Descriptive statistics
Definition: Descriptive statistics - statistics that focus on describing, summarizing, or explaining data. After data collection in a quantitative study is complete, analysis can be done to develop descriptive statistics that can be used to give a better picture of the results. A starting point would be to identify the mode, median, and mean of the data. The __mode__ is the most frequently occurring value within the data set. The __median__ is the value that represents a midpoint of the ordered set of data when arranged lowest to highest. The __mean__ is an average of all represented values. Standard deviation will represent the average distances of values from the mean. Normal distribution is when the mean, median, and mode fall roughly at the same point in the data distribution. This may also be referred to as a “bell-curve”. Variations in the data distribution may alter mean, median, and modes and lead to positive or negative skews when plotted. In addition to curves created from graphing data on x-y axes, descriptive statistics may also include visual depictions such as pie charts, bar charts, and histograms.

Image accessed 11/23/08 courtesy of the [|CenterforPublicEducation.Org] Looking at the measures of central tendency (mean, median, and mode) as mentioned above tells you something about what is most typical in your data. In addition to this, you can also make sense of your data by looking at frequency distributions, which tells how frequently a certain variable occurs. Looking at measures of variability tells you how spread out your data is. Measures of relative standing tell you where a particular value falls in your data relative to other values (like a percentile score).

4.2 Inferential statistics
Definition: Inferential statistics – statistics that go beyond the immediate data and infer the characteristics of populations based on samples: they use the law of probability to make inferences and draw conclusions. There are four major points of inferential statistics: ***Sample- a subset of cases from a population
 * Population- a complete set of cases
 * Statistic- a numerical characteristic of a sample
 * Parameter- a numerical characteristic of a population**

Examples of inferential statistics found in content areas for teachers: Reading and Language Arts - cause and effect/making inferences/reading between the lines Math – estimate/think through/add two and two/problem solve Science – Newton’s law: for every action there is an equal and opposite reaction/analyze/dissect/investigate.  As we are now dealing with “Value Added” and “Accountability”, isn’t it logical to use inferential statistics to read test results in order to determine why some students do well and why others do not do well on standardized tests? Teachers’ informal assessment and observations should also play a major role in data analysis in order to help administrators understand data results. Central office administrators automatically assume and categorize teachers as accomplished or struggling according to test results. They do not take into consideration the student population of individual classrooms or campuses. They compare campuses with other campuses with similar population groups and come to unrealistic conclusions for failure. Not all campus are designed the same, no matter how similar the populations groups may be; nor are all classrooms within a campus. Central, regional, and school administrators might look at inferential statistics in order to see the whole picture and not just numbers. Color coded data only shows the surface of results but not the ingredients that produced the results.

There are two major branches in inferential statistics: estimation and hypothesis testing. __Estimation__ - In **point estimation** you use a single number in your estimate. For example, if a teacher historically has his/her students score a mean of 75% correct on the TAKS test and you are estimating the same this year that would be point estimation. The other type of estimation is **interval estimation**. This is where a range of numbers provide the estimate. Researchers construct confidence intervals. For example: the district office probably looks at how many school have 100% passing on the TAKS test, how many have 90% passing, etc. They then would make an estimation that students in the future at these schools should achieve relatively similiar results.

__Hypothesis Testing__ - Hypothesis Testing is concerned with how well the sample data support a null hypothesis and when the null hypothesis can be rejected. For example, a principal may wish to see if a certain method of a third-grade teacher is causing a difference between that class and another third-grade class. The null hypothesis is that this method is not causing any difference. The alternative hypothesis is that this method is causing the difference. The principal assumedly is hoping to prove the null hypothesis incorrect and the alternative hypothesis as correct.

An important concept is “statistical significance.” If your finding is statistically significant, that means the result was probably not just due to chance. You can also calculate an “effect size indicator.” This tells you how much the effect on the dependent variable is due to the independent variable.

5. Summary
Quantitative research involves the collection of numerical data. Researchers may choose to conduct a quantitative research study to test a theory. Quantitative research is often conducted in educational research to see how a variable affects the outcome in a classroom. Constructs are generally not directly measurable. It is important in a research proposal, that the researcher clearly defines the construct and clearly states how it is to be measured. The ultimate purpose of experimental designs is to determine cause-and-effect relationships. Examples of experimental designs includes prettest and posttests control groups and the posttest-only control groups. Experimental design does not work well in educational research due to the inability for the teacher to have anonymity amongst the control group. Single-case experimental design is used with a single participant. This is a time-series design involving a large period of time and repeated investigation. Nonexperimental research designs are ones where the independent variable is not manipulated by the researcher. This occurs because the variables cannot be controlled by the researcher. The researcher is looking for patterns which occur naturally. Most educational research is conducted this way. Casual –comparative designs are nonexperimental research in which the researcher studies relationships between variables looking for three types of evidence. Correlational design is an approach which tries to find the nature of relationships between a set of variables. This is not determined by the researcher and cannot be manipulated. Survey research is a nonexperimental method using questionnaires or interviews. Cross-sectional surveys are given to a large group of people at one time and the data is used to compare groups. Longitudinal surveys are given over time so that the data collected can be compared across time. There are three types of longitudinal designs. Statistical analysis can be divided into two areas descriptive statistics and inferential statistics. Descriptive statistics focus on describing, summarizing, or explaining data. Inferential statistics are statistics which go beyond the immediate data and infer the characteristics of populations based on samples. They use the law of probability to make inferences and draw conclusions.