Research question
A formal statement of the goal of a study. The research question states clearly what the study will investigate or attempt to prove. It is a logical statement that progresses from what is known or believed to be true (as determined by the literature review) to that is unknown and requires validation.

Operational definition
Variables must be defined in terms of measurable behaviors. The operational definition of a variable describes the variable. There are two ways by which we can operationally define a variable; by how it is measured and by how it is used to classify subjects.

Variable
Variables are the observable phenomena that can be studied. A variable varies, that is it can be observed to change, or can take on different attributes. Gender is a variable and it can take on two different attributes, male and female.

Concept of control
Keeping all variables affecting the study constant so that the pure effect of the independent variable on the dependent variable is measured. See internal validity.

Causality
Knowing that the independent variable effects the dependent variable in a predictable manner; knowing that one activity causes another to occur.

Hypothesis
A statement that expresses the probable relationship between variables.

Hypothesis - Descriptive hypothesis
Asks a specific question regarding some phenomenon.

Hypothesis - Directional hypothesis
Expresses the effect of an independent on a dependent variable.

Treatment
The levels of the independent variable manipulated during a study, e. g., students were given either one or two weeks of skill training.

Observation
A measurement made of the dependent variable, e. g., measuring the subject's level of proficiency in a skill.

Ethics - Consent
This is the procedure by which research subjects choose whether or not they wish to participate in a research study. Consent involves three elements: capacity, information, and voluntariness. All three elements must be satisfied for consent to be given.

Ethics - Capacity
One of three elements of consent. The other two are information and voluntariness. The ability to acquire or retain knowledge, and the authority, or legal qualification, to perform an act. Is the subject able to decide if he wants to participate? Does a child or a person who is mentally retarded have capacity?

Ethics - Information
One of three elements of consent. The other two are capacity and voluntariness. This element consists of insuring the subjects are told, and they understand, the purpose of the study and their roles as subjects.

Ethics - Voluntariness
One of three elements of consent. The other two are information and capacity. The subject chooses to be in the study of his/her own free will and is free to withdraw from the study at any time. There must be no element of force, fraud, deceit, duress, ulterior form of constraint or coercion to get a subject to participate. Paying subjects is usually OK., but offering prisoners a parole to participate in a study is not.

Ethics - Harm
One of the most important issues in all of research ethics is that subjects not be harmed by your study. To avoid physical harm is obvious, but other areas need to be avoided also. These areas are: psychological stress, personal embarrassment, and humiliation.

Ethics - Privacy
Every subject has the right to keep private the fact that he/she participated in your study, and the right that information given to you not be linked to them. Research often is based on information obtained from the subjects. The information will be used in the study, and perhaps published, but it must be done in a way that insures the individual's anonymity.

Ethics - Deception
Deception in research involves the misrepresentation of facts related to the purpose, nature, or consequences of a research study. The omission of facts is the same as misrepresentation. Subjects need to be fully informed in order to give consent.

Designs - Differences design
A design using two or more independent groups of subjects, or multiple observations of one of more groups, where the purpose is to compare the groups to see if they are different after the treatment.

Designs - Pre/Post design
A design where the dependent variable is measured before and after treatment.

Designs - Two group design
A design with two groups of subjects.

Designs - Three group design
A design with three groups of subjects.

Designs - Factorial design
A design with two or more independent variables. See analysis of variance.

Designs - Co-variance design
A design where a mathematical attempt is made to compensate for the effect of a variable on the interaction between the independent and dependent variables, e. g., we might use a measure of pre-treatment language skills to attempt to eliminate the effects of prior learning in a study of the relationship between teacher years-of-experience and student learning.

Designs - Repeated measures design
A design where the dependent variable is measured three or more times.

Designs - Time series design
A design characterized by daily, or very frequent, observation of the dependent variable. A variant of the repeated measures design often used in studies of the effect of behavior modification treatments.

Baseline
Observations made on the dependent variable before any treatment

Designs - Clinical trials
A category of designs, used in health care settings, where care is taken to control for placebo effects. Commonly called "double blind" studies because usually the subjects do not know if they are getting a real treatment or a placebo, and the health care provider does not know which he/she is administering.

Clinical trials
A category of designs, used in health care settings, where care is taken to control for placebo effects. Commonly called "double blind" studies because usually the subjects do not know if they are getting a real treatment or a placebo, and the health care provider does not know which he/she is administering.

Outcomes research
A category of research inquiry where the focus is on the objective measurement of changes in function or performance. A program or treatment is judged based on the degree to which it meets a criterion of terminal performance.

Action research
A category of research inquiry that focuses on measuring the outcomes of a program or intervention applied to non-random samples and in the presence of extraneous variables using multiple outcome measures. The multiple measures, if showing consistent outcomes, can compensate for the reduced internal and external validity, and allow for valid conclusions to be drawn.

Designs - Relationship (correlation) design
A relationships question would be: Is the relationship between diet and heart disease strong enough that we can predict the onset of heart disease from a knowledge of diet? This type of research question leads to correlation/regression designs. Correlation/regression designs are ex post facto designs because while we may define IVs we do not manipulate them. We do not manipulate the IV because we gather all the data at one time, hence there is no time to manipulate the IV. These designs are similar to observational studies except that we hypothesize a correlation between variables, rather than simple descriptions.

Designs - Descriptive design
Descriptive research questions give rise to observational designs. Observational designs use three general ways to gather data: observation, interview, or survey. These three methods are obvious in their application. In the first we observe behavior, in the second we ask people to describe their behavior orally, and in the third we seek written replies.

Designs - Survey
Descriptive research questions give rise to observational designs. Observational designs use three general ways to gather data: observation, interview, or survey. These three methods are obvious in their application. In the first we observe behavior, in the second we ask people to describe their behavior orally, and in the third we seek written replies.

Designs - Observational
Descriptive research questions give rise to observational designs. Observational designs use three general ways to gather data: observation, interview, or survey. These three methods are obvious in their application. In the first we observe behavior, in the second we ask people to describe their behavior orally, and in the third we seek written replies.

Validity
See external or internal

External validity
A study has external validity when you first define your population, then randomly select a large sample. With a random sample of sufficient size research findings can generalize to the larger population. A study that generalizes has external validity.

Internal validity
This refers to the adequacy of our study design and the degree of control we have exercised in our data gathering. Good internal validity is insured by application of the concept of control. This concept is very important in research. By control we mean that all variables except the dependent variable are controlled by the experimenter. In this way if the dependent variable changes during the study then that change is due to the changes the experimenter made in the independent variable(s).

Internal validity - Pretest influence
The effects (e.g., cueing and practice) of taking one test upon the results of taking a second test. Sometimes the subjects can learn about the dependent variable by just taking the pre-test. This additional learning can confound the effect of the independent variable on the dependent variable.

Internal validity - Hawthorne effect
Being in an experiment sometimes changes the response of the subjects. New treatment methods may be exciting, and people improve due to the thrill of it all, the increased attention, and other extraneous variables. Sometimes also called the placebo effect, but they are not quite the same.

Internal validity - Bias in group composition
Biases or conveniences in creating comparison groups that cannot be assumed to be equivalent (e.g., the groups are not equal because they were not randomly chosen. For example, if one school uses one teaching method and a second school uses a second method, then the groups are biased because it is unrealistic to assume the school populations, or hospitals or work groups, are the same).

Internal validity - Experimental mortality
Loss of more subjects from one group than from the other. This may make groups unequal.

Internal validity - Test practice
The effects (e.g., cueing and practice) of taking one test upon the results of taking a second test. Sometimes the subjects can learn about the dependent variable by just taking the pre-test. This additional learning can confound the effect of the independent variable on the dependent variable.

Internal validity - Instrumentation error
Error of measurement due to: 1) Changes in the assessment instrument (e.g., shortening a test, adding different items, changing the scoring procedure), 2) Changes in the observers (e.g., different observers at O1 and O2, some observers using different standards than others, or training of observers changes from one treatment to the next), and 3) Changes in the equipment (e.g., a fault in the equipment, non-standardization of equipment prior to study, loss of calibration).

Internal validity - Statistical regression
Changes in scores over time due to unreliability of measuring devices; especially troublesome when using subjects selected on the basis of extreme scores.

Internal validity - Selection-Maturation Interaction
Biases in the selection of groups to be included in the study may differentially be affected by the time between assessments. For example, if the subjects are children and the average age of one group is older than the others, then the maturation process will effect the older group differently than the younger groups. If changes due to maturation can be confounded with changes due to the independent variable then the internal validity of the study is reduced.

Internal validity - Maturation
General events/experiences occurring to participants over an extended period of time, e.g., growing older, fatiguing, etc.

Internal validity - History
Specific events unrelated to the study, occurring between the first and second measurements in addition to the experimental treatment.

Sample
A group of people or events drawn from a population.

Population
Any set of people or events from which the sample is selected and to which the study results will generalize

Sample source - Random sampling
People or events are selected from a population in such a way that every member of the population has an equal chance if being selected.

Sample source - Simple random sampling
Selection from a population using a Random Number Table or some other random process (slips of paper in a hat).

Sample source - Stratified random sampling
Sampling from sub-groups in the population, i.e., to have a random sample of 100 people evenly divided by gender, you would divide population into male and female groups and randomly select 50 from each group.

Sample source - Proportional random sampling
To insure maintenance of sub-group proportions, i.e., divide population of a school into male and female groups. Suppose a certain school has 8 women to every man, in order to have a random sample of 100 people balanced on gender we need to randomly select 80 women and 20 men.

Sample source - Systematic random sampling
Drawing every kth person, i.e., to get a random sample of voters you select every 10th person from the Voter Registration Roles at the courthouse.

Sample source - Cluster random sampling
A method to get random samples when the population is large, there are important control variables, and you can only study a small sample (i.e., to get a random sample of 60 administrators of hospitals in the United States, you could group hospitals into clusters based on private/public ownership, and big/medium/small hospitals and then randomly select ten subjects from each cluster. This method is a more elaborate version of stratified sampling.

Assignment - Random assignment to groups
This method insures distribution of extraneous variables and is usually the best way to assign subjects. This method is best if the group size is twenty or more. Random assignment assures an equal distribution of all extraneous variables. For any extraneous variable (age, weight, distribution of blood types), you will find the average of the variables (or the distributions) to be the same for each group if you use random assignment.

Assignment - Matching of subjects across groups
This method insures distribution of control variables by matching pairs of subjects in the different groups, i.e., you find pairs of subjects who are very similar to each other on control variables, such as age, sex, race, etc., and then you randomly assign one of the pair to one group and the other to the second. Good when the sample has to be small.

Assignment - Pre-existing groups or non-random assignment
Often a study uses groups that are pre-existing, i.e., people who die or who live after a heart attack. Other times you use the first ten patients for treatment A, and the second ten for treatment B, or you wish to compare schools or classrooms. These groups have non-random assignment and are also not necessarily representative of the population you want to study. Non-random assignment reduces the internal validity of a study, because the groups are different at the start of the study.

Generalizability
If the sample is representative of the desired population then our results will generalize.

Variable - Independent (experimental) variable
There are two types of independent variables: Active and attribute. If the independent variable is an active variable then we manipulate the values of the variable to study its affect on another variable, e. g., practice, the amount of time spent in training. An attribute variable is a variable where we do not alter the variable during the study, e. g., gender, being male or female.

Variable - Dependent (criterion measure) variable
This is the variable that is affected by the independent variable, e. g., If I praise you, you will probably feel good, but if I am critical of you, you will probably feel angry. My response to you is the independent variable, and your response to me is the dependent variable, because what I say influences how you respond

Variable - Control variable
A control variable is a variable that effects the dependent variable. You control for a variable by holding it constant, e.g., keep humidity the same, and vary temperature, to study comfort levels.

Variable - Extraneous variable
This is a variable that probably does influence the relationship between the independent and dependent variables, but it is one that we do not control or manipulate. Often research studies do not find evidence to support the hypotheses because of unnoticed extraneous variables that influenced the results.

Variable - Confounding variable
Extraneous variables which influence the study in a negative manner are often called confounding variables.

Variable - Continuous variable
Continuous variables can be measured using either the interval or ordinal scale of measurement.

Variable - Categorical variable
Categorical variables can be measured using either the ordinal or nominal scale of measurement. It is often desirable to call your variable categorical when you have an ordinal measurement scale.

Scale of measurement - Interval
Interval measurement uses a scale where the distances between the points on the scale are equal across the scale, i.e., measuring with a ruler is an interval measurement because inches are carefully defined to be a uniform length.

Scale of measurement - Ratio
A ratio scale is almost the same as an interval scale, e.g., the distances between the points on the scale are equal across the scale.

Scale of measurement - Nominal
A nominal scale consists of categories with no order. The variable, gender of student, is a nominal variable with two categories: female and male.

Scale of measurement - Ordinal
An ordinal scale is a scale where phenomena are ordered or ranked, i.e., arrange a group of ten people by height and number the tallest person one and the shortest person ten.

Measurement Scale
See scale of measurement

Data (datum)
A datum is one piece of information, a score, a value, a number. Data are several such pieces.

Normal (Symmetrical) distribution
Many variables that exist in populations express a singular shape when shown as a frequency graph. This shape is a curve that has the shape of an inverted bell. Approximately 70% of the data fall within one standard deviation above and below the mean. Height and weight are variables that express a normal distribution when plotted for a large, random sample of people.

Measures of central tendency
There are three methods used to find the midpoint of a set of values: mean, median, mode.

Mean
The average score in a distribution.

Median
The score that lies in the middle of a distribution

Mode
The score or category in a distribution that occurs the most often.

Dispersion measures
These are values that describe how the data are distributed around the middle of a distribution. The three common measures of dispersion are: variance and standard deviation, and the range.

Variance and Standard deviation
Variance is a measure of how scores are distributed around the mean. The variance is the average squared deviation around the mean. The standard deviation is the square root of the variance, and is the more meaningful statistic. These statistics are measures of the differences between the subjects in a sample or population.

Standard deviation
See variance

Range
The spread of scores in a distribution between the lowest and highest scores. This is a coarser measure of variance than the standard deviation.

Standard error of the mean
The standard error of the mean is the expected standard deviation of the means for a large number of people taking a test many times. It is an estimate of the error of measurement.

Error of the mean
The standard error of the mean is the expected standard deviation of the means for a large number of people taking a test many times. It is an estimate of the error of measurement.

Correlation (Regression)
The correlation coefficient tells how accurately we can predict one variable from another, and it ranges from -1.00 to 0.00 to 1.00. A correlation coefficient of 0.00 means that we cannot predict one variable from the other. As the correlation coefficient gets larger (approaches 1.00) or smaller (approaches -1.00), the ability to predict one variable from another improves. When the value is 1.00 (or -1.00) then you can predict one variable from another with perfect accuracy. Whether the value is positive or negative refers to the nature of the relationship. Negative correlation coefficients are obtained when as one variable increases the other decreases, i.e., the older I get, the fewer hairs I have on my head. Regression is a mathematical method for describing the relationship between two or more variables. You can use the regression equation to generate a straight line that shows all the predictions of Y from all values of X. You can compute a regression line for any two variables, but the predictions may or may not be accurate. It is how near the correlation is to 1 or -1 that determines the accuracy of prediction.

Parametric/Non-parametric
Parametric statistical tests use continuous data, e.g., data gathered using ratio or interval scales of measurement. Non-parametric tests use categorical data, e.g., ordinal or nominal.

Pretest/Posttest
Measurements taken before and after a treatment or intervention.

Treatment
A period of time when the independent variable is manipulated. Often call an intervention.

Validity of a test or measure
Test Validity refers to the degree to which our measuring strategy (instrument, machine, or test) measures what we want to measure.

Validity - Content validity
Content validity is established if your measuring instrument samples from the areas of skill or knowledge that compose the variable, i.e., if a test on addition has a good selection of 2 + 2 type problems then it is probably valid.

Validity - Construct validity
Construct validity is based on designing a measure that logically follows from a theory or hypothesis. For example: suppose creativity is defined as the ability to find original solutions to problems. I design a test for creativity where subjects are to list as many uses for a paper clip as possible. I designate subjects who list more than 30 uses as creative. I have developed a test with construct validity. The test is valid to the extent that the task (uses for a paper clip) is a logical application of my theory about creativity. If my theory is wrong or if my measure is not a logical application of the theory, then the measure is not valid.

Validity - Predictive validity
Predictive validity refers to the ability of my measure to separate subjects who possess the attribute I am studying from those who do not. If I design a test of aptitude for flying an airplane, it has predictive validity if subjects who score high learn to fly, and if subjects who score low crash.

Validity - Concurrent validity
Concurrent validity is used when a valid measure exists for your variable but you want to design another measure that is perhaps easier to use or faster to take. Suppose you design a short test for manual dexterity to replace a much longer one. In this case you have subjects take both the old and new tests. Your new test has concurrent validity if the subjects make similar scores on both tests. Concurrent and predictive validity are similar.

Reliability
Reliability is the consistency with which our measure measures. If you cannot get the same answer twice with your measure it is not reliable.

Nominal scale
See scale of measurement

Ordinal scale
See scale of measurement

Interval scale
See scale of measurement

Ratio scale
See scale of measurement

Distribution
A set of observations of a variable, usually arranged in a frequency graph (See Graph - Frequency distribution of a variable).

Distribution - Normal distribution
Many variables that exist in populations express a singular shape when shown as a frequency graph. This shape is a curve that has the shape of an inverted bell. Approximately 70% of the data fall within one standard deviation above and below the mean. Height and weight are variables that express a normal distribution when plotted for a large, random sample of people

Distribution - Skewed distribution
When a distribution has more values at one end of its range, then the distribution is said to be skewed.

Graphs
Visualization of numerical data.

Graph - Frequency distribution of a variable
An xy plot with the categories of a variable on the bottom (x) axis and the frequency with which the category was observed on the vertical (y) axis.

Graph - Box plots of groups on one variable
A visualization to compare groups using the median and quartiles.

Graph - Scatter plot of two variables and regression line
An xy plot with the values of an independent variable on the bottom (x) axis and the values of a dependent variable on the vertical (y) axis. Points are drawn at the intersection of the x and y values for each subject. It gives a picture of how two variables relate to each other. The regression line is a mathematical expression or summary of the relationship.

Null hypothesis
The condition of no differences or no relationship. In statistics, when a test statistic indicates that no differences between groups are present or no relationship between variables exists then it is said that the null hypothesis is supported.

Alternative hypothesis
The condition of a difference or a relationship. In statistics, when a test statistic indicates that differences between groups are present or a relationship between variables exists then it is said that the alternative hypothesis is supported.

Level of significance
The probability of obtaining a test statistic by chance alone. If this probability is small then it is unlikely that the statistic was obtained by chance. If the statistic was not obtained by chance then the difference or relationship was caused by something other than chance. If the study has good internal validity, the manipulation of the independent variable caused the difference or the variables are meaningfully correlated.

Chi-square test
The chi-square test is used with relationship studies where you have categorical variables. Chi-square is a non-parametric test because the variables are categorical. The Chi-square test will help us decide if a relationship exists between the variables.

t-test
A parametric statistical test used to compare two groups or one group measured twice.

t-test - Independent means
Use this test when you have two-groups and a single measurement of the dependent variable (DV). This test tells you if the mean of one group is different from the mean of the other.

t-test - Dependent means or matched pairs
Used when you have one group and you measure the DV twice. It tells you if the mean of the first measurement is different from the mean of the second.

t-test - One group
If you only have one group, a descriptive research question, and a single measure of the DV you can see if your sample mean is different from the mean of the population (that is you can if you know the population mean). Since you have only one group you do NOT have an independent variable. This test is useful to see if your sample is representative of the population.

Analysis of variance
Used when you have three or more groups and measure the DV once. It tells you if one of the group means is different from the other means. If you have one independent variable (IV) it is a one-factor AOV, and if you have two IVs it is a two-factor AOV. The two-factor AOV tells you three things: 1) if there is a difference between the DV means for the first IV, 2) if there is a difference between the DV means for the second IV, and 3) if there is a difference between the group means. An AOV design with two factors has one group for each combination of the categories of the two IVs. If one IV has 2 categories and the second IV has 3 categories then the study will have 6 groups (3 x 2). AOV designs can be used with as small as four subjects per group and still obtain valid results.

Analysis of variance - Interaction
Interaction means that there is one combination of Factor A and Factor B that produces a bigger difference than another combination. For example, perhaps women do better with group therapy, and men do better with individual therapy.

Analysis of variance - Factor
Each independent variable that is part of and analysis of variance design is called a factor.

Interaction
See analysis of variance

Factor
See analysis of variance

AOV
See analysis of variance