This chapter deals with the most important aspect of the whole research process. This aspect is drawing conclusions from your data. After going through all the steps of thinking through your hypotheses, designing the study, gathering and analyzing the data, you now can see if any new knowledge was gained.
If you recall from the first chapter, step seven of the research process is: Conclusions drawn regarding the hypotheses. The process of drawing conclusions has three steps and is best approached by laying out all your information in front of you:
First, gather all your information in one place. Do the following:
Always look at all three aspects of validity.
The sample can be affected by the loss of subjects. for example, if a number of subjects from one group dropped out, the results could be thrown off since the groups may no longer be equivalent. Here again, unless you can verify that the groups were still equal, even with the loss of subjects, you should not assume the internal validity was intact.
The reliability and validity need not be perfect for a study to provide useful information. Remember with imperfect reliability and validity you cannot decide on the hypotheses in complete confidence, but you can decide with reservations. Thus, an imperfect study can contribute new knowledge, if it is interpreted with the limitations of the study always in mind.
Third, if the reliability and validity are acceptable, you can go on to decide about each hypothesis. Based on the statistical test results decide if each hypothesis is supported or denied (true or false). Below is an example of how to interpret statistical test information and then apply it to a hypothesis.
Example. Suppose you are studying the effect of occupational stress on health. Two work settings are involved; a OT department and a MT department in the same large, urban hospital. A journal article said that MT's had very stressful jobs. You wonder if a MT's job is more stressful than an OT's. The professional employees in each department took the Work Stress Test to assess the level of stressful agents in their work environment. On the Work Stress Test scale, zero represents no stress and 100 represents maximum stress.
The data are as follows:
Mean Standard Number of employees Deviation Men Women Total ------------------------------------------------------------ OT department 76.8 9.3 9 10 19 MT department 85.9 6.5 3 19 22A t-test gives the following results: t = 4.12. This value is significant using the .05 probability level as a criterion. The t-test is the appropriate statistical test to use to see if there are differences between the means of two groups when the DV is measured once.
As a measure of level of health, you obtain the number of sick days from personnel records for the four month period prior to administering the stress test. These data are as follows:
Mean Standard Number of employees Deviation Men Women Total ------------------------------------------------------------ OT department 4.1 2.1 9 11 20 MT department 5.2 3.5 12 19 31A t-test gives the following results: t = 1.65. This t-value is not significant at the .05 level.
The research hypothesis is that the department with higher stress levels will have poorer health. Do the data support or not support this hypothesis?
To evaluate the research hypothesis given in the example above, we first need to evaluate the two t-tests that were done. Each statistical test has its own pair of hypotheses: the null and alternative. The null hypothesis is that there will be no difference between the means except due to chance. The alternative hypotheses are that the MT department will have 1) higher stress levels and 2) more sick days. The alternative hypothesis is based on the journal article.
The t-test value for the stress test indicates that the probability that the null hypothesis is true is smaller than one-in-twenty. Since the null hypothesis is probably not true, you decide to accept the alternative hypothesis that the stress levels of the MT's were higher than the OT's.
The t-test value for the number of sick days indicates that the probability that the null hypothesis is true is larger than one-in-twenty. Since the null hypothesis has a reasonable probability of being true, you decide to accept the null hypothesis that the there was no difference in the number of sick days.
The research hypothesis states the department with higher stress levels will have poorer health. Given that there is no difference between the departments on the number of sick days, it is best to conclude that the data do not support the research hypothesis? Even though the MT department had higher stress levels, and the mean number of MT sick days was higher than OT, since the difference was not significant it is best to decide the research hypothesis is not true.
A possible problem with the internal validity of the study is present due to the sample. There are fewer subjects from the MT department who took the Work Stress Test than had the number of sick days reported. The study would be more valid if the sick days data were restricted to those who took the stress test. Because sick days are taken based on the judgment of the employee they may not be the best measure of a person's health status. These concerns reduce the internal validity.
Suppose the research hypothesis above were supported by the statistical analysis of the data. Should we then conclude the research hypothesis was true? The answer is no. Just because a significant difference is found does not automatically mean you should accept the research hypothesis. Even though a difference or a correlation may be statistically significant, it may not be meaningful or have a practical utility. With a very large sample size a trivial difference can be statistically significant. For example, suppose the average total cholesterol levels for two groups were 160 and 180 respectively, and that the difference was statistically significant. Since both cholesterol levels are below accepted levels that indicate the presence of arteriosclerosis the fact that the values are different may have no practical utility.
Once you have a hypothesis that is true, both statistically and practically, then you have verified new knowledge. New knowledge is the whole purpose of research. When you obtain new knowledge from research, then you can revise the current theory. From the revision, you can see if further research is needed.
You need to publish your new knowledge so that others will be able to use what you found. We have come full circle through the research process. We began with a problem and discussed how to discover new knowledge. In the end our new knowledge leads to further questions about people and their universe. Research is a process that, thankfully, is never completed.