Some further thoughts on qualitative trends

Analogous procedures can be devised for comparable qualitative psychological hypotheses. The details need not be specified here (see Hager, 1992 [22]; Hager & Hasselhorn, 1995 [25]). Nor will details on testing statistical predictions concerning bitonic or tritonic trends be presented here. A hypothesis like the Yerkes-Dodson law addressed above postulates to a bitonic trend. If one chooses J = 5 experimental conditions (degrees of motivation) the respective prediction may take the following form, where the 's refer to some measure of performance:
 
These and other qualitative trends can also be tested using the method of planned contrasts, since the predictions always refer to certain rank orders, but not to any sizes of distances among means. Moreover, the method can additionally be used to achieve appropriate and exhaustive tests of other statistical predictions encountered in research practice (see the examples in Hager, 1995 [24]) such as:
 
or for a two-factorial design with equal n's per cell:
 
The examples considered here should suffice to demonstrate the versatility of the method of planned or focussed contrasts which can (and should) be applied in a multitude of different empirical situations where researchers wish to test psychological hypotheses by means of statistical ones. It leads to easy interpretations, which are exclusively test-based and need not be 'corrected' by data-based inferences. The cumulation of error probabilities can be compensated for by adjustments referring to the tests one has actually planned to perform or actually performed; adjustments do not refer to a fixed set of potentially relevant partial hypotheses as called for in numerous techniques of multiple comparisons. Furthermore, power analysis for them can be based on wide-spread tables like those presented by Cohen (1988) [11]. The versatility of the method offers the additional advantage that many empirical tests of psychological hypotheses can be handled successfully by applying a single statistical method, if the researcher derives her or his predictions carefully, appropriately, and exhaustively. In addition, this derivation should take into account the general postulate in choosing statistical tests: 'We generally prefer to carry out the minimum number of significance tests required to evaluate our theory' (Myers & Well, 1991, p. 216) [51]. With respect to this demand the criterion of exhaustiveness has been defined to assure that the tests indeed 'required' or necessary with respect to the statements (or empirical content) of the theory or hypotheses are carried out as well. In the parametric case considered here the tests refer to the same definition of a 'difference or distance among means,' whereas the more widely used F tests use a squared function of all distances among the means, this detail being responsible for the fact that several well-known techniques of multiple comparisons when applied after a significant F, do not necessarily lead to decisions in agreement with the overall result (see, e.g., Betz & Levin, 1982 [4]; Gabriel, 1969 [18]). As a consequence, the method of focussed contrasts employed without a preceding overall test is often (but not always; see the overview by Thompson, 1994 [62]) recommended, since the respective tests 'usually result in increased power and greater clarity of substantive interpretation,' as Rosnow and Rosenthal (1989, p. 1281) [57] state.

Another consideration refers to power analysis for planned contrasts to test statistical predictions concerning monotonic (or other qualitative) trends. Power analysis enables the determination of the sample size necessary to detect population effect sizes with pre-chosen error probabilities. In the t test situation, the effect size is the standardized difference among two population means, the values of which have to be selected for each hypothesis on a pair contrast. Some authors (e.g., Bredenkamp, 1984 [8]) argue that specifying these values for each adjacent pair of population means implicitly leads to an upgrading of a monotonic trend to a strictly linear trend. This belief may lead to the recommendation that the monotonic trend may and should be statistically handled as linear trend (see above). Bredenkamp's argument, however, overlooks the fact that predicted effect sizes for a quantitative trend refer to exact values which are functionally dependent on the values of the quantitative independent variable. If at least one of these exact values is (substantially) larger than predicted the strict definition of linearity is violated. On the other hand, when dealing with qualitative trends effect sizes such as are minimum values, which cannot be predicted, but are chosen according to methodological or economic reasons (see above). If one or more population values are larger than prespecified, this would not disagree with the prediction of a particular qualitative trend as long as the other values are still large enough. Referring to the samples, the empirical effects for pairs of means must be large enough to reach statistical significance which, in turn, allows assignment of different ranks to the means. Thus, the choice of some minimum values for should not be interpreted as upgrading a monotonic trend, especially as this interpretation would violate the criterion of exhaustiveness: The statistical hypothesis then actually tested comprises more information than can be derived from the original psychological hypothesis referring to a qualitative variable and trend (see above).

Considering J - 1 or J(J-1)/2 pair contrasts will always lead to contrasts which, considered as a whole, are not orthogonal to one another. Although Hays (1988, p. 415) [28] demands that planned contrasts or comparisons have to be orthogonal, many other textbook authors do not share this opinion as 'after all, contrasts are tested because they are of psychological import, not because they are independent of each other. ... in many and perhaps most cases the contrasts of interest will not be orthogonal' (Myers, 1972, p. 362 [50]; see also Thompson, 1994 [62]; Winer et al., 1991 [68]). In addition, pair contrasts do not contain the complete information inherent in the sums of squares between in an analysis of variance on the same means. But as long as the only (statistical) information needed for examining a psychological hypothesis in a valid manner consists in knowing whether the means are in the predicted order or not, there is no need for further information. But if there is any interest in further information not (directly) related to the examination of the psychological hypothesis each additional test may be performed which is thought to deliver insightful information. But these additional tests should be separated from those which directly refer to the psychological hypothesis of interest.