It is established methodological standard in cognitive psychology to treat stimuli - e.g. words - as a random factor, just like subjects. Indeed, cognitive psychologists often carry out two distinct statistical analyses, one with subjects as random effects (and stimuli treated as fixed effects) and the other one with stimuli as random effects (and subjects treated as fixed effects). The rationale for this approach goes back to Coleman (1964) and Clark (1973). For instance, Coleman stated that
many studies of verbal behavior have little scientific point if their conclusions have to be restricted to the specific language materials that were used in the experiment. It has not been customary, however, to perform significance tests that permit generalization beyond the specific materials, and thus there is little evidence that such studies could be successfully replicated if a different sample of language materials were used (1964, p. 216, italics added).This methodological requirement has since been extended to other research areas (e.g. Fontenelle, Phillips & Lane, 1985; Hopkins, 1984; Kenny & Smith, 1980; Richter & Seay, 1987; Santa, Miller, & Shaw, 1979; Wickens & Keppel, 1983) and the statistical treatment of different experimenters (Bonge, Schuldt, & Harper, 1992; Scheirer & Geller, 1979). The arguments in favor of this methodological requirement are generally not substantially different from those of Coleman (1964). The following paragraph from Fontenelle, Phillips, and Lane (1985) may be taken as representing this view:
In conclusion, generalizing research findings is a major aim of all experimentation. Although experimenters as a rule are very careful to make sure that their results generalize to the population of subjects, the problem of generalizing to the population of stimuli had been neglected (p. 106, italics added).Before the validity of these arguments can be discussed further, it is necessary to briefly present the widely held conceptualization of inferential statistics in experimental psychology as following the logic of randomization tests (rather than parametric or population based statistics).