Journal of Humanistic Psychology, Ahead of Print.
The main objective of this study is to evaluate the utility of a new qualitative scale development methodology—Three-Step Test-Interview (TSTI)—in its first application in the validation of a psychotherapy scale: The Relational Depth Frequency Scale (RDFS). The TSTI is a cognitive pretesting method designed to uncover potential problems in scale construction. The RDFS is a six-item unidimensional scale of in-depth therapeutic relating, designed for use in large-scale outcome studies. Following the creation of an item pool and “expert ratings,” a purposive sample of four therapists and four clients (five females, three males, mean age: 49 years) was recruited to take part in the TSTI with the view to refine the original 36-item RDFS prior to psychometric exploration. Structured observations pointed to problems in test-takers’ patterns of responses in relation to theoretical knowledge of the relational depth construct. Issues uncovered and addressed included some misinterpretations of instructions and items, redundant content, double-barreled items, and test-takers’ reactions to intimate content wording. The method supported the refinement of the RDFS including amendment to its instructions and the removal of problematic items. TSTI results produced knowledge on the scale which could not be captured with statistical methods.