Objective: This paper describes the development of a practice-based evaluation (PBE) tool that allows instructors to represent their student’s clinical performance in a way that is sufficiently authentic to resonate with both instructors and students, is psychometrically sound, and is feasible in the context of real practice. Method: A new online evaluation tool was designed to address several of the problems associated with previous methods of evaluation, and was tested on 190 field instructor–student pairs. Results: Results demonstrated feasibility of the tool, high acceptability from students and faculty, high internal consistency, and clearly reduced ceiling effect, when compared with a traditional competency-based evaluation (CBE) tool. It did, however, continue to result in a strong skew toward positive evaluation and did not increase the identification of students at risk. Conclusions: The online PBE tool demonstrates promise in redressing some of the evaluation issues posed by the previous CBE model of evaluation.