Section 8.1.2 Validity

Section 8.1.2 Validity

Section 8.1.2 Validity

There are several different types of inter-related validity:

"They all address the same point: whether what is being measured is what the researchers intended."
(Clark-Carter, 2002, p28)


In relation to SET the question can be reduced to: can the methods used really measure what they purport to measure in an unbiased way – i.e., teaching effectiveness (Chen & Hoshower, 1998; Hobson, & Talbot, 2001)


Beran and Rokosh (2009) report Messick’s assertion that validity extends beyond meanings and interpretations of SET scores to the inferences and social consequences that result from the evaluation. They must, therefore, be demonstrably useful for all stakeholders involved in the SET process, namely students, lecturers, and university administrators/managers.


There are numerous ways in which the validity of SET measures have been accepted as relatively stable, including correlation with other credible indicators of effective teaching, and the measurement of the association between student ratings and student learning (Lemos, Queirós, Teixeira, & Menezes, 2010).


Some have suggested that although results are not always consistent across studies, there is general support for both the reliability and validity of student ratings as measures of teaching performance (Greenwald 2002).


However, Stake (1975) challenged the assumption that people could be objectively measured as evidenced by the Hawthorne Effect. Coined by Landsberger in 1950 this posits that individuals instinctively alter their behaviour in some manner in response to the fact that they’re being observed, and not in response to any particular experimental manipulation.


While lecturers have traditionally accepted formative SET (Ballantyne, Borthwick, & Packer, 2000) there remains a general suspicion towards summative SET and in its ability to fulfil its prescribed role (Light & Cox, 2001; Martinson, 2000; Sproule, 2002). At a theoretical level, this draws into question the ability to objectively and accurately measure any facet of human behaviour. More specific concerns about validity are explored in Sections 8.2 to 8.5.


Back to 8.1.1 Continue to Section 8.2 Back To Section 8

Page tools