Category talk:Evaluation

Hi Wolfgang, Your outline sounds very interesting. It looks like it will be a very comprehensive evaluation of how, both the current and future requirements of the policy makers as set out in the Hunt report, could be built into a realistic framework to address all the issues outlined. It might be interesting to look at the recent roll out of the UCD SFM as a case study. As a recently designed model, it appears to take a more summative format, while leaving scope for the use of a formative approach. It could be interesting to address the type of questions it uses and the possible feedback it could provide over previous formats used? Gwen


Welcome to the Discussion Page


Come along here to ask those stupid questions, draw attention to something strange, or instigate insurrection.


If everyone pops back here every so often and contributes this could be an interesting resource and area for general discussion



Hi Wolfgang,

This proposal seems very interesting in terms of reconciling the goals of formative and summative evaluation and your critical approach to the Hunt Report. It would be helpful to see what kind of questions on a summative SET can feed into SET as an instrument of both accountability and improvement.

Anna

Hi Wolfgang, The article you posted is interesting. I am suprised that 3 out 4 respondents said both paper and online survey methodologies are a requirement at their institutions. The "in class" and "out of class" argument is quite convincing, I agree that the response rate is likely to be higher if the survey is done in class. In fact I tend to agree with most of the arguments the author presented against online surveys! Gillian

Hi Wolfgang,

I like the proposal. It deals with the formative/summative link very well. I have a problem with linking these two approaches, so your essay would be an interesting read.

Are your ideas linked to a module you are teaching at the moment?

I wasn't quite sure whether your two goals were accountability/improving teaching or summative/formative. My feeling would be that the accountability/improving teaching are conflicting goals. If evaluation is used for accountability I think the lecturer will focus on making his teaching look good in the eyes of the evaluation. So I would be interested to hear if there are any arguments for how these two goals can be achieved by one evaluation strategy.

Jon

Why online surveys alone don't work

Hi all,

Someone just sent me an interesting link to an article about online evaluation surveys.

Would you agree or disagree with any of the points raised in the article?

http://ericbohms.wordpress.com/2011/08/09/why-online-only-course-and-module-surveys-dont-work/



Quality Control versus Accountability – a Reconcilable Paradox in SET?

Assignment Proposal Wolfgang Marx

In recent years student evaluation techniques have become an instrument in the more and more detailed attempts to establish quantifiable quality assurance and accountability systems in third/level education. This means that alongside their original purpose of improving aspects of teaching and learning (thus being a tool of communication between students and lecturers, with the results having an immediate impact on future student cohorts), they increasingly also serve as instruments to establish accountability of academics in administrative decision-making processes (thus becoming a tool of communication between academics and managers, as well as between university managers and politicians and other stakeholders). On the OER website, these two purposes are described as formative and summative SET (although it appears possible to me that summative SET can also have its merits in a non-accountability context, if only in conjunction with formative SETs). In this essay I want to investigate to what extent these two possible goals of SET exclude or complement each other.

- What types of accountability are requested by policy makers? Which of these can be supported by SET? - What kind of information is requested to establish SET as an instrument of accountability, and does this information fulfil its task? - In what way does the theory of SET focus on formative rather than summative evaluation? How do they complement each other? - How do the original, “formative” SET side with the new goal? - Which SET could fulfil both roles (at least in part)?

My hypothesis would be that both goals can indeed be reconciled, yet that the way in which accountability is requested shows that managers and politicians have no primary interest in quality assurance but use SET for their own, more political and economic ends. This will be looked at particularly in an Irish context with reference to recent policy papers like the Hunt Report or the implementation plans for the “Cork Park Agreement.”


Literature (preliminary) Jeroen Huisman, Jan Currie, “Accountability in Higher Education: Bridge over Troubled Water?”, Higher Education 48/4 (December 2004), 529-551. Andreas Hoecht, “Quality Assurance in UK Higher Education: Issues of Trust, Control, Professional Autonomy and Accountability,” Higher Education 51/4 (June 2006), 541-63. Graham D. Henry, Sarah J. Dean, “Accountability, Evaluation of Teaching and Expertise in Higher Eduaction,” The International Journal for Academic Development 7/1 (2002), 76-82. Ginns/Prosser/Barrie Steve Rayner, Mary Fuller, Lindsey McEwen, Hazel Robert, “Managing Leadership in the UK University: a Case for Researching the Missing Professoriate?”, Studies in Higher Education 35/6 (September 2010), 617-631. Jill Blackmore, “Academic Pedagogies, Quality Logics and Performative Universities: Evaluating Teaching and what Students want,” Studies in Higher Education 34/8 (20), 857-872.

Hi Wolfgang, Your proposal seems very interesting. How formative and summative evaluation complement each other should be an interesting piece. Do you currently engage in both forms of evaluation as part of your current modules? Sinéad

Hi Wolfgang,

Your proposal was a very interesting read and you have brought up some intriguing points. I would be very interested in reading your findings, in particular your findings and views in relation to any SET which could fulfill both roles of interest.

Fiona

Page tools