An exploratory analysis of major dropdowns in Student Evaluation of Teaching ratings in higher education

Authors

  • Santiago Leguey Universidad Rey Juan Carlos
  • Ana I. Cid-Cid Universidad Rey Juan Carlos
  • Rocío Guede Cid Universidad Rey Juan Carlos
  • Jaime Prieto Universidad Rey Juan Carlos

https://doi.org/10.17583/remie.10419

Keywords:


Downloads

Type:

Text file

Abstract

We present an exploratory study of Student Evaluation of Teaching (SET) surveys that focused on analyzing the major dropdowns that sometimes occur in SET scores of those university teachers who teach the same subject for at least two consecutive years. Specifically, the purpose of this study was to identify those SET survey items associated with major dropdowns in teacher evaluations. The sample comprised SET surveys ratings from one Spanish University over five years, for a total of 2,284 different teacher-subject pairs under analysis. The 5% percentile of the differences between the teacher and subject SET ratings in two consecutive years was used as the cut-off point to identify major dropdowns. Frequency tables were used for analysis. The results highlighted SET survey items related to teaching methodology as those most associated with major dropdowns in teachers’ ratings. In particular, the item concerning teachers’ compliance with class schedules showed the highest frequency of occurrence. Beyond the specific scores obtained or the specific items in which major dropdowns may take place, in this study we discuss the importance of incorporating major dropdowns analysis to SET surveys reports in the future.

Downloads

Download data is not yet available.

References

Aguillo, I., Bar-Ilan, J., Levene, M., & Ortega, J. (2010). Comparing university rankings. Scientometrics, 85(1), 243-256. https://doi.org/10.1007/s11192-010-0190-z

Google Scholar Crossref

ASA, American Sociological Association (2019). Statement on Student Evaluation of Teaching. https://www.asanet.org/sites/default/files/asa_statement_on_student_evaluations_of_teaching_sept52019.pdf

Google Scholar Crossref

Asare-Nuamah, P. (2017). International Students’ Satisfaction: Assessing the Determinants of Satisfaction. Higher Education for the Future, 4(1), 44-59. https://doi.org/10.1177/2347631116681213

Google Scholar Crossref

Bedggood, R. E., & Donovan, J. D. (2012). University performance evaluations: what are we really measuring? Studies in Higher Education, 37(7), 825-842. https://doi.org/10.1080/03075079.2010.549221

Google Scholar Crossref

British Educational Research Association (BERA) (2018). Ethical Guidelines for Educational Research. London: BERA.

Google Scholar Crossref

Boring, A., Ottoboni, K., & Stark, P. B. (2016). Student evaluations of teaching (mostly) do not measure teaching effectiveness. Science Open Research. https://doi.org/10.14293/s2199-1006.1.sor-edu.aetbzc.v1

Google Scholar Crossref

Capa-Aydin, Y. (2016). Student evaluation of instruction: comparison between in-class and online methods. Assessment & Evaluation in Higher Education, 41, 112-126. https://doi.org/10.1080/02602938.2014.987106

Google Scholar Crossref

Casero, A. (2008). Propuesta de un cuestionario de evaluación de la calidad docente universitaria consensuado entre alumnos y profesores. [Proposal for a consensual questionnaire for evaluating the quality of university among students and teachers]. Revista de Investigación Educativa, 26(1), 25-44. https://revistas.um.es/rie/article/view/94091

Google Scholar Crossref

Castro-Morera, M.; Navarro-Asencio, E. & Blanco-Blanco, Á. (2020). The quality of teaching as perceived by students and university teachers: analysis of the dimensionality of a teacher evaluation questionnaire. Educación XX1, 23(2), 41-65. https://doi.org/10.5944/educXX1.25711

Google Scholar Crossref

Centra, J. A., & Gaubatz, N. B. (2000). Is there gender bias in student evaluations of teaching? The Journal of Higher Education, 71, 17-33. https://doi.org/10.1080/00221546.2000.11780814

Google Scholar Crossref

Chan, C. K. Y., Luk, L. Y. Y., & Zeng, M. (2014). Teachers’ perceptions of student evaluations of teaching. Educational Research and Evaluation, 20(4), 275-289. https://doi.org/10.1080/13803611.2014.932698

Google Scholar Crossref

Ching, G. (2018). A literature review on the student evaluation of teaching: An examination of the search, experience, and credence qualities of SET. Higher Education Evaluation and Development, 12(2), 63-84. https://doi.org/10.1108/HEED-04-2018-0009

Google Scholar Crossref

Clayson, D. E. (2009). Student evaluations of teaching: are they related to what students learn? A meta-analysis and review of the literature. Journal of Marketing Education, 31(1), 16-30. https://doi.org/10.1177/0273475308324086

Google Scholar Crossref

Clayson, D. E., & Sheffet, M. J. (2006). Personality and the student evaluation of teaching. Journal of Marketing Education, 28, 149-160. https://doi.org/10.1177/0273475306288402

Google Scholar Crossref

Griffin, B. W. (2001). Instructor reputation and student ratings of instruction. Contemporary Educational Psychology, 26(4), 534-552. https://doi.org/10.1006/ceps.2000.1075

Google Scholar Crossref

Griffin, B. W. (2016). Perceived autonomy support, intrinsic motivation, and student ratings of instruction. Studies in Educational Evaluation, 51, 116-125. https://doi.org/10.1016/j.stueduc.2016.10.007

Google Scholar Crossref

Guilbault, M. (2016). Students as customers in higher education: reframing the debate. Journal of Marketing for Higher Education, 26(2), 132-142. https://doi.org/10.1080/08841241.2016.1245234

Google Scholar Crossref

Henard, F. (2010). Learning our lesson: Review of quality teaching in higher education. Institutional Management in Higher Education, OECD Publishing.

Google Scholar Crossref

Johnson, R. (2000). The authority of the student evaluation questionnaire. Teaching in Higher Education, 5, 419-434. https://doi.org/10.1080/713699176

Google Scholar Crossref

Komotar, M. H. (2019). Global university rankings and their impact on the internationalisation of higher education. European Journal of Education, 54(2), 299-310. https://doi.org/10.1111/ejed.12332

Google Scholar Crossref

Marsh, H. W. (2007a). Students’ evaluations of university teaching: A multidimensional perspective. In R. P. Perry and J. C. Smart (Eds.), The scholarship of teaching and learning in higher education: An evidence based perspective (pp. 319-384). New York: Springer.

Google Scholar Crossref

Marsh, H. W. (2007b). Do university teachers become more effective with experience? A multilevel growth model of students’ evaluations of teaching over 13 years. Journal of Educational Psychology, 99(4), 775-790. https://doi.org/10.1037/0022-0663.99.4.775

Google Scholar Crossref

Mau, R. R., & Opengart, R. (2012). Comparing Ratings: In-class (paper) vs. Out of Class (online) Student Evaluations. Higher Education Studies, 2(3), 55-68. https://doi.org/10.5539/hes.v2n3p55

Google Scholar Crossref

Molero, D., & Ruiz, J. (2005). La evaluación de la docencia universitaria. Dimensiones y variables más relevantes. [The evaluation of university teaching. Dimensions and most relevant variables]. Revista de Investigación Educativa, 23(1), 57-84. https://revistas.um.es/rie/article/view/98341

Google Scholar Crossref

Moreno-Murcia, J. A., Silveira, Y., & Belando, N. (2015). Questionnaire Evaluating Teaching Competencies in the University Environment. Evaluation of Teaching Competencies in the University. Journal of New Approaches in Educational Research, 4(1), 54-61. https://www.learntechlib.org/p/150692/

Google Scholar Crossref

Mortelmans, D., & Spooren, P. (2009). A revalidation of the SET37-questionnaire for student evaluations of teaching. Educational Studies, 35, 547-552. https://doi.org/10.1080/03055690902880299

Google Scholar Crossref

Neumann, R. (2000). Communicating Student Evaluation of Teaching Results: Rating Interpretation Guides (RIGs). Assessment & Evaluation in Higher Education, 25(2), 121-134. https://doi.org/10.1080/02602930050031289

Google Scholar Crossref

Richardson, J. T. E. (2005). Instruments for obtaining student feedback: a review of the literature. Assessment & Evaluation in Higher Education, 30(4), 387-415. https://doi.org/10.1080/02602930500099193

Google Scholar Crossref

Rindermann, H., & Schofield, N. (2001). Generalizability of multidimensional student ratings of university instruction across courses and teachers. Research in Higher Education, 42(4), 377-399. https://doi.org/10.1023/A:1011050724796

Google Scholar Crossref

Scheepers, A. (2019). SET Project: Student Evaluations of Teaching, Measuring and Enhancing Course Quality and Teaching Quality. Rotterdam School of Management. https://equal.network/wp-content/uploads/2019/03/SET_Ad-Scheepers-.pdf

Google Scholar Crossref

Spooren, P., Brockx, B. & Mortelmans, D. (2013). On the validity of student evaluation of teaching: The state of the art. Review of Educational Research, 83(4), 598-642. https://doi.org/10.3102/0034654313496870

Google Scholar Crossref

Stark, P. B., & Freishtat, R. (2014). An evaluation of course evaluations. Science Open Research. https://doi.org/10.14293/S2199-1006.1.SOR-EDU.AOFRQA.v1

Google Scholar Crossref

Sulis, I., Porcu, M., & Capursi, V. (2019). On the Use of Student Evaluation of Teaching: A Longitudinal Analysis Combining Measurement Issues and Implications of the Exercise. Social Indicators Research, 142, 1305-1331. https://doi.org/10.1007/s11205-018-1946-8

Google Scholar Crossref

Surgenor, P. W. G. (2013). Obstacles and opportunities: Addressing the growing pains of summative student evaluation of teaching. Assessment & Evaluation in Higher Education, 38, 363-376. https://doi.org/10.1080/02602938.2011.635247

Google Scholar Crossref

Downloads

Published

2023-02-15

Almetric

Dimensions

How to Cite

Leguey, S., Cid-Cid, A. I., Guede Cid, R., & Prieto, J. (2023). An exploratory analysis of major dropdowns in Student Evaluation of Teaching ratings in higher education. Multidisciplinary Journal of Educational Research, 13(1). https://doi.org/10.17583/remie.10419

Issue

Section

Articles