After students have completed their Learner Experience of Unit (LEU) surveys it is time to consider the results and focus on areas for improvement. We provide an overview of how teaching staff can interpret the results and provide some pointers to resources relevant to each LEU item.

In case you missed it : The first post “Myth busting the reporting of Learner Experience of Unit surveys” outlined how reliability and validity are considered, the criteria used for the reporting of results based on minimum response rates and what is included in each LEU results report.

Consider evaluation outcomes

Having received your TEDS survey results it is time to reflect on the results. Be sure to read student’s comments because these often reveal the reasons behind scale item results.

It is worth noting that LEU and LET surveys represent the students’ experience of what occurred during the session. These surveys should not be taken to represent the quality of teaching per se. However – the experience at Macquarie University since the 1980s with using student feedback surveys is that the responses to scaled items are generally skewed towards the positive (>70% explicit positive, >85% non-negative). This leads to the conclusion that very poor results with narrow spread are a good indicator of the need for intervention in relation to teaching and/or unit design practice, especially if such scores are evident in more than one survey. Similarly, a uniform distribution of scores across negative, neutral and positive, is perhaps indicative of divided opinions, but is also a reason to investigate further.

When interpreting your results, you may want to consider:

  • How representative was the response to the survey? What are the error margins? (See Myth busting the reporting of Learner Experience of Unit surveys for further information).
  • Were there any unique circumstances that may have impacted the results (positively or negatively)?
  • How do your results compare with those of others? You can compare your TEDS results with others via:
    • Comparison data reports. Aggregated item level results covering 2015-2017/2018 is available for LEU and for LET (requires login to MQ iLearn).
    • MQ BI Unit monitoring dashboard includes filtering and drill-down of aggregated LEU Headline response data for 2021 onward by session, faculty, department and groups of units. The MQ Wiki has information on accessing the LEU Headline data (requires MQ login).

Additional guidance on interpreting TEDS survey results is available from the TEDS site.

Focus on areas for improvement

Decide areas where you need to take action by asking:

  • What went well? Was this expected? Why?
  • What did not go well? Was this expected? Why?
  • What will you continue to do?
  • What will you stop doing?
  • What can be done better next time?

To save your sanity, focus on viable and feasible changes in a staged manner as to not become overwhelmed. Improvement efforts are a cyclical process of making changes and reviewing the outcomes. The needs of students and the context will also change over time so a continuous improvement stance is more effective in the long run.

If a wholesale change to a unit is warranted then consideration of the unit in the context of the course will be needed. This should involve discussions with the course leader and colleagues. Additional resources and time may be required for a larger scale change. Therefore it is best to discuss with the leaders in your course, department or faculty to see what support is available. It may also be appropriate to leverage unit and course reviews to gather resources to support a redevelopment effort.

Sources of help

The Learning and Teaching professional development team have complied a set of resources available in the PLaCE resources repository. Table 1 below provides links to relevant resources according to the items present in LEU surveys. Consider these as a starting point when tackling improvements aligned to particular questions. You are also welcome to get in touch with the team for further advice.

Table 1: LEU items and matching development resources
Theme and LEU itemsResource
Learning outcomes: The unit learning outcomes were clear to me.Quick Guide: Learning Outcomes (writing them) https://goto.mq/qg-lt
Writing ULO’s (MQCMS wiki) https://wiki.mq.edu.au/x/sICqEg
Unit Guide: I was provided with accurate information about what to expect in this unit in the Unit Guide.Quick Guide: Unit Guides (what to include) https://goto.mq/qg-lt
MQ Assessment procedure “communication of assessment requirements” https://goto.mq/assessment-proc-comms and “Requirements for Unit Guides System” https://goto.mq/assess-unit-guide-reqs
iLearn site: i) I was provided with accurate information about what to expect in this unit in iLearn. ii) The unit was structured in a logical way for learning. iii) The unit’s iLearn site was easy to navigate.About iLearn standards https://goto.mq/about-ilearn-standards and a unit template checking tool https://goto.mq/ilearn-template-check
iLearn Quick Guides https://goto.mq/ilearn-quick-guides
Discover iLearn for Convenors (self paced module in iLearn) https://goto.mq/discover-ilearn-conv
Assessment: i) Assessment tasks in the unit assisted my learning. ii) The assessment criteria were clear to me. iii) The information provided enabled me to attempt assessment tasks with confidence.Quick guide: Unit Assessment (defining assessment methods in the CMS) https://goto.mq/qg-lt
Workday module: Turnitin: Rubrics https://bit.ly/3PIQVZr
Rubrics resource repository https://teche.mq.edu.au/rubrics/
PLaCE Assessment resources https://goto.mq/place-assess-curric
Transforming Assessment webinars https://transformingassessment.com/
Feedback: Overall, I received assessment feedback in time to guide my subsequent study / assessment work. Overall, the feedback provided in this unit:  i) helped me understand my strengths & weaknesses; ii) helped me understand how to improve my performance; iii) related closely to the rubric/marking criteria provided; iv) appeared to be inconsistent between classes/teachers/markers; v) was not detailed enough to assist my learning; vi) was infrequent or non-existent.Timely provision of assessment criteria and feedback https://goto.mq/provide-criteria
Feedback for learning – 5 tips https://goto.mq/5-feedback-tips
Workday module: Turnitin Feedback Studio https://bit.ly/3ckQW88
Workday module: Turnitin Rubrics https://bit.ly/3dW8BmU
PLaCE Assessment resources (see Feedback to students section) https://goto.mq/place-assess-curric
Rubrics: Why, What & How https://goto.mq/rubrics-why-what-how
First time marking and feedback https://goto.mq/1st-time-marking-n-feedback
Marking: Rubrics, feedback & team marking https://goto.mq/marking-rubrics-feedback
Participation / discussions: The unit included learning activities that encouraged my participation in discussions (online or face to face).Why are they not participating? https://goto.mq/encourage-online-communities Encouraging students to use their webcams https://goto.mq/encourage-webcam-use
Group work: This unit provided opportunities for me to work with other students.Team based learning student guide https://goto.mq/tbl-student-guide
Team based learning staff guide https://bit.ly/3PP1H00
Small group activities https://goto.mq/group-activity-ideas
What is the best way to allocate groups? https://goto.mq/group-allocation-options
Engagement: Overall, I found the learning activities in this unit engaging.Active learning self-paced modules 1 & 2 (in iLearn) https://goto.mq/lt-staff-dev-ilearn-modules
PLaCE Student learning and support resources (see Engaging students section) https://goto.mq/place-student-support
Relevance: I found this unit relevant to my course of study.PLaCE Curriculum design resources https://goto.mq/place-assess-curric
Constructive alignment quick guide and article in MQCMS wiki https://wiki.mq.edu.au/x/AYF_Fg
Indigenous perspectives:
This unit contributed to my understanding of First Nations perspectives, cultures and knowledge.
Embedding Indigenous perspectives https://goto.mq/embed-indigenous-perspectives Indigenous connected curriculum https://goto.mq/indigenous-curriculum 
Indigenous student engagement https://goto.mq/indigenous-student-engagement
The following resources and tools assisted my learning in this unit:
Set texts, prescribed readingsUnit readings made easy with Leganto https://goto.mq/unit-readings-leganto
Lecture slides / study notes / handoutsPreparing and structuring lecture notes https://bit.ly/3chTOCE
Lecturing effectively https://bit.ly/3TgjmAL
Preparing lectures https://bit.ly/3ApBsro
Echo360 and Zoom recordings (lectures, tutorials, seminars)Workday module: Echo360 Universal capture https://bit.ly/3PNBCPc 
Zoom for teaching self-paced iLearn module https://goto.mq/lt-staff-dev-ilearn-modules
Echo360 Beyond the basics https://goto.mq/advanced-echo360
Teaching in-class and online simultaneously https://goto.mq/blend-sync-top10
Videos and podcastsInteractive videos: comparing H5P and Echo360 https://goto.mq/h5p-and-echo360
Discover your voice and online presence series https://goto.mq/discover-your-voice
Computing and ITiLearn Quick Guides https://goto.mq/ilearn-quick-guides
Laboratory and clinical equipmentStaff Lab OH&S: https://goto.mq/staff-labs-ohs Students Lab OH&S: https://goto.mq/student-labs-ohs
Skills: In this unit I have developed my:
Critical thinking skillsStudywise (iLearn site for students) https://goto.mq/mq-studywise
Ability to solve complex problemsStudywise: Think and write critically https://goto.mq/studywise-critical-thinking
Numeracy centre https://goto.mq/numeracy-centre
Ability to work with othersStudywise: working in groups https://goto.mq/studywise-groupwork
Team-based learning (MQBS) https://bit.ly/3wpM7RR
Confidence to learn independentlyHEA independent learning guide https://goto.mq/hea-independent-learning
Written communication skillsStudywise: academic style https://goto.mq/studywise-academic-style
Studywise: written structure https://goto.mq/studywise-written-structure
The writing centre https://students.mq.edu.au/support/study/writing
Spoken communication skillsStudywise: presentations https://goto.mq/studywise-presentations
Digital skillsStudywise: ‘going digital’: https://goto.mq/studywise-going-digital
Library teaching support resource: https://goto.mq/library-teaching-support
Knowledge of field(s) I am studyingLibrary subject guides https://libguides.mq.edu.au/homepage
Work related/ professional knowledge and skillsEmployability transferrable skills for students https://goto.mq/career-skill-dev
PACE resources wiki for staff https://wiki.mq.edu.au/x/1YOuD

Explore this topic further …

This article is available in summary form as a two page LT Quick Guide:

[1] Compare LEU to other units https://goto.mq/leu-compare

[2] Compare LET to other units https://goto.mq/let-compare

[3] About LEU Headline data https://wiki.mq.edu.au/x/Nw0UGQ – For access, first login to MQ office 365 online, then go to the MQ BI Dashboard https://app.powerbi.com/home then the ‘Unit monitoring…’ dashboard. Then find the LEU section.

Other Quick Guides: ‘Evaluation surveys‘, ‘Informal Evaluation‘, ‘Evidencing your teaching‘ and more, see https://goto.mq/qg-lt

Acknowledgement:

MQ L&T Staff development team, MQ TEDS Team.

Prepared by Dr Mathew Hillier, Macquarie University, Nov 2022

Posted by Mathew Hillier

Mathew has been engaged by Macquarie University as an e-Assessment Academic in residence and is available to answer questions by MQ staff. Mathew specialises in Digital Assessment (e-Assessment) in Higher Education. Has held positions as an advisor and academic developer at University of New South Wales, University of Queensland, Monash University and University of Adelaide. He has also held academic teaching roles in areas such as business information systems, multimedia arts and engineering project management. Mathew recently led a half million dollar Federal government funded grant on e-Exams across ten university partners and is co-chair of the international 'Transforming Assessment' webinar series as the e-Assessment special interest group under the Australasian society for computers in learning in tertiary education. He is an honorary academic University of Queensland, Monash University and an adjunct academic at University of Tasmania.

Leave a reply

Your email address will not be published.