Proof of Concept Projects: 2017 Update

2017 has been a very productive year for the Learning Innovation Hub. Part of this productivity came in the form of testing new products and services to solve existing “pain points” in learning and teaching. The potential of these emerging learning technologies to truly allow for more engaging and effective education in the future is also important. These proof of concept projects span University wide and range over several service areas. This post will briefly touch on some of these new and innovative approaches.

Online Electronic Exams

Final examinations have not changed much over the decades, they are still conducted in large halls or classrooms on campus at set times with paper exam booklets. In recent times however there has been an ever increasing interest in making final exams electronic. There are several projects across campus including Matt Bower’s Transforming Exams project involving student devices with a USB lockout, and Matt Roberts’ E-Exam project which will be designed to make use of existing computer labs across campus.

The Learning Innovation Hub also investigated an electronic exam solution called Examity. It is a third-party service which makes use of an online proctor who monitors students via webcam and microphone while they undertake a final exam on their own computer from wherever they may be, whether that’s home, the library, or in another country. Examity was piloted in FBE (Faculty of Business and Economics) in a post graduate unit with around 40 students during session 1 2017.

Examity was largely a success, however numerous technological issues limited the effectiveness of the service, such as a heavy reliance on fast internet speeds, device compatibilities, and concerns around scalability as the service did not cope well even with small numbers. This project finalised in mid 2017, with the final recommendation by the Learning Innovation Hub to not continue with the Examity service because of these limitations. The electronic exams space will continue to be explored, so stay tuned!

Study Skills Support

Another area experiencing increasing levels of interest is the Learning Skills team at the Library. The Academic Advisors there have had a 400% increase in student consultations this year alone, with expectations that demand will continue to increase as more students become aware of the service.

The Learning Innovation Hub worked with Macquarie University International College (MUIC) and Widening Participation to trial a third-party service called Studiosity in March of 2017 (at that time the company was called YourTutor, but had a rebrand half way through the year as it was suggested that the name caused confusion among students). This company provides two lines of afterhours study support, one being a live chat and the other a written feedback service with a 24-hour turnaround. Students click through iLearn via an LTI to the Studiosity service where they use uncapped minutes provided for free by the University.

The service was offered from MUIC Term 2 through to 7 with some promising results. Student surveys indicated that 95% of students rated the service as being relevant and useful, 82% were satisfied with the amount of feedback, 88% felt more confident to complete their assignment, and 91% said they would like to have the service in all their units.

Teaching staff also had positive feedback. While there was no decrease in teacher workload (they still had to give the same level of feedback to students), marking assignments however was easier as submissions were easier to read.

Some interesting findings of this pilot were that upon deeper investigation into marks and GPA’s, it was revealed that generally stronger students were using the service. The average GPA was 3.79 across 2017, comparatively the average GPA of students who used the service was 4.07. Furthermore, students who used the service for 100 minutes or more had an average GPA of 4.78. Another interesting point to note was that comparing the two trials of MUIC and Widening Participation, it was noted that Studiosity links were most efficiently used when they were on a unit iLearn space and tied to an assignment through the teacher advising students to submit drafts to the service.

The result of this pilot will be a second phase, rolling out to more students for a better understanding of how the service will deliver.

Learning Management Systems

The Learning Management System (LMS), or as those of us at Macquarie University have come to know as iLearn (based on Moodle), is the most accessed University system. It is the platform where all resources and activities are hosted, and is becoming more relevant as the shift toward blended learning becomes stronger. Educational institutions are constantly keeping an eye out for the next best technology that may replace their current system, so every now and again an LMS review may be called for.

Over the course of the year there were three pilot projects that trialed Desire2Learn (D2L) Brightspace LMS. Two of these projects were designed to provide courses to external users as iLearn is not currently set up to allow such a feature. One was from the Faculty of Arts for external high school students, and the other was by the Faculty of Human Sciences to provide an external fee paying course qualification for school teachers. The third project was undertaken by the Applied Finance Centre and assessed D2L as a potential LMS replacement, if that time came.

These pilots revealed that as a potential LMS replacement, D2L is not ideal. There were multiple issues affecting integrations with key systems such as iTeach, AMIS and iShare. Another heavily limiting factor was the fact that D2L is built using proprietary code whereas currently the University operates on open source Moodle. What that means is currently the University has the freedom to add plugins and modifications with relative ease, versus being heavily restricted to what a vendor would allow using a proprietary product such as D2L. There were several other issues related to migration of content from existing iLearn spaces, rendering the migrated spaces unusable, which on a scalability level proved unfeasible. As a result, the Applied Finance Centre made the call to not go ahead with their D2L implementation, remaining on Moodle for the time being.

The external spaces were more successful, however there were other restricting factors which limited them to building overly basic courses with a recommendation from the LIH that they be non-fee charging as there were numerous issues which could not be resolved.

Video Assessments

Video assessments are another growing area with developments in technology and increasing focus on blended learning. Through utilising video assessment platforms new assessment types can be created, or simply tutorial time can be freed up by shifting presentations online. They also have the added benefit of being able to keep a record of assessments as required by assessment policies.

YouSeeU is an LTI third-party service which has numerous video assessment types. One of these assignment types was trialed as an option for video assessments by a second-year unit in FBE with around 700 enrolments. As a video assessment platform, feedback was mostly good, however there were some concerning issues such as massive delays while uploading (since students like to leave submissions to the last minute it seemed YouSeeU’s servers lacked the bandwidth capacity), tutors being unable to sort through tutorial groups leaving some of them lost, and no inbuilt video editor where students could possibly cut their videos down to size. The platform was also found not to be suited for general video hosting or other video related needs such as desktop capture.

As a result, the search for a video assessment tool continues with the potential for further YouSeeU testing in the future along with other products.

What’s coming in 2018?

More learning technologies! There will be plenty more proof of concepts running in the next year so keep an eye out on Teche!