Safeguarding academic integrity, connecting law students with markers: Assessment via viva voce

In the era of generative artificial intelligence, such as ChatGPT, academics may find that more chat, not less, is part of the solution. Find out how a growing community of law academics are applying principles of authentic assessment through interactive orals – a.k.a the viva voce (‘living voice’).

The ‘industrial scale’ problem

News stories have laid bare the extent and sophistication of the ‘industrial scale cheating’ taking place in Australian higher education, with one investigation into ‘ghost writer’ services ballparking a ‘basic’ service of $149 per 1000 words. TEQSA has taken action to block 40 known commercial cheating websites, however learning designers have also been put to task to embed counter-plagiarism methods into assessment design while being resource efficient.

Enter ChatGPT

Generative AI has added yet another layer of complexity to assessment design. Academics need to ensure that students, and not chatbots, are obtaining the skills and knowledge required for their careers ahead. ChatGPT is a major challenge, particularly for those in programs tied to professional accreditation such as law. In the legal profession, the Law Society Journal (Online) has offered its thoughts and LawyersWeekly has flagged that ‘authentic’ assessments are set to become the first line of defence. Fortunately, the Macquarie Law School has cultivated an affinity for authentic assessment in recent years in response to feedback from industry and students. Our team has focused on running viva voces to-scale and are eager to share what we’ve learned here.

Authentic Assessment 101

Authentic assessment has been hailed as an assessment approach that can advance several learning objectives such as promoting academic integrity, engagement, and, perhaps most significantly, skill development and employability. 

Described by Mueller (2010) as: “[a] form of assessment in which students are asked to perform real-world tasks that demonstrate meaningful application of essential knowledge and skills”, assessment tasks are considered authentic when students are asked to construct their own responses rather than select from ones presented; and the task replicates real-world challenges. Practically speaking, these tasks may include: assessment in a professional setting; completion of a workplace task; and simulation or role play of a scenario or hypothetical.

The Viva Voce as Authentic Assessment

Our team has collaborated to combine the pedagogy of authentic assessment with the viva voce as an assessment type, to produce three variations (so far) adopted across undergraduate and postgraduate units, including core units and electives. The viva voce seemed the most appropriate assessment type to adapt into authentic tasks given the importance of oral communication in both traditional and non-traditional careers for law graduates.

‘Viva voce’ derives from the Medieval Latin for “living voice” and involves assessing a student in real time through an oral exchange of varying degrees of interactivity depending on the assessment. The work of Sotiriadou and colleagues best reflects the rationale behind our design: ‘An interactive oral (previously known as a viva voce exam) is an opportunity for genuine and unscripted interaction between a student and other students or a student and an examiner. Specifically, students can demonstrate knowledge verbally in an authentic setting representative of what would be encountered in the workplace.’

When pairing authentic assessment with interactive oral assessments, students are presented with learning activities that not only allow for the assessment of discipline-based knowledge and core skills, but allow them to engage with contemporary real-world sociolegal issues that build their confidence as they transition to professional life post-study.

From concept to execution

Our initial discussions around introducing viva voce style assessments occurred in 2018-2019 and included conversations with members of the profession, junior and senior colleagues, recent graduates, and students. We learned the following:

1. The increase in student-teacher ratios had left both staff and students wanting more dedicated face-to-face time for individualised feedback and engaging L&T experiences;
2. The profession reported back that law graduates were not sufficiently trained in ‘job-ready’ communication (oral and written);
3. The Law School was stressing the need for more ‘authentic assessment’ in our curriculum to better prepare graduates for the transition to full-time work and range of work open to law graduates; and
4. With the move away from sit-down invigilated exams, staff were challenged to consider assessment methods that would minimise opportunities for plagiarism while being resource efficient.

So, there were both internal and external drivers for introducing some variation to the typical assessment regime. The three Viva Voce assessment modes we have designed and executed across units since 2019 have taken all four of these considerations into account.

With a boost from the tech: Viva la Zoom!

When conducted via Zoom and coordinated through an online sign-up function on iLearn, we found additional benefits, including:

1. Flexibility around student availability, including after work hours;
2. Resilience to pandemic, flood, and other disruptions to physical attendance;
3. Capacity to record sessions for record-keeping, grade moderation, assessor training, student self-reflection, etc; and
4. Students developing soft-skills associated with functioning in virtual workplaces/ workspaces (e.g. professional and persuasive communication via web-conferencing, effective screen-use, setting up an effective working-from-home space, etc).

In keeping with the spirit of the viva as an authentic assessment, we chose to name each of the variations according to the role we require students to adopt: ‘The Critical Case Commentator’; ‘The Model Junior Solicitor’; and ‘The Expert Witness’.

1. Students as ‘Critical Case Commentators’

The ‘Critical Case Commentator’ variation of the viva is particularly useful for building confidence in developing original ideas around critical concepts and consolidating skills around case analysis.

We trialled this assessment in LAWS8001: Foundations of Law considering it to be an interactive means to assess skills formative to the rest of the law program, including language proficiency. LAWS8001 is our compulsory, first-semester, first-year unit in the JD program. We asked students to select from a list of cases and argue whether the case upheld or undermined the Rule of Law. We conducted these vivas at the end of semester and into the exam period which meant they also served as an informal check-in with students to cap-off their first semester at law school before walking into the rest of their degree program.

2. Students as ‘Model Junior Solicitors’

‘The Model Junior Solicitor’ variation was piloted in Session 2, 2022 and Session 3 2022-2023 in LAWS5051/LAWS8099: Professional and Community Engagement: Individual Placements for both UG LLB and PG JD students. These units are most often undertaken in the penultimate or final years of study and provide the opportunity for students to engage with the legal profession and community through participation in a variety of workplace experiences including, but not limited to, law firms, legal centres, community-based legal organisations and services, government agencies and not-for-profit organisations. The viva in these units takes the form of an Oral Research Report from a junior solicitor to their principal selected from a range of topics upon which the firm will be running a series of legal ethics-focused Continuing Legal Education seminars.

Teaching staff select topics drawing from contemporary issues surrounding the practice of law. These may include questions arising from recent cases or trends in issues addressed to recent graduates in job interviews and practice. ChatGPT came up frequently in student responses to one of the topics and offered staff an insight into how firms of various sizes were responding in real time. The overarching aim of the assessment is to acclimatise students to collegial communication within the typical hierarchy of traditional legal careers within the context of a unit that shapes their identities as burgeoning legal professionals. As such, this variation is suitable for professional skills-focused units or could be adapted for other law units seeking to introduce a practical component to substantive law as currently practiced.

3. Students as ‘Expert Witnesses’

‘The Expert Witness’ variation on the viva voce was borne of designer Zara’s experiences as an expert in parliamentary inquiries and criminal proceedings, coupled with a keen interest within our cohorts for more law reform/advocacy relevant assessments. This viva variation is used in two elective units in the LLB program undertaken in the penultimate or ultimate year: LAWS5057: Conflict of Laws and LAWS5071: Health Law and Ethics. The convenor in each unit adopts the role of Chair of an Inquiry conducted by the Attorney General and House of Reps Standing Committee on Health, Aged Care and Sport respectively. Chairs post the Terms of Reference for inquiry to which students are to respond with written submissions written on behalf of an entity interested. The viva component is conducted during exam period mirroring an in-camera appearance by an expert witness before an Inquiry lasting 15 minutes total.

This more in-depth viva, positioning students as legal experts offering assistance on complex matters of interpretation and reform, lends itself to elective units in latter years (students having themselves self-selected into areas in which they possess particular interest). ‘Inquiries’ called to date include: the Attorney Generals’ Inquiry into an issue affecting personal injury liability issues concerning cruise ships and passengers; and House of Reps Standing Committee on Health, Aged Care and Sport Inquiry into the impact of the COVID‐19 pandemic on the law and regulation of healthcare in Australia and Inquiry into the legal regulation of innovative treatments in Australia.

What do students think?

The team will be looking to conduct more rigorous empirical analysis of student experience as we refine our assessments and test multiple cohorts with variations moving forward. For now, comments from LEUs, written correspondence, and post Q&A debriefs appear to be positive:

“I enjoyed the viva voce assessment that reflected a task you are likely to experience in a professional environment. It is great to get an opportunity to undertake a different type of assessment that is more practical in nature as we don’t often get to do this in our study. I also enjoyed this assignment as it gave us an opportunity to receive immediate oral feedback on an assignment. This allowed us to ask any questions or clarify areas to get a more in-depth overview of our strengths and weaknesses.”

“Although when I saw it in the unit guide, I panicked initially, the viva voce was an excellent assignment that allowed me to develop my communication skills further and see more into the world of legal practice. I hope they are employed more readily throughout the rest of a LLB because I truly believe they are beneficial. Really thankful to Paul and Zara for their time, patience and assistance throughout this course.”

“I did Health Law and Ethics in Sem 2 2022 and it really left an impression on me, specifically the Expert Submission assignment on the social determinants of health and public health. I never realised how passionate I was about public health until I learnt about it and did the assignment. In my assignment I focused on the disparities on health outcomes between affluent and non-affluent areas, as well as Indigenous populations.”

“This unit was extremely beneficial in developing my understanding of the legal industry and the importance of ongoing self-development. The oral report assessment was very different to other assessments I have completed throughout my course and I found it to be a very interesting and engaging way of completing an assessment, where I was able to develop my communicational skills in a professional manner. Furthermore the feedback received following the oral report was really encouraging and inspired myself to undertake further research into the ideas and feedback discussed.”

Final thoughts from the teaching team

Creating space within an assessment regime for every student to visualise and actualise themselves into these professional roles can make for transformative learning experiences that shape their personal and professional identity.

Zara Bending

Zara Bending: The potential for further variations of the viva in law education is limitless given the array of professional tasks performed by graduates requiring a strong communication skillset. However, we are positive that there is room for uptake for both authentic assessment and the viva voce more specifically across all disciplines in higher education.

Paul Maluga: Integrating authentic tasks into assessment regimes prepares students to excel in an increasingly competitive graduate market. We deliberately feed graduate and industry feedback into our assessment themes. It also provides an opportunity to invite professionals literate in the evolving needs of the profession into the learning environment. I recommend giving a viva assessment a go, particularly in PACE units.

Harry Melkonian: In this era of recorded lectures and Zoom tutorials, students can become disengaged and discouraged. The viva demands one-on-one interaction between student and teacher and, even if conducted by Zoom, creates a personalised learning experience.

George Tomossy: Students have consistently fed back that they both enjoy and value the experience, which is unlike anything they have had in their law degree. I would much rather participate in an interactive oral assessment any day, which is by far more enriching than grading a written paper or review a pre-recorded effort.

---

This article draws on work presented at the 2022 Professional Legal Education Conference: LawTech, Newlaw and NetZero: Preparing for an Uncertain Future.

Acknowledgements: Text provided by Zara Bending, Paul Maluga, Harry Melkonian and George Tomossy. Author photos courtesy of authors. Banner image by Shutterstock. Photo by Headway on Unsplash. Post edited and reviewed by Karina Luzia.