In the era of generative artificial intelligence, such as ChatGPT, academics may find that more chat, not less, is part of the solution. Find out how a growing community of law academics are applying principles of authentic assessment through interactive orals – a.k.a the viva voce (‘living voice’).
Post authored by Zara Bending (lead), Paul Maluga, Harry Melkonian, & George Tomossy, Macquarie Law School
The ‘industrial scale’ problem
News stories have laid bare the extent and sophistication of the ‘industrial scale cheating’ taking place in Australian higher education, with one investigation into ‘ghost writer’ services ballparking a ‘basic’ service of $149 per 1000 words. TEQSA has taken action to block 40 known commercial cheating websites, however learning designers have also been put to task to embed counter-plagiarism methods into assessment design while being resource efficient.
Generative AI has added yet another layer of complexity to assessment design. Academics need to ensure that students, and not chatbots, are obtaining the skills and knowledge required for their careers ahead. ChatGPT is a major challenge, particularly for those in programs tied to professional accreditation such as law. In the legal profession, the Law Society Journal (Online) has offered its thoughts and LawyersWeekly has flagged that ‘authentic’ assessments are set to become the first line of defence. Fortunately, the Macquarie Law School has cultivated an affinity for authentic assessment in recent years in response to feedback from industry and students. Our team has focused on running viva voces to-scale and are eager to share what we’ve learned here.
Authentic Assessment 101
Authentic assessment has been hailed as an assessment approach that can advance several learning objectives such as promoting academic integrity, engagement, and, perhaps most significantly, skill development and employability.
Described by Mueller (2010) as: “[a] form of assessment in which students are asked to perform real-world tasks that demonstrate meaningful application of essential knowledge and skills”, assessment tasks are considered authentic when students are asked to construct their own responses rather than select from ones presented; and the task replicates real-world challenges. Practically speaking, these tasks may include: assessment in a professional setting; completion of a workplace task; and simulation or role play of a scenario or hypothetical.
The Viva Voce as Authentic Assessment
Our team has collaborated to combine the pedagogy of authentic assessment with the viva voce as an assessment type, to produce three variations (so far) adopted across undergraduate and postgraduate units, including core units and electives. The viva voce seemed the most appropriate assessment type to adapt into authentic tasks given the importance of oral communication in both traditional and non-traditional careers for law graduates.
‘Viva voce’ derives from the Medieval Latin for “living voice” and involves assessing a student in real time through an oral exchange of varying degrees of interactivity depending on the assessment. The work of Sotiriadou and colleagues best reflects the rationale behind our design: ‘An interactive oral (previously known as a viva voce exam) is an opportunity for genuine and unscripted interaction between a student and other students or a student and an examiner. Specifically, students can demonstrate knowledge verbally in an authentic setting representative of what would be encountered in the workplace.’
When pairing authentic assessment with interactive oral assessments, students are presented with learning activities that not only allow for the assessment of discipline-based knowledge and core skills, but allow them to engage with contemporary real-world sociolegal issues that build their confidence as they transition to professional life post-study.
From concept to execution
Our initial discussions around introducing viva voce style assessments occurred in 2018-2019 and included conversations with members of the profession, junior and senior colleagues, recent graduates, and students. We learned the following:
- The increase in student-teacher ratios had left both staff and students wanting more dedicated face-to-face time for individualised feedback and engaging L&T experiences;
- The profession reported back that law graduates were not sufficiently trained in ‘job-ready’ communication (oral and written);
- The Law School was stressing the need for more ‘authentic assessment’ in our curriculum to better prepare graduates for the transition to full-time work and range of work open to law graduates; and
- With the move away from sit-down invigilated exams, staff were challenged to consider assessment methods that would minimise opportunities for plagiarism while being resource efficient.
So, there were both internal and external drivers for introducing some variation to the typical assessment regime. The three Viva Voce assessment modes we have designed and executed across units since 2019 have taken all four of these considerations into account.
With a boost from the tech: Viva la Zoom!
When conducted via Zoom and coordinated through an online sign-up function on iLearn, we found additional benefits, including:
- Flexibility around student availability, including after work hours;
- Resilience to pandemic, flood, and other disruptions to physical attendance;
- Capacity to record sessions for record-keeping, grade moderation, assessor training, student self-reflection, etc; and
- Students developing soft-skills associated with functioning in virtual workplaces/ workspaces (e.g. professional and persuasive communication via web-conferencing, effective screen-use, setting up an effective working-from-home space, etc).
In keeping with the spirit of the viva as an authentic assessment, we chose to name each of the variations according to the role we require students to adopt: ‘The Critical Case Commentator’; ‘The Model Junior Solicitor’; and ‘The Expert Witness’.
1. Students as ‘Critical Case Commentators’
The ‘Critical Case Commentator’ variation of the viva is particularly useful for building confidence in developing original ideas around critical concepts and consolidating skills around case analysis.
We trialled this assessment in LAWS8001: Foundations of Law considering it to be an interactive means to assess skills formative to the rest of the law program, including language proficiency. LAWS8001 is our compulsory, first-semester, first-year unit in the JD program. We asked students to select from a list of cases and argue whether the case upheld or undermined the Rule of Law. We conducted these vivas at the end of semester and into the exam period which meant they also served as an informal check-in with students to cap-off their first semester at law school before walking into the rest of their degree program.
2. Students as ‘Model Junior Solicitors’
‘The Model Junior Solicitor’ variation was piloted in Session 2, 2022 and Session 3 2022-2023 in LAWS5051/LAWS8099: Professional and Community Engagement: Individual Placements for both UG LLB and PG JD students. These units are most often undertaken in the penultimate or final years of study and provide the opportunity for students to engage with the legal profession and community through participation in a variety of workplace experiences including, but not limited to, law firms, legal centres, community-based legal organisations and services, government agencies and not-for-profit organisations. The viva in these units takes the form of an Oral Research Report from a junior solicitor to their principal selected from a range of topics upon which the firm will be running a series of legal ethics-focused Continuing Legal Education seminars.
Teaching staff select topics drawing from contemporary issues surrounding the practice of law. These may include questions arising from recent cases or trends in issues addressed to recent graduates in job interviews and practice. ChatGPT came up frequently in student responses to one of the topics and offered staff an insight into how firms of various sizes were responding in real time. The overarching aim of the assessment is to acclimatise students to collegial communication within the typical hierarchy of traditional legal careers within the context of a unit that shapes their identities as burgeoning legal professionals. As such, this variation is suitable for professional skills-focused units or could be adapted for other law units seeking to introduce a practical component to substantive law as currently practiced.
3. Students as ‘Expert Witnesses’
‘The Expert Witness’ variation on the viva voce was borne of designer Zara’s experiences as an expert in parliamentary inquiries and criminal proceedings, coupled with a keen interest within our cohorts for more law reform/advocacy relevant assessments. This viva variation is used in two elective units in the LLB program undertaken in the penultimate or ultimate year: LAWS5057: Conflict of Laws and LAWS5071: Health Law and Ethics. The convenor in each unit adopts the role of Chair of an Inquiry conducted by the Attorney General and House of Reps Standing Committee on Health, Aged Care and Sport respectively. Chairs post the Terms of Reference for inquiry to which students are to respond with written submissions written on behalf of an entity interested. The viva component is conducted during exam period mirroring an in-camera appearance by an expert witness before an Inquiry lasting 15 minutes total.
This more in-depth viva, positioning students as legal experts offering assistance on complex matters of interpretation and reform, lends itself to elective units in latter years (students having themselves self-selected into areas in which they possess particular interest). ‘Inquiries’ called to date include: the Attorney Generals’ Inquiry into an issue affecting personal injury liability issues concerning cruise ships and passengers; and House of Reps Standing Committee on Health, Aged Care and Sport Inquiry into the impact of the COVID‐19 pandemic on the law and regulation of healthcare in Australia and Inquiry into the legal regulation of innovative treatments in Australia.
Q&A with the teaching team
Q: How does the viva in your unit authentically contribute to a law student’s graduate capabilities?
Harry: The viva better prepares law students for the practice of law. Law does not just involve writing memoranda because a key aspect of written work is the oral questioning that inevitably follows in any work environment.
Paul: As a legal educator and practicing solicitor, it’s incumbent on me to ensure that the students I teach are well-versed in the profession’s expectations. Legal education and legal practice are not divorced of each other and should be aimed at addressing the core function of the legal profession, being the resolution of legal disputes. Clear communication, responsiveness to queries, and collegiality go a long way to getting the job done so it’s crucial to develop these capacities before graduation.
Q: We’ve spoken a lot about authentic assessment, but, in addition, how has introducing a viva made for more personalised learning in your units?
George: The viva in LAWS5071 has been a game changer. I am able, in real-time, to assess knowledge and critical thinking, with the ability to ramp up the difficulty of questions to challenge stronger students on the fly. I am also able to provide feedback on their performance at the end, which helps with their learning as they can ask questions to clarify my comments on the spot. This is far more efficient than providing written feedback, which most students never engage with constructively.
Harry: I have been teaching law for over 15 years and have found that the viva experience presents an exciting mode for teaching. While the viva is an assessment, I suggest that it actually affects and enhances the learning process itself. From the perspective of assessment, it presents some challenges. Because the Viva Voce aspect is based on a written submission [in LAWS5057], I think that it is essential that the person who graded the essay also conduct the viva portion. That way, the questioning is individualised and builds on the strengths and weaknesses of the essay and yields a personalised student experience – something appreciated by students.
Q: Are vivas really a resource efficient method to assess students?
George: Aside from addressing concerns about plagiarism and academic integrity, this assessment provides a much-needed bit of personal/direct contact, which with rising class sizes is at a premium. As the assessment can be carried out within normal marking workload allocations (in lieu of reviewing written work), the viva simply provides better value for students — and a more rewarding experience for assessors.
Paul: Just like the assessable component of the viva, the provision of feedback is an exchange where students can query any qualitative feedback received in real time. This aims to maximise the usefulness of our feedback and minimise the likelihood of grade appeals which require more staff to expend additional professional time.
Q: Do you have any practical tips on how to administer this sort of assessment for course convenors?
Zara: Like administering any assessment, how smoothly it runs comes down to how well you prepare your students and markers. A well-designed rubric + detailed instructions (including some mock questions to help students gauge the sophistication of responses expected) are must-haves. This gets everyone on the same page. It also controls variables to counter impression-based marking (and/or the impression of impression-based marking).
In addition, I find that running a designated Assessment Briefing + Q&A Zoom Session ahead of viva assessments is a useful way to address any student anxieties about undertaking ‘novel’ assessments.
Aside from going through timelines, the rubric, instructions, and FAQs, I recommend introducing students to your team of prospective markers in that Zoom Session. This way, students can be assured that they’ll be meeting with someone with requisite interest and knowledge regardless of who ends up grading their session. It also helps to set the tone for the sort of professional-collegial approach we want within the task. Pop a recording on iLearn and direct all further queries to the Discussion Forum and you’re good to go!
What do students think?
The team will be looking to conduct more rigorous empirical analysis of student experience as we refine our assessments and test multiple cohorts with variations moving forward. For now, comments from LEUs, written correspondence, and post Q&A debriefs appear to be positive:
“I enjoyed the viva voce assessment that reflected a task you are likely to experience in a professional environment. It is great to get an opportunity to undertake a different type of assessment that is more practical in nature as we don’t often get to do this in our study. I also enjoyed this assignment as it gave us an opportunity to receive immediate oral feedback on an assignment. This allowed us to ask any questions or clarify areas to get a more in-depth overview of our strengths and weaknesses.”
“Although when I saw it in the unit guide, I panicked initially, the viva voce was an excellent assignment that allowed me to develop my communication skills further and see more into the world of legal practice. I hope they are employed more readily throughout the rest of a LLB because I truly believe they are beneficial. Really thankful to Paul and Zara for their time, patience and assistance throughout this course.”
“I did Health Law and Ethics in Sem 2 2022 and it really left an impression on me, specifically the Expert Submission assignment on the social determinants of health and public health. I never realised how passionate I was about public health until I learnt about it and did the assignment. In my assignment I focused on the disparities on health outcomes between affluent and non-affluent areas, as well as Indigenous populations.”
“This unit was extremely beneficial in developing my understanding of the legal industry and the importance of ongoing self-development. The oral report assessment was very different to other assessments I have completed throughout my course and I found it to be a very interesting and engaging way of completing an assessment, where I was able to develop my communicational skills in a professional manner. Furthermore the feedback received following the oral report was really encouraging and inspired myself to undertake further research into the ideas and feedback discussed.”
Final thoughts from the teaching team
Creating space within an assessment regime for every student to visualise and actualise themselves into these professional roles can make for transformative learning experiences that shape their personal and professional identity.Zara Bending
Zara Bending: The potential for further variations of the viva in law education is limitless given the array of professional tasks performed by graduates requiring a strong communication skillset. However, we are positive that there is room for uptake for both authentic assessment and the viva voce more specifically across all disciplines in higher education.
Paul Maluga: Integrating authentic tasks into assessment regimes prepares students to excel in an increasingly competitive graduate market. We deliberately feed graduate and industry feedback into our assessment themes. It also provides an opportunity to invite professionals literate in the evolving needs of the profession into the learning environment. I recommend giving a viva assessment a go, particularly in PACE units.
Harry Melkonian: In this era of recorded lectures and Zoom tutorials, students can become disengaged and discouraged. The viva demands one-on-one interaction between student and teacher and, even if conducted by Zoom, creates a personalised learning experience.
George Tomossy: Students have consistently fed back that they both enjoy and value the experience, which is unlike anything they have had in their law degree. I would much rather participate in an interactive oral assessment any day, which is by far more enriching than grading a written paper or review a pre-recorded effort.
This article draws on work presented at the 2022 Professional Legal Education Conference: LawTech, Newlaw and NetZero: Preparing for an Uncertain Future.
Acknowledgements: Text provided by Zara Bending, Paul Maluga, Harry Melkonian and George Tomossy. Author photos courtesy of authors. Banner image by Shutterstock. Photo by Headway on Unsplash. Post edited and reviewed by Karina Luzia.