Key words: student-centred assessments, reflections and evaluation.
Is Portfolio Assessment in Language Learning worthwhile?
In other words is there enough of a positive effect on students and the learning outcomes, considering the energy required from both sides, students and tutor, for portfolio introduction, production and assessment?
Languages for All (LfA)
LfA is one of ten Strands in the Discovery Themes offered to all UG students at the University of Leeds. We teach four Roman script and three non-Roman script languages, with all languages covering Beginners and Elementary level. Intermediate and Advanced levels are offered in most languages. All modules are credit bearing and most are taught in a two hour weekly seminar over one or two semesters. In this article we focus mainly on Beginners and Elementary levels to reflect on developments in LfA assessments.
We teach the following languages and levels:
Figure 1: All modules are assessed via continuous assessments and a final speaking exam during the exam period(s).
Until recently one of the continuous assessments was a summative portfolio. We have now come to a point where we can take stock and evaluate our eight-year journey. The question raised is becoming particularly relevant for our subject area at this stage, as we are in the process of further streamlining and reducing assessments to avoid over-assessing (University of Leeds, 2016). At the beginning of our journey we did not consider data collation nor comparison of pre and post portfolio assessment results in students’ achievements, as all our efforts were focused on portfolio design and acting on student feedback. We also did not anticipate the complexity that emerged from developing and introducing this new type of assessment.
In this article we intend to outline the background that led us to introduce summative portfolios. The various formats we developed over a number of years and our underlying rationale and reflections that lead to subsequent changes will be explained. We will attempt to answer the posed question and also invite the readers to share their own experience in relation to assessments and challenges in language learning. It is very likely that colleagues in the School and further afield have their own experience in using Portfolio assessment. We hope to encourage a dialogue and discussion on this aspect of teaching and learning, which has been the focus of ‘Transforming Assessment in HE’ to further develop good practice in students’ assessment (Council of Europe, 2001). We would like to point out that this article represents our personal perspective on developments in LfA and some colleagues may have differing opinions.
In 2010-11 a process of standardising our assessment structure and formats began, starting at Beginners and Elementary levels followed by the higher levels in subsequent years. We felt that this would facilitate students’ engagement in the assessment process and open a dialogue within our language community about independent and lifelong learning. This process of standardisation allowed us to share an understanding of assessment principles by focusing on a common approach to assessment, benefiting our whole teaching team.
Our common assessment practice, up till this point, was continuous assessment in the form of in-class tests at all levels. It entailed two assessments in semester one: reading/writing/listening mid-semester (15%) and listening/speaking at the end of the semester (15%). A third continuous assessment: reading/writing (20%) took place just before Easter, followed by the final listening and speaking exam (20% each). The remaining 10% was given for homework, attendance and participation in a holistic form.
Although this structure allowed students to gauge their progress at regular intervals, drawbacks to this tutor-centred approach were evident. Tutors had to design many assessment components and the mid-semester 1 assessment was found to be rather too early in the students’ learning. It also created undue pressure on tutors to complete topics in time for the assessment. The need to reduce assessments and more importantly to unify the diversity of formats that had been used till that point became ever clearer. Furthermore, some external examiners felt that the 10% holistic homework element was unreliable since it could be viewed as subjective, therefore it was abandoned even though many tutors believed that this element encouraged and motivated students. At this stage the Foreign Language Teaching Unit (FLTU) Director initiated the standardisation process which lead to the introduction of portfolio assessment.
Upsurge in Portfolios
Within our teaching unit only one tutor was using portfolio assessment in her higher level modules, following the European Language Portfolio guidelines (launched in 2001 in parallel with the Common European Framework of Reference for Languages). Her positive experience was encouraging and informed the start of LfA standardisation by focusing our attention on portfolios as a new form of assessment: a collection of students’ work in the target language, accompanied by reflections and action plan for improving the language further.
The variety and many formats and uses of portfolio assessment was showcased in numerous Conferences and Language Fora which was inspirational. This encouraged us to opt for a Portfolio approach which was to be innovative since it was based on principles of reflective learning, learner autonomy and aspects of self-assessment (Mhlauli and Kgosidialwa, 2016).
The term Portfolio in language learning was for most LfA tutors, at that time, a new concept. In order for our team to understand the potential of such a learning and assessment tool a focus group was formed in the summer 2010. We explore its application as a form of summative assessment and decided to replace the ‘traditional’ in-class assessments for our Beginners and Elementary modules. The task involved numerous staff workshops during which different options were considered and the following was agreed on: Two portfolios, one per semester. Each would contain five tasks of directed learning that tutors devised and that students would hand in for feedback at set intervals. Students were encouraged to make improvements based on feedback and re-submit the original task alongside the final amended version. In addition students could also include any type and amount of private study they chose to demonstrate their progress in the language.
Importantly a reflective element had to accompany these collections of tasks. To do so students were asked to keep a log of the set core homework and private study and to comment on their learning outcomes and future plans. Furthermore, students were required to self-evaluate the four skills and grammar developed at the point of submission. In order to engage the students with the portfolio grading criteria, they had the option of giving a holistic impression mark of their portfolio. As most students were not well acquainted with portfolio assessment, they were guided throughout the semesters, supplemented by information in the student handbook.
Stocktaking two years later (Summer 2012)
With the introduction of the portfolio assessment initially at Beginners and Elementary level we tried to address what could be seen as ‘shortcomings’ of in-class assessments as described above. Two summative end of semester portfolios meant fewer components overall, no re-designing of continuous summative assessments, relieving pressure to complete topics at particular points in the semester and freedom to be more creative in teaching.
To help us evaluate our new assessment structure we conducted a short survey to gauge opinion from student and teaching staff as can be seen below:
Table 1: Students’ view of Portfolio (Beginner and Elementary)
Table 2: Tutors’ view of Portfolio
Feedback gathered from both sides was vital in closing the gap between what was expected or hoped for and the reality of this new form of assessment. In addition to the above views, we as a team of staff had to also contemplate unexpected developments:
Students tended to favour quantity over quality of learning evidence and therefore in many cases portfolios were becoming very large.
Rationale and reflections were often superficial and/or done at the last minute rather than ongoing, as was desired. It became evident that, as Jennifer Moon (1999, p.3 and p.5) indicated, ‘reflective capacity varies among individuals’ and the idea of the student ‘taking an overview or sitting back from a situation to review it’ did not always occur.
A further development became clear. Considerable amount of time was required to explain to students the format and rationale of portfolio assessment, reducing valuable teaching time which, in fact, we had strongly hoped to increase by way of introducing student-centred assessments.
Also, the portfolio was not designed to be graded holistic as it became in later portfolios. It was the sum of individual tasks which was complex and the workload involved was felt to be extremely demanding, particularly for hourly paid staff.
Mindful of the feedback received, in a number of summer workshops, ways forward were sought to address the students’ and tutors’ response. We decided to use a portfolio in semester 2 only and to drop the need to include any private study. Furthermore, we re-introduced tutor-lead class tests in semester 1 to prepare students for the type of texts and listening material they could then choose in semester 2 for their portfolio. These class tests were called ‘Core Tasks’ and covered the four skills. Our intention was to address the students’ perceived difficulty of ‘too much freedom’ when choosing materials commensurate with the level and ‘not clear what is expected’ (see Table 1).
This new, semester 2 only portfolio format required one Private study piece per skill, including students’ reflection and future plan (i.e. how a particular skill could be improved further) for each item of learning evidence. We also had developed detailed grading criteria for the productive skills as well as templates with detailed guidance for each skill item. Importantly, we had moved away from individually marked components to a holistic evaluation of the portfolio which required devising appropriate holistic grading criteria. Overall, these changes seemed to make the portfolio assessment easier to manage and addressed some of the concerns that students and tutors had mentioned in that tutors felt more comfortable with this hybrid of teacher-centred and student-centred assessment approach.
With refreshed confidence about portfolio assessment, we aimed to foster autonomous learning and encourage reflective learning strategies (Pilkington, 1997) now also at higher levels. A pilot portfolio was introduced 2013-14 in Italian and German, in Lower (exit B1) and Upper Intermediate (exit B2) level modules respectively. We kept to the tried and tested format of the lower levels. Templates for the Independent Tasks (at higher levels no longer called Private Study) as well as a comprehensive Assessment handbooks were produced to instruct and direct students and new tutors.
Despite improved detailed guidance on how to set out a rationale (i.e. ‘In this task I intend to explore….) and reflect on tasks and samples of effective portfolios shown in class, many students at Beginners and Elementary levels still felt insecure in being the producer of their own assessment. Students found it particularly challenging to locate appropriate resources for the listening private study evidence. Also, some deemed writing a rationale and reflections confusing as they could not always link the task they had chosen with their rationale. For example a listening extract or a reading passage they had located in course books and other sources and on which they had to comment was at times seen as challenging or even irrelevant.
Likewise tutors found the speaking skill difficult to grade since students had to upload a pre-recorded dialogue and were tempted to read from their script instead of producing mainly un-rehearsed speaking. Furthermore with the portfolio assessment having been rolled out to all levels, tutors felt strongly that the volume of marking had now increased radically since the Core Tasks and a Project assessment for higher levels had to be assessed and moderated also. To address some of these issues, at Beginners and Elementary levels, we reduced the portfolio components required to the reading and writing skill as shown below:
Table 3: Assessment details
Stocktaking 5 years later
In Higher Education many institutions experimented with different formats of assessment via Portfolio, as did we over the mentioned 5 years, with the aim of seeking the ‘ideal model of assessment’. To share LfA’s experience, we showcased our portfolio journey at the London Imperial College Conference in April 2015. Here we were keen to show the outcome of the most recent questionnaire i.e. our second stocktaking.
Table 4: Students response
We were pleased about the resulting figures and reasonably sure that relevant analysis and subsequent changes to the portfolio meant that students saw the portfolio overall as a positive tool to assess their skills and motivate them to learn. It is clear to see that over 70% in the higher levels would rather have this form of assessment than in class testing. But the yes/no split for or against replacing the portfolio was extremely close in the lower levels.
Table 5: Tutors’ response
Although two thirds of tutors regarded a portfolio to be a good form of assessment, a small majority favoured traditional class testing. Finding the right balance between not over assessing and still motivating students to learn ‘with a focus’ we continued to search for still better ways to accommodate students’ and tutors’ needs. In doing so, it became ever clearer that challenges we were dealing with were not merely to develop the ‘best portfolio’ option but that we had to factor in other impactful aspects – the nature of our cohorts is very complex, compounding the effect of a demanding assessment format such as the portfolio. In LfA classes, first, second and final year students study alongside each other, leading to a widely varying need for supporting students in their organisation and time management. Also, some students, i.e. with language learning experience can draw on previous language learning strategies, more than ‘non-linguists’. Some found portfolio assessment and the requirement to reflect on their learning puzzling or even arduous since they had never encountered portfolio assessment before. Equally, students with good language learning strategies, who may intuitively understand and use language structures, may find it difficult to unbundle their internalised understanding of the (new) language and it can be cumbersome to explain their learning process. From many tutors prospective the volume of marking remained an issue but more so the challenge to mark a portfolio holistically with skill evidence and rationale and reflections.
Slimming down from 2015-16
In a radical move to address the concerns mentioned above, changes to our assessment were developed. A number of tutors across the higher level trialled the introduction of a group Speaking task at the end of semester one, replacing the portfolio entirely. A Writing task in class, in semester two, replacing the project.
At lower levels we reduced the Core tasks of the first semester to one Reading and one Listening task only and focused in the second semester portfolio solely on the writing skill, i.e. two writing tasks, including rationale and reflections.
Further issues clouding the portfolio at Beginners and Elementary emerged nonetheless. Online learning and translation tools have become very sophisticated and although of great benefit for language learning in and outside the classroom, we ultimately did not succeed in guiding students as to when and when not to rely on such online aids in writing for their portfolio. Tutors were now faced with portfolios that often entailed language structures that went beyond what had been taught and was expected at beginners and elementary level. Complex sentences, passive voice, subjunctive forms etc. are demanding structures that, if used in their evidence, have to be explained in the rationale and reflections to demonstrate their origin. If there is insufficient evidence for this, grading a portfolio becomes extremely challenging resulting in disappointing marks.
What did we learn?
Over the past nine years we had the pleasure of reading wonderfully composed portfolios and it was clear that portfolios helped a great deal with students’ engagement in the language learning process. It freed up some teaching time and helped avoid ‘teaching to the task’ when compared with previous continuous assessments. Digital uploading and text scrutiny via Turnitin was helpful.
Students’ feedback as well as views from external examiners and the teachers involved was invaluable in on-going rigorous reflections on our part as to why, what and how we assess in portfolio. An enormous amount of energy was required by the team for analysis, proposals, amendments, changes and intensive workshops, mainly through the summer periods.
In the summer of 2018 we asked ourselves finally, if Portfolio Assessment in Language Learning is worthwhile. Perhaps this is not the right question to ask. Rather, we need to modify it: ‘Is a portfolio worthwhile within the complexities of an assessed language discovery module?’ The majority in our teaching team have come to the conclusion that a Portfolio, particularly at lower levels, does not lend itself well as a tool for summative assessment. Learner autonomy, creativity and meaningful reflection is too difficult to reconcile with the rigor and warranting of ‘high-stakes assessment’ (P. Knight, 2006). After all, portfolios by their nature should be a way to display a student’s achievement in a variety of skills and abilities and should be a document that captures the students’ reflective practice without it being graded. In our modular approach and with mixed students’ cohorts in LfA an assessed portfolio has become counterproductive. We would, however, recommend a Portfolio for formative learning.
Council of Europe. 2018. European Language Portfolio. [Online]. [Accessed 16 July 2019].Available from: https://www.coe.int/fr/web/portfolio/home
Knight, P. 2006. Grading, classifying and future learning. In: Boud, D., N. Falchikov. eds. Rethinking assessment in higher education. London and New York: Routledge Taylor & Francis Group, pp. 72-86.
Kolb, D. A. 1984. Experiential learning: experience as the source of learning and development. New Jersey: Prentice Hall.
Mhlauli, M. and Kgosidialwa, K. 2016. The use of a portfolio to enhance authentic assessment among in-service student-teachers in social studies education at the University of Botswana. Journal of Education and Human Development. 5(3), pp.84-96.
Moon, J. A. 1999. Reflection in learning & professional development. London and New York: Routledge Falmer, Taylor & Francis Group.
University of Leeds. 2016. Report from Faculty taught student education committee. [Online]. [Accessed 16 July 2019]. Available from: https://workspace.leeds.ac.uk/sites/committees/TSEB/Shared%20Documents1/2016-17/2%20-%208%20February%202017/15%20TSEB%2016-26%20FTSECtoTSEB%20Oct16.pdf
Pilkington, R. 1997. Survey of non-specialist language provision in further and higher education institutions in the UK. University of Central Lancashire/Translang.
 Taken from FLTU Student Handbook level 0&1 2010-11
The portfolio is designed to encourage you to study regularly and consistently over the semester/year. It encourages you to focus on the process of learning as well as the product which will be assessed i.e. the portfolio. You are actively involved in planning your learning, monitoring you own progress and evaluating the learning outcomes. It gives you the opportunity to develop your independent learning skills by allowing you to take control over what you need to focus on, make your own choice of materials and employ your preferred learning style. Your tutor will of course provide guidance particularly at the beginning of the module.
As with any learning which has deadlines, you will need to manage your time effectively. Effective language learning requires regular consolidation and practice so you are encouraged to get into the habit of setting aside time each week for private study.