Created: 4.11.2022, Last edited: 5.11.2022

For the forth webinar we had to plan another lesson. This time the topic was: What are the amendments and guidelines regulating, governing and providing the framework for assessment in vocational and higher education?  We also had to illustrate our teaching session with real or imaginary examples from actual assessment cases.

PREPARATION

Our group prepared as usual: meeting at 5:00 in the mornings. This time, however, I had to skip one of the planning sessions, so I prepared material for others to review during that session. I made few slides (on the SOLO taxonomy [1], about the applicability of Kahoot and the Bologna declaration [2]) and created a Miro board for an interactive session where students would choose assessment methods that they are familiar with in preparation for an in-lesson discussion.

After my colleagues had the planning session I listened to the recording while walking to work and noticed that 2 of them were had difficulties using Miro, so one colleague opened it and shared her screen. To me, this showed that Miro has drawbacks: it requires user authentication, it has a limited number of boards (3), and Miro was used in earlier webinars by other groups, so students would need to remove some of them before being able to join our own. These inconveniences can make a significant difference in student willingness to participate (I mean, if my colleagues were hesitant, think about the students...). So, I decided to design a custom software for the purpose: MyMiro. Now, to be fair, Miro is capable of much more than what I was planning to use it for, and my version simply allows the drag and drop of assessment methods and storage in a remote database. But no need for login, student can jump right in and do the task making it much more appealing. The visual design is not fantastic, but fitted the purpose. In a later planning session, my colleagues completed the task and I got some feedback from them: to add instructions at the top of the page, so I did. Then I submitted it as a pre-task to the students. After some days, my colleague pointed out that there was a problem with MyMiro on mobile devices: instead of scrolling the page, it would drag the selected items of other students instead. This wasn't a problem when I tested because only few students made selections, but now, all students at the top of the page selected things, and there was almost no room to 'grab the page and scroll' (mobile devices don't have a scroll wheel). I fixed this by reordering the students so that those who haven't completed the task yet are at the top (students who completed the task don't need to visit the page anymore, anyway). After this, everything went smoothly and almost every student completed the task (1 missing)..

Because I was preoccupied with this software, I didn't contribute as much to planning the theoretical part, apart from those 3 themes mentioned earlier and my example of how to handle urgent assessment when a student misses the exam for a valid reason.

PRESENTATION

My presentation started well, with the discussion about Kahoot: if it is suitable as an assessment tool. We concluded that it lacks a formative component and the assessment is not constructively aligned with the learning objective. After the discussion, I presented a history of how the education system evolved in Finland. At first, I thought this part was interesting to talk about, as I didn't know many of the details. In hindsight, however, I think we insisted too much on the history because I had to skip few slides at the end. This was also pointed out by our tutor: that we focused on history too much. However, understanding the evolution of education in Finland is important; we need to know who we need to assess and how / why. History teaches us how to act in similar unexpected situations: like when the number of students increased dramatically after the Winter war: there was a challenge to fit all the students in the large age groups into primary schools (see Figure 1). In these turbulent times (Covid-19, Russo-Ukrainian war...) the education systems (including the ways we do assessment) are being challenged, and using past experience helps us prepare for the future as well.

Figure 1: Number of students per year (source: https://www.stat.fi, edited to emphasize the war time)

After the history section ended we checked the result of MyMiro. To make things easier for my colleagues, I created 4 links, one for each room.

We then had discussions about our individual experiences with assessment and concluded that assessment should be tailored to the learning objectives and we often must assess using a mix of methods.

After the discussion, I began the theoretical section. I gave the following guide for doing assessments: they should be recorded and kept for at least 6 months after grading; students must have the opportunity to re-show their ability; students may request a review of their assessment at any time; we must be able to give arguments of the evaluation to the student; if a student who has a decision on special support is still unable to achieve the skill, the competence assessment can be adjusted. I then mentioned that after the 2018 VET reform [3], all vocational skills requirements and assessment criteria must include the following key competences:

Digital and technological
Mathematics and science
Competence development
Communication and interaction
Sustainable development
Cultural
Social and citizenship
Entrepreneurial

We talked about assessment scales applicable to vocational education (1-5 and pass/fail) and grading techniques like using the SOLO taxonomy [1]. I explained how I use it in practice. The conversations with the students were really good so I decided to skip the FiNQF framework [4] (only showed it briefly).

I also mentioned the Bologna declaration [2] and how it helps to unify education all around Europe by providing the credit system (promotes mobility), comparable degrees and two main cycles: Undergraduate (min 3 years) and Graduate (leads to master and/or doctorate).

Finally, I wrapped up by sharing the Flinga link to give us feedback, which was all positive.

OTHER SESSIONS

I really enjoyed the other teaching sessions. I liked the start of the first one, using the mentimeter app to make a word cloud. We had discussions related to assessment according to different learning theories. The constructivistic assessment made me consider the field of mathematics and how the information within a chapter is naturally connected to previous chapters so assessing the final topic only is, theoretically, sufficient if the student is asked to demonstrate all the steps starting from the axioms towards the final result. In practice, however, this is too demanding so, intermediate assessments are done along the way as the theory unfolds. I also enjoyed the conversation related to connectivism and believe that a dialog between the students is important. I pointed out that some learning theories cannot be applied in some fields, like humanistic self-assessment is not useful when evaluating computer science students who tend to be fooled into considering their code good just because it work (that is not enough when working in a team).

The second teaching session started with an interesting discussion about the 'freedom to teach what we want', which increases towards higher education. While high school curriculum is decided at the national level, we at universities / AMKs can decide the curriculum ourselves. At even higher levels (Master / PhD), students can be taught according to the teacher's own personal judgement (from selected research papers). The case study about defining assessment criteria for taking care of a horse was great! It made us get outside our comfort zones and we all agreed that demonstration is probably the most suitable way to assess the competences, however, it cannot cover everything. I related the case study to the similar one of taking a driving examination, where a written examination is also used because the demonstration cannot possibly assess all scenarios. Some kind of simulation could also be helpful (like practicing driving on the snow during summer time).

The final teaching session started with a relatively shocking example (from a movie) where the teacher resorted to violence to prove a point. Some of us were left speechless, however, we are from different cultural backgrounds and violence was experienced by some of us in the past, making the experience less shocking to. We spoke about what makes an assessment good in vocational education and higher education. It should be valid, reliable, practical and have no negative backwash, the latter referring to situations where instead of making assessment criteria for the end goals, we think about what kind of evaluation to use instead because it is more convenient (like using multiple choice questions to assess large groups of students because they are convenient). We also had an interactive session where we decided how to assess the English level of a student. We concluded that they should give a presentation in their professional field. Through the Q&A they demonstrate the ability to communicate. The final assessment should be formative, so they know what they still need to improve.

REFLECTION

I added reflection throughout the blog post this time. Here I only want to briefly mention that I will consider more formative ways of doing assessment in the future and that I want to use connectivism more in my sessions.

REFERENCES

Biggs, J.B., & Collis, K.F. (1982). Evaluating the quality of learning: The SOLO taxonomy. New York: Academic Press.
Bologna Declaration http://www.magna-charta.org/resources/files/BOLOGNA_DECLARATION.pdf
2018 VET Reform: http://data.europa.eu/doi/10.2801/841614
https://www.oph.fi/en/education-and-qualifications/qualifications-frameworks

Webinar 4