EQUIPE

Report from Task Force 1 on Mutual Virtual Evaluation Visit


Background and aims

One of the targeted outputs of the EQUIPE project is a couple of 'Mutual Virtual Evaluation Visits' (MVEV). Carol Stewart and Toril Eikaas Eide in Task Force 1 took the responsibility for organising and monitoring one of them.

'Visit' was taken as a discussion or reflective dialogue on a certain topic where the participants exchange experience on a chosen theme. Afterwards a diverging concept of MVEV has come up.

In our visit the starting point was decided to be Wolfgang Jütte's case study 'Studies on Adult Students as Contributing Part to Quality Assurance' (German title: 'Teilnehmerforschung als ein Beitrag zur Qualitätsentwicklung'). Wolfgang Jütte kindly agreed to facilitate the visit and also prepared a set of questions related to his article / case study in order to structure and stimulate the dialogue.

On 19 May nine partners in the project were invited to participate in the evaluation visit, some of them responded positively. After having contacted a couple of additional persons altogether 10 colleagues had agreed to participate in the ‘mutual visit’, which took place during week 24 (13 - 17 June).


Performance and content

The communication was asynchronous, and was performed in the discussion forum in a ‘room’ created for that purpose in a learning management system called Classfronter. The participants were given a short introduction about how to access and use this tool.

Wolfgang Jütte opened the discussion introducing a number of sub themes to structure the discussion:

1) Target Group

2) Function of Participation Research

3) Dimension / Question Items

4) Results / Consequences

The colleagues participating focussed mainly on the first point above seeking an adequate or functional definition of our target group. Suggestions brought up were ‘adult students’, (University) Continuing Education Students, Postgraduate Lifelong Students, and University lifelong students. The other topics focussed and also connected to each other were Students’ motivation/aims (illustration from Wolfgang) and Function of Participation Research as

a) Planning instrument
b) Marketing instrument
c) Instrument of self-reflection
d) Instrument of educational policy.

The following example is a key message in this discussion:

“ (…) it is very important to know the goals and the motivation of the adults to prepare and organise the University Continuing Education courses, (…) [however] we also need to know the strategies and policies from the industries, as well as from the key stakeholders (employment services, local authorities, educational system, social partners and so on...) “


Participation

Six out of ten participants took part and visited the room 8, 10, 9, 2, 4, and 2 times respectively. Five responded, 4, 5, 2, 1, and 1 times, the facilitator included. The organisers/monitors are not included in these figures. Four colleagues did not log in at all.

Those participating:

Jütte, Wolfgang, (facilitator/chair) University of Krems

Collot, Andrée University Henri Poincaré, Nancy I

Jazbec, Marijana University of Ljubljana

Canu, Giuliana M. University of Genova va

Soeiro, Alfredo University of Porto

Urponen, Helka, University of Lapland


Output/benefit

It is difficult to say what has been the benefit of this MVEV. As the figures above may indicate, the discussion - even if going on for five working days - was comparatively limited as to participation and content. Time pressure certainly is one and maybe the main reason for this (some of those participating expressed this explicitly; some were on travel parts of the week). Other reasons may be that the participants were unfamiliar with the MVEV genre and/or the Classfronter tool. The latter is quite clear as the participants mix up the Forum (discussion) and the Message tool and also have problems when trying to introduce a new title/topic, so that in the Forum all the messages are in one main thread.

The following comments from one of the most active participants clearly describe the core challenges and dilemmas related to our activity in particular. Experience shows, however, that they apply to such types of electronic, asyncronic communication in general.:

“ The positive thing is that I read with a great interest the contribution of Wolfgang and I talked about that with a colleague of mine. But further, it's really frustration because I really took time to put on some ideas (when I was completely overbooked by that time) but there was no discussion engaged during 2 days and after that I had not real time to make sure that I still could share some ideas with other colleagues in Europe. There certainly is technical matter but not only, there is a matter of availability of people.”

Probably the most important output of the MVEV is the very experience of participating in such a discussion, which may not be very different from many of those forums we prepare for our students in continuing/distance education. This is also commented on by the above mentioned participant:

“But anyway, it was an interesting experience and gives a lot to think about concerning the way we do with our students when we tell them ‘just go on the forum’ and we disagree when they prefer talk to us in a face to face way.”

Time constraints and lack of familiarity with the virtual discussion tool were a slight hindrance to full participation. In the future this could perhaps be remedied by the organisers allowing a period of familiarisation in advance of the actual discussion and, possibly, by scheduling the discussion for an alternative time of the academic year. Demands of the project however limited flexibility in this instance.

Although participation was limited by lack of user familiarity and time constraints, the MVEV was a valuable exercise in that it raised valid and interesting questions about the characteristics of adult participants in University Continuing Education. The discussion hopefully has laid the groundwork for further dialogue between partners.


Future challenges

From our effort with the MVEV we have learnt that when arranging MVEVs in future projects the following challenges should be particularly observed and aimed at:

  • Establish agreement within the project about what is meant by and expected from a MVEV (re footnote 1).
  • Reserve a period of familiarisation with the electronic discussion tool before the very MVEV.
  • Aim for and clearly communicate the expected structure of the discussion.
  • Try to establish a strong commitment and loyalty among the participants so that they really participate, and
  • Spend some time and energy to find an optimal period of time for the MVEV to try to secure involvement and continuity among the participants.

The two latter bullet points are both related to the problem of time constraint, which is extremely difficult, not to say impossible, to overcome. However, it may always be possible to improve a little at each attempt.

After the MVEV had been accomplished and the report drafted the following comment was made by one of those participating “ (…) your concept of MVEV is different from mine. I was thinking of a real evaluation with

assessment, diagnosis and conclusions. Gathering people once a year is not enough to make ideas flow. A lesson for me from this project is……. that communication must be established more often between partners.