These notes, taken during the course of filming for the Birmingham City University video case study, provide a further insight into the implementation strategies outlined in the Toolkit case study.
Professor Alan Staley, Head of Learning Technology Development
e-Portfolios mean different things to different people: at one end of the spectrum, there are e-portfolio tools that are quite restrictive while Mahara is at the other end. Mahara is a blank canvas on which students can portray their own personalities. An e-portfolio tool is different to the VLE. It’s owned by students who have the opportunity to work with it at a personal level. By these means, we find out so much more about our students. It’s really valuable to get them to divulge more about themselves in their e-portfolios.
Why Mahara?
The drive towards using Mahara started with a small number of staff interested in PDP and employability. They needed a vehicle to achieve their ends, plus we had built up experience with Moodle and knew we could work with another open source tool.
Different faculties have different criteria – for example, health is very different to engineering – so the blank canvas approach of Mahara was appealing. We felt we would put in scaffolding via Moodle and leave the e-portfolio blank. Some institutions provide scaffolded support within the e-portfolio tool, but it hasn’t been that easy here as each faculty has a different take on what an e-portfolio should look like. We let people play with the tool, then set up an open forum in Moodle so that they could tell us what they want. From that experiment, a requirements list was drawn up and an external company commissioned to adapt Mahara to suit.
Outcomes
We have had mixed success so far. Those staff that ‘get it’ really get it, others don’t see why you need an e-portfolio. Some want to teach via Mahara ie create content for it. It’s taken some time for staff to understand that it’s a tool for students to showcase what they have done. That’s not an easy message to get across.
Student response has been mixed too. In the Business School and in Law and Education where academics have designed activities that regularly use e-portfolios, it works well. If it’s left as optional, students are confused as to why they are not using Facebook. In other words, students see the benefit only if its use is designed into learning activities appropriate to that discipline.
There isn’t a lot of connection yet between the e-portfolio tool and Shareville (a 3-D environment developed at Birmingham City that is populated with problem-based learning opportunities). But there is no reason why students can’t document what went on in Shareville in their e-portfolios.
Implementation
We installed Mahara in 2008 and have not done too much more development yet. However, there have been developments in the way governance takes place. There is now more formality in how we manage both Mahara and Moodle. There has also been more investment – two new servers to support Mahara for example. The infrastructure is catching up and funding is happening now. Mahara was originally installed on one of our existing servers, but now it’s mainstream we cannot afford to have failures. That’s how we have worked here… we have allowed growth to occur organically then, because greater use can lead to more instances of failure, we have invested more.
Our first significant pilot was in the Business School, ranging from 1st year PDP modules to use on postgraduate courses. Mahara has also been used in Law, Education and Social Science but elsewhere, less. There is some use of Mahara in Performance and Media but we have yet to see it adopted in Health. But overall, there has still been a significant take up – 4000 students last January (2011) despite a patchy distribution. In many ways it is similar to the implementation of Moodle – patchy to start with but catching up over time.
We had two multimedia developers working on Shareville and another works on training staff, but in terms of pushing out Mahara, I have only had one person apart from the IT team who administer it and provide the helpdesk.
Drivers
The drive came from key academics as well as from us, the learning developers. It was a bottom up process which then got senior management to follow by demonstrating the potential to them. We had to get a demonstration first as so few people really understood what e-portfolios were.
Challenges
There are a number. Because Mahara looks and feels a bit like Facebook, people can ask why not just use Facebook? But we need an institutionally based e-portfolio that we can integrate into Moodle and so close the loop on assessment. Students can submit aspects of their e-portfolio through Gradebook in Moodle. They can also create a secret URL to give an employer access. You cannot do this on Facebook. But the Web 2.0 mentality (ie we don’t need institutional systems) can be a hard one to fight.
Closing the loop on assessment
What happens is that students collect evidence, complete a blog in Mahara, then go back into Moodle where they can chose from a list of views or pages as they are called now. All that they have created can be seen by the academic in Moodle, it can be graded, then the Mahara view is released so the student can carry on. It goes from Private >Staff view>Private again. This is a cycle that really sells it to me, it plugs a gap.
Evaluation
We have had no formal evaluation because we have let people work in their own way. As a result, faculties will do their own evaluations. I think that is better than a standardised survey. There are no criteria for success in the classic sense; success comes if students are doing good stuff. Is Moodle a success? In uptake terms, yes, but how much of that is good quality is difficult to measure. The best approach is to ask if it works at a local level. Student motivation is different too in different parts of the university.
The future
I would like to see e-portfolios more deeply embedded in learning activities. We need to do this so that students can more easily sell themselves to employers by demonstrating what they have done. We will probably soon see Mahara on mobile devices so that content can also be captured in the workplace.
Lessons learnt
Starting again, I would try to get other stakeholders involved such as careers and central services. I could also have benefited from setting up a group of different stakeholders with an interest in Technology-Enhanced Learning and Teaching.
Key messages
You will need champions to demonstrate how e-portfolios work.
Finding real examples is also important to make clear to people what the benefits of e-portfolios are.
Students need clear explanation and support – the assumption that they will just get on with it is false.
Encourage the use of multimedia. In a good e-portfolio you will see a variety of media and the student shining through.
Jon Curwin, Principal Lecturer, Business
An e-portfolio is more than an online CV. In an e-portfolio, students can link evidence to what they claim. The software allows the generation of more than one page (or view) and that can be what you want it to be. It can be designed for your tutor, employer or yourself or you can produce a single page to meet professional body requirements with underpinning evidence.
Introducing students to e-portfolios
Students don’t need a lot of technical instruction and in any case it’s good for them to do it in their own way. For example, you can drag and drop features in Mahara, you can upload images or spreadsheets to demonstrate numerical skills, or a video to evidence communication skills. These possibilities make an e-portfolio a voyage of discovery.
I introduce the use of Mahara in class so that I can go round and talk to individual students. We work through a range of topics such as how to make a presentation. I then ask how they would evidence the skills they describe. If you are building up an e-portfolio, you are soon made aware of the need to collect evidence and argue your case.
Historically we have used paper portfolios in some basic business modules. These could be very big and it was always an issue as to what they told you about the student anyway. I felt it was a risk to stay with what we had always done. Implementing an e-portfolio involves some degree of risk, like any innovation, but there is also a risk to be had from not changing. I took the view it was worth changing and told my students about the risk I was taking. We had no examples and 500 students, but it was the students who had to know how to use the tool, not the tutors. Staff advise only on the content. As managers of the future, students will need to take such decisions and assess such risks.
Outcomes
The outcomes of our first implementation were stunning. It might have been because the students knew it was an experiment but their work was far better, more selective and original. Seeing the creativity of students was an insight.
I had to have a fallback position and that was paper – we have to have other options for students with disabilities, for example - but Mahara delivered the benefits we were hoping for. The content was usually created elsewhere so could be found if anything had gone wrong. Mahara was just where they located their work.
Benefits
Students using an e-portfolio have to consider their content from different viewpoints. We introduced it because we felt it was important to evidence skills and present to different audiences such as tutors and employers. There was also a general consensus that we should be moving away from paper. e-Portfolios help students to engage with the processes of learning. They get students thinking, they are motivational. It’s not just a matter of acquiring content but engaging with the process.
Students also like using something new; it’s nice that there are no ‘experts’. I introduce e-portfolios as part of a learning journey, an experiment with new software. Then the creative process gets going. Students start to evidence their skills in ways that are unique to them even, in one case, by means of a poem.
Employability
Mahara gives students choices. You don’t want people to have to scroll down so you have to be succinct; in this way, students begin to be selective about their content. Employability is also about confidence in your skills and the ability to take a risk. You have to make choices: is this adequate and fit for purpose? If not, then you have to address the weaknesses in your case.
People ask for completed examples but an e-portfolio is confidential. The person who writes it makes the decision as to who sees it. This is different to paper portfolios which take the form of a large file containing text-based materials that are largely pre-determined by the course. In comparison, Mahara offers an electronic page that can host multimedia materials and offer an insight into the way the student works.
Assessment
The feedback on the assessed use of e-portfolios has been positive so we are committed to continuing.
I am involved with two groups in the Business School: a year 1 undergraduate module and a postgraduate one. Both involve assessed e-portfolios, but quite differently. In the undergraduate module, students submit a formative view designed for their tutor at an early point in the module some 5 weeks after coming to university. This can help the tutor write a reference and allows students to articulate their skills and gain feedback. When they come to repeat the process at the end of the module, they are likely by then to be more reflective. Each presentation is submitted through Moodle where the tutors can give feedback. This process is of benefit to both students and tutors; students can try out the tool and we can see how well they are engaging. For a 15 credit module, students must submit 3 views; an additional employer view engages them in a debate about what is expected of a new graduate employee.
When we first used Mahara, we transferred over the previous marking scheme. This had been adequate for the paper-based skills portfolios but did not reflect the fact that students could attach files such as PowerPoint presentations and Excel spread sheets, link to other external sources such as YouTube and become more creative with the use of images. The marking scheme needed to be revised to reflect the fact that students could do so much more with Mahara.
Perhaps because of this increased potential, we have been surprised at the quality of the work. It’s slightly different for the postgraduates who take a zero-rated module. However, you cannot progress without passing it. The student response to this has been interesting. I haven’t come across too many who are purely task focused. I can see they are reflecting.
Key messages
Students can see the value of building an electronic profile that can evidence the range of work they do.
Building and improving electronic pages over times encourages students to be more reflective about their work and their learning.
Students can be more creative in the way they present their work and themselves.
Building an e-portfolio can be an adventure; it can also be seen as challenge and a chance to develop creative problem-solving skills.
The outcome is about the student. Staff are in a supportive role to help students develop an effective presentation.
Sarah King, Senior Lecturer, Law
At the most basic level, an e-portfolio is a collection of material that demonstrates aspects of yourself. This can include both personal and professional information and you can reflect and plan, so an e-portfolio is much more than just storage.
As part of an e-learning strategy, we looked for resources that would support students undertaking law, and identified e-portfolios as valuable. We implemented e-portfolios for PDP in a first-year module where previously paper-based portfolios were used. Last year, we rolled out Mahara to 200 students as a pilot and achieved some success.
Learning points
We learnt that you need staff development in advance and that exemplars are valuable. We also had to respond more quickly to any technical issues that arose – we assumed that students would find the tool intuitive and easy to use. The reality was that this was not always the case.
Benefits
e-Portfolios help students gain skills of employability and professionalism. Students can present themselves in different ways, starting with their social selves then building in other aspects. In this way they can demonstrate ‘a rounded self’. For employability, it’s also key that students develop skills of selection. An e-portfolio is a vehicle for students to reflect on and select material so that by their second year they have a more professional page (view) which demonstrates the ability to reflect plus their academic aspirations and achievements.
From the staff perspective, what was a paper exercise has now moved online so we now longer get faced with boxes of folders stuffed with papers. The need to be selective focuses the students’ minds and makes it easier for us.
I think it’s brilliant that we are now sending students out into the working world with relevant skills. Now lots of law students apply for jobs online and the old-fashioned CV is disappearing. The e-portfolio can be a feature of the application: if you want to find out more about me, follow this link. I have heard second hand of employers looking at e-portfolios but have no evidence myself.
But there is an intrinsic value for law students in two ways. Firstly, e-portfolios enable students to store PDP evidence in one place that is accessible 24/7 from any location. Secondly, because law is a profession, it is important to think about how you present yourself professionally. In the first year, we encourage students to use it as a social tool and to collect evidence of the competences in the one module, then reflection on the feedback they receive is submitted at end of year 1. In the second year, the e-portfolio is developed as a means of presenting themselves to an employer.
On the postgraduate diploma course, Mahara is used for optional self-assessment as students get closer to leaving the university. We film role-play interviews which are uploaded into the students’ e-portfolios to share and reflect on. They can use these to track their progress at the end of the year and see how far they have come. The postgrads in fact use their e-portfolios for lots of purposes including written exercises uploaded as examples of their work, feedback from tutors and so forth.
Students and reflection
I have found that when you ask students at the end of a session to reflect on their progress, they are unsure and aren’t really thinking deeply. The advantage of an e-portfolio is that they can return when and where they want to, developing important reflective skills.
There has, however, been a mixed response to reflection in e-portfolios. Some students produce a lot, others don’t easily appreciate what they are asked to do. At first, we set them the task in September to be submitted in May the next year. We didn’t recognise that some students would wait until April before getting down to the task. We learnt that structured interim checks are needed.
Key messages
The best thing about e-portfolios is that they can be visual and engaging.
Tutors learn things about their students that they wouldn’t otherwise know.
Before you implement e-portfolios, get everyone in the staff team to sign up to the idea.
Think carefully about how students will adapt – they need support too.
Think about how students could present information.
Be willing to adapt and respond quickly when needed.
Graham Lowe, Learning Technology Champion and Senior Lecturer, Education
My role as a learning technology champion is to look at how e-portfolios can be implemented and push forward e-learning in Education by finding new tools and demonstrating what can happen as a result.
The embedding of e-portfolios on the courses is still in the early stages. In three years’ time, we will have most students doing something in an e-portfolio. e-Portfolios need to be student-led but they are such an important part of our pedagogy that they are likely to be embedded in every course at some level.
Encouraging others
As a learning technology champion, I aim to make others feel confident that they can use these tools. So I devised a way of getting staff to engage with e-portfolios by setting an action research project for students while they are on teaching practice. Because the projects are submitted through Mahara, a greater number of tutors have become exposed to working with e-portfolios. Students can use their e-portfolios for social purposes, but they can also use the tool to present in a more visual way work that would otherwise be presented traditionally. For example, graphs and videos can be included, as can face-to-face interviews… there are criteria to be met but they can present their work in any way they feel appropriate to meet these criteria.
Benefits
Students have more freedom in how they present their research. They are presenting themselves as much as the data they have collected. Being a professional is about making decisions. e-Portfolios come from them and show their decisions – for example, ‘this is how I did it’.
Challenges
There have been none really. Most staff are keen to see how things work. It’s more a case of how you present a new technology to them.
Assessment
Staff have links to their students’ e-portfolios by email and share the marking criteria with their students. It is easier then to keep a check on their students’ work. It’s no different to assessing any other piece of work.
The student response has been very positive. It’s different to what they are used to but we deliberately avoid training them, just give them enough input to enable them to experiment. Reflection on their practice is part of the PGCE course and is built into the standards they have to meet. An e-portfolio enables them to do this on an on-going basis. By the time of submission, it’s usually a work of art.
Key messages
Think about which of your current systems can cross over easily and which need a culture change. Work with what is easy first. Resistance will occur but work around this. Sometimes work with it, sometimes be subversive.
A key point is that e-portfolios encourage selection and develop audience awareness. Everyone has a lot of ‘stuff’. e-Portfolios help students organise and manage their online presence… this is the bit you can see, this is the bit only my friends can see…
Some staff have fears about e-learning. It’s best to introduce e-portfolios to students first. Staff then have to work with the tool, they become interested and want to know more.
The processes involved in marking e-portfolios are no more challenging than marking any other format. The criteria are the same.