Being There: Course design, Teaching and Learning, Assessment and other issues

Customised Programmes

Virtual Learning Environments The whole curriculum

High quality work

'Learner Centred Approach'

Institutional policies on assessment

**Assessment of and for learning

Feedback on Assessment**

Making assessment more effective and efficient

Promoting and supporting student engagement

Personal Tutor systems

Students with disabilities

Programmes should be customised to suit the different needs of a new generation of students. This flexibility should extend to degree structures and learning programmes.

Given the changes in the make up of student populations, and especially with more mature students, it is argued that there is an ever growing need to customise programmes. This includes the idea of allowing people to remain at work while they study, with HEIs acting as learning providers. Other suggestions include flexible degree structures and learning programmes for existing students. Further details.

This ‘continuum of flexibility’ i.e. in terms of time, place, content and mode of learning and assessment implies a partnership between staff and learners with greater student choice and autonomy. (More details also here). This in turn reflects growing student autonomy and individualised learning which is itself rightly founded on growing indications of interdisciplinarity in many programmes.

The implications for assessment in all this are that it too needs to be flexible and ‘inclusive’ for all students, including those from non traditional backgrounds. More details here and here)

Although VLEs are widespread across HE institutions, it is argued that a more sophisticated use of them is required. In addition to their advantages in integrating distance learners, other forms of technology can also help.

There is considerable criticism in the Flexible Delivery Theme’s report of VLEs. In particular it stresses the high percentage of courses badged as VLE-supported, but which inevitably disguises an uneven terrain, with some highly inventive course materials and others of low quality. The need for staff training in this area is emphasised. . On the other hand VLEs and other aspects of technology are seen as a highly effective way of supporting distance learners.

There is a widespread recognition of the whole curriculum as something that extends beyond formal classroom experiences.

In particular an emphasis on employability requires a changing mindset on the part of HE institutions as well as students and employers.

It is argued that employability is at the heart of the curriculum (also here and Scottish Funding Council’s Learning to Work) but it goes further than that. It is possible that from a student-centred perspective, the things learnt and achieved outside the formal curriculum might not only aid an individual’s employability but might also mediate the manner in which the formal curriculum is understood.

.
The Employability Enhancement Theme surveyed 60 projects involving collaborations between institutions and students’ associations in the UK and abroad and developed five themes as a result:

Development of sports clubs and societies
Services for student welfare and diversity
Volunteering
Entrepreneurship and enterprise
Issues of recognition.

Further details and case studies

There are suggestions as to some general principles of best practice in engaging employers in the curriculum. These include the Central coordination of relationships with business and industry, discrete support from the careers service, funding for an employability champion; access to project finding and a mechanism for coordinating activities (more details here and here)

Ways of engaging academic staff and students with issues of employability are discussed. It is suggested that there is a great deal of good practice currently being undertaken by staff across the sector and these staff should be used as a training resource. Links with the Higher Education Academy, especially at subject level, should also be exploited. From a student point of view the literature expresses disappointment at the under-exploitation of the Erasmus exchange programme.

One of the issues surrounding student integration into the HE experience is that they are not always aware of what counts as high-quality work.

It is suggested that more clearly stating what is, and what is not considered to be high quality is part of a move to a more interactive briefing and training of student about assessment expectations and requirements.

The Integrative Assessment Theme suggests that it is increasingly recognised that if students are to attain high standards, it is essential that they develop a good grasp of what counts as high-quality work in a given subject and at a given level. But it is also becoming apparent that making explicit to students the criteria used to assess their progress and performance, valuable though that is, is not in itself enough to help students to come to hold 'a concept of quality roughly similar to that held by the teacher' (Sadler 1989). An important development in rebalancing assessment has therefore been the emergence of initiatives to close this gap by more interactive briefing and training of students about assessment expectations and requirements. One prominent focus has been dialogue about assessment, but there have also been attempts to involve students in generating their own criteria for assessing an unfamiliar task, and to offer them training in evaluating their own and others' work. For Further details

Changing forms of teaching brought about by technology.

The increasing use of VLEs is emphasised, along with other forms of technological advance which are altering the whole notion of the learning environment. In particular it is argued that the result will be a greater personalisation of learning, including the idea of a learner-centred approach.

The personalisation of the learning environment is expected to increase the possibilities for a learner-centred approach becoming the prevailing pedagogy. The notion of a social-constructivist approach emphasising the taking of responsibility for own learning will be increasingly prevalent. Consistent with this will also be an increasing use of e-portfolios for formative self-assessment and self-diagnosis and emerging social software tools on the Internet, such as weblogs, podcasting and wikis. Alongside these changes are likely developments of mobile phone technology and PDAs to facilitate more integration and ‘just for me’ information for students. This has implications for students with disabilities. For example, the use of SMS messages is quoted as a major breakthrough for those with hearing impairments

Alongside these technological developments, a likely result will be the demise of the lecture as a staple of learning. In its place will be VLEs (cross reference VLE section above) which allow repeated attempts at mastering the same information, somewhat in the same manner of the hierarchies of increasing difficulty in a PC game VLEs will therefore shift from being regarded as an adornment to being acknowledged as one of the foundations of core business. Furthermore, VLE development is largely student-led rather than being driven by policy, though staff champions, strategic aim and technology itself are also cited as drivers. The extensive and developing use of VLEs well beyond content is mentioned. But content-based approaches are also becoming more sophisticated (e.g. use of video streaming parts of lectures and critical incidents, digitised materials from HERON; simulations and virtual experiences of labs and field trips etc). Altogether there is a need for an increasing supply of flexible learning spaces.

In addition to the implications for students, there are also implications for staff. There is mounting evidence that electronic interaction is just as consuming of academic time - or more so - though less visible than conventional contact. In general, it is more individually targeted and more likely to lead to enhanced learning outcomes for students, but not obviously to increased cost-effectiveness. The time spent by staff in participating in on-line discussions or responding to emails needs to be recognised and formally timetabled. Currently, many staff are working from home in their own time to carry out these activities.

The need for more investment and the inevitable rise in costs will have to be recognised by both senior management and by the Funding Councils.

The following are cross referenced from ‘more efficient and effective assessment’ below

Use of technology in student support email and text – pros and cons (more details here and here) and use at Wolverhampton for assessment feedback

Institutional policies on assessment.

The Reflections on Assessment Theme quotes Mantz Yorke’s self reflective questions aimed at Institutional policies on assessment. Yorke proposes that leadership needs to be taken by senior management in the sector in assisting teaching staff to develop enhanced student assessment practice - Yorke encourages institutions (the sector) to take this forward based on the construction of responses to a number of strategic questions:

Does the institutional policy or mission imply that its general approach to assessment should be changed? And, if so, in what way(s)?
Are there any general institutional weaknesses in assessment (such as might have emerged from subject reviews or from internal reflection on practices and procedures, which need to be tackled?
Since assessment is, by general consent, the least well-understood and enacted aspect of curricula, what developmental activity needs to be instigated?
In dealing with the preceding questions, is best use being made of existing expertise, both 'in-house' and from outside. And if not, why not?
What, if anything, needs to be done to make the institutional system that surrounds assessment function effectively and be compliant with external expectations?
How does the institution keep abreast of developments in assessment both nationally and internationally?
How does the institution learn from its diverse experiences regarding assessment, and develop?

Answers to these questions will assist in the ongoing development of institutions’ teaching, learning and assessment strategies.

Further details

Assessment of and for learning and the rebalancing of the two, the assumption being that the former has superseded the latter and is crowding it out. This trend is best exemplified by the growing emphasis on summative, at the expense of formative assessment.

The Integrative Assessment Theme offers a diagrammatic version of Strategies designed to rebalance assessment of and assessment for learning. There are four elements that support the strategies. They are

Suggestions on redressing the balance een formative and summative tasks were provided by the Assessment Theme summary document. This suggested that there was a widespread belief that a major step forward for assessment practices will be to provide more opportunities for students to learn from their mistakes through, for example:

the progressive weighting of assignments so that at the start of a course the summative element is a relatively small proportion compared to the formative, to a situation at the end where the proportions are reversed
the use of computer-aided assessment (CAA), which was seen as an unthreatening environment for students and one that can provide instant, high-quality feedback
the application of self and peer-assessment, which are ideal for formative purposes
the development of personal development planning (PDP), which offers an opportunity to embed the notion of an ongoing process, using a variety of sources as evidence, including formative assessment tasks.

Another, more detailed view of the summative / formative balance was provided at a workshop as part of the Assessment Theme.

The importance of accurate and speedy feedback on assessment

There are concerns that formal exams rarely include feedback, other than in the crudest form of marks or grades, and the diagnostic assessment of skills is rare. Solutions to the challenges of feedback are offered, some of which rely on technology.

Improving feedback and providing a link between formative and summative assessment was the subject of workshop in the Assessment Theme.

It is argued that technology can be used to improve individualised feedback on assessment An example is given at Wolverhampton showing how they applied SMS text and VLEs to support ‘at risk’ students.

There is a need for more diagnostic assessment, especially for Internet and study skills and to enhance the targeting of individualised support. In this context it is felt that PDPs will be useful

One idea proposed in the Integrative Assessment Theme is to Introduce ‘cumulative coursework’, where an assignment evolves over the span of a semester or longer, and can therefore reflect and benefit from the student’s improving grasp of the subject matter and from ongoing feedback from tutors, fellow students or placement supervisors and work Colleagues

Making assessment more effective and efficient.

It is strongly argued that assessment practices should themselves be assessed for efficiency and effectiveness. This topic was directly addressed in Workshop 1 of the Assessment Enhancement Theme, which also emphasised the need to reduce the assessment load. A number of suggestions on ways of doing so were offered here. The Flexible Delivery Theme argued strongly in favour of allowing students to take control of the assessment process (perhaps through the used of computerised tests). Other points raised by the Themes include: the unintended consequences of assessment (also known as ‘backwash’), the need to undertake surveys of students’ experience of assessment; the importance of assessments that provide ‘feedforward’; the need to ensure consistency of marking and feedback and strategies for inclusivity in assessment.

The objective of making assessment more effective and efficient was examined in depth in Workshop 1 of the Assessment Enhancement Theme, which also emphasised the need to reduce the assessment load on staff and students. A number of suggestions on ways of doing so were offered. In the overview to the workshop on making assessment more efficient and effective, a number of key issues and dilemmas were identified. These might provide a template for action.

The Integrative Assessment Theme examined the ‘backwash’ effect of assessment, that is to say the impact upon learning produced by a particular method of testing. This notion was illustrated using research evidence that showed how multiple choice tests (MCQs) encouraged a surface approach to learning, whilst essays encouraged a deeper approach. In a further twist it was shown that those students who took a deep approach to learning for the MCQ actually performed less well than those who had taken a surface approach

Alongside the ‘backwash effect’ effect of assessment, the impact of tests in providing feedback (or perhaps more properly ‘feedforward’), was examined in a number of places in the Themes. The Integrative Assessment Theme argued that it was possible to convert feedback into feedforward by interconnecting assignment and assessment tasks and creating a recursive cycle, a ‘feedback loop’, in which feedback comments on one task, draft or set of questions could be fed directly into a subsequent task or draft, or will aid preparation for an exam. The importance of consistent marking and feedback was emphasised, and such consistency can only be achieved through systematic oversight, analysis and evaluation

The Integrative Assessment Theme also argued that conventional student surveys tend to focus on specific courses or module and do not typically ask students about their assessment experience across units, across and between subjects, and across and between successive years of study, which is far more typical of their actual experience. This omission could and should be rectified in order to improve the student experience. Certainly there was a view strongly expressed in the Flexible Delivery Theme that students should be allowed to take more control of the assessment process and that technology offered a number of opportunities for them to do just that via computerised tests and effective interactive teaching tools.

The Integrative Assessment Theme raised a number of questions pertaining to the speed of change of teaching, as opposed to assessment, citing, inter alia problem based learning, where, perhaps, assessment has not kept pace with new styles of teaching, or the converse, where innovative assessment has not been matched by teaching. It was also suggested that students may need some training or at least better preparation for some of the new assessment techniques that are being introduced.

Promoting and supporting student engagement with the wider academic community.

The Responding to Student Needs Theme dealt with the issue of promoting and supporting student engagement with the wider academic community in some detail. It outlined different areas of engagement: engagement with technical and information services; engagement with governance; discipline-based engagement and student engagement in specialised support roles. It suggested that there are three levels of engagement with their institutions outside of their formal programmes of study. The first level involves providing general support through employment or volunteering within, for example, university retail outlets, the university library, office administrative support, as well as establishing informal study support groups (possibly in liaison with the students' association or student support services).

At the second level, students undertake a consultant or technical support role, working or volunteering as a student mentor, peer or ambassador; as a representative on institutional consultative bodies (such as committees and working groups); or in a technical support role within the university's computing service (including website maintenance, loading software etc).

At the third level, students become involved in development roles or specialised support roles such as working alongside academic and support staff, contributing to student support activities such as help desks (for example IT and/or library), study skills support and tutoring, academic advising and counselling.

The level of training and development required increases at each level, with a corresponding increase in investment made by the institution. By the third level it is desirable to ensure that an institution gains a return on its investment by providing training and development early in a student's learning programme so that they can contribute to the institution before graduation.

The Flexible Delivery Theme argued that in certain circumstances there was a fundamental misalignment between the student’s expectation of the HE experience and what actually took place One way of bridging this gap might be through improved student surveys. For instance, the Integrative Assessment Theme offered a number of ways in which surveys might be improved to find out more about the student experience of assessment.These were:

1 Plug gaps in monitoring students' experiences

eg ask students about:

their experiences of exams and tests
the consistency of feedback and marking
how different types of assessment compare with one another
any other aspects of assessment normally overlooked.

2 Tap into their wider assessment experiences

eg ask questions about:

their experiences across modules/course units
their experiences across different years/levels of study
their experiences across different subject areas.

3 Combine questionnaires with other methods

eg explore students' experiences and perceptions via:

focus group interviews
an open forum to which students are invited
web-boards or virtual learning environments which invite students' comments and suggestions.

4 Focus in on changes in assessment practices or procedures

eg ask students to comment:

where procedures change (for instance, a new marking scheme is adopted)
when a new method of assessment is introduced (for example, when oral presentations or portfolios are introduced)

5 Ask different kinds of questions

eg:

what one thing would really improve how your work as a student is assessed?
which aspects of assessment seem to work really well/less well/could be improved?

6 Rethink when to ask students for their views

eg:

carry out a brief survey mid-term or mid-semester, while there is still time to address major
concerns raised by these students
once students have been given feedback on their coursework by their tutors, invite comments on its helpfulness to them.

7 Review what background information you ask of students

eg to enable you to relate differences in students' perceptions to whether

they have studied the subject before, and how well they did
they are likely to take further courses in the future
they live on/off campus
they have a job in term-time
they come from an English-speaking background.

8 Focus in on areas of known student concern

eg where past evaluations have indicated student discontent with the provision of guidance and feedback, make use of items from existing resources, such as the FAST inventory, Weaver's (2006) questionnaire, or a typology of potential trouble spots in guidance and feedback to probe the issue more searchingly

9 Survey staff as well as student experiences and perceptions of assessment particularly where teaching and assessment responsibilities are spread across a large and diverse course team (eg mainstream lecturers, postgraduate teaching assistants, part-time tutors or demonstrators).

Personal tutor systems and their alternatives.

The Flexible Delivery Theme suggested that technology-based PDPs could link with further development of personal tutoring systems.

The section of the Responding to Student Needs publication dealt in detail with personal tutor systems and their alternatives. It used research evidence from a number of studies to outline the requirements needed to satisfy student support needs. These are: the availability of the advisor; the level of knowledge and the level of interest of the advisor. It suggested, however, that one of the problems is a lack of recognition for those who perform the role and that there may be a conflict of interests if the tutor is engaging with personal problems but also responsible for assessing the student.

It argued that a flexible model of personal tutoring is probably the best one, rather than a one-size-fits-all but that a gap exists between theory and practice; this being most apparent in student surveys of tutors failing to meet tutees at least once a semester. The Flexible Delivery Theme argued that technology might offer a way forward, especially in terms of PDPs , but caution was expressed in Responding to Student Needs in terms of a possible ‘technology gap’ between staff and students, or a growing and not always helpful growth of informality. The use of both e-mail and text messaging was cited in this context.

Students with disabilities

The challenges of streamlining assessment to cope with increased massification, whilst at the same time diversifying assessment techniques to cater for specific learning needs, such as those presented by students with disabilities was highlighted here.

The Employability Enhancement Theme looked at enhancing employability through the co-curriculum. Under Services for student welfare and diversity student-run schemes that are aimed specifically at black and minority ethic or disabled students were explored. (see also). Two specific schemes were instanced in this document. The first is a mentoring approach to providing students with disabilities career information and advice. This idea lies at the heart of the ‘Willing and Able’ Mentoring Program based at Deakin University in Australia.

'Recognising that students with disabilities can often face particular hurdles finding employment after graduating, the mentoring programme was started by a then PhD student. The Graduate Careers Council of Australia 2001-02 survey found that 39.6 per cent of graduates with a disability had found employment compared with 53.4 per cent of those without a disability. The mentoring programme was launched in 2000 with 12 student participants; by 2004 the scheme had 100 student participants. The mentoring scheme works by linking a student with disabilities to a mentor in the career area that the student wishes to enter. Through the mentoring programme many students have managed to find employment but according to the founder of the scheme the real benefit would appear to be 'demystifying of disability on both sides', with one of the mentors commenting that they had learnt more from the scheme than the student'. (Taken from Working together: enhancing students' employability).

The second describes what it calls one of the most simple devices for attempting to ensure that careers information is available to a diverse range of students. The example is the Victoria University of Technology's web page of diversity career information resources, which contains links to appropriate sites for the seven following different groups:

indigenous students
women
disability
gay, lesbian, bisexual, transgendered and intersex
mature age
cultural and
postgraduate.

Guide Home Page

Being There: Course design, Teaching and Learning, Assessment and other issues

Institutional policies on assessment Assessment of and for learning Feedback on Assessment

Institutional policies on assessment

**Assessment of and for learning

Feedback on Assessment**