Value & Impact overviewThe AMOSSHE community has been increasingly aware of the need for an evidence base to demonstrate its value and impact over recent years. Whilst there is anecdotal evidence to confirm that student support makes a positive difference to the student experience, there is little solid evidence to support such statements, as demonstrated in the FSSG report ‘The sustainability of learning and teaching in English higher education’ (HEFCE 2009). AMOSSHE’s 2009 Supplement to the HEFCE FSSG report (PDF 230 KB) rehearsed this argument in greater detail. We were all too familiar with NASPA’s critical work on assessment and learning reconsidered, but very conscious of its focus on learning outcomes and the United States context. AMOSSHE felt there was a real need for materials that would enable student services managers in the UK to adopt a more professional approach to understanding their value and impact. In turn, this would enable an improved student experience and facilitate better management. In December 2009, AMOSSHE proposed the Value & Impact projec to HEFCE’s Leadership, Governance and Management fund. It was awarded £73,231 toward developing the fifteen month project. AMOSSHE is delighted to be championing the value and impact approach, and in particular believes that:
| Welcome"I am delighted to introduce this new resource for the higher education student services community - the result of a highly collaborative project led by AMOSSHE, the Student Services Organisation during 2010/11. Value & Impact evaluation represents a rich, reflective process paying strong dividends in terms of improved user understanding, professional learning and service enhancement. The evaluation journey begins here - I wish you well in your travels!" Dr. Andrew West "Now more than ever, it is essential that we understand the value that our services add to the student experience, and the impact that high performing services can have. I commend the outcomes of this project to you and look forward to its wide dissemination." Professor John Craven |
About the toolkitThere is a dearth of evidence to demonstrate the cost and value of student support services and student development activities (AMOSSHE, 2009; HEFCE 2009). Further, there are no appropriate sector-wide agreed or understood tools or sufficiently developed measures for evaluation. This makes benchmarking difficult, and restricts any comprehensive understanding of the effectiveness of services. Thus, the Value & Impact project has been established to fill this gap by embedding awareness of value, impact and good practice to encourage improvements in the overall quality of service delivery at higher education providers. The toolkit developed by the Value & Impact project is aimed at helping departments undertake performance monitoring, assess what is most appropriate to their organisation and students’ needs, and support professionalisation in the sector. This toolkit is designed to help Student Services professionals undertake evaluations of value and impact in their service areas. It aims to support and inform evaluations rather than constrain or prevent alternative approaches, which might suit different institutional contexts. It has been developed from the literature review, which was the first output of the Value & Impact project (CHERI, 2010). The aim of the literature review was to identify and produce a thematic analysis of existing literature and other published resources on the policy and practice of undertaking assessment / evaluation of student services provision, which is broader than process monitoring and user satisfaction surveys, and asks ‘how services help students to develop, grow, achieve, learn and succeed’. Many of the publications found for the review point to the fact that, although much has been written about assessment / evaluation theory and why it is important to assess / evaluate the value and impact of student services, there is a lack of research into the extent to which evidence-based assessment / evaluation in student services is indeed taking place. The evidence that does exist remains ad hoc and anecdotal and stems primarily from single institutional case studies. Thus, the literature identified is stronger on conceptual frameworks and approaches than on practical solutions for real life settings. Furthermore, the review failed to find relevant studies of value – specifically value for money – which meant that the search for studies continued outside the literature review phase. With the exception of value for money evaluations, the majority of literature found for the review originates in the United Sates. Much of it is focused on student learning outcomes-based assessment rather than on other impacts on the student experience that may have only an indirect relation to academic success (for example health and wellbeing, finance, housing, careers, part-time employment). Being of US origin, the literature used for the toolkit has had to be modified and adapted to the UK context and the different institutional circumstances found in this country. Using the toolkitThe toolkit is intended to support evaluations of the value and impact of services that support students. They are aimed at providing:
The toolkit is not meant to be prescriptive. It has been designed to help stimulate ideas and raise questions when thinking about and planning value and impact evaluations. Further informationVarious software packages are available to help create and support online surveys, including SurveyMonkey, smart-survey and surveygizmo. Packages such as these provide a quick and easy way to produce online surveys as well as basic analyses of data, which can be viewed graphically (although anyone considering using them should first refer to the data protection policies of these packages). More sophisticated packages are available to analyse data of various kinds. For quantitative data, SPSS is commonly used; for qualitative data, available software packages include NVivo and ATLAS.ti. However, to use these packages effectively, training is required. A number of UK universities provide training in these packages and indeed in basic and advanced social science research methods (including questionnaire design, analysing qualitative data, focus groups). |