INTRODUCTION
This is an index to a set of reports been produced during a two-year a research project which set out to discover what difference bursary support providers (BSPs) may make to student success at universities – and what their (unique) contributions to this might be.
Funded by the DG Murray Trust and commissioned by the Rural Education Access Programme (REAP), it was carried out in active close collaboration with members of the National Bursary Support Providers Forum (NBSPF), a network of about 25 organisations which support university students in South Africa.
Information, ideas, insights and perceptions were obtained through desktop research, key informant interviews and substantial work undertaken by three working groups comprising volunteers from the bursary support sector. These focused on academic, psycho-social and employment outcomes.
The research was characterised by robust engagement with differences in approaches and awkward findings. We hope this intensive work has contributed to the sector’s better understanding of its work as well as provides a platform for further discussion, especially with national institutions.
A full research report was produced, as was a shorter version comprising the executive summary and findings. These are both posted on this website.
In addition, a range of sub-reports have also been made available here as they contain the detailed thinking behind some of the exacting work which was undertaken – probably for the first time in this sector.
MAIN REPORTS
Report 1: Optimising student success in higher education: The (unique) contributions of the bursary support provider sector: Full report.
The first section of the 270-page main report maps the terrains in which the sector operates. It rehearses the nature of current university students – globally and in South Africa – particularly describing the challenges faced by first generation students. The profile of the approximately nine thousand students supported by organisations in the National Bursary Support Provider Forum (NBSPF) is described.
The diverse higher education sector in which these students study in South Africa is then briefly described, noting the different kinds of difficulties faced by universities, particularly since access has been promoted through the provision of loans – and now grants – from NSFAS. Crucially, the research takes place in a context in which success – and throughput – has replaced mere access. The various student support services offered by some universities and bursary support provider organisations (BSPs) are some described, and the concept of ‘wraparound’ or comprehensive support is examined.
The role of universities in society is briefly reviewed, suggesting that the BSPs promote both individual and public goods. A concern about the impact on first generation students of ‘blended learning’ (comprising online and in-person provision) is briefly explored.
Given that the research question was about what difference the BSP organisations may be making to student outcomes (success), measurement of academic and psycho-social outcomes is addressed in considerable detail.
The research finds that certain elements of support are central to student success generally – while other elements are important for addressing individual needs which might otherwise derail individual students.
The report concludes with findings and recommendations – for national and state institutions, for those interested in student success, for the BSP sector and for individual organisations themselves.Date:
April, 2022
Report 2: Optimising student success in higher education: The (unique) contributions of the bursary support provider sector: Summary report.
This report presents the executive summary and findings drawn from the main report.
Date:
April, 2022
SUB-REPORTS: ACADEMIC OUTCOMES
Report 1: Terms used in the bursary support provider sector when measuring academic outcomes: Towards developing common usage / definitions.
Given that the research question was about what difference the bursary support provider organisations may be making to student outcomes (success), considerable detailed work on measurement was undertaken – especially the measurement of academic outcomes.
In the past decade, the focus in the higher education sector has shifted from access to success – foregrounding the measurement of throughput and, in some instances, progression rates. Organisations in the sector had hoped to be able to compare their outcomes with one another and to national statistics – to show that the extra support makes a difference to academic success. In order to do so, a common way of measuring academic outcomes (referenced to national measures where this was possible) was developed for use within the sector – in the process revealing that these had been defined differently within and beyond the sector.
This detailed report provides definitions of a range of terms including ‘cohort’, ‘proceed/progress’, ‘year-on-year progression rate’, ‘organisational success rate’, ‘throughput’, ‘ failure rate’, ‘dropout’ – as well as how time to completion is counted in the case of course changes, extended curricula etc.
It also problematises the ability to compare academic outcomes, given the range of variables in how cohorts are selected/constructed which includes accessing datasets – both organisational and national – that are sufficiently comparable.
Date:
November, 2022
Report 2: Guidelines for organisations compiling selected retrospective stats: throughput, time to completion and year-on-year progression rates.
Given that the research question was about what difference the bursary support provider organisations (BSP) may be making to student outcomes (success), considerable detailed work on the measurement of academic outcomes was undertaken.
Once the definitional work on measuring throughput and progression rates had been finalised (see Terms used in the bursary support provider sector when measuring academic outcomes: Towards developing common usage / definitions), four BSPs undertook a retrospective analysis of their own data using the agreed methods of counting (Academic outcomes of students supported by four bursary support provider organisations). In order to do so, a detailed guide on what and how to count was produced – so that variables were excluded as far as possible and real comparisons could be made. This is that detailed guide.
Date:
November, 2020
Report 3: Academic outcomes of students supported by four bursary support provider organisations.
Given that the research question was about what difference the bursary support provider organisations (BSPs) might be making to student outcomes (success), considerable detailed work on the measurement of academic outcomes was undertaken.
Once the definitional work on measuring throughput and progression rates had been finalised (see Terms used in the bursary support provider sector when measuring academic outcomes: Towards developing common usage / definitions), four BSPs undertook a retrospective analysis of their own data, using the agreed methods of counting (outlined in Guidelines for organisations compiling selected retrospective stats: throughput, time to completion and year-on-year progression rates) – with some unexpected results. This is a report of those findings.
While a comparable national data set was difficult to access, comparisons with a notionally comparable dataset of NSFAS students suggested that half of these BSP projects obtained better outcomes while the other half may not have. This needs further work to be conclusive however, and recommendations about national data and developing agreed ways of measuring are made so that more secure comparisons can be made.
Date:
October, 2021
SUB-REPORTS: PSYCHO-SOCIAL OUTCOMES
Report 1: Towards measuring psycho-social outcomes in the bursary support provider sector.
Working on measuring psycho-social outcomes was the most innovative part of this research. As the claim has historically been that psycho-social support is what this sector can uniquely offer, finding out the extent of the difference they make – and which bits are the most effective – was a key question, but also the most evasive.
This sub-report examines the complexities – both ethical and technical – of trying to measure psycho-social factors, especially when linked to any changes that the bursary support providers (BSPs) might claim to have caused / influenced. It elaborates on issues of causality and the thorny issue of variables that cannot be excluded.
Most importantly it starts from the premise that BSPs do not employ psychologists or psychometrics and therefore may not make formal assessments as a matter of course. The competencies identified – and the indicators for each are, therefore, observable behaviours drawn from those deployed in workplaces including those proposed by the World Economic Forum.
The research details the offerings of seven BSPs who agreed to be key informants – commenting on issues of centrality and dosage. In addition to funding, support offered includes individual mentoring, development programmes focussing on various lifeskills, including financial management; supplementary orientation (to that offered by the universities); monitoring academic progress; referrals to the systems and resources that exist within universities; job search skills. The report briefly reviews the obstacles to students accessing services.
This difficulty in comparing their effectiveness lies in differences in the choice and dosage of what is offered by each organisation, which is compounded by varieties in their funding models, selection criteria, the universities to which they go, and the fields they study, among others.
Nonetheless the research undertook to identify psycho-social competencies / outcomes that would both support academic success and be an outcome in their own right inasmuch as they contribute to employability. Eight outcomes were eventually agreed to following a review of international workplace competencies – and 29 indicators with rating tools were developed. These are being piloted by two organisations and a third which is piloting their own set of outcomes, will be participating in the conversations about what is possible.
Date:
November, 2020
Report 2: Eight psycho-social outcomes and their indicators.
Working on measuring psycho-social outcomes was the most innovative part of this research. As the claim has historically been that psycho-social support is what this sector can uniquely offer, finding out the extent of the difference they make – and which bits are the most effective – was a key question, but also the most evasive.
Nonetheless the research undertook to identify psycho-social competencies / outcomes that would both support academic success and be an outcome in their own right inasmuch as they contribute to employability. Eight outcomes were eventually agreed to following a review of international workplace competencies – and 29 indicators with rating tools were developed. These are being piloted by two organisations and a third which is piloting their own set of outcomes, will be participating in the conversations about what is possible.
This sub-report presents the eight psycho-social outcomes and the associated 29 indicators.
Date:
May, 2021
Report 3: Matrices for assessing the psycho-social outcomes and indicators.
Working on measuring psycho-social outcomes was the most innovative part of this research. As the claim has historically been that psycho-social support is what this sector can uniquely offer, finding out the extent of the difference they make – and which bits are the most effective – was a key question, but also the most evasive.
Nonetheless the research undertook to identify psycho-social competencies / outcomes that would both support academic success and be an outcome in their own right inasmuch as they contribute to employability. Eight outcomes were eventually agreed to following a review of international workplace competencies – and 29 indicators with rating tools were developed. These are being piloted by two organisations and a third which is piloting their own set of outcomes, will be participating in the conversations about what is possible.
This sub-report presents the four-scale matrices for assessing all indicators of all psycho-social outcomes.
Date:
May, 2021
Report 4: A selection of compulsory indicators for psycho-social outcomes: An example.
Working on measuring psycho-social outcomes was the most innovative part of this research. As the claim has historically been that psycho-social support is what this sector can uniquely offer, finding out the extent of the difference they make – and which bits are the most effective – was a key question, but also the most evasive.
Nonetheless the research undertook to identify psycho-social competencies / outcomes that would both support academic success and be an outcome in their own right inasmuch as they contribute to employability. Eight outcomes were eventually agreed to following a review of international workplace competencies – and 29 indicators with rating tools were developed.
Given that 29 indicators are too many to assess for each student each year, the proposal is that each organisation selects those that are compulsory at different phases of university life – to which they add electives suitable to each student’s profile and needs. This is an example of one organisation’s selection of compulsory indicators.
Date:
May, 2021
Report 5: Measuring the psycho-social outcomes and indicators of: A Guide.
This guide outlines how an organisation might implement a system for assessing psycho-social outcomes based on the eight competencies and their related indicators. It was developed for those organisations who are piloting the measurement of two psycho-social outcomes in 2021/2022.
Date:
May, 2021
SUB-REPORT: EMPLOYMENT OUTCOMES
Report 1: Terms used in the bursary support provider sector when measuring employment outcomes: Towards developing common usage / definitions.
Bursary support providers (BSPs) are increasingly aware of the importance of supporting graduates to access the workplace or income-generating opportunities – if higher education is to intervene in the cycles of poverty as some would intend. Not many BSPs offer this support within their programmes, however, and only a few are accountable for employment outcomes.
Nonetheless, ways of measuring employment outcomes were developed during this research – much of which was about defining levels and durations of work suitable for graduates. No tools were developed for implementing this, however.
Date:
August, 2020
