Thanks for sharing what you have done David, I think we might copy some of that! Would you be willing to share the questionnaires?!

I agree also that this is a crucial area and it would be good to pool resources/approaches to be able to benchmark what we do and how we do it as we all face the inevitable questions of "how have you increased retention?"!

Becka

Becka Currant
Dean of Students
University  of Bradford
Tel: 01274 236821
Mob: 07917 241214 or 1739 internally


-----Original Message-----
From: learning development in higher education network [mailto:[log in to unmask]] On Behalf Of David Bowers
Sent: 04 September 2009 09:36
To: [log in to unmask]
Subject: Re: Evaluation Project

Absolutely spot on, Julia!

Please (please) can this generate a good response from the LDHEN and
related communities? 

From this end, I can only contribute a small drop into this ocean:

1) Last year we decided to directly email a brief questionnaire, at 4
points in the year (end Oct, end Dec, end Mar, end May), to all students
who had used our Learning Development drop-in sessions during the
previous 2 months. (For context - we run around 30 hours of open-access
drop-ins and 15 hours of tutor-led topic-based workshops each week. We
only offer individual one-to-ones if there is a specific need, by
appointment). We felt that this more regular issue, rather than waiting
for an end-of-year survey, would capture it more freshly in the
students' mind. We didn't mind the "risk" of regular users being
captured more than once. I can't say the rate of return was as good as
we hoped - around 40% - we put this down to students not regularly
accessing their uni email account. And (of course?) it was likely to be
only the grateful/ensthusiastic students who replied.....

2) We also included a question along the lines of "has the support
resulted in a noticeable improvement in the quality of your work?" and a
question along the lines of "would you say the support has helped keep
you on the course whereas otherwise you might have considered leaving?"
Pretty non-scientific, but as Julia notes, this is the nitty gritty that
grabs the attention! We too got a surprising amount of affirmatives.


David Bowers,
Head of Learning Development,
University Campus Suffolk,
Neptune Quay,
Ipswich,
IP4 1QJ,
United Kingdom.

Tel: 01473 296339
Fax: 01473 343696 

 

-----Original Message-----
From: learning development in higher education network
[mailto:[log in to unmask]] On Behalf Of Julia Braham
Sent: 03 September 2009 18:06
To: [log in to unmask]
Subject: Re: Evaluation Project

Hello,
This is an interesting question and very timely. There has been some
discussion on the lists over summer about 'impact evaluation of study
support' and evaluation of learning to learn modules, but the response
to Nick Hooper's call in June suggests that we are all still skirting
around this question. 
For some reason we rarely begin an academic year at Leeds thinking
strategically about evaluation - it is usually towards the end of the
year when we try to make sure we have captured plenty of 'how was it for
you' evidence to populate annual reports. Our evaluation mechanisms seem
relatively simplistic and still focus on questionnaires given to
students attending workshops and drop ins. We have tried to find out
from students the extent to which attending workshops may have helped
them change their practice, and what they found useful (or not) about
the workshop and use their responses to adapt our delivery - if they are
consistent enough!
A few years ago we distributed an end of year evaluation to students who
had attended our drop in sessions. One question asked if students felt
that attending drop in sessions had encouraged them to remain on their
course. The responses to this were significant enough to convert into
FTEs and inserted into a report that argued that our drop in facility
had maintained x amount of income for the university. The extent to
which this figure was quoted (and not interrogated) was surprising.
When we launch a new learning resource or website, we usually run focus
groups to capture impressions and again, take account of responses in
resource development.   We ask different questions and seek different
answers depending on who needs to use the information our evaluation
provides us with. We know that our students want to see the continuation
of one to one drop in sessions and that they find them effective, but
others ask questions about their cost effectiveness.      
I would suggest, by the responses to the email question, that may other
institutions are the same as us. We know that learning development is an
activity that is difficult to evaluate and we all seem to be unsure how
to grasp the nettle to produce meaningful and reliable data (whatever
that is), and then know how best to use the information that it
provides.  
Does anyone feel that they are using more sophisticated evaluation
mechanisms? Are there any more innovative example of current evaluation
practice being used? It will be interesting to see the outcome of the
Queens project, but I can see why the Project Leaders would benefit from
a clearer understanding of what we are all currently doing to evaluate
our delivery.

Julia Braham
Senior Academic Skills Adviser
Skills@Library
University of Leeds

________________________________________
From: learning development in higher education network
[[log in to unmask]] On Behalf Of Paula Moran [[log in to unmask]]
Sent: 17 August 2009 14:25
To: [log in to unmask]
Subject: Evaluation Project - Small Grant Funding

Hi
Project DEVELOPMENT OF A GENERIC EVALUATION PLAN AND EVALUATION TOOLS
FOR LEARNING DEVELOPMENT ACTIVITY

We are currently working on our literature review for the above project
and after initial discussions on our first draft with our mentor we
think it would be very beneficial to include a section that scopes
current practice within the LDHEN network.
I would be very grateful if you would be willing to provide me with
information on the rationale and/or practice you use for learning
development activities.
Also it would be useful if you could highlight what you have experienced
as most effective and any issues you have experienced in the area of
evaluation of learning activities which may include one-to-one
provision, workshops and resources.

Any feedback on the above or other information you think would be
relevant will be much appreciated.

Best wishes

Paula


University Campus Suffolk is one the UK’s newest higher education institutions and officially opened on 1 August 2007. The main campus is located in Ipswich with centres in Bury St Edmunds, Great Yarmouth, Lowestoft and Otley. The new Waterfront building in Ipswich opened in September 2008.  Visit www.ucs.ac.uk for further details.

University Campus Suffolk (UCS) is the trading name of University Campus Suffolk Ltd, a company registered in England & Wales, registered number 5078498, registered office Waterfront Building, Neptune Quay, Ipswich, IP4 1QJ