Ben Johnson
Graphic Science

Part of The Training Group - Supporting public engagement in the research community

( +44 (0) 1275 47 44 44
È +44 (0) 7521 690 724
› [log in to unmask]
: www.graphicscience.co.uk
Disclaimer: The author of this email never meant a word of it.

do you really need to print this email?

This is very true! We need to be more proactive in sharing failures as well as successes. 

Thanks for the resources Dom G. I know your colleagues worked extremely hard on Connect Physics and it’s wonderful that you’re making it publicly available. 

Best wishes,

Sheila

Royal Astronomical Society 

Hi Dom

 

Notwithstanding Eric’s critique of the evaluation methods used by the external agency I want to give a bit of Psci-Com love to SEPnet for having the gumption to post some relatively negative evaluation findings. Gold star.

 

I especially enjoyed the phrase “…our aims may have been a little impossible…”.

 

Dom

Founding partner: The Guild of Sci-Comm Doms

 

From: psci-com: on public engagement with science [mailto:[log in to unmask]] On Behalf Of Jensen, Eric
Sent: 24 September 2018 12:10
To: [log in to unmask]
Subject: Re: [PSCI-COM] New SEPnet Physics Resource for KS3 Science - Connect Physics.

 

Hi Dom, 

 

Thanks for sharing. Unfortunately, on the quantitative side of the evaluation, this report does not provide the key details that would be needed to learn anything about impact patterns.

 

Just two key points amongst a range of issues:

1. There are several references to ‘significant’ findings in the report, but no indication of which inferential statistical tests were used. 

(This statement in the methods make no sense to me at all: 'Where responses were in the form of scores out of 5, based on the size of the achieved samples, those scores where there is difference of 0.3 or more between the base line and endline can be considered significant.’)

 

2. There are some interesting Likert scale outcome measures (e.g., 'Anyone can become a scientist’ and ‘I want to become a scientist’; also some problematic ones, e.g. 'It is important to study science even if you don’t want a science job in the future’). However, measuring change on the outcome measures with the entire pre (n = 474) and post (n = 85) samples is not okay. These are repeated measures items and the analysis should only include data where the same individual completed both pre and post measures. So unfortunately, none of the reported ‘change’ analyses are valid as this analytic error is repeated throughout. 

 

It sounds like you are finishing this project so might not be able to go back and fix these issues on the evaluation. If this is the case, perhaps you could share the underlying data as well? Could be a nice small-scale project for a student to re-analyse the data using appropriate statistical methods. 

 

I am sorry to hear that your investment in evaluation did not yield the answers you had hoped. This is a shame, as it looks like it was a great opportunity to contribute some valuable knowledge to the field. I would advise on future projects it may be worth involving a methodological advisor to the risk of this outcome. I can suggest some people off list if you (or others) are considering a major evaluation investment. 

 

Best wishes, 

Eric

 

---------------

Dr Eric Jensen, Fellow Higher Education Academy

Associate Professor (Senior Lecturer), Department of Sociology, University of Warwick

http://warwick.academia.edu/EricJensen

Latest book: - Doing Real Research (SAGE): https://us.sagepub.com/en-us/nam/doing-real-research/book241193

Check out a sample chapter here - https://us.sagepub.com/sites/default/files/upm-binaries/73894_Jensen_Chapter_6.pdf 

 

Sociology at the University of Warwick ranked:

The Guardian, Complete University Guide and The Times Good University Guide – 3^rd

QS World University Ranking - 23rd

 

On 24 Sep 2018, at 11:38, Dominic Galliano <[log in to unmask]> wrote:

 

Hi all

 

Apologies for cross posting, and multiple messages today! Final week at SEPnet, so it’s a tad manic….

 

Over the last few years SEPnet has been developing and testing resources aimed at bringing Physics into the context of Key Stage 3 Students’ lives and career paths. After testing across over twenty five schools in the South East over two years, we are finally ready to launch the resources! Everyone, here’s Connect Physics:http://www.sepnet.ac.uk/outreach/schools/connect-physics/

 

The ideas were developed by SEPnet Officers and teacher, then taken to full workshops by the fine folk at Science Theatre.

 

We set out to develop a set of three workshops which can be delivered over a school year which aim to maintain and raise the science capital of students. We found in testing, after throwing a lot of budget and people time at the evaluation, that we actually couldn’t bring in sufficient data to measure any changes in Science Capital. Also our aims may have been a little impossible to carry out and measure.

 

However the outcomes after each of the workshops were delivered, along with observations from teachers showed there was some short-term impact. The full evaluation report, carried out by Hope-Stone research, can be found here:http://www.sepnet.ac.uk/wp-content/uploads/2018/09/Connect-Physics-Complete-Evaluation-Report-2017-18.pdf

 

So in the end, these workshops do encourage the students to think of the bigger picture through connecting different ideas, such as topics from KS3 science, the latest physics research or their everyday lives. The workshops also help students find out about careers that are available after studying physics and also develop the students’’ skills using the scientific method and the peer-review process by tackling an open-ended problem with no given solution. Over a third of students had a raised interest in Physics after completing each workshop. 

The workshops are available for all to use under Creative Commons Attribution-Non Commercial 4.0 International License. The workshops can be delivered on condition that:

 

·         The resources are to be used for non-profit purposes only.

·         The SEPnet logos are not to be removed from any of the materials. However you are free to add additional logos.

·          A completed reporting spreadsheet is sent back at the end of the school year.

 

To register please complete the form linked below: http://www.sepnet.ac.uk/outreach/schools/connect-physics/user/  

 

All the best

 

Dom

 

Dr Dominic Galliano

(Pronouns:  he/him/his)

 

SEPnet Director of Outreach & Public Engagement

www.sepnet.ac.uk/outreach/

 

[log in to unmask]

@PhysicsDom

T: +44(0)207 882 5063

M: +44 (0)7702 363731

 

********************************************************************** psci-com how-to: Once subscribed, send emails for the list to [log in to unmask]. If not subscribed, either subscribe here https://www.jiscmail.ac.uk/cgi-bin/webadmin?A0=psci-com or send requests for items to be posted on your behalf to [log in to unmask] To unsubscribe (or silence messages while away) send an email (any subject) to [log in to unmask] with one of the following messages (ignoring text in brackets) • signoff psci-com (to leave the list) • set psci-com nomail (to stop receiving messages while on holiday) • set psci-com mail (to resume getting messages) Contact list owner at [log in to unmask] Small print and JISCMail acceptable use policy https://sites.google.com/site/pscicomjiscmail/the-small-print Contact Jisc Helpline: Email: [log in to unmask], Telephone: 0300 300 2212 **********************************************************************

 

********************************************************************** psci-com how-to: Once subscribed, send emails for the list to [log in to unmask]. If not subscribed, either subscribe here https://www.jiscmail.ac.uk/cgi-bin/webadmin?A0=psci-com or send requests for items to be posted on your behalf to [log in to unmask] To unsubscribe (or silence messages while away) send an email (any subject) to [log in to unmask] with one of the following messages (ignoring text in brackets) • signoff psci-com (to leave the list) • set psci-com nomail (to stop receiving messages while on holiday) • set psci-com mail (to resume getting messages) Contact list owner at [log in to unmask] Small print and JISCMail acceptable use policy https://sites.google.com/site/pscicomjiscmail/the-small-print Contact Jisc Helpline: Email: [log in to unmask], Telephone: 0300 300 2212 **********************************************************************

********************************************************************** psci-com how-to: Once subscribed, send emails for the list to [log in to unmask]. If not subscribed, either subscribe here https://www.jiscmail.ac.uk/cgi-bin/webadmin?A0=psci-com or send requests for items to be posted on your behalf to [log in to unmask] To unsubscribe (or silence messages while away) send an email (any subject) to [log in to unmask] with one of the following messages (ignoring text in brackets) • signoff psci-com (to leave the list) • set psci-com nomail (to stop receiving messages while on holiday) • set psci-com mail (to resume getting messages) Contact list owner at [log in to unmask] Small print and JISCMail acceptable use policy https://sites.google.com/site/pscicomjiscmail/the-small-print Contact Jisc Helpline: Email: [log in to unmask], Telephone: 0300 300 2212 **********************************************************************