Dear John,
Thank you for providing the references. Regarding our current discussion, I'm
not specifically querying a single approach or a multiple approach. What I
am trying to clarify is the specific evaluation question relating to this
instrument, or whether there may be more than one relevant, (according to
the active stage of the project) - which in turn may mean you are effectively
developing a series of different evaluations with different groups to lead you to
the information that you need to define your final process/approach (and
which you have provided the references to back up what it might become).
If this is the case, there may be a range of outcomes that are independently
of value to the community, and should probably be extracted as such,
irrespective of the final approach that is suggested/adopted by the project.
From this point of view, I then ask:
If the 'evaluation question' is a single one, namely 'How may we best evaluate
online learning?' (which I am assuming as this is not explicitly stated
anywhere but seems to cover the list of framework questions which you
asked the group to reflect on in your mail of 4 June)
-this is quite general and does not define audience at this stage i.e.
students, practitioners, the project and so can contain a range of diverse
questions not specifically designed to test learning but to poll the
population's opinion on methodology
-the first part of your document makes more sense in attempting to
determine relevance of models we may like to use to specifically test learning
and so on.
But this question is very general - and there is not yet a breakdown of the
components of this question that need to be examined, or in what order, to
which each instrument can be focused. (Your framework questions could
probably be regrouped to partly do this, but some relate to aspects of
process management which don't directly map.) If the sections of the
document you provide go some way towards this breakdown, then that
should be explicitly articulated.
However, I understood your request to be to examine a document that would
consider the kinds of questions that might be put to students (to test across
the range of approaches you describe) - which would be a different outcome
for the project, and which this instrument would not achieve in its own right.
Also from what I have seen so far, this must come further down the line -
after the kind of groundwork you are currently laying has provided some
concrete feedback from which to work. (I probably misconstrued who you
meant by 'participants' in the document introduction - I initially assumed this
to mean participants in an online learning intervention, which made me begin
to think more carefully about what is trying to be achieved here!)
To me that means you have two things (and maybe more to come as it gets
refined) the project is testing, or planning to, so far (and which require
different tools and approaches):
1. Some kind of determination/measurement of what the community believes
it would be appropriate and acceptable to either test or use to test online
learning - and which the document presented goes some way to doing
2. An actual mechanism (yet to be defined) that may help to determine
elements of online learning that actually take place, and the factors that
affect this, (and perhaps with some kind of measure as to what extent).
These are quite different - and the first will inevitably mould the second.
Therefore if there are disagreements as to the outcome of the final
methodology, and the two processes and outcomes are recorded separately,
it should be possible to determine which parts of the methodology are
traceable to which assumptions by the community. That may prove of more
value in beginning to ensuring that any methodology developed by the project
can be adapted to a range of online learning circumstances, even where the
underlying assumptions about the nature of the learning may differ.
Hope this is of some help
All the best
Gayle
On 14 Aug 2003, at 12:02, Professor John KONRAD wrote:
> Dear Gayle,
>
> Thanks for this very thoughtful analysis. I'm grateful for your taking the
> time.
>
> I think that you are querying the concept of a SINGLE approach, which would
> be valid regardless of context? I am unclear what you mean by a "single
> task" - would you please give me an example.
>
> I started from the perspectives of the US investigation which implicitly
> took that view, having reflected on the E-learning evaluation toolkit and
> the Cook-book approaches!
>
> For me the biggest problem is that, in my experience, E-learning can involve
> a largely "objectivist" approach which assumes that required learning can be
> necessarily and sufficiently described by statements of measurable
> participant learning outcomes. See for example
> http://www.ecc.org.sg/cocoon/ecc/website/services/article/designed2teachIII.
> article An approach to evaluation that would work would be related to
> classic summative product evaluation. See
> http://hagar.up.ac.za/catts/ole/eel/concept-map/types.html for a typology.
>
> It seemed to me that the instrument might also relate to some aspects of a
> more constructivist approach which appears to be of increasing importance.
> See http://www.e-learningguru.com/articles/art3_6.htm and
> http://www.bath.ac.uk/e-learning/objectivist.htm For a useful approach to
> integration, see:
>
> This article proposes a model to integrate the traditionally conflicting
> objectivism and constructivist approaches to curriculum design. It is
> argued that these two are not opposing paradigms, but complementing
> approaches. A number of analyses of learning programs are discussed to show
> that learning events contain both objectivist and constructivist elements.
> Plotting the two approaches at right angles to one another produces four
> quadrants of conditions of learning. These four quadrants are discussed
> together with the rationales for each. Finally, a challenge is issued to
> members of ITForum to identify learning events that are high in both
> instances. http://it.coe.uga.edu/itforum/paper48/paper48.htm
>
> However, the "blended learning" approach seems to be growing in importance
> and needs to be taken into account. See
> http://www.learningcircuits.com/2002/aug2002/valiathan.html
> http://www.usdla.org/html/journal/MAY02_Issue/article03.html and
> http://www.blended-learning-symposium.de/ [in German]
>
> [All sites visited 14/08/2003]
>
> Kind regards,
>
> John
>
>
> Professor John Konrad
> Honeysuckle House
> 48 Keswick Road
> Cringleford
> Norwich NR4 6UQ
> Great Britain
>
> Phone/Fax: 01603 455501
> International: 0044 1603 455501
> -----Original Message-----
> From: Gayle J Calverley [[log in to unmask]]
> To: [log in to unmask]
> Sent: 13/08/03 15:42
> Subject: Re: First draft of online learning evaluation instrument
>
> Dear John,
>
> I've been looking at your document and the past list correspondence - and as
> the document stands it puzzles me somewhat.
>
> Taking the main concept of the document as "to develop a useable methodology
> for the evaluation of online learning" the questions you have presented do
> not quite seem to fit this remit - although they appear to be designed to
> some different yet related focus.
>
> I would anticipate such a document to be an instrument that questioned a
> particular methodology that had been chosen and used within a particular
> case, rather than an instrument that explored the various types of
> methodology that are more generally available for use, and whether or not
> they might be relevant to an approach for a specific task or remit. Here
> there seems to be a mismatch between what is being described as tested (the
> evaluation context) and what is actually presented for test (the
> questions or perhaps appropriateness of the requested responses) or perhaps
> to whom in what role (the audience/test sample). I think if this can be
> defined more closely for each instrument, the appropriateness of the
> questions and the responses will be clearer. This in turn will allow more
> effective commenting.
>
> For example, I can think of ways in which each methodology described can be
> used to evaluate aspects of an online learning project depending on the
> purpose of the evaluation, but must be asked of different people. Then you
> state relevant/not relevant as a response, I have to ask - under what
> circumstances, under what context, by which people in which role - before I
> can even begin to answer. That in turn implies a problem with the underlying
> assumptions of the tool that need to be explored further, as there are
> likely to be several concepts tied up as a single entity that need to be
> further untangled for success.
>
> To me, this means there is potential for developing a range of instruments
> according to a wider range of questions and so extend the potential
> usefulness of your project. Alternatively it could allow you to focus more
> closely on one specific extremely well defined task - so that it becomes
> more effective -although this would limit transferability.
>
> I hope this helps, and look forward to seeing the project progress.
>
> All the best
> Gayle
>
>
>
>
________________________________________________________________
Dr Gayle J Calverley,
Distributed Learning, 186 Waterloo Place,
University of Manchester, Oxford Road, M13 9GP
Email: [log in to unmask]
Telephone: 0161 275 8106 Internal: 58106
Fax: 0161 275 2221 Internal: 52221
Resources and Technology Advisor, Distributed Learning
LifeSign JISC DNER Project http://www.lifesign.ac.uk/
ALT Occasional Publications Editor http://www.alt.ac.uk/
|
