JiscMail Logo
Email discussion lists for the UK Education and Research communities

Help for PHD-DESIGN Archives


PHD-DESIGN Archives

PHD-DESIGN Archives


PHD-DESIGN@JISCMAIL.AC.UK


View:

Message:

[

First

|

Previous

|

Next

|

Last

]

By Topic:

[

First

|

Previous

|

Next

|

Last

]

By Author:

[

First

|

Previous

|

Next

|

Last

]

Font:

Proportional Font

LISTSERV Archives

LISTSERV Archives

PHD-DESIGN Home

PHD-DESIGN Home

PHD-DESIGN  March 2014

PHD-DESIGN March 2014

Options

Subscribe or Unsubscribe

Subscribe or Unsubscribe

Log In

Log In

Get Password

Get Password

Subject:

Re: Design Theory - (long post)

From:

Arjun Dhillon <[log in to unmask]>

Reply-To:

PhD-Design - This list is for discussion of PhD studies and related research in Design <[log in to unmask]>

Date:

Tue, 25 Mar 2014 04:01:09 -0400

Content-Type:

text/plain

Parts/Attachments:

Parts/Attachments

text/plain (466 lines)

Dear Terry et al, 

Also a long post:

I appreciate your mention of the tactic of making the familiar strange. This is an approach important in sociology, and either through adoption or through convergent evolution, it is also important in design. In design, we often talk about being skeptical about the known and optimistic about the unknown, just as in sociology it is often said to trust what is alien and alienate what is trusted (there is a citation here, forgive me for not having it on hand).  

I also appreciate that you are attempting to apply this approach to wicked problems. That said, I think your application leaves much to be desired. 

Indeed, your approach doesn’t seem to make any of Rittel and Webber’s work strange, rather it is just a strange way to try to argue against it. To make the familiar strange isn’t to interpret the familiar in a strange way, as you have done here. Rather, it is to seriously question that which we take as given (indeed, that which we take and forget as given). It is to examine those things tacitly accepted, those things that aren’t explained by reason or evidence but are instead accepted as familiar, as standard, as needing no explanation. 

If we are to point this approach at “wicked problems”, it seems that our first step is identify what within this theory/position/hypothesis/whatever is familiar. What is the most familiar? What do we know, but cannot explain? What would make total and unquestioned sense to us, but would confound an alien? 

The answer, I believe, is not our approach at looking at problems – wicked or otherwise. Rather, it has something to do with our concept of “problem” itself. 

As pervasive as our concept of ‘problem’ is, there is surprisingly little literature published about *what*, exactly, it is to have a problem. Maybe it is such a simple and intuitive concept that it needs nothing more than the dictionary entries that already exist. According to Merriam-Webster, ‘problem’ is defined as “something that is difficult to deal with: something that is a source of trouble, worry, etc”. This definition is based on the ancient Greek word *problema*, which means “anything thrown forward, hindrance, obstacle, anything projecting, a headland, promontory” (wiktionary). 

But these definitions cannot account for our common usage of the term, nor can it account for Rittel and Webber’s use of it. Indeed, design seems to have a strange relationship with “problems” and “solutions”. It has been said that design involves the co-evolution of problem and solution (Dorst and Cross, 2001, and others). According to Rittel and Webber’s original paper, "problems can be described as discrepancies between the state of affairs as it is and the state as it ought to be”. If we translate the philosophically dubious “ought to be” part of this statement, and put “discrepancies” into more accessible terms, we get something like: to have a problem is to have a gap between our current state and our preferred state (sounds familiar, right?).

Now we have something that we don’t need a tactic to make strange. To have a problem is to have a known current state (one that we disvalue in some way), a known preferred state (in the form of some mental representation – some goal or ends), and a gap or discrepancy between the two.

With wicked problems, what do we have? We could have a known current state, and we certainly know we want something better than our current state, so I suppose you can say there is a gap. But a gap between our current state and what? If we knew the end-state we desired, wouldn’t the “wickedness” of the “problem” dissipate? Does it even make sense to call the phenomenon we name “wicked problems” as problems at all?

Maybe more importantly, does it make sense to try to define design in this context – in the context of problems (wicked or not)? Simon expanded the definition of design to include any problem solving activity – hell, to include any activity directed towards some a priori ends. This seems to include a great deal of intentional human action as a form of design. To me, this feels wrong not because it is too inclusive, but because it still fails to account for the evidence and the phenomenology of design. 

Ask a practiced designer how often their work is to realize some preconceived goal or end-state. For most of us, when we have to do this it is unfortunate, and usually the result of a failure (on our part or the clients’). 

Maybe design can be formalized as a mathematical language; I doubt it, but I might be wrong. Either way, it certainly isn’t going to happen in terms of problems and solutions. This approach is an artifact of our goal-directed strategies of thinking and acting in the world. It, along with our habit of problematization, is the result of the assumption that all intentional human action is goal-directed action – is action aimed at preconceived end-states. Design isn’t the realization of preconceived ends. Design is the practice of bringing about deliberately emergent ends – of producing outcomes that cannot be imagined up front. This is a practice that has nothing to do with “problems” or “solutions”.

In the end, I agree we ought to make the familiar strange, especially regarding wicked problems (and “problems” and “problem solving in general). I do not think, Terry, that you have revealed what is truly strange in this context. 

Thanks,
Arjun




On Mar 24, 2014, at 9:06 PM, Terence Love <[log in to unmask]> wrote:

> Dear Lubomir, Birger, Ken and all,
> 
> My apologies for the delay in getting back to you in response to questions
> about my assertion there are significant benefits for design research in
> translating  design theories into mathematical language.
> 
> I would add to my previous claim that such translation offers insights,
> clarification and identification of faulty theory that are almost impossible
> to identify using the approaches common in much of the literature of design.
> 
> Lubomir, what I wrote is not positivist. In philosophy of science terms it
> is simple, straightforward and uncontroversial. In my posts to Ken and all.
> I've been very clear about the sort of mathematical representation I have
> proposed.  It is different from what Ken  and you seem to have inferred. I
> pointed out there were two possible positions on creating mathematical
> representations for  design theories.
> 
> The first position is a language translation. It is the exact equivalent of
> translating  a design theory written in English into Finnish, Arabic,
> German, Yiddish,  Yupic, or Archi. What  I proposed IS exactly that - to
> convert  a design theory written in English into the language used in
> mathematics (in exactly the same manner that one might convert it into the
> language structures  of  Finnish, Greek  or Yupic ). I maintain there are
> benefits in doing this for analysing  and testing the language-based aspects
> of design theories and their implications as well as identifying
> implications. This is neither positivist nor non-positivist.  It simply
> offers an alternative basis for linguisitic, conceptual and reasoning
> analyses that offers a better basis to identify  some of the linguistic
> oversights in the English version . 
> 
> The second position would be to create a mathematical dynamic model that
> represents the physical behaviour of the phenomena outlined by the word
> version of a design theory. This is NOT what I proposed. It appears,
> however, to be what some think  I proposed. 
> 
> One of the reasons for doing the language translation described in 1.  is
> because we can't easily perceive errors in theories described in our own
> language. An essential part of research is to continuously try to find
> errors and gaps in theories and to find out the theoretical and real regions
> they don't apply and the reasons they do not apply.  Finding these errors in
> theory  is not easy nor straightforward due to human habits. We tend to
> bridge the gaps mentally to make theories appear to work.  This is in the
> same manner that underpins the problems of witness reliability. We can see
> it practically in discussions about the current airline disaster. We ignore
> uncritically under-justified generalities when  they seem like they should
> apply.   There are of course methods of critical conceptual analysis for
> making the familiar strange in conceptual terms. I used straight critical
> conceptual  analysis in a previous post to phd-design analysing  and
> identifying the weaknesses in Rittel and Weber's 10 points about what they
> had called 'wicked problems'.
> 
> An alternative way to make the familiar strange in order to see the errors
> in theories is to put them in a language that has different structures. In
> doing that,  mathematics is particularly useful because its structures are
> unusually well defined and hence there is less opportunity to blur over
> unjustified aspects of a theory.
> 
> Let me give an example of the use of  the kind of language translation into
> mathematics of the above position 1 to analyse a design theory by
> translating it from English  into mathematical language.  Note in doing this
> I've assumed a Popperian viewpoint that is  the opposite of  positivism.
> 
> Take the design theory claim:     'Wicked problems are not solveable' (part
> of what Ken proposed earlier that I mathematically represent)
> 
> It is possible to translate this simply into  mathematical language along
> the lines of the following. What follows translates the  above design theory
> claim into the language of set theory. 
> 
> "There exists a set W that contains, and only contains, theoretical
> entities, w, in which the boundaries of the abstract space containing, and
> only containing, set W, and the members, w of set W, are defined by criteria
> Cw (1… n), listed by Rittel and Weber as the characteristics of what they
> call ‘wicked problems’. 
> Each entity, w, in W, is assumed to be a functionally complete abstract
> representation of a real situation z, from the universal set of real
> situations, Z. The set Z contains and only contains all real situations
> comprising real and virtual entities u, from the universal set of phenomena,
> U.
> Each real situation zi, defined as a ‘wicked problem’ situation by the
> theoretical information in its corresponding theoretical representation wi,
> also can be described by a different theoretical representation, dp, as a
> design problem, that is, a member of a set DPr, which in turn is a member of
> the super set DP. The super set DP contains and only contains, entities
> identified as design problems appropriate to being addressed by design
> professionals. The subset DPr of DP contains, and only contains, those dp
> entities that describe real world situations for which a theoretical
> solution entity s can be identified.
> Each solution entity, s, is an abstract representation of a real
> intervention, i, in which s is a member of the universal set S, of
> solutions, and i is the corresponding member of the set I of real world
> interventions. The boundaries of set I are defined by all real world
> physical constraints Cphys (1…m); by theoretical constraints Cth (1…p), e.g.
> laws, customs, habits, bounds of knowledge and theory; and by constraints of
> human limitations Chuman (1…q), e.g. as a human being unable to identify or
> understand or predict the behaviours of particular classes of solutions or
> sub-sets of interventions. The boundaries of the abstract set S that
> contains all possible theoretical solutions are also defined by the abstract
> and real constraints and also by the nature of the entities it encloses
> being solutions. 
> The above theory claims  that for those members of  w in W, that describe
> corresponding members z, from Z, consisting of combinations of entities u
> from U, of which the members z are restricted to those in which each
> theoretical entity  w can be also described by a corresponding theory entity
> dp, which is a member of subset DPr and superset DP, THEN each of those
> entities z do not have a corresponding entity, s, of the set S, on the basis
> that it is implicitly claimed there does not exist a corresponding entity,
> i, of the set I."
> 
> The above representation in mathematical language is useful in design
> research terms in several ways:
> 
> 1. It Identifies the multiple different theoretical entity types that are
> intrinsic to  the above design theory claim but are hidden and overlooked by
> its presentation in English language
> 
> 2. It identifies the bounds of the variety of entities within each of the
> above intrinsic entity types.
> 
> 3. It identifies the criteria that create those bounds 
> 
> 4. It identifies the specifics of the relationships between entities, sets
> of entities, bounds and criteria
> 
> 5. It identifies and provides the basis for testing the above theory claim
> in terms of the epistemological integrity of the entities, entity types,
> sets,  relationships, criteria - as well as by exclusion (i.e does the
> theory set representing the claim exist, and in practical terms (by
> reversing and using null hypothesis)
> 
> 6. It identifies and provides a basis for testing the criteria (including
> Rittel & Weber's characteristics of 'wicked problems') in terms of their
> internal and external epistemological validity and completeness. I.e. do
> they match with the epistemological characteristics of the entities to which
> they refer, are they internally consistent, do they provide a set of
> complete bounds for set W (and by implication appropriate bounds for the
> other sets, is there redundancy etc?
> 
> 7. It enables easier and more explicit  careful searching for implicit
> tautologies, unjustified assumptions, internal contradictions etc
> 
> Incidentally, the above translation reveals at least one internal
> contradiction in Rittel and Weber's  theory claim.
> 
> To maintain all aspects of this analysis in a single place, I have
> deliberately refrained from removing the prior posts. I have however trimmed
> them of address details.
> 
> Best wishes,
> Terry
> 
> ---
> Dr Terence Love
> PhD(UWA), BA(Hons) Engin. PGCEd, FDRS, AMIMechE, MISI
> Director,
> Love Services Pty Ltd
> PO Box 226, Quinns Rocks
> Western Australia 6030
> Tel: +61 (0)4 3497 5848
> Fax:+61 (0)8 9305 7629
> [log in to unmask] 
> --
> 
> ==
> Dear Terry,
> 
> I was going to ask you a similar question, but Birger got ahead. Show me one
> design theory that can be mathematized. But before that, we have to settle
> on what is design, what is design theory, how a theory can be expressed in
> mathematical terms, and so forth. 
> 
> It seems to me that you enter very boldly into a philosophy of science
> minefield and on top of that, you make very extreme statements. If you say
> that some design theories can be mathematically expressed, it might be
> easier to defend such a position. 
> 
> I am a bit astonished that you take such an extreme positivist view. What
> you say is not new and not unheard of, but people try to stay away of it for
> several decades. Is there are resurgence of Positivism? My observations are
> that the Positivism has "softened" a lot under the pressure of the
> renaissance of the humanistic paradigms. What is going on here? 
> 
> Best,
> 
> Lubomir
> 
> ==
> Dear Terry
> Please show us one mathematically expressed design theory.
> 
> Best regards
> 
> Birger Sevaldson 
> 
> ==
> Hi, Ken,
> Again,  you are conflating the two different ideas. you can't make a silk
> purse out of a sows ear. . .  If the original word version of a theory
> doesn't provide a complete functional  model of all aspects of design, then
> how would representing that same theory in mathematical terms do so?
> 
> E.g. '"Wicked problems" have a large number of solutions' can be transformed
> into mathematical notation into ' The number of entities in the solution set
> of the set of problems bounded by the following criteria tends towards a
> large number'.
> Why would I expect the mathematical representation of this  to be a working
> model of design activity? I don't get your logic. Goedel's is a different
> argument.
> 
> You could think  of it slightly differently. If you  gave me a word-based
> description of a motorcycle you had seen; the word-based descriptions could
> be equivalently be created as a visual-representation, as a sketch of the
> motorcycle;  or the words  or sketch could be represented  (as above) in
> mathematical propositions. I'm at a loss as to why you think the words,
> sketch or mathematical representation  would  be a working motorbike.
> descriptions result in
> 
> Rittel & Weber's  wicked problem 'theory' is a their proposal for a
> collection of boundaries on the characteristics of the concept they have
> called 'wicked problem'. In that sense it has the characteristics of a
> definition not a theory.  As you say, it's a set of theory-related
> (theoretical) propositions. Propositions, though. In spite of their nature
> as parts of a definition, most of them don't define anything. The most that
> can be said is they are partial pointers. In some cases they are unrelated.
> I'm happy to critique  Rittel and Weber's propositions, andnd they are easy
> enough to represent in set terminology. It's value, however, will probably
> more that it will demonstrate they ain't a theory, nor do they make sense
> except perhaps as a political statement, or a proposal  for avoiding
> responsibility.
> 
> But, lets take it from the top on  Rittel and Weber's 10 comments on what
> they called  'wicked problems'
> 1. Means  there's no definition here so what follows in the remaining 9
> items is irrelevant. Non-sequitor.
> 2.  Is simply confused language.  *Activities* have stopping rules.
> Creating solutions to problems have rules and boundaries. *Problems*,
> however, are simply problems. They don't go about stopping or starting.
> Non-sequitor.
> 3. Means there is an undefined (good-bad) spectrum of optimisation criteria.
> Weird -  of course solutions are not 'true or false. Do you know ANY
> *solutions* that are true or false, except in the realm of binary logic?.
> 4. Contradicts 3.  In 3, Rittel and Weber have already said they are
> defining the success of a solution to a 'wicked problem'  on the criteria of
> an undefined spectrum of good-bad. Regardless of it being undefined, they
> have indicated they are testing solutions against a criteria  - which
> contradicts 4.
> 5. Apart from the faulty logic (confusion between creating the solution and
> the solution itself) , its true, less goes at getting things right means
> each attempt is more significant. This isn't really a major theory item.
> 6. Means there are lots of possible solutions (i.e. the set of possible
> solutions tends towards a large number) - the second part of 6 doesn't seem
> to make sense.
> 7. Each problem is different (is that unusual?) 8. All problems result from
> the outcomes of other problems. Is this unusual?
> all events result from other events.
> 9. Its confused language with faulty logic but it seems to mean ' The
> outcomes resulting from prior events (that if R&W had been trying to  create
> a solution to them, they would have called a wicked problem) can be
> explained in lots of ways.  Is this not obvious?
> 10. The professional identifying the solutions  is responsible. This is
> clearly untrue for most planners and designers. I've yet to see the costs of
> social problems from poor designs being charged against planners personal
> bank accounts.
> 
> So? Rittel and Webers collection  of sentences and comments about wicked
> problems is  a theory? It doesn't look like it to me. It doesn't have
> predictive power, it doesn't define, and it isn't coherent. It had value at
> the time to draw attention to some characteristics  of planning decisions,
> but a theory  - no.
> 
> Warm regards,
> Terry
> 
> ==
> 
> Dear Terry,
> 
> Thanks for your reply.
> 
> Before offering the example you request, I'll state that there is no
> straw-man argument in my notes. If a theory meets the criteria you propose,
> it will permit a complete mathematically-based functional model of all
> aspects of design.
> 
> You make this statement about ALL theories in any discipline. Since the set
> of ALL design theories must incorporate the sub-set of those theories that
> model ANY design activity, the sub-set of ALL design theories will afford us
> a functional, mathematically rigorous model of all aspects of design. As I
> wrote, I do not think this is possible.
> 
> You wrote,
> 
> -snip-
> 
> 1. All theories (any discipline) can be wholly and exactly represented as
> mathematical functions (this is the set of ALL theories regardless of valid,
> useful or not). The relevant mathematical functions are typically found in
> complex non-linear multivariable multidimensional spaces.
> 
> 2. There are significant benefits in terms of validity and usefulness if
> theories are in the set represented by continuous well-behaved mathematical
> function.
> 
> 3. If a theory cannot be represented by a well-behaved continuous
> mathematical function, this is an indication that the phenomena being
> theorised about needs representing in a different theoretical manner
> typically by more than one theory.
> 
> 4. A simple test for whether a theory is represented by a well-behaved
> mathematical function (and hence is a 'good' theory) is whether the
> phenomena and the mathematical theories are free from discontinuities or
> singularities.
> 
> 5. The existence of discontinuities and singularities in the mathematical
> space field  that represents a theory indicates the need to represent
> phenomena on each side of the discontinuity or at the point of the
> singularity differently. That is, it indicates that what was previously one
> theory actually requires several theories of which the minimum number is
> (n+m+1) where n is the number of discontinuities and m is the number of
> singularities.
> 
> 6. The above applies to a single characteristic of a phenomenon. Where
> discontinuities and singularities occur at the same point in respect to
> multiple characteristics of the same phenomenon this indicates the presence
> of (n+m+1) different phenomena and the need for (n+m+1) different vector
> space fields (bodies of theory) to describe them).
> 
> Conceptual analysis, the work of Foucault and other theorists in realms of
> sociology, communications and design theory can each be seen as a sub-sets
> of the above.
> 
> -snip-
> 
> Now, as requested, I will give an example of a design theory that you feel
> cannot be represented mathematically, at least not in full - that is, I do
> not believe that it can be wholly and exactly represented as mathematical
> functions. Despite this, the theory I am about to put forward is one of the
> most important sets of theoretical propositions in design and design
> thinking. You have used it yourself, so I'd argue that it has to count as a
> theory of some kind.
> 
> These propositions meet my criteria for a theory: a model that illustrates
> or describes how something works by showing its elements in their dynamic
> relationship to one another. This is the dynamic demonstration of working
> elements in action as part of a structure.
> 
> The theory I put forward with the challenge that you wholly and exactly
> represent these statements as mathematical functions is Rittel and Weber's
> (1973: -166) theory of the wicked problem:
> 
> "1. There is no definitive formulation of a wicked problem.
> 
> 2. Wicked problems have no stopping rule.
> 
> 3. Solutions to wicked problems are not true-or-false, but good-or-bad.
> 
> 4. There is no immediate and no ultimate test of a solution to a wicked
> problem.
> 
> 5. Every solution to a wicked problem is a 'one-shot operation'; because
> there is no opportunity to learn by trial-and-error, every attempt counts
> significantly.
> 
> 6. Wicked problems do not have an enumerable (or an exhaustively
> describable) set of potential solutions, nor is there a well-described set
> of permissible operations that may be incorporated into the plan.
> 
> 7. Every wicked problem is essentially unique.
> 
> 8. Every wicked problem can be considered to be a symptom of another
> problem.
> 
> 9. The existence of a discrepancy representing a wicked problem can be
> explained in numerous ways. The choice of explanation determines the nature
> of the problem's resolution.
> 
> 10. The planner has no right to be wrong."
> 
> Later, I hope to add a few comments on your note. In several debates, you
> have used a tactic in which you decline to answer a challenge directly.
> Rather, you answer a question with a question, or you propose trading
> questions and answers.
> 
> Since you have asked for a design theory that I feel cannot be represented
> mathematically, I am starting with this.
> 
> There are other problems, however, with the notion that "all theories (any
> discipline) can be wholly and exactly represented as mathematical
> functions." This is the shadow of the logical positivist program. Kurt
> Goedel (1931, 1962 [1931]) put an end to that program, and even to the idea
> that one can build all mathematical propositions that way.
> 
> For now, I'll be curious to see Rittel and Webber's theory of wicked
> problems - a model, as I've stated - "wholly and exactly represented as
> mathematical functions."
> 
> Best regards,
> 
> Ken
> 
> Reference
> 
> Goedel, K., 1931, "Ueber formal unentscheidbare Sätze der Principia
> Mathematica und verwandter Systeme I." Monatshefte fuer Mathematik Physik.
> Vol. 38: 173-198.
> 
> Goedel, Kurt. 1962 (1931). On Formally Undecidable Propositions of Principia
> Mathematica and Related Systems. Translated bv B. Meltzer. New York: Basic
> Books.
> 
> Rittel, Horst W J, and Melvin M. Webber. 1973. Policy Sciences, Vol. 4,
> (1973), 155- 169
> 
> 
> -----------------------------------------------------------------
> PhD-Design mailing list  <[log in to unmask]>
> Discussion of PhD studies and related research in Design
> Subscribe or Unsubscribe at https://www.jiscmail.ac.uk/phd-design
> -----------------------------------------------------------------



-----------------------------------------------------------------
PhD-Design mailing list  <[log in to unmask]>
Discussion of PhD studies and related research in Design
Subscribe or Unsubscribe at https://www.jiscmail.ac.uk/phd-design
-----------------------------------------------------------------

Top of Message | Previous Page | Permalink

JiscMail Tools


RSS Feeds and Sharing


Advanced Options


Archives

April 2024
March 2024
February 2024
January 2024
December 2023
November 2023
October 2023
September 2023
August 2023
July 2023
June 2023
May 2023
April 2023
March 2023
February 2023
January 2023
December 2022
November 2022
October 2022
September 2022
August 2022
July 2022
June 2022
May 2022
April 2022
March 2022
February 2022
January 2022
December 2021
November 2021
October 2021
September 2021
August 2021
July 2021
June 2021
May 2021
April 2021
March 2021
February 2021
January 2021
December 2020
November 2020
October 2020
September 2020
August 2020
July 2020
June 2020
May 2020
April 2020
March 2020
February 2020
January 2020
December 2019
November 2019
October 2019
September 2019
August 2019
July 2019
June 2019
May 2019
April 2019
March 2019
February 2019
January 2019
December 2018
November 2018
October 2018
September 2018
August 2018
July 2018
June 2018
May 2018
April 2018
March 2018
February 2018
January 2018
December 2017
November 2017
October 2017
September 2017
August 2017
July 2017
June 2017
May 2017
April 2017
March 2017
February 2017
January 2017
December 2016
November 2016
October 2016
September 2016
August 2016
July 2016
June 2016
May 2016
April 2016
March 2016
February 2016
January 2016
December 2015
November 2015
October 2015
September 2015
August 2015
July 2015
June 2015
May 2015
April 2015
March 2015
February 2015
January 2015
December 2014
November 2014
October 2014
September 2014
August 2014
July 2014
June 2014
May 2014
April 2014
March 2014
February 2014
January 2014
December 2013
November 2013
October 2013
September 2013
August 2013
July 2013
June 2013
May 2013
April 2013
March 2013
February 2013
January 2013
December 2012
November 2012
October 2012
September 2012
August 2012
July 2012
June 2012
May 2012
April 2012
March 2012
February 2012
January 2012
December 2011
November 2011
October 2011
September 2011
August 2011
July 2011
June 2011
May 2011
April 2011
March 2011
February 2011
January 2011
December 2010
November 2010
October 2010
September 2010
August 2010
July 2010
June 2010
May 2010
April 2010
March 2010
February 2010
January 2010
December 2009
November 2009
October 2009
September 2009
August 2009
July 2009
June 2009
May 2009
April 2009
March 2009
February 2009
January 2009
December 2008
November 2008
October 2008
September 2008
August 2008
July 2008
June 2008
May 2008
April 2008
March 2008
February 2008
January 2008
December 2007
November 2007
October 2007
September 2007
August 2007
July 2007
June 2007
May 2007
April 2007
March 2007
February 2007
January 2007
2006
2005
2004
2003
2002
2001
2000
1999
1998


JiscMail is a Jisc service.

View our service policies at https://www.jiscmail.ac.uk/policyandsecurity/ and Jisc's privacy policy at https://www.jisc.ac.uk/website/privacy-notice

For help and support help@jisc.ac.uk

Secured by F-Secure Anti-Virus CataList Email List Search Powered by the LISTSERV Email List Manager