Dear Terry,
Mike Zender’s note to you [1, below] is serious and appropriate. The place for your paper is a peer-reviewed journal, not another self-published paper that will generate endless debate on this list. There is no need to post the material in your offer [2, below]. While you have offered masses of data, there is no way to know whether the data is meaningful.
While I did not plan to enter this debate again, you addressed your comments to me and to Klaus. In my view, Mike is right. (Sorry, Jonas, but Terry addressed this to me by name.) The masses of data you offer cannot provide the demonstration many of us have asked you to provide.
There is a difference between showing “mathematically-based predictive methods” and making specific predictions that permit a test that determines whether the predictive methods work.
You plan to post “new clean models and dashboards (rather than working models), anonymising the models and their data, and writing anonymised case descriptions to suit.” This is the equivalent of giving business students sanitised and anonymised classroom case studies to simulate business decisions. These cases provide no evidence for decisions that will work in the real world. These kinds of cases allow students to simulate decision processes. They do not show that the processes will be effective in the real world. Even more significant, simulations offer no way to compare the actual outcome of simulated decisions use one simulation method against similar decisions using other methods. It’s all guesswork — it’s useful for teaching, but not for prediction.
The data you offer to provide are the material that case study publishers offer in an instructor’s pack: "Outline description of each design case, image of the Vensim predictive model structures, list of the equations and data used in the models, binary program code for the predictive models and dashboards, pointer to downloading the Vensim reader to use the models on your own computer.” It is easier to learn about Vensim on the Ventana Systems web site:
http://vensim.com
Why would anyone provide binary program code to 2,600 list members? Who on this list is going to read hundreds of lines of binary code to see whether a simulation is properly coded? Masses of irrelevant information are as useless as no information at all.
What your do not offer to provide is a series of actual predictions in real working situations. Actual predictions would 1) show convincingly 2) with real-world data that 3) you can use mathematical methods to design outcomes that 4) behave as you predict 5) despite the complexities and perturbations that inevitably arise through complex loops of behavioural action and interaction in the world. This will validate your methods by showing that your methods overcome the problems that others cannot overcome.
Most of us appreciate the complexity of situations that you claim you can control. For us, these complex situations have inherently unpredictable outcomes. Only by demonstrating a real series of predictions can you demonstrate that you can predict outcomes where others cannot.
Simulations resemble case studies in business school or law school, war games in the military, or modelling in geoscience or meteorology. They do not have genuine predictive power. They are useful in helping us to understand the properties and behaviour of systems and processes. Even predictive modelling in geoscience and meteorology has limited success. Physics permits greater predictive power, but physics also requires evidence that predictions are valid.
If Albert Einstein had merely shown “mathematically-based predictive methods” for his theory of relativity, physicists would never have adopted his views. Einstein made specific predictions. Other scientists tested these predictions. These tests demonstrated that the predictions were accurate. This demonstrated the valid predictive capacity of Einstein’s new theory of physics.
Anyone can make predictions. Making valid predictions is another matter entirely. Einstein’s friend and sometime theoretical opponent Niels Bohr famously said, “Predictions are very difficult — especially about the future.”
If your models have predictive power, you don’t need to control the situation whose outcomes you predict. You should be able to study any situation, and — using your “mathematically-based predictive methods” — determine the outcome of the actions that others take.
During the last debate, one colleague wrote me off-list with a valid point. If your mathematically-based predictive methods were truly workable, he wrote, you could make a fortune in commodities, financial futures, stocks, or just about any market. If you were to do that, you could endow your own research institute to study, teach, and work with your approach to mathematically-based predictive methods.
If you can really predict the future behaviour of complex adaptive systems that operate through multiple complex loops of feedback and interaction, you can model markets.
The Encyclopaedia of World Problems and Human Potential demonstrates the complexity of describing and modelling the kinds of systems for which you make your claims. Anyone who works with these kinds of problems recognises that this is generally impossible in the real world. The links between and among problems generate more variables than anyone can control, not even with massive government resources. It’s difficult to model the problems. Solving them with the certainty you claim is beyond the capacity of those who work with these kinds of issues. If you or other subscribers to this list would like to see what these kinds of problems look like in a descriptive model, go to:
http://www.uia.org/encyclopedia
Several people I know work with these kinds of problems at the World Bank, the United Nations, and other organisations. No one who does this kind of work claims the certainty and predictive power that you claim to possess. Knowing such people and the work they do, your offer of sanitised case studies and masses of data is not impressive. You offer nothing to demonstrate the predictive power of your methods.
Demonstrating the predictive power you claim for your methods requires more than masses of code applied to anonymised teaching cases.
Rather then launch another debate, why not take Mike’s suggestion seriously? Publishing your work in a peer-reviewed journal would be a good first step.
Yours,
Ken
Ken Friedman, PhD, DSc (hc), FDRS | Editor-in-Chief | 设计 She Ji. The Journal of Design, Economics, and Innovation | Published by Elsevier in Cooperation with Tongji University Press | Launching in 2015
Chair Professor of Design Innovation Studies | College of Design and Innovation | Tongji University | Shanghai, China ||| University Distinguished Professor | Centre for Design Innovation | Swinburne University of Technology | Melbourne, Australia
Email [log in to unmask] | Academia http://swinburne.academia.edu/KenFriedman | D&I http://tjdi.tongji.edu.cn
—
[1]
Mike Zender wrote:
—snip—
Terry:
Might I suggest that at this point, with your previous papers being questioned by creditable people, that you refrain from posting papers to this list until they have been published in peer-reviewed journals?
If you want to run partially developed and unvalidated ideas by peers, always a good practice, I suggest for sake of time and quality that you do so first with a smaller number of trusted colleagues than that represented by this full list.
Many might assume, I do, that any paper posted to this list has either been peer-reviewed OR will be clearly identified as an unvalidated concept.
—snip--
[2]
Terry Love wrote:
—snip—
I hope to post 3 case studies to phd-design using mathematically-based predictive methods in design before the New Year.
These use a predictive participatory design process that echoes but differs from traditional collaborative/participatory approaches to design.
To do it involves producing new clean models and dashboards (rather than working models), anonymising the models and their data, and writing anonymised case descriptions to suit. This is a significant task.
The three design cases using the predictive models will most likely be:
1. Design of local government strategy for design-based support to address a specific problem of local economic development (research for an LGA).
2. Design of commercial property investment strategies (commercial project).
3. Design of crime control strategies in high-crime neighbourhoods in areas of relative socio-economic disadvantage (part of as yet unpublished Design Out Crime project), or, possibly, design of optimal design strategy in competition between two countries and their design infrastructures.
The case studies will include:
Outline description of each design case
Image of the Vensim predictive model structures
List of the equations and data used in the models
Binary program code for the predictive models and dashboards
Pointer to downloading the Vensim reader to use the models on your own computer.
—snip--
-----------------------------------------------------------------
PhD-Design mailing list <[log in to unmask]>
Discussion of PhD studies and related research in Design
Subscribe or Unsubscribe at https://www.jiscmail.ac.uk/phd-design
-----------------------------------------------------------------
|
