Dear Ken and all,

> On 6 Nov 2014, at 3:11 pm, Ken Friedman <[log in to unmask] <mailto:[log in to unmask]>> wrote:
>
> You have put evidence-based design to use on many occasions, and you describe an evidence-based approach in several works and projects. One valuable resource you have developed using this approach is the book Writing About Medicines for People: Usability Guidelines for Consumer Medicine Information. May I ask you to share your thoughts on the book — How did you developed it? How did you gather the evidence that informs the content? How can others put this approach to work?
>
> Admittedly, this is a specific kind of issue, but once you get into the details of evidence-based design, it is the specifics that count for specific kinds of situations. Evidence-based design choices are a function of the specific situations within which a designer must practice.

Happy to do so. And this is a timely moment to do so because I have just completed a chapter for a forthcoming book which gives an account of the history of the work within which Writing About Medicines for People (WAMFP) took place, and the publisher has given us permission to make it available to our Subscribers, Members and Fellows. Also we are at a moment when many of our achievements in this area could be lost. I’ll place some of the URL’s that explain this along with that for the chapter at the end of this post.

To cast this work in it’s larger framework (for us), it’s useful to see it as part of a program of institutionalising good EBDP in a variety of government, and industrial contexts. By the time we got to working on WAMFP, we had already demonstrated through published case histories (e.g Fisher and Sless 1990) and through a number of high profile public sector projects (eg. Australian Bureau of Statistic Guidelines for designing data collection forms and publications) that were not published but widely known about and used, that it was possible to institutionalise good EBDP. Moreover, because these were EBDP projects there was a lot of data to demonstrate that these led within administrations to massive (90%+) improvements in productivity and when they were applied in commercial contexts they led to increases in profitability through increased sales and market share—all of which came as a bit of surprise to us because the principle motivation for us to undertake the work then and now was to improve equity: to make the relationship between organisations and the people they served fairer. It’s also important to note that much of this work preceded the now popular fashion for so called “User Centred Design” (UCD) and prior to the WWW and internet.

WAMFP came about specifically because of a major development in public policy in Australia. The Federal Government introduced a Quality Use of Medicines (QUM) policy, We were alerted to this by one of our Board Members who was a senior medic and ergonomics advocate—Michael Patkin—who asked us to take an active interest in this area. So I began positioning CRI as an interested organisation by attending some of the conferences on the subject and meeting many of key stakeholders from the professions, industry, consumer groups and regulators who were jockeying for position in relation to QUM.

As an aside from the WAMFP project, I mention this because one of the key lessons we had learnt by then (starting in the mid 1980s) was the need to engage with the key stakeholders in a sophisticated way at the commencement of a project. I say 'sophisticated’ because we had over a number of previous projects investigated and tested a variety of stakeholder engagement methods and narrowed it down to a few highly productive tools which gave us systematic methods across a range of design projects: we used a combination of 'Appreciative Dialogue’ methods and ‘Logic of Positions’ analysis to ground our engagement with stakeholders. Sorry to deal with this by naming topics only, but to go into detail would take up all of this post and more and is much more suited to coursework that lists. The important things to note at this moment are:
1. Stakeholder engagement accounts for at least 50% of the project effort.
2. The highest risks to successful outcomes comes from poorly managed stakeholder engagement
3. Even though no-one had commissioned us at this stage to undertake this project, from our own technical and procedural point of view, we had already started work.
4. Superficially our methods in this area look like co-designing, collaborative and participatory, design. And this is the language we often use to describe our approach publicly, but they are not, even though many of the participants would think that they have taken part in this type of project.

Back to WAMFP. We tendered for and were successful in winning the contract to develop the Usability Guidelines for Consumer Medicine Information for the Commonwealth Government of Australia, Department of Health.

As an aside, many of you will be familiar with our model of the design process, so I will not go into detail here except to say that we commence this type of project with the Scoping Stage which consists of about 20 different investigations seeking evidence about all aspects of the problem domain, including investigations at and beyond the problem boundaries as they were defined in our brief. These boundary investigations are some of the most interesting because they move us into the unthought territories that I know many of you are interested in. We have to weigh up the extent to which our stakeholders will collectively or individually enable us to breach these boundaries. Needless to say, many of the results of these investigations do not find their way into the public reports.

Here are some examples which I snuck out in the form of blogs at a later stage after the projects were complete:
http://communication.org.au/medicine-information-panacea-or-prosthesis/ <http://communication.org.au/medicine-information-panacea-or-prosthesis/>
http://communication.org.au/tax-and-complexity/ <http://communication.org.au/tax-and-complexity/>
http://communication.org.au/there-are-always-externalities/ <http://communication.org.au/there-are-always-externalities/>
These are, of course, recognisable parts of the design process.

We use the term Scoping to describe the 20 or so investigations we undertake as part of our design process. Much of this stage is what Ranajan describes as involving:

> …the role of insights in shaping design directions while the design team is immersed in the task and usually meandering along in search of a firm direction, very iterative”
The outcome of which
> …comes from deep immersion in reality with our senses wide open seeking, not just eyes wide open, I include here the mind, body and soul, all are in play here, and the seeking is for a variety of possible directions that lead to several alternate resource maps being processed into numerous compositions that set the stage for ideas to emerge from these juxtapositions of real sense data that is as yet un-apprehended by all of science, since that is still in the future.

Needless to say, this is a highly creative, exciting and entertaining part of our work.

One of the key things to emerge out of this process, once we have synthesised our discoveries and inventions into some possibly achievable outcomes in the current environment, is a list of desirable outcomes agreed by the stakeholders. In this particular project we expected a huge divergence among the stakeholders about the list of desirable outcomes. Nurses, Doctors, Pharmacists, Industry, Regulators and Consumer Advocates are notorious for their disagreements and we had set aside a considerable amount of time to consult with them all to come up with an agreed list. But because of the stakeholder management methods we had previously tested, this stage went very smoothly and quickly.

It’s at this point that we took a step backwards before moving forward (and I would regard this step as one of our innovations in design methods, though there are notable precedents in the work of Buckminster Fuller and others). We test these desirable outcomes against the existing solutions within this problem domain as it is currently defined. We cal this the Baseline Measurement (BM) stage, though we used to call it benchmarking. The EVIDENCE we collect at this stage is critical in shaping what happens next both in development of new design prototypes, then testing them refining them. It is also used in managing the implementation of the new designs in the environment in which they have to work.

We used a testing method we developed in-house which we call Diagnostic Testing at this BM stage and in subsequent stages of the development of WAMFP. This is described in detail in WAMFP and many other publications so I will not repeat it here, but a few points are worth highlighting.
1. Superficially this method looks like usability testing familiar in HCI and UX design. It is not. Its origins are in information design not human factors and psychology.
(See: SLESS D. 1979. Image design and modification: an experimental project in transforming. Information Design Journal 1: 74-80.)
2. At its heart, it is based on Appreciative Dialogue methods and is therefore open ended, not closed as in scientific hypothesis testing.
3. It derives from an epistemology rooted in dialogic and linguistic thinking such as Buber, Bhaktin, and Wittgenstein.
4. We use it to collect both quantitative and qualitative evidence in the same sessions.
5. We find it one of the most creative parts of our work.

The last point may come as a surprise to many designers who eschew an EBDP. What I can report, however, is that sitting alongside people who are struggling with our crude prototypes and talking with them about their use of the prototypes provides us with insights that we could not get in any other way. It’s like an epiphany, a moment of revelation. Indeed, if more than one of us is conducting the testing, the first question we ask each other when we emerge from the testing is “what struck you?”. We go into the testing with the expectation of being surprised, and we are seldom disappointed. This is obviously quite different to the rather cliched view of EBDP that designers without this experience sometimes have of testing. All I can say is try it and see.

One of the key insights we had in WAMFP was to do with the way people navigated through medicines information. With the existing material, such as pack inserts, we found that the way people tried to navigate through the material was totally different to the way it was presented. This led us in the prototype stage to put a lot of emphasis on the navigational structure. In previous work on totally different projects to do with letter systems, billing systems, forms, and legal documents, we had developed techniques for writing document grammars. We incorporated a medicine information specific grammar into our prototypes as a basic structural aspect of the design. We tested and retested that structure a number of times using diagnostic testing until it worked and met the performance targets set in our list of desirable outcomes. I cannot overstate the importance of this iterative testing in EBDP, but then many of you who work in this area will know this. One final point on this: comparing the results from the final prototype to the results from the BM gave us the best indication we had that we had brought about a desirable change. Without the before, there is no after. We do this as a matter of routine in all our projects, if we are allowed to. I know that most designers, even those who use EBDP, don’t do so. I hope this will change because it is the single most important public legitimiser of EBDP and it may turn out to be the most important legitimiser of design in general. Think about it.

The final stage for us, implementing the new way of writing ands testing medicine information was done in collaboration with people who were already writing consumer medicine information leaflets so that the book WAMFP was usable. It’s now in its third edition and is widely used as a de-facto standard around the world.

From our point of view, though, it never reached its full potential. The original design used a very crude typographical output system because of the technical constraints of the day, which have not changed since 1994 when it was first published. Despite all the advances since then, which we anticipated and mapped out in the architecture, it has never been lifted above the level of a crude word processor typography. We envisaged a document that could be highly customised for different platforms and different users: customised, responsive design. It has yet to happen. Nonetheless, we have succeeded in lifting the overall standard of such documents in Australia. We have institutionalised better design practice than went before it, some of which has been copied by other jurisdictions, most notably the EU in their regulation of patient information leaflets (PIL). In the EU, testing of PIL is now mandatory and some of the document grammar we developed in Australian has been reinterpreted into a different and not entirely usable dialect. I would have to conclude that it falls short of our objective of institutionalising good design practice, but it’s a step in the right direction.

We haver monitored the performance of these designs for some years and we now see an emerging possibility that all this good work will be lost, partly because the Australian regulator of such documents has had a major attack of amnesia in recent years and has begun dismantling much of our work in this area, and we fear it’s only a matter of time before they turn their sights onto WAMFP. It is also partly because our society is changing and becoming less inclusive and egalitarian.

For those of you with a stomach for the unfolding tragedy, here are a few of my recent blogs on the subject and the chapter of the book I mentioned at the beginning of this long post which is my personal attempt, as is this post, to prevent this work from being entirely lost.

Thank you Ken for the suggestion.

David

http://communication.org.au/guidelines-for-medicine-labelling/ <http://communication.org.au/guidelines-for-medicine-labelling/>
http://communication.org.au/bad-design-by-tga/ <http://communication.org.au/bad-design-by-tga/>
http://communication.org.au/tga-ignores-consumer-research/ <http://communication.org.au/tga-ignores-consumer-research/>
http://communication.org.au/comments-to-tga-on-proposed-tgo79/ <http://communication.org.au/comments-to-tga-on-proposed-tgo79/>
http://communication.org.au/product/regulating-information-for-people/ <http://communication.org.au/product/regulating-information-for-people/>


blog: http://communication.org.au/blog/ <http://communication.org.au/blog/>
web: http://communication.org.au <http://communication.org.au/>

Professor David Sless BA MSc FRSA
CEO • Communication Research Institute •
• helping people communicate with people •

Mobile: +61 (0)412 356 795
Phone: +61 (0)3 9005 5903
Skype: davidsless

60 Park Street • Fitzroy North • Melbourne • Australia • 3068

-----------------------------------------------------------------
PhD-Design mailing list <[log in to unmask]>
Discussion of PhD studies and related research in Design
Subscribe or Unsubscribe at https://www.jiscmail.ac.uk/phd-design
-----------------------------------------------------------------