Mauricio:
Great question (how we can tell which methods are better).
We can in a crude way, and we can't in a rigorous way.
We can because our first design concepts done following the 'typical' design process were received with mixed reviews in focus groups and other informal tests. Our final solutions following iterative revision and interaction with parents, patients and medical professionals were much more successful. We can compare the results of those two, suggesting the enhanced methods were better - they improved the design.
And then again, we can't tell at all because the first testing methods (limited to focus group and interview methods I think) were different from the final evaluation tests (pre and post knowledge, decisional conflict and final decision methods). It could be that simple iteration accounts for most of the improvement, or it may be the interaction directly with the patients was largely responsible for the improvement (methods). It may be that made the latter work was the changed written content, or it may be that what worked better was the format size change (parameters). We just don't know, we can only 'suspect.' It would be ideal to run the first prototypes that used the abbreviated design process through the same tests as the final prototype to get a direct comparison of two methods. Maybe we can do this, but those tests are currently covered by someone else's research dollars and aims! Our post hoc interest as designers to learn about methods/parameters are different than our medical colleagues which was to reduce decisional conflict and increase knowledge on a sensitive decision.
Complicating any valid comparison based on this study is that this study was not designed from the start to test design methods at all, it was not controlled for methods and other design method factors, so even an additional test as described above will not be very good. This gives me the idea for our next collaborative projects (currently under draft discussion) to identify our design aims for the study and write them into it so that both parties are sharing aims and pursuing aims specific to their disciplines. I'm not sure how this will look to the NIH (National Institutes of Health) - too many aims make them nervous. But it's worth a try.
As you suggest, proving that a design worked is good, but generating from that some transferable/generalizable knowledge about method or process or technique that can be applied to other medical or other design problems is desperately needed and in short supply. As this case shows, when we collaborate with medicine using their research funding against their aims it is hard to get answers to our specific design questions. There is so little specific design research funding that we (I'm speaking of UC right now as we), for now, are often just tagging along with others learning as we join them in their research, working against their aims. I know your dissertation is more carefully designed and hopefully, in part because it's a specific design research method study, will help fill in the blanks with not only some generalizable knowledge but also some methods for design research in design methods.
I'd love to have a current discussion on this list on ways to generate more available research funding for design research. Not a lamentation, but some actionable and positive ideas to get funding in the hundreds of thousands and millions of dollars rather than thousands or tens of thousands.
Blessings to you in your work...
Mike
-----------------------------------------------------------------
PhD-Design mailing list <[log in to unmask]>
Discussion of PhD studies and related research in Design
Subscribe or Unsubscribe at https://www.jiscmail.ac.uk/phd-design
-----------------------------------------------------------------
|