Todd,

You come close to persuading me but not quite. When I speak of
>> publicly accessible evidence based on agreed criteria of what  
>> counts as 'working'
I have in mind something a little more demanding than you offer.

Let me give you a concrete example. If you look at the Australian  
Code of Practice for Medicine Labelling (available on our web site)  
you will see that it requires consumers to be able to perform an  
agreed set of tasks at an agreed level. So, if we design a new label  
we have to provide evidence both to the client and the regulator that  
consumers can use the label appropriately at the agreed level. The  
kind of evidence admissable is agreed by the parties involved.

What you seem to be talking about with personas is a possible process  
by which you arrive at the agreed set of tasks and the level at which  
they might be performed. In our type of work this is work we do with  
the stakeholders at the scoping stage (again see the code of practice  
and other stuff on our site). But we still have to provide evidence  
that our designs meet the agreed performance requirements.

You come close to something similar when you say that:
> For example, in the old design, most participants couldn't  
> determine the number of new voice mails within 10 seconds of  
> looking at the screen. In the redesign, every participant was able  
> to complete this task in under 3 seconds.
This is good, but on it's own not persuasive. You need to give many  
more cases to be convincing.

The labelling area mentioned above is highly formalised, the general  
list of agreed tasks is quite small (11), and it is only concerned  
with one type of user. So it is not representative of information  
design projects where there are many more varied and complex tasks  
and many more users, but the principles are the same, as is the  
requirement for before and after evidence.

But you also need to do something else, and that is demonstrate that  
using Personas is better at achieving these outcomes--more reliable  
and cost effective-- than other techniques. There I remain unpersuaded.

You say:
> This is one of the problems with such a "theory-driven" mentality  
> (not directed at you personally, but one of the problems we see  
> regularly in academia)
This is not my 'theory driven' approach. I was trying to encapsulate   
what I take to be YOUR THEORY. (BTW, as other members of the list  
will know, I don't take any of this personally, so you can be as  
blunt as you like. I will certainly be blunt, and it is not directed  
at you personally, but about the matter we are discussing). I am  
trying to help articulate some of the assumptions involved in what  
you describe so that you can drive the theory, rather than it driving  
you. You say, for example:

> The fact is that personas are characters. No, they do not have  
> social security numbers. But they are as real to the product as you  
> and I. GOOD personas are built on real people who use the product.

This is exactly the way in which a good author or playwright  
construct their characters, using real people as a basis. But unlike  
an author or a playwright I think you are confusing real people with  
plausible constructs and you are missing the fact that constructing  
plausible personas is an art in much the same way that constructing a  
character in a play is an art. And like the playwright you have to  
deal with the impondrable relationship between art and life. It may  
seem that by using personas you are avoiding such invention and  
uncertainty. I think you are jumping straight into it, but unlike the  
playwright you want to say that art and life are the same.
So when you say:
> we can control how we use them, we design and test with personas  
> the same way we would with real live people. Our personas attend  
> our project meetings and they have a voice just as loud, sometimes  
> louder, than you and I. They are as real to the product as anyone  
> else in the room.
I have serious doubts about your own understanding of what you are  
doing.

But I recognise the political nature of what you are doing with  
personas--representing the normally unrepresented constituencies--in  
a decision making process. That is something that is familiar to many  
designers. For example, we estimate that this type of political work  
is roughly 50% of the effort in any successful design project, and,  
when left unmanaged, it is the most likely reason for project  
failure. But is using 'personas' the way to deal with it? There, I  
remain skeptical.

BTW, the UK marketplace, like all marketplaces, is probably  
unforgiving too. The research and professional work we do is in many  
markets, across and in many countries.  It is also in environments  
which are far more unforgiving than the marketplace.

 From smog ridden Sao Paulo in Brazil.

David
-- 
Professor David Sless BA MSc FRSA
Director • Communication Research Institute of Australia
• helping people communicate with people •

60 Park Street • Fitzroy North • Melbourne • Australia • 3068

Mobile: +61 (0)412 356 795
Phone: +61 (0)3 9489 8640
web:    http://www.communication.org.au


On 27/10/2005, at 1:59 PM, Todd Warfel wrote:

> A key advantage to personas is that they help eliminate design/ 
> development of unnecessary features. That is better design.
>
> On Oct 27, 2005, at 10:52 AM, David Sless wrote:
>
>> [...] These are all examples of 'buy in' by the client.
>>
>> At best, these are descriptions of a satisfying process. But on  
>> the question of whether this leads to successful design outcomes-- 
>> for example, improved products or services--you remain silent.  
>> When I ask for evidence, I am asking for evidence that takes a  
>> simple form.
>
> Well, a satisfying process does lead to improved designs. To  
> clarify, a successful design marries business objectives with  
> customer goals. So, for instance:
>
> Business goals
> * increase usability
> * increase revenues
> * decrease support costs
>
> Customer goals
> * show me products I'm interested in
> * don't show me items I'm not interested in
> * make my shopping process easier, faster, less frustrating
> * help me find what I want easier, faster
>
> If the design can accomplish these things, or assist in the  
> accomplishment of these things then it's a better design. Personas  
> assist in this by allowing us to focus our discussions and design  
> decisions around things like:
>
> Q: How do we improve usability
> A: Well, what does Sheldon think a more usable product is?
>
> Q: If we provide the customer with more targeted products, will  
> they buy more?
> A: Well, let's ask Sheldon if that theory will work.
>
>> For example:
>>
>>> Before we did our design, x was the case.
>>> After we did our design, y was the case.
>
>> This is the kind of evidence I'm called upon to provide. Does this  
>> not apply in your work?
>
> Actually, this was the case with a recent redesign project for  
> Comcast (largest telecom here in the states).
>
> Another Philadelphia based firm (who has offices in the UK) did an  
> initial design for the Comcast Digital Voice (VoIP). They were on  
> their seventh iteration when we were called in to clean up the  
> mess. They had copied Outlook and slapped it on VoIP.
>
> At first glance, it didn't look so bad. However, once you tried to  
> perform tasks like "How many new voice mails do I have? And which  
> one do I want to listen to first?" well, it quickly fell apart.  
> They were not in touch with the customer.
>
> We did several types of user research and testing (e.g. contextual  
> interviews, behavior observations, card sorting, paper prototype  
> testing) and developed personas based on the marketing info we had  
> for audience demographics, characteristics from the customers we  
> interviewed, and people we knew who fit that profile.
>
> We developed five personas for the DVC and based our design  
> decisions and patterns based on how the personas would attempt to  
> perform a task, or complete one of their goal.
>
> We tested paper prototypes of the old version against the new  
> version and the redesigned version, based on personas, patterns,  
> etc. outperformed the old design every single time. Participants  
> were tested on typical tasks like:
> * Determining if there was anything that needed my attention (e.g.  
> new voice mail)
> * Determining how many new voice mails I had
> * Play a voice mail
> * See if anyone important called who doesn't leave a voice mail  
> (e.g. grandma)
> * Block a number that I don't want to take calls from (e.g.  
> solicitors)
>
> For example, in the old design, most participants couldn't  
> determine the number of new voice mails within 10 seconds of  
> looking at the screen. In the redesign, every participant was able  
> to complete this task in under 3 seconds.
>> But your final comment is just another example of 'buy in'
>>
>>> With personas it's a Sheldon thinks this - end of discussion.
>
> But, David, it's our experience that buy-in is critical to good  
> design and a good process. So, that does lead to a better design.  
> Not to mention it makes the design process go faster.
>
> As I stated above, the biggest thing with personas is that they  
> help eliminate design/development of unnecessary features. That is  
> better design.
>
>> [..] If we move this to a discussion of theory, it seems to me  
>> that the use of personas is based on a rather simple theory,  
>> namely that plausible fictions can be a substitute for actual  
>> people. I don't doubt the capacity of fictions to offer us  
>> insights and illumination. Are they a guide to how people will  
>> actually behave? I have no idea, but I have some doubts.
>
> This is one of the problems with such a "theory-driven" mentality  
> (not directed at you personally, but one of the problems we see  
> regularly in academia).
>
> The fact is that personas are characters. No, they do not have  
> social security numbers. But they are as real to the product as you  
> and I. GOOD personas are built on real people who use the product.  
> I'm not talking about Marketing personas - I don't trust them, as  
> they're designed with the "Who can we sell this to?" mentality.  
> That's bogus.
>
> We design personas based on real customer information, real people,  
> real habits, real behaviors. So, while they are characters, you'll  
> find yourself saying "I swear I know this guy and he uses our  
> product."
>
>> Part of my doubt grows out out of an appreciation of the  
>> performing and visual arts where the achievement of plausible  
>> verisimilitude involves great artifice [...] The 'trick' of  
>> plausible fiction involves what Gombrich describes as the  
>> beholders share, and what in literary and theatre criticism is  
>> called 'the suspension of disbelief'. In other words 'buy in'.
>
> Well, that's not the case here. You're going to need to step back  
> from theatre, pretend, and theory a bit here and understand that  
> the reality is, Personas are a design tool, which, when created  
> properly, lead to better designs (see above).
>
>> My doubt then is based on the my suspicion that the 'test' of a  
>> successful persona is similar to the 'test' of successful fiction.  
>> If this is the case, I can understand why it may seem satisfying,  
>> but I am also concerned that such satisfaction may be misplaced.
>
> Well, that's not the case. I can understand skepticsm if this were  
> the case. But for us, and I can only speak for my firm, because we  
> can control how we use them, we design and test with personas the  
> same way we would with real live people. Our personas attend our  
> project meetings and they have a voice just as loud, sometimes  
> louder, than you and I. They are as real to the product as anyone  
> else in the room.
>
> It's kind of like having a conference call. You can't see the  
> persona on the other end of the phone, you can only image what they  
> look like. But you know they're there, because they speak up.  
> Personas give you that effect. They might not talk verbally, but  
> they've given their testimony and they are present.
>
>> As I said in my last post and now elaborate:
>> I have a preference for publicly accessible evidence based on  
>> agreed criteria of what counts as 'working'--a far more  
>> unforgiving place than the marketplace, or the shared stories of  
>> designers and clients.
>
> Well, I don't know what the market place in the UK is like, but in  
> the US, it's not that forgiving thanks to competition.
>
> Hope this helps.
>
> Cheers!
>
> Todd R. Warfel
> Partner, Design & Usability Specialist
> Messagefirst | making products & services easier to use
> --------------------------------------
> Contact Info
> Voice:    (607) 339-9640
> Email:    [log in to unmask]
> AIM:       [log in to unmask]
> Blog:      http://toddwarfel.com
> --------------------------------------
> In theory, theory and practice are the same.
> In practice, they are not.
>
>
>
>