Ken - I appreciate the distinction making in your critique. I agree that we have several different purposes for critical, bibliographic, and narrative review of sources. Because the methods for producing these formats and outputs are quite similar (bibliographies, annotated, with summary, narrative, or multiple attributes) people often produce an adequate artifact and can confound the purposes. I would say that if we don't teach good practice at the MDes level, those that pursue a PhD will find this an especially difficult undertaking. We may teach critiquing, but critical review writing and literature reviews are pitiful in much of the design literature. 

And I agree there's a real need for disciplinary development and conceptual mapping of literature and concepts to theoretical and historical development. Developmental concept mapping through the literature is a PhD level task. But the outcome of this work should not be "just" individual learning. As I noted with respect to graduate medicine - Review articles are not only a primary means of practitioner and advanced resident study, they are also a significant output of fellows and faculty (and MD/PhD's) who have requirements for publishing, and are advancing their disciplines. I think we have some parallels to medical education, but at the PhD level design is being treated more like a social sciences PhD.  I'm not convinced this is the only or best model myself.

Medical professionals move into fellowships or PhD programs to pursue advanced study or pure research. At that stage, but not in residency as much, they are producing review articles. Residents in their research rotation often work on ongoing research projects, but as a PGY3 resident they do not initiate research, and they often join projects that are mid-stream and have their literature base well established.  Therefore, they may have the opportunity to write review article or produce critical literature reviews, but it's not that common in my observations of US programs.

So if our purpose is to strengthen the research base of our field, the tools you've indicated are ways to do promote those purposes, of course. I think there is room for different types of commitments in developing the concepts from literature.  One of them is a research-based approach I've been developing with a Pharmacy professor in U Toronto's Knowledge Media Design Institute. The Interpretive Collaborative Review is a process and a system (prototype) in search of funding. I can appreciate why something simple like Zotero achieves adoption (which is nicely articulated as a Web 2.0 design in many ways). Zotero meets 80% of the need while leaving the advanced features to academics. The ICR is described as:

Collaborative Discovery of Information Significance: A Framework for Making Sense of Healthcare Research

Peter Pennefather and Peter H. Jones. Laboratory for Collaborative Diagnostics, Leslie Dan Faculty of Pharmacy, University of Toronto

We present a framework for collaborative sensemaking by a problem-focused community using electronically accessible scientific journal articles and other digital information artifacts. The framework guides collective structured evaluations of the significance of information sources associated with a given problem. The Interpretive Collaborative Review (ICR) framework is designed as a social informatics process. It is motivated by a need for researchers and practitioners to ascertain a current, collective interpretation of electronically accessible information and collectively generated propositions for problem understanding in complex and rapidly developing domains. Healthcare related information domains are used as an example where there is a need to integrate information derived from biomedical sciences, evidence-based measures of clinical outcomes, and health systems socio-economic analysis.

The ICR framework establishes a conceptual model and a process for explicit human assignment of reviews and scores to information sources within an online dialogical environment, enabling collaborative evaluation, discussion, and recording of significance relationships.  At least three necessary dimensions of significance relationships are recognized and evaluated with respect to each source considered: 1) match, 2) standing, and 3) authority. Match = Claims in the source (meaning), Standing = 
Warranted linking of claim to evidence (agency), and Authority = Evidence in source (power). These referents have both objective data (associated with a publication) and subjective interpretations. 

Each dimension is further characterized by collective scoring for three qualities of value in the source: 1) knowledge validity, 2) precedence, and 3) maturity.  The resulting matrix of scores, specific comments, group editorial commentaries, and references are all woven into an electronic sensemaking narrative publication designed to be indexed, retrieved, and reviewed along with the associated corpus of prioritized sources. 

ICR makes a strong appeal for the dialogic construction of knowledge about collective problems using intentional human assignment of scores and reviews. We find that algorithmic relevancy scores are insufficient when considering the significance of materials in the context of collective problem solving. Human interpretation is needed to determine the relevance of a given information source to a problem context and to understand the range of equally valid perspectives in the recognition of that relevance. The authenticity of a source’s authorship can only be determined by another human being with contextual knowledge of the problem domain and of human motivations and ethical sensibilities. The credibility of a source to a problem situation represents another interpretive context, as the perception of the credibility of the source is a complex function of trust, expertise and of quality.

This is the ICR in summary, which serves some of the purposes we are discussing. It will publish review results electronically, yet also is compatible with peer-review and with new forms of editorial review.

I am quite in agreement with your purpose to address the gaps in our literatures and "to do the hard yards and actually write and develop some of these tools." I will just note that there's a lot more funding available to do this in medicine than in design!

Best, Peter

Peter Jones, Ph.D.
Associate Professor, Faculty of Design
Strategic Foresight and Innovation

OCAD University
http://DesignDialogues.com