Analysis: Peer review for digital scholarship
<p>The feedback and critique process that allows a scholar to refine their work was identified as a theme of scholarly practice after the Project Bamboo Workshop 1 series. The most formal-- and arguably most important-- channel for feedback is peer review. Peer review is the "seal of approval" for written articles, indicating that they have passed a certain threshold of quality. While the longstanding system of peer review for articles continues to flourish due to a promotion and tenure system that necessitates the publication of traditional forms of scholarship, workshop participants felt there was a need for peer review to evolve, in order to allow non-traditional manifestations of scholarship (including digital humanities projects, tools, and mashups) to receive that same seal of approval.</p> <p>The extent to which the quality of digital scholarship is doubted by default can be seen in the case of on-line journals, even those that are the same as their print counterparts in every way, save printing paper volumes. One workshop participant explained: "If you manage online publications, must be sure all standards are in place, and must be higher than standards of print. Without clear standards, people will not publish to online journals, feeling that this is the "bottom" in terms of publishing." (Ex 2, 1c-A).</p> <p>Workshop participants were interested in the potential for developing a set of criteria or peer review techniques for digital tools and projects. The success of these tools requires institutional buy-in, including changes to the dossier to make the inclusion of digital projects feasible. While some participants noted that the MLA has already put together standards for evaluating digital scholarship, others felt those guidelines were unable to serve as a guideline for writing university policies because of their generality. Successfully opening the door to digital scholarship as part of the tenure dossier would, at least initially, require more work on the part of the evaluators: "At Research 1 universities, it isn't uncommon for tenure cases to be turned down at the higher levels even when these same tenure cases have been wholeheartedly supported by specific departments; and so candidates, external reviewers, and chairs face the added burden of having to educate deans and provosts through documentation and explanation. At the moment, for better or for worse, the onus is mostly on the candidates to explain what institutional bodies have ratified your work, and exactly what form has that ratification taken." (Education working group, <a href="http://www.loriemerson.com/">Lori Emerson</a>, 1/12/2009 post)</p> <p>Peer review for digital scholarship was also discussed in the context of a potential directory of tools, data and projects. Reviews could increase the usefulness of the directory, and could additionally serve as both as feedback for the scholar, and material that could be included in a tenure dossier.</p>