Peer review

"How will the Bamboo Project "peer review" what it selects ... including review by persons and institutions that are not formally participants in Bamboo?" (Ex 1, 1b-C)

"How do we get to an industry standard of quality? What is authority?" (Ex 1, 1b-D)

"want Bamboo certification on data or sites. Want industry wide standard for content." (Ex 1, 1b-D)

"Interating feedback into research. Rethinking and imagining what hasn't been explored? Rethinking what has been done, critiquing your own work; happens when you share work with colleagues, revising book, "iterative loops"; There is a org called "virtual knowledge studio" that provides a metastudy of our practices; Looking at iterations, versions of texts." (Ex 2, 1a-B)

"Ask new questions inspired by others, by self, serendipity. Refine existing questions by sharing work with others, getting feedback, etc." (Ex 2, 1a-C)

"Reviewing articles. Submitted for publication in a journal. First step is to filter for initial value, then send on to reviewers, etc. Question raised whether this is actually supporting own scholarship" (Ex 2, 1a-C)

"citation, attack or plaudit for a published work" (Ex 2, 1b-A)

"Bamboo: to introduce a set of standards and practices, a peer-review equivalent" (Ex 1, 1b-B)

"If you manage online publications, must be sure all standards are in place, and must be higher than standards of print. Without clear standards, people will not publish to online journals, feeling that this is the "bottom" in terms of publishing." (Ex 2, 1c-A)

"does the open access model work as well in humanities as it does in the sciences. The whole system of scholarly communication needs to be revisited, esp. in the humanities. but it's threatening and needs to be done carefully. If ou think the model for open access is computer science, without peer review, then its OK. But if you're in a field with a tradition peer review then it's a problem. Comes down to the question of authority of the source" (Ex 2, 1d-G)

"have been putting together scholar maintained communities; participation evaluated by peers. Researcher driven collaborative space. Have graduate level of access - "increasing levels of less and less trusted readers." " (Ex 2, 1d-G)

"Need for assessment tools to determine quality, plus track into careers" (Ex 1, 1d-G)

"Evaluation: Solicit feedback on research; question assumptions;" (Ex 3, 1a-E)

"intellectual networking: share & discuss pre-pub, converse with divergent communities, collaboration -- find collaborators, find & form invisible colleges, talking about/giving feedback about research, organize/share research, engage in thoughtful writing & converssation with colleagues, participate in several interest groups" (Ex 3, 1b-A)

"Building a formal/informal community of readers, commentators, advisors" (Ex 3, 1b-C)

"Get feedback. How are other people thinking about somewhat formulated concepts." (Ex 3, 1c-A)

"As the discussion unfolded the group seemed to be concerned about the dangers of imposing or creating a digital publishing standard via bamboo. They realized that at the basic level it was desirable to have some seal/ certification that would give digital humanities research and production greater valence in the academic world. However, too strict a set of rules or too regulated a world would mean the loss of true creativity, or political anarchy." (Ex 4, 1b-D)

"Provide and solicit peer review and critique ("getting people to do peer review is one of the hardest aspects of running any publication ... there are so many ways to do a useless peer review")" (Ex 5, 1b-B)

"We need a peer review for the stuff that matters to us now. Way of referencing a vetting process. Attempts to do that: Merlot w/ pedagogical tools. More than a social network. Work and digital scholarship for peer review. Rather than a parallel peer review system, what we should do is look for senior established colleagues who can take risks and who would do something different - persuade them to publish something in a digital form. By taking that step it makes it more respectable for junior people to do. Peer review in place for this digital journal. There are forms of scholarly work that don't fit - might be service, scholarship at same time. Need other way of having this work outside of our own institutions. Resistance not just review; metrics not there to evaluate online edition. Faculty numbers are not growing but nontenured track is growing significantly. Depts need research professionals - nonclass of academic staff. Shouldn't create a separate academic class but how to measure the merit." (Ex 6a, 1d-F)

"Danger is still internalizing assumptions: a print body and a digital tail. Need to build a whole new world from scratch, rather than an add-on. If Bamboo could identify pools of faculty who could shepherd thru nontraditional projects, that were assessed on intellectual merit. Not rewarded or penalized for being technological. We could do it at my institution." (Ex 6a, 1d-F)

"there's lots of talk about tools, but tools need to prove their usefulness. If Bamboo builds, provides tools, but how you judge them." (Ex 7, 1a-E)

"Bamboo could be a clearing house for what's happening, and what's happened. A place to get knowledgeable people who can review work. A specialty and competence space." (Ex 7, 1d-C)

"Be more pragmatic; it's hard to evaluate digital scholarship. PB could provide set of criteria, peer review techniques for evaluating it. Tools for understanding the value, rather than saying "it's good" - show people why it's important and how to demonstrate the importance." (W2, discussion of advocacy)

"Need to be cognizant that there's standards for evaluation already; MLA already developed standards. PB should consider who are the faculty who are coming in to work on these things. If you don't have any advocacy, you can't get junior faculty to do this." (W2, discussion of advocacy)

"the documents for P & T put together by the MLA for digital work are too general to serve as policy manuals at any given institution: they sometimes state the obvious, e.g., that a peer-reviewed journal online should count as much as a peer-reviewed article appearing in print, valued according to the online venue's stated number of submissions and percentage of rejections. But still, she makes clear that unfortunately the obvious still needs to be said. She also notes that one of the highlights of the MLA workshop was when she realized that department chairs need to offer external and internal reviewers detailed instructions on how to evaluate what they see. At Research 1 universities, it isn't uncommon for tenure cases to be turned down at the higher levels even when these same tenure cases have been wholeheartedly supported by specific departments; and so candidates, external reviewers, and chairs face the added burden of having to educate deans and provosts through documentation and explanation. At the moment, for better or for worse, the onus is mostly on the candidates to explain what institutional bodies have ratified your work, and exactly what form has that ratification taken. Perhaps one form of that ratification could be an online written evaluation composed by scholars constituting particular editorial boards, perhaps housed at Bamboo." (Education working group, Lori Emerson, 1/12/2009 post)

"Regardless, overall, most of the participants that I corresponded with agreed that one of the main problems to evaluating digital work is that the artificiality of the dossier form required by most institutions lacks flexibility for explaining the relevance of a digital project. My colleagues also noted that there is currently an uncomfortable channeling of so-called Digital Humanities work into established tracks whose establishment may itself be a subject of reconsideration (for example, the continued relevance of the term "humanities" is up for debate for many people currently in so-called humanities departments). The general consensus is also that organizations such as the MLA or such as Bamboo might want to find ways to bring the review process into the open and use the affordances of networking technologies to create a profession-wide, peer to peer, network for active and public evaluation." (Education working group, Lori Emerson, 1/12/2009 post)

"MLA documents on tenure/promotion state the obvious, but that's needed. Could PB have a role in ratification of digital work. Dossier form is not amenable to digital work ... lacks way to express relevance of a digital project. Profession-wide active/public evaluation - can PB enable that?" (W3, Education, Progress and Demonstrators)

Bamboo tags: 

Add new comment