Universities are being encouraged to make their hiring criteria more inclusive.Credit: skynesher/Getty
A new online resource brings together universities’ research-assessment and career-development policies to help the academic community make hiring, promotion and tenure procedures fairer, more robust and more diverse.
The Reformscape database, launched today by the Declaration on Research Assessment (DORA), collates publicly available policies, action plans and other documents from research institutions worldwide.
Its aim is to provide examples of institutions that have changed their assessment systems, and to show administrators practical, actionable ways to improve their own policies. DORA, a global advocacy group, says that the policies included in the data set move beyond conventional, narrow assessment criteria and metrics, such as an overreliance on journal impact factors.
“It can be difficult to convince people from a standing start that change is possible,” says Zen Faulkes, DORA’s programme director. “This is a tool to help them get unstuck.”
Online treasure trove
James Wilsdon, who studies research policy at University College London, says Reformscape is “a timely and practical addition to the toolkit of resources now available to help organizations to embed responsible assessment in their operating cultures”. He adds that he would have found it invaluable when he was in a university-management role.
Dashboard will track hiring and promotion criteria
“One common grumble you hear when encouraging institutions to adopt responsible approaches to research assessment in hiring and promotion decisions is that there aren’t enough good examples of how to do this in time-efficient and robust ways,” says Wilsdon. “DORA’s new Reformscape database goes a long way to plugging this gap, by assembling an online treasure trove of policies, HR guidance and case studies of how others have grappled with these agendas.”
Reformscape was developed as part of Project TARA (Tools to Advance Research Assessment), a multi-pronged effort to gain a clearer picture of researcher evaluation. The database turned out to be less wide-ranging than originally planned; when the project launched in 2021, it aimed to identify, collect and publicize all the criteria that universities use to hire and promote researchers. That proved to be too ambitious, says Faulkes, because many of those policies are not publicly available and institutions can be reluctant to share them.
However, the database does include documents from more than 200 institutions in 20 countries, and it is expected to grow. Faulkes says that users are encouraged to add information as it becomes available. “We hope this becomes a community resource that people will contribute to,” he says.
Starting point
Kamden Strunk, a higher-education researcher at Virginia Commonwealth University (VCU) in Richmond, says that despite the narrowed scope, Reformscape could be a very useful starting point for universities looking for examples of how others are reforming their assessment policies — if it continues to grow as Faulkes predicts.
Eight ways universities can make career assessment more equitable
“The first thing universities want to do is benchmark against peer and aspirant institutions,” he says. VCU, for example, would look at what is being done at universities rated equal to or above it on various international rankings, as well as other institutions in Virginia and the wider region. As it stands, the database contains the global information, but not more localized data. The university “would need to do a deeper dive to find credible comparisons”, he says.
Loosening Reformscape’s inclusion criteria might help the database to grow, Strunk suggests. For example, institutions could be allowed to submit lightly redacted copies of their policies, rather than the database relying on publicly available information.
Faulkes says there has been a positive response from those who have seen the database, and that some have added extra information. “We are already seeing contributions from people who had early access, and over time that will grow,” he says.
