Australian Computing Research Alliance
The University of Sydney's School of Computer Science is part of the Australian Computing Research Alliance (ACRA) group, an informal alliance of computing schools from across Australia.
Other members include the Australian National University, University of Melbourne and University of New South Wales.
This alliance encourages publication of quality peer-reviewed research, in both conferences and journals.
The ACRA group aligns with the Declaration on Research Assessment (DORA) and the Leiden Manifesto for Research Metrics.
This alliance advocates practical and robust approaches for evaluating research, aligned to those of DORA.
Venue impact factors and rankings are not measures of the scientific quality nor impact of an article’s research.
ACRA strongly discourage inclusion of such rankings in job applications, promotion applications, and other career(-progression) and evaluation processes.
They acknowledge that such rankings may serve as a guide for early career researchers, or newcomers to a research area, towards finding quality publications.
However, venue rankings have limited value in comparing one research area with another, they do not discriminate specialist from generalist venues, nor the distinct values of different venues, and they often replicate information contained in standard bibliometric tools.
This ACRA grouping proposes that career processes support academics and assessment panels in:
- evaluating research in context, providing scope for applicants to highlight their key publications and the quality and impact of these key publications;
- encouraging evaluators to read an applicant’s key publications;
- recognising that, occasionally, some of the best research, especially in emerging or interdisciplinary fields, might be announced in peer-reviewed workshops.
To assist our colleagues in transitioning, ACRA advocates that research leaders offer specific support for writing research quality and impact cases.
As an example first step, the UK Research Excellence Framework (REF) proposes consideration of the importance of the research problem solved, the approach taken and properties of the solution, the output describing such an approach, and how the approach in the research output has been built on or applied, including concrete evidence of impact.
Proposed wording for announcements and documentation includes: “Applicants are actively encouraged not to include conference/journal/venue rankings, but should instead focus on the impact of their research outputs in describing the excellence of their research”.