Tagged / impact

Bibliometrics need not be baffling!

What are bibliometrics?

Bibliometrics are a set of methods used to study or measure text and information. Citation analysis and content analysis are the most commonly used bibliometric methods. Bibliometric methods can help you explore the academic impact of a field, a set of researchers or a particular journal paper.

What is citation analysis?

Citation analysis looks at where a document has been referenced by others since it was originally published – this information can be used when searching for materials and in analysing their merit. Undertaking citation analysis on yourself is useful for assessing your own research performance. Specialist databases such as Web of Science and Scopus provide various tools for doing this analysis.

Searching for citation information on the Web of ScienceSM

Web of ScienceSM is hosted by Thomson Reuters and consists of various databases containing information gathered from thousands of scholarly journals, books, book series, reports, conference proceedings, and more:

  • Science Citation Index Expanded (SCI-Expanded)
  • Social Sciences Citation Index (SSCI)
  • Arts & Humanities Citation Index (A&HCI)
  • Index Chemicus (IC)
  • Current Chemical Reactions (CCR-Expanded)
  • Book Citations Index – coming soon!

These databases enable you to perform a variety of tasks, such as search published literature, undertake citation analysis, track new research within a particular field, and identify chemical compounds and reactions. Data is available from around 1990, and even earlier in some cases.

By producing a Web of ScienceSM Citation Report for yourself (or for others), you can find out who is citing your work and how it is being used in other people’s publications so that you can get a feel for the overall citation activity around your outputs. Search for an author’s name and then click on ‘Create Citation Report’ from the results page.

Producing this report will give you information such as the number of items published in each year, the number of citations to those items for each year, the average number of citations per item, and your h-index based on this information. Click here for web tutorials on how to use the Web of ScienceSM.

Searching for citation information on Scopus

Scopus, part of Elsevier’s SciVerse facility, was launched in November 2004 and is an abstract and citation database containing around 19,500 titles from more than 5,000 publishers. Scopus enables researchers to track, analyse and visualise research, and has broad coverage of the scientific, technical, medical and social sciences fields and, more recently, the arts and humanities. Data is currently largely available from 1996 but it does go back further than this in some cases. For more information about Scopus, click here.

By searching for yourself (or others) on the Scopus database using the author search facility, you can use the ‘View Citation Overview’ function to get a feel for the citations activity around your outputs. The information is presented and can be analysed in a number of ways, including pie charts, graphs and tables, and shows the breakdown of citation activity over a number of years and your h-index based on this data. Various tutorials on using Scopus can be accessed here.

Scopus and the Research Excellence Framework (REF): HEFCE has announced that Elsevier have been chosen as the provider of citation data services to the REF sub-panels that have chosen to make use of citation information as part of the assessment process. Using the Scopus database, HEFCE will provide the relevant sub-panels with raw citation data (i.e. not normalised) accompanied by contextual information, which will assist those panel members in making decisions about the outputs part of the REF submissions.

What is the h-index?

The h-index was conceived by Professor Jorge Hirsch in 2005 within the field of physics and is fast becoming one of the most widely used metrics for research evaluation. It is also becoming increasingly used as a measure of research activity and academic prominence across various subject areas.

The benefit of the h-index over other citation measures is that it is not influenced by a few highly cited papers and it ignores any papers that remain uncited. It is calculated based on the number of papers by a particular author that receive h or more citations. Therefore, an h-index of 15 means that a person has at least 15 papers that have been cited 15 times or more. Fortunately, the Web of Science and Scopus both automatically calculate the h-index as part of their citation analysis functions so there is no need to work it out manually.

If you’d like to know more about the h-index, the original research document can be accessed from the Cornell University Library webpage.

What are journal impact factors?

Journal Impact Factors are published annually on the Web of Knowledge and provide a way of ranking journals based on the citation performance of articles published by those journals from the previous two years. For more information about how impact factors are calculated and how they can be used, see my previous blog post.

Other methods of ranking journals also exist, such as the ABS Academic Journal Quality Guide and the ERA journal ranking list. Journal rankings can be useful when choosing which journal to publish in, for example.

Exploring research impact

Why do I need to think about the impact of my research?

Given the current economic situation, tighter spending reviews and increasing constraints on public spending, there is more of a need than ever to demonstrate the economic, social and cultural benefits of publicly funded research to wider society. This broad definition of research impact is gradually being adopted and used in a number of ways by various funding bodies that need to be accountable for the money they distribute, such as the Research Councils UK (RCUK), the Higher Education Funding Council for England (HEFCE) and some charities and trusts.

Consequently, there is now greater impetus to involve researchers more directly in demonstrating the impact of their research. Researchers therefore need to actively think about how to demonstrate the value of their research and its wider impact, from the application stage through to project completion, dissemination and beyond.

This impact pathway is a fluid process and research impacts can occur at any stage in the research life cycle – often they can stem from unexpected or unintended outcomes as well as from planned activity. The key is to start thinking about potential beneficiaries and pathways to impact during the project planning stage and to continue to monitor the outcomes in an ongoing way. This will help you to make new connections and partnerships beyond the project itself, and to put in place resources and activities that enable you to make the most of opportunities for achieving impact when they arise. Keeping a record of activity related to a project, and gathering evidence to support impacts and outcomes achieved, is recommended to enable you to effectively fulfil any current and future reporting requirements.

What are funders looking for in terms of impact?

Many funding bodies, particularly larger ones such as RCUK, have a section on their funding application form that specifically asks you to consider the potential pathways to impact as appropriate for the nature of the research you’re proposing to conduct. This is to enable funders to support you in undertaking these activities – you’re not being asked to predict the actual outcomes that the research will achieve.

Each funder understands that there is great diversity in the kinds of impact that are possible; they also acknowledge that this diversity is a great strength of the research community in addressing such things as urgent social issues, remaining competitive in global markets and improving quality of life. It is about embracing the ways in which research-related knowledge and skills benefit individuals, organisations and nations.
In thinking about potential impacts, you might find it useful to consider the potential beneficiaries of the research – innovative and creative approaches to engaging beneficiaries and fostering impact are generally strongly encouraged by the funders. For more specific information about completing the impact sections on the RCUK application forms and for an indication of the potential range of impacts that can be generated from research, visit the RCUK impact web-pages.

Furthermore, the RCUK has just launched the Research Outcomes Project, which requires all RCUK grant holders to upload information about the various outcomes that have resulted from each of the RCUK-funded projects they are responsible for, and one of those categories is impact.

What is HEFCE looking for in terms of research impact for the Research Excellence Framework (REF)?

As part of submitting to the REF, HEFCE requires higher education institutions to provide evidence of research impact that has been realised within the assessment period but which stems from research undertaken at that institution within a number of years prior to the assessment period. Therefore, rather than looking forward to the kinds of impact that might stem from a research project, HEFCE is asking for information about impacts that are being, or have already been, achieved within a set timeframe. More information about how HEFCE is approaching impact in the REF is available from the HEFCE REF web-pages.

RCUK Research Outcomes Project is ready to launch!

Following my previous post about the development of the Research Councils UK (RCUK) Outcomes Project, the launch of the new system for collecting information about research outcomes from all RCUK grant holders is nearly here. Assuming all goes to plan with the final phase of user testing, the system will go live from 14 November 2011. Grant holders will be required to upload information about the following for each of the RCUK-funded projects they are responsible for:

  • Publications
  • Other research outputs
  • Collaboration
  • Communication
  • Exploitation
  • Recognition
  • Staff development
  • Further funding
  • Impact

Grant holders will be able to log in to the system using their Je-S login and will be responsible for maintaining the outcomes information about the grants they have been awarded, even if they move institution. RCUK have issued a list of FAQs to help answer some common queries.

Southern Universities REF event at BU – 22 February 2012 – SAVE THE DATE!

Research assessment

REF logoBU will be hosting a half day Research Excellence Framework (REF) event for southern universities, supported by the REF Team, on 22 February 2012 to which all staff are invited to attend.

This event follows hot on the heels of the first REF Team-supported event held at BU on 19 May 2011, to which over 150 delegates from 39 institutions attended (see our previous blog post – The excellent HEFCE REF event at BU!).

The sector-wide consultation on the proposed REF panel criteria closed earlier this month and the final documents are due to be published in January 2012. This event will provide an update on the current developments with the REF and the confirmed REF panel documentation, focusing specifically on the assessment of impact within each of the four Main Panels.

The event will be open to BU staff and external delegates and the provisional programme is shown below.

Provisional programme:

09:30 – 10:00    Coffee and registration

10:00 – 10:30    REF Team overview of the assessment framework

Chris Taylor, Deputy REF Project Manager, REF Team

10:30 – 11:00    Similarities between the 4 Main Panel criteria

Professor Stephen Holgate, Chair of Main Panel A

11:00 – 11:30    Morning break

11:30 – 12:15    Panel specifics, differences and impact assessment

There will be concurrent sessions, one for each of the four Main Panels. Attendees choose which one to attend.

Main Panel A:    Professor Stephen Holgate, Chair of Main Panel A

Main Panel B:    Professor Philip Nelson, Chair of sub-panel 15 (General Engineering)

Main Panel C:    Professor John Scott, Chair of sub-panel 23 (Sociology)

Main Panel D:    Professor Bruce Brown, Chair of Main Panel D

12:15 – 13:00    Panel Q&A session with all participants

13:00 – 14:00   Lunch, networking and close

 

The event is free to attend but booking is essential. Booking will open very soon – further details to follow!
 

Funding Opportunity – Assistance with Impact Analysis

Funding opportunities

JISC have recently announced a funding opportunity “Embedding impact analysis good practice in research, using BCE practitioners”.

This Call is designed to encourage skills sharing between researchers, engagement practitioners and information management specialists. It is aimed at research groups looking to bring in external expertise and support to help them develop improved capability and process to analyse the benefits and impact of their research, in order to enhance their impact and sustainability.

The work, which will be facilitated by the National Co-ordinating Centre for Public Engagement (NCCPE), focuses on the highly topical area of research impact. It invites people to register online to form partnerships to synthesise and share learning about how to embed technology-enabled good practice in impact analysis in research groups, using the expertise of Business and Community Engagement staff.

For any queries please contact David Owen.

The deadline for receipt of proposals is 31 October 2011. Call details can be found here.

What makes a good impact section?

Guidance

In writing an FP7 bid the marks allocated for Impact are the same as those for Science & Technological Excellence. So, how do you make sure you score top marks?

Beta Technology (sponsored by DEFRA) are the UKs National Contact Point for three of the FP7 themes and offer a number of good tips. They’ve also provided real-life examples of a good and not so good Impact Section together with the evaluators’ mark and feedback – these are essential reading for any propective FP7 applicant! 

Impact section examples can be found on the I drive at the following address: \\Lytchett\IntraStore\CRKT\Public\Research Blog Docs\Impact Summary

If you would like more information on the impact advice from Beta Technology please contact Shelly Maskell.

REF update from the VC and PVC

Research assessment

In the latest Vice-Chancellor’s email, Prof John Vinney gave an update on the Research Excellence Framework (REF) and BU’s preparations for the exercise. The email also introduced this month’s VC video in which he and Prof Matthew Bennett discuss the REF in more detail – and what it means for BU – with Sue Eccles from the Media School.

You can watch the video here:

httpv://youtu.be/7s8RTlOOPnU.

Knowledge exchange and business links are increasingly important in academic promotion

Research commissioned by the Association for University Research and Industry Links (Auril) indicates that experience of knowledge exchange and links with business have become increasingly important to academics trying to win promotion.

Philip Graham, executive director of Auril, stated that “In some job applications, (knowledge transfer) is now rated as desirable, although not essential. Ten years ago it would not have been rated as desirable.”

The majority of survey respondents said that academic promotion depends more on academic teaching and research reputation, however, business links are becoming increasingly more important.

The government’s White Paper on higher education, published in June, says that universities should “look again at how they work with business across their teaching and research activities, to promote better teaching, employer sponsorship, innovation and enterprise.”

HEIF (Higher Education Innovation Funding) exists to support and develop a broad range of knowledge exchange activities between universities and colleges and the wider world, which result in economic and social benefit to the UK. BU submitted its HEIF-5 strategy to HEFCE in July; further information will appear on the blog in due course.

You can read the full story on the AURIL website: Build up the Business Assets if you Want to Get Ahead

BU REF preparations – next mock exercise underway

Hot on the heels of our REF light-touch review of outputs last winter, our subsequent mini-mock exercise in two UOAs, and the release of the final REF Guidance on Submissions and draft panel criteria documentation, BU’s next mock REF exercise is now underway.

The BU winter 2011 mock will concentrate solely on the impact and environment elements of the submissions (i.e. individual outputs will not be reviewed). During the autumn term BU Unit of Assessment (UOA) Leaders have been asked to produce 2-3 impact case studies, an impact statement and an environment statement for each UOA. These will be sent to the external reviewers (at least 2 per UOA) in early December, and feedback will be shared with the UOA Leaders in February 2012.

The next review of outputs is planned for spring 2012.

These mock exercises are intended to shape and craft our submissions to REF2014 to ensure we put forward the strongest submissions possible.

If you have any questions or suggestions regarding our internal preparations for the REF then send me an email 🙂

Overview of the REF draft panel criteria – what are the subtleties between panels?

Guidance, Research assessment

At the end of July the REF team released the draft panel working methods and criteria documentation (see our previous blog post for access to the documents).

We’ve spent the week wading through the four main panel documents and have produced a very brief overview of the subtleties between the panels on key criteria (such as the use of citation data, co-authored outputs, additional environment data, etc) in a tabular format.

You can access the overview table here: REF – draft panel criteria comparison table

Unfortunately this is no substitute for reading the actual documentation (sorry!) but does highlight the key points and differences between panels.

These documents are currently open to sector-wide consultation until 5 October 2011. BU will be submitting a single institutional response coordinated by the Research Development Unit. BU staff are invited to submit feedback for consideration as part of this response. Please email all comments to Anita Somner by 20 September 2011.

Latest journal impact factors

Research assessment

Following the release of the latest Journal Citation Reports® on the Thomson Reuters ISI Web of Science database, we have compiled a list of the top ranking journals in various fields related to BU research. BU staff can access these lists by going to the designated folder on the collaborative I-drive: I:\R&KEO\Public\RDU\Journal Impact Factors 2011. If there are any additional subject areas that you would like to see included, do send me an email.

Related blog posts that may be of interest:
Journal impact factors explained

Google Scholar’s new citations tracking tool

As the demand for information about the performance of research outputs increases, Google Scholar’s latest online citations tracking tool could prove to be a useful resource. Google Scholar Citations is currently being launched with a small number of users before widespread dissemination, but the early signs are that it could provide a simple way for academics to calculate key metrics and monitor them over time.

According to Google Scholar, the tool will ‘use a statistical model based on author names, bibliographic data, and article content to group articles likely written by the same author’. You can then identify your articles using these groups and Google Scholar will then automatically produce the citation metrics. Three metrics will be available: the h-index, the i-10 index (which is the number of articles with at least ten citations), and the total number of citations to your articles. Each of these metrics will be available for all outputs as well as for articles published in the last five years, and will be automatically updated as new citations are available on the web.

You will be able to update your ‘profile’ of outputs by adding missing items, merging duplicates or correcting bibliographic errors. You’ll also have the choice to make your profile public, the idea being that you’ll make it easier for colleagues globally to follow your research because your outputs will appear in Google Scholar search results.

To get a glimpse of Google Scholar Citations in action, there are some sample profiles available via the Google Scholar blog. You can also register your email address so that you will be notified when the tool is finally available to everyone.

REF draft panel criteria is now available

Research assessment

Two important REF documents are now available on the REF website:

1. The draft panel criteria and working methods for consultation. A summary of this will be added to the blog next week.

2. The analysis of panel membership is now available.

Panel criteria and working methods consultation – the consultation runs from now until 5 October. BU will be submitting a single institutional response coordinated by the Research Development Unit. BU staff are invited to submit feedback for consideration as part of this response. Please email all comments to Anita Somner by 20 September 2011.

The REF Guidance on Submissions document was released on 14 June, a summary of which can be accessed on the I-drive from: I:\CRKT\Public\RDU\REF.

Continuing importance of ‘Impact’ highlighted by HEFCE and RCUK

Research assessment

RCUK logoThe Higher Education Funding Council for England (HEFCE), Research Councils UK (RCUK) and Universities UK (UUK) have issued a joint statement highlighting the continuing importance of the impact of research and making a commitment to developing common frameworks.  Both the REF (undertaken by HEFCE) and the Research Councils place a strong emphasis on the impact of research.

Their key aims are:

  • ensuring that their approaches to supporting knowledge exchange are joined up
  • securing greater impact from publicly funded research whilst ensuring their funding policies are complementary and work coherently together
  • ensuring that their monitoring, assessment and evaluation as well as the collection of statistical data from HEIs is proportionate.
  • supporting the training of new researchers, at both PhD and post-doctoral level, to ensure that they acquire the skills and expertise needed to maximise the impact of their own research.

The full statement is available on the RCUK website.

ESRC Knowledge Exchange Opportunities

Funding opportunities

ESRC logoThe ESRC has announced two new calls as part of its Knowledge Exchange Opportunities scheme

The scheme exists to enable researchers to work with individuals and organisations in the private, public and civil society sectors. Knowledge exchange can involve a range of methods but is ultimately about sharing and applying good ideas, research results, experiences and engagement skills. The ESRC fund and manage a range of schemes to support collaborative projects and create a dialogue between researchers and individuals/organisations that have the potential to benefit from social science research.

The two calls that have been announced are:

 – the Follow on Fund scheme

 – the Knowledge Exchange Opportunities scheme

The Knowledge Exchange call is open from 1 September until 27 October 2011 and is aimed at social science researchers at all stages of their career and at organisations in the business, public and civil society sectors, with the intention of encouraging dialogue and collaboration between these groups.

Future calls are scheduled as follows:

  • 12 December 2011 – 6 February 2012
  • 9 April 2012 – 4 June 2012
  • 20 August 2012 – 1 October 2012

Further information is available from the ESRC website: http://www.esrc.ac.uk/funding-and-guidance/collaboration/knowledge-exchange/opportunities/index.aspx

If you are interested in applying to one of these schemes, please contact CRE Operations who will happy to support your application.

Excellent PI development resource available from Vitae

Guidance

Earlier this year Vitae launched an excellent development resource for principal investigators (PIs). The Leadership Development for Principal Investigators training is available online, free of charge from here: http://www.vitae.ac.uk/policy-practice/263521/Leadership-Development-for-Principal-Investigators.html

The website provides information in the following sections:

  • What is expected of a principal investigator
  • Research environment
  • Impact
  • Managing people
  • Project management
  • Networks

Information is provided for both pre-award and post-award stages of the research lifecycle.

This is a fantastic resource suitable for PIs at all stages of the research career.

If you have used the resource to access information then let us know what you think by commenting on this blog post and share your tips with your colleagues!

REF Guidance on Submissions document released

Guidance, Research assessment

The REF2014 Guidance on Submissions document was released on Thursday and can be accessed on the HEFCE website here: REF Guidance on Submissions

We have prepared a summary document of the key points that can be accessed on the I drive: I:\CRKT\Public\RDU\REF

At the end of July the REF team will publish the draft panel working methods and criteria documents which will be open to consultation until the autumn. The Research Development Unit will be coordinating the BU response to the consultation – further details will be available once the documentation is released.