Annual impact reporting using Researchfish®

What is Researchfish®?

Researchfish® is an easy-to-use, standard electronic tool for capturing and analyzing impacts of the spectrum of research and innovation activity in the province.

The Researchfish® system will replace the previous Word-document forms used for AIHS annual impact reports.

Researchfish® will allow AIHS-funded researchers to do the following:
• enter an outcome once and then re-use the data in reports to other funders;
• download research outputs into several formats, so that researchers can load the data into other documents like CVs;
• enable members of a research team to view the PI’s award and allow them to assign data from their own portfolio to the shared award; and
• track achievements and notable successes over time to use when reporting to their institution, funders, the public and others even after their award has ended.

Researchfish® is not being used for interim reporting; current interim reporting processes and schedules remain in effect.

You can visit Researchfish online for more information about the tool and how it is used.

Who will use Researchfish®?

AIHS will roll out access to Researchfish® in a phased approach, starting with a few selected programs. By April 2016, all AIHS-funded researchers and teams who need to submit annual impact reports will do so through Researchfish®.

The first phase of users to have access are the CRIOPRIHS, or Translational Health Chair awardees. By mid-April 2015, the collaborative leads, principle applicants, and recipients of these award programs will be sent an email from admin@Researchfish.com with account and submission details.

• If you already have a Researchfish® account (i.e. because you have used Researchfish for another funder), please click on the link in your email, log in to your existing account, and follow the instructions to add the AIHS award(s) to your existing account.
• If you do not have a Researchfish® account then you will need to register by following the instructions in the email, it is essential that you click on the link provided in the email.

* (Please see the table below for specific award requirements and submission timelines.)

Why gather information on impacts using Researchfish®?

Our provincial health research and innovation community has been telling us for years that they need a simpler, more flexible and more useful way to do mandatory reporting on their activity.

With the launch of Researchfish®—developed by the UK’s Medical Research Council and used by 6,500 researchers globally—we’ll have an easy-to-use, standard platform for capturing and analyzing impacts of the spectrum of research and innovation activity in the province.

The tool will let multiple funders to track and collect comparable impacts and results and let researchers compare themselves internationally.
Researchfish® also allows researchers to track their achievements and notable successes over time, information that may be invaluable when reporting to their institution, funders, the public and others.

What does this mean for AIHS and Alberta?

The information collected will be invaluable to AIHS and will be used in a multitude of ways, from contributing to the evidence submitted to make the continued case for sustained investment in health research and innovation, assessing progress to AIHS’s goals informing internal funding processes or communication and importantly in making the information available to universities.

The impacts collected in Researchfish® are aligned with AIHS’s Health Research to Impact Framework, which is based on the Canadian Academy of Health Sciences (CAHS) framework.

Specific award and submission information:

AIHS Award start date

PRIHS with award start date of January 1, 2014

Translational Health Chairs with award start dates of April 1, 2014 and earlier

A. CRIO Team, Program and Project awards with start date of April 1, 2013

B. CRIO Team awards with start date of April 1, 2014

C. CRIO Project Cancer awards with start date of March 28, 2013

D. CRIO Project Cancer awards with start date of March 28, 2014

Reporting Period

Researchfish® will be used to report impacts achieved  between award start date of January 1, 2014  and March 31, 2015

Researchfish® will be used to report impacts achieved between award start date  and March 31, 2015

A. Word-processor based reports are being replaced with Researchfish® to report impacts achieved between April 1, 2014 and March 31, 2015.

B. Researchfish® will be used to report impacts achieved between April 1, 2014 and March 31, 2015

C. Word-processor based reports are being replaced with Researchfish to report impacts achieved between March 28, 2014 and March 31, 2015

D. Researchfish® will be used to report impacts achieved between April 1, 2014 and March 31, 2015

Submission Period

April 10, 2015 to June 5, 2015 (18:00 MST)

Researchfish® FAQs

What is Researchfish®?
A standardized electronic reporting tool to capture and analyze common research outputs and outcomes (impacts). This tool gives Alberta Innovates – Health Solutions (AIHS) the ability to report on the impacts achieved through our research investments over time. There is also the opportunity for this information to be viewed by academic institutions in Alberta and across funders internationally.

How do I log into my Researchfish® portfolio?
All researchers who are required to submit their annual impact reports to AIHS using Researchfish® will receive an invitation to do so from admin@researchfish.com. The invitation will include the information you require to set up your Researchfish® portfolio. Once you have set up your portfolio, you will be able to access it at www.researchfish.com. If you have forgotten your Username or Password, help is available through the “Help and Support” tab on the Researchfish® home page (www.researchfish.com).
What can AIHS see in my Researchfish® portfolio?
AIHS can only see information that you have attributed to your AIHS research award/grant, and this pertains to both “live data” and “submitted date.” We cannot see any data in your Researchfish® portfolio that has not been attributed to AIHS.

When I invite others as assistants or delegates to my Researchfish® portfolio, what can they see?
If you add an assistant or delegate to your portfolio, they will be able to view everything in your portfolio. Your assistant/delegate will be able to add, amend, and save data. However, the PI who received the initial invitation for the award must verify the report content and submit the report to AIHS.

How do I invite others to my Researchfish® portfolio?
Once you have logged into Researchfish®, you will see the “My Network” window on the right hand side of your home page. Click on “Add new…” and follow the instructions provided to send invitations to delegates, assistants, administrators or colleagues.

When I invite others as co-investigators or team members to my Researchfish® portfolio, what can they see and do?
When you add a team member, you can select the specific award(s) you would like the member to access in your Researchfish® portfolio or provide them with the ability to add outputs into your portfolio. If given access to a specific award, they will be able to view only the information that you have attributed to the selected award. You can also provide co-investigators or team members with the ability to re-use outputs from either a specific award or your portfolio in their awards.

Why is AIHS collecting this data?
The information submitted to AIHS in annual impact reports is used to communicate to the public, government, and academic institutions about AIHS’s investments in health research. The annual impact reports are used to demonstrate AIHS’ accountability to the government and the public with respect to funding relevant, high-quality health research and innovation across the research continuum and to advocate for continued support for health research and innovation activities.
The information is also used to better understand how research leads to impact and how to optimize impact, thereby informing organizational strategic decisions at AIHS.

Why can’t AIHS look at the data provided through interim reports instead of collecting annual impact reports?
The information provided in your interim report is compared against objectives and milestones for your research group as outlined in your Research Management Plan. Whereas interim reports are quarterly progress updates, annual reports measure the impacts achieved by yourself or your research group on an annual basis in relation to the AIHS award. The Performance Measurement and Evaluation unit at AIHS is responsible for administering, analyzing and reporting the results of the information submitted through annual impact reports.

When can I update my Researchfish portfolio?
You have access to your portfolio all year and can update it at any time.

Why do I need to attribute impacts to AIHS?
You can use your Researchfish portfolio as a centralized record of all your research activities, outputs and outcomes (impacts) resulting from various grants and/or awards. However, when reporting about a specific award such as one from AIHS, it is important to only report the impacts that were achieved through the support of that award. Specifically, AIHS is interested in knowing what impacts have resulted from our investment in your research. This is done by attributing the relevant impacts in your portfolio to the AIHS award.

When is the annual report submission period?
All annual impact reports must be submitted to AIHS in the period of April 1 – 30 each year. If April 30th falls on a weekend, AIHS may extend the deadline to 17:00 MST on the first Monday in May of that year; this slight extension will be communicated to your via email and will also be visible during the submission period on the top right side of the screen near the “submit” button. Please note that while you can enter data in your Researchfish portfolio and attribute it to AIHS at any point during the year, this is not the same as submitting your report.

What if I forgot to submit an impact in my report?
You can add impacts to your portfolio and attribute them to an award any day of the year. If you have submitted your report and then attribute another impact, you can resubmit the award to AIHS if the submission period is still open (i.e., April 1 – 30). If the submission period has closed before the impact is attributed to the award, the newly attributed impact will not be received by AIHS for that reporting period.

What if the Researchfish® categories do not capture an impact of my research group?
Researchfish® includes an “Other Outputs and Knowledge/Future Steps” category. Use this category for an impact that you feel is important to report to AIHS and which you cannot include in the other categories provided. Using this category, you have the opportunity to provide details about the impact as well as whether you think it should be captured in one of the existing Researchfish® categories.

Do I have to report confidential information (e.g., intellectual property)?
Because the reported impact information is used by AIHS in public reporting, we ask that you only attribute information to the AIHS award that you are comfortable having in the public domain. Some questions will allow you to indicate that certain information is held in confidence.

Where do I get help with Researchfish?
Many forms of help are available on the Researchfish® website, including videos, a download of the Common Question Set, FAQs and opportunities to sign-up for a webinar. Check the “Invitation Help” and “Help and Support” tabs for further information.

Can I get feedback on the information I reported through Researchfish®?
If you have questions about the information you have submitted to Researchfish® or AIHS’s annual impact reporting process, please contact the Performance Management and Evaluation unit at AIHS: pme@aihealthsolutions.ca using the subject heading “Annual Impact Report.”

What are “exemplars”?
The qualitative data collected through annual impact reports provides us with important insights into the specific aspects of the research process that are not always captured by the quantitative data. This data has been used to create “exemplars”, i.e., narrative examples of research impacts, that have been shared publicly through our annual impact report and other presentations (http://www.aihealthsolutions.ca/our-impact/assessing-research-impact/aihs-annual-impact-report/). We also use exemplars to help illustrate how certain impacts in one area (e.g., advancing knowledge) are interconnected with those in another area (e.g., informing decision making). Exemplars further provide us with the opportunity to showcase some of the achievements of our funded researchers to various audiences.

Why was my research group not included as an exemplar in last year’s AIHS Annual Impact Report?
Not being selected as an exemplar for the AIHS Annual Impact Report is not a reflection of the quality of research conducted by a research group. As only a limited number of exemplars can be included in the Annual Impact Report, criteria (HYPERLINK) were developed to ensure fairness and transparency in exemplar selection.

What criteria are used to select exemplars?
The following criteria are considered when selecting exemplars for our annual impact report.
• Alignment to Canadian Academy of Health Sciences (CAHS) impact categories (http://cahs-acss.ca/making-an-impact/)
o Exemplars can focus on one or more CAHS categories. In fact, a strength of exemplars is that they can demonstrate connections across CAHS categories (e.g., how activities to inform decision making may have led to collaborations that, in turn, built capacity for research)
• Diversity with respect to grant/award type and location
o This is to show the breadth of AIHS investment as well as to inform the grant/award and university-specific reports
• Richness of data with respect to context
o Why is this research important? What is novel/innovative about the research?
• Richness of data with respect to the pathway to impact (i.e., does the researcher provide the connections to link activities to outcomes to impact(s)?)
• Key topic areas of interest to AIHS (e.g., patient oriented research, vulnerable populations, specific disease states).

Can I use technical language?
The information submitted to AIHS in annual impact reports is used to communicate about AIHS’ investments in health research to the public, government, and academic institutions. You are therefore writing your report for a broad audience. Wherever possible, avoid jargon and include plain language descriptions of technical terms. It is important that you use your own words in your report. Do not copy and paste descriptions from media reports, for example. Also, try to have your writing convey what your impacts mean to a broad audience (i.e., what does your research mean to them?)

Can I refer the reader to information reported elsewhere by using statements like ‘see Section X’?
No. Include all relevant information in each section rather than reference information elsewhere (e.g., in another report, in another section of the report).

What kind of information are you looking for in the open-ended fields asking for descriptions of impact?
• Provide details about the actual (rather than anticipated) impacts of your work and the role of your research group in achieving these impacts.
o For example, the statement that trainees in your research group are benefiting from an interdisciplinary training environment provides limited detail about impact. What kinds of professional and research skills have your trainees gained as a result of this environment? Have any new collaborations resulted from working in an interdisciplinary environment?
o For example, if you have prepared policy briefings: for whom were they prepared? Did they impact specific policies? What was the reach of these policies? Did these policies result in change?
o For example, if you presented at a prestigious conference, describe why the conference is prestigious. If the conference was interdisciplinary in nature and this is important in understanding the impact of your work, report this.
o For example, if a story about your work was published in the media, include details about the audience for the story and its reach (i.e., how widely it was disseminated).
o For example, if you identified “audience reported change in views, opinions or behaviours” as a result of an engagement activity in which you were involved, provide details about the particular change that resulted.

Can I use acronyms or do I need to spell these out in full each time in my report?
There are many acronyms used by our funded researchers and, in some cases, the same acronym can be used by different groups to mean different things. For accuracy, we want to be sure what each acronym means. Spell out acronyms the first time they are used in each category. We analyze across reports by category, so it is not enough to only spell out each acronym once in your report. Common acronyms do not need to be spelled out (e.g., CT scan, MRI).

Where do I report funding obtained by trainees in my research group?
In the ‘Further Funding’ category. Indicate the name of the trainee who received the award next to the title of the funding scheme. For example:

How do I add collaborator(s) and/or partners in the “Collaborations and Partnerships” category?
Do not list collaborators and partners in the description fields of the collaboration. They will not be counted if entered in these open-text fields. You must enter each collaborator and/or partner using the “Add a Partner” button:

Can I combine multiple engagement activities in a single entry?
No. Please enter each engagement activity separately. If multiple activities are related to each other, indicate this in the open-ended description field. For example, if a presentation at a conference led to a journal publication, a media article, and additional presentations, each of these should be entered separately. However, the description of the initial conference presentation can detail how all these outcomes are linked.

How do I report conference presentations?
Report conference presentations under “engagement activities” as “a talk or presentation or debate” and describe the activity in the open-ended field provided. If your presentation is included in conference proceedings, also report this under “publications.” A single activity can have more than one impact and therefore be reported in more than one category of Researchfish®.

How do I report a poster I presented at a conference?
Report poster presentations under “engagement activities” as “a talk or presentation or debate” and describe the activity in the open-ended field provided. If the abstract for your poster presentation is included in conference proceedings, also report this under “publications.” A single activity can have more than one impact and therefore be reported in more than one category of Researchfish®.

What is the difference between “engagement activities” and “collaborations and partnerships”?
Collaborations and partnerships denote a more formal, mutually beneficial and longer term relationship in which all members contribute towards a common goal. Collaborations and partnerships do not necessarily have to be governed by formal agreements. Engagement activities refer to the meaningful involvement of stakeholders of your research and are often one-time events.

How do I calculate in-kind financial support for a collaboration or partnership?
In-kind support refers to free-of-charge goods or services to support your research (e.g., databases, use of equipment, personnel). Use your best efforts to estimate the monetary value of in-kind supports. For example, if a partner contributed personnel support to your research group, report how much if would have cost your research group to hire someone to do the work.

How do I report the use of online resources (e.g., Alberta SPOR Support Unit, server support for website hosting)?
Report the use of online resources under “facilities and resources.” We are interested knowing what resources you have used to support your work. This includes physical facilities, databases, biorepositories as well as online resources and services such as the Alberta SPOR support unit (http://www.aihealthsolutions.ca/initiatives-partnerships/spor/), which provides access to experts and knowledge.

I submitted an annual impact report last year and now see an error message and red alerts next to the engagement activities I reported. How do I resolve these error messages?

There were several changes made this year to the Common Outcome questions for Engagement Activities. This error is likely due to the fact that questions pertaining to engagement activities with academic peers have been amended or removed (e.g., “scientific meetings, conferences and symposia” is no longer a response option). Click on each output marked with a red alert, edit the information, and reattribute it back to your award. See the following FAQs to see how you can report engagement activities with academic audiences in Researchfish: How do I report conference presentations? How do I report a poster I presented at a conference?

Can I download a copy of an annual impact report I submitted in a previous year?
No, you cannot download a copy of a report that was submitted in a previous year. We recommend that you download a copy of your annual impact report for your records each year after it is submitted.

Why is the submission notice on the right-hand side of my portfolio red (or green)?
When the submission period for Alberta Innovates – Health Solutions is open (i.e., April 1 –30), the submission notice in your portfolio will at first be RED. Red indicates that you have not submitted an annual impact report to AIHS. Once you have submitted a report, the submission notice will turn green for the remainder of the submission period. Green indicates that you have submitted an annual impact report, but the submission period is still open. If you have forgotten to report an output, you may update your portfolio, attribute the output to your award, and resubmit your annual impact report. You may resubmit your report as many times as you like until the submission period closes.

How do I know if an output has been attributed to AIHS?
In your portfolio, select your award on the right-hand side under “View” to see the outputs currently assigned to your award. By selecting a Common Outcomes category, you can see a summary list of the outputs in that category. For example, you can see in the screenshot below that two funding awards have been attributed to the award “Test5_PME Evaluator” in the “Further Funding” common outcomes category. The number “1” to the left of each description indicates that that particular output has been attributed to one award. By selecting the magnifying glass next to the output description, you can see to which awards in particular that output has been attributed.

Tips

  • Check for gaps. For example, if you state that a conference presentation led to a collaboration and a publication with your collaborators, the collaboration should be entered under “Collaborations and Partnerships” and the publication should be entered under “Publications.”
  • Check for repeated information. For example, ensure that each engagement activity is only reported once.
  • Remember that one activity may lead to more than one impact. All resulting impacts should be reported.
  • Wherever possible, highlight what your impact(s) mean to a broader audience (i.e., to the public and the government).

Have some of the questions and response options changed from 2015 to 2016?
Yes. There have been some important changes to the Common Outcomes and Additional Questions. Changes to the Common Outcomes (i.e., Common Question Set) were determined by an international user group and implemented systematically by Researchfish. These changes have been posted on the AIHS website for your reference. A summary of key changes is provided below.

 

Category Changes

Self Service Support

Publications

Engagement Activities

Spin Outs

Use of Facilities & Resources

Medical Products, Interventions & Clinical Trials

Research Tools & Methods

Research Databases & Models

Artistic & Creative Products

Software & Technical Products

Self-service tools have been added so that researchers can resolve common issues without having to contact support, including invitation resend and password reset.

Publications search has been improved with added publication repositories and improved interface.

The range of search providers has been expanded.

› To add a new publication, a single “add new entry button” has been added which will take you to a new publications search page.

› A feature was added so that hovering over the green arrows by bulk uploaded publications will display the origin of the publication information.

Engagement activities with academic peers have been removed. For example, “scientific meetings, conferences and symposia” are no longer part of this section.

 

Multiple questions and response options have been amended or removed. Some response options have been added to better capture researchers’ engagement activities.

The three questions asking about the purpose, results and impacts of each activity have been combined into one.

The response option “Commercial in confidence” has been added to the question “Enter the number of salaried people employed.”

Two questions asking the years in which a service/facility/centre was first and last used were removed from this section.

Reporting a clinical trial has been improved with added clinical trials registries.

Opportunity to report Digital Object Identifiers (DOIs) has been added.

Opportunity to report Digital Object Identifiers (DOIs) has been added.

Opportunity to report Digital Object Identifiers (DOIs) has been added.

Opportunity to report Digital Object Identifiers (DOIs) has been added.

For more information:z
Deanne Langlois-Klassen, Associate Director, Performance Management & Evaluation (PME)
Deanne.Langlois-Klassen@albertainnovates.ca, (780) 429-6892