Improving Our Program Through Evaluation
California taxpayers deserve to have the funds they provide for breast cancer research spent wisely. That's why the California Breast Cancer Research Program is conducting a multi-year, formal evaluation of our entire program. Evaluation helps us target research dollars where they will do the most good.
During 2002, we evaluated two of our ground-breaking grant programs: our New Investigator Awards and our Innovative, Developmental, and Exploratory Awards (IDEAs).
New Investigator Awards
Our New Investigator Awards go to entry-level scientists to carry out their own breast cancer research projects. The grants are designed to recruit high-quality new researchers into the breast cancer field. Since 1995, we've invested over $13 million in New Investigator grants.
During 2002, we conducted a survey among researchers who received New Investigator Awards between 1995 and 1999.
We learned that 95% of the new investigators we funded felt they got a chance to do breast cancer research that they would otherwise not have been able to do. In addition, 85% said the New Investigator Award allowed them to stay in breast cancer research after the award ended. While some of the scientists were already working in breast cancer research when they received their New Investigator grant, 35% were recruited from other fields.
One goal of these grants is to help new researchers develop their careers. Our new investigators reported that the grant did help them do this. New Investigator awards allowed some scientists who had been assisting other more senior researchers to conduct their own research for the first time. Others received job advancement, or learned enough about breast cancer to be able to pursue further research.
Our grants also gave some of these scientists the opportunity to launch significant research that was later funded by other agencies. Another measure of the research's significance is that peer-reviewed scientific journals published papers about the CBCRP-funded work of 65% of our new investigators.
We drew three recommendations from this evaluation. First, our former new investigators suggested that we host an annual gathering where they can exchange ideas with each other, and also with senior researchers and breast cancer experts. Second, we should provide more consultation to new investigators to help them better manage their grants and employees. Finally, we plan to evaluate this program again in four or five years, when we'll have a larger group of former new investigators to survey and we'll be likely to learn more.
IDEAs
This year we also evaluated our Innovative, Developmental, and Exploratory Awards (IDEAs). These grants are designed to push breast cancer research in promising new directions. They provide seed money to test new ideas that have a high potential for scientific payoff, but also a risk for failure.
In interviews with 35 scientists who have received IDEAs since 1995, we learned that we are meeting many of our goals.
Almost half of the scientists we funded said the results of their research confirmed their original hypothesis. Some of the projects failed, but we expected this to happen with high-risk research. Almost half of the researchers who received IDEAs also made a new discovery or developed a new tool. One study led to two new patents. Some studies have led to increased scientific interest in a new area of breast cancer research.
One of our goals for the IDEAs was to jump-start new research that other funding agencies—which generally have larger budgets —would then continue to fund. We have had some success. Our past IDEA researchers have received $16.7 million in additional funding for breast cancer research that is at least partially based on the project we funded—five times the total amount we invested in these grants.
However, the IDEAs have not done what the CBCRP hoped for most—created a major breakthrough to prevent, treat, or cure breast cancer. Perhaps time is a factor, and one of these studies will be seen in the future to have led to a breakthrough. But we also want to learn if we can improve these grants to make breakthroughs more likely. Over the coming year, we plan to ask deeper questions in our continued evaluation of our IDEAs.
Evaluation Spurs Improvement
We use formal evaluations like those we conducted this year to improve the CBCRP. Here are some of the ways we've changed our grant-making as a result of evaluations:
-
During 2000, we evaluated our Community Research Collaborations. As a result, we've engaged in a successful, ongoing, multi-year effort to increase the number of community organizations and scientific researchers collaborating on breast cancer research. These researchers investigate questions of interest to communities of California women. We've changed the awards to make them more user-friendly to people new to the research world. We've done extensive outreach to community organizations and to researchers. We've also held special training workshops to prepare interested scientists and members of community groups to embark on research collaborations.
-
This year's evaluation of the CBCRP's New Investigator Awards, combined with a 2001 evaluation of the program's postdoctoral awards, led us to develop a new award. During the evaluation, scientists who had received these grants, which are designed to increase California's pool of talented breast cancer researchers, told us, “You need to encourage people to start doing breast cancer research even earlier in their careers.” Responding to this suggestion, the CBCRP now makes two new awards, Diversity Supplements and Dissertation Awards. Diversity Supplements go to promising graduate or undergraduate students who face economic or social barriers to embarking on a career in breast cancer research. The students work under a CBCRP-funded scientific investigator. Dissertation Awards fund dissertation research conducted by masters or doctoral candidates working under breast cancer research mentors.
More Evaluations in the Future
We plan to continue evaluating our program and to keep making improvements as a result of these evaluations.
