Home > Publications > CBCRP Annual Reports — 2008 > Improving the CBCRP through Evaluation

Improving the CBCRP through Evaluation

California taxpayers deserve to have the funds they provide for breast cancer research spent wisely. That’s why the California Breast Cancer Research Program is conducting a multi-year, formal evaluation of the entire program. Evaluation helps the program target research dollars where they will do the most to reduce and end the suffering caused by breast cancer. Over the past several years, the CBCRP has evaluated several of its award types: the Community Research Collaboration awards, the Postdoctoral Fellowship awards, the New Investigator awards, and the Innovative, Developmental, Exploratory Awards (IDEAs). The results of these evaluations were used by the CBCRP’s advisory Breast Cancer Research Council to set priorities. These evaluations are available in print to the public and can also be viewed on the Program Web site.

Dissertation Awards Evaluation
During 2008, the CBCRP evaluated the Program’s Dissertation Awards, which fund research performed by graduate students as part of the requirements they complete to receive a Ph.D. degree. The evaluation found that these awards are meeting several goals they were established to achieve. Receiving a CBCRP Dissertation Award helped develop the students’ careers. The CBCRP funding was also used by the students’ mentors to leverage large amounts of additional funding for breast cancer research. However, an important goal of the Dissertation Awards is to train the breast cancer researchers of tomorrow. Only a minority—26 percent—of those who received CBCRP Dissertation Awards are staying in the breast cancer research field.

Postdoctoral Awards Evaluation
This year, the CBCRP also conducted the second evaluation of the Program’s Postdoctoral Fellowship Awards. These fellows—including graduates having recently completed their Ph.D.s, physicians continuing research activity, and individuals in transition to breast cancer research from another field—receive CBCRP financial support to obtain their postdoctoral training under a designated mentor experienced in breast cancer research. The evaluation found that these awards are meeting important goals set by the CBCRP. The majority (63 percent) of those who received these fellowships continued to be involved in breast cancer research after their fellowships ended, and a total of 84 percent were involved in some type of work related to breast cancer. One third of those receiving CBCRP Postdoctoral Fellowships used their grant to switch to breast cancer research from another field. These awards also allowed the fellows to leverage millions in additional funding for breast cancer research, assuring that the lines of inquiry they are pursuing will go forward in the future. In addition, almost three quarters of CBCRP Postdoctoral Fellows said the award gave them the opportunity to do relevant breast cancer research that they would otherwise not have been able to do.

The results from these evaluations will contribute to the CBCRP’s current three-year priority setting process, which will be completed in 2010. Previous priority-setting evaluation processes have led to major improvements in the type of research the CBCRP funds.

Evaluation Leading to Improvement

Formal evaluations are used to improve the CBCRP. Examples of changes in the program made as a result of evaluations include:

The CBCRP’s first formal evaluation of the program’s Community Research Collaborations, in 2000, led to a multi-year effort that has increased the number of community organizations and scientific researchers collaborating on breast cancer research questions of interest to communities of California women. This effort is discussed more fully in this report in the section titled “Collaborating with Breast Cancer Activists and California Communities.”
The CBCRP’s second formal evaluation of the Community Research Collaborations, conducted in 2005, highlighted a problem facing the research teams. Once they had successfully tested an intervention, they encountered difficulty applying their research results because of lack of funds. This led to the CBCRP providing a new grant opportunity, where successful research teams can apply for an additional grant to make their results available to other programs, apply their results to changing public policy, or make the public more aware of their results. The evaluation also resulted in the CRC grant amount being increased to $150,000 for pilot awards and $600,000 for full awards.
The CBCRP’s third formal evaluation of the Community Research Collaborations, conducted in 2007, led to the Program modifying the application process for these grants. Some parts of the application process were helpful to only a portion of the applicants, and these parts have been made optional.
A previous three-year priority-setting process led the CBCRP to discontinue award types that were not meeting the program’s goals. It also led to the CBCRP investing 30 percent of its funds in the Program’s Special Research Initiatives, in order to answer crucial questions about the influence of the environment on breast cancer, and to uncover the reasons why some groups in California bear more of the burden of the disease. For more on the CBCRP’s Special Research Initiatives, see the previous section of this report titled, “The CBCRP’s Strategy for Allocating Research Funds.”
CBCRP staff and the Program’s advisory council informally evaluated how CBCRPfunded research gets translated into new medications, new detection methods, new programs to support patients, policy changes, or other actions that have an impact on breast cancer. As a result, applicants for CBCRP research grants are now required to describe the steps necessary to translate their research project into action that impacts the disease. This has enabled the Program to target its limited funds toward research most likely to lead to progress against breast cancer.