A New Evaluation Strategy for Measuring the Returns on Investments in Medical Research
MetadataShow full item record
As part of the continued efforts of the Center for Innovation to develop new evaluation strategies, we developed a new framework and proposed this in an invited white paper for a major assessment called for by the Canadian Academy of Health Sciences. What makes this framework new is that is focuses on the treatment sector because this is where the greatest variations in both research investments and potential returns are concentrated. By focusing on the treatment process, a very fine-grained evaluation is possible, which then can be aggregated to the macro level, the level of policy makers. But since it starts with the treatment process, it can pinpoint where either too much money in research has been invested or not enough. This is a very important question for policy-makers. One critical component of this framework is that it provides precise definitions of what is the treatment process as distinct from the research process, the differences in where treatments are provided and where research is conducted, and the distinctions between the micro, meso, and macro levels of evaluation. These definitions are important because of the mismatch among them. The specific metrics of the framework are: 1. Metrics of health care impact by stage in the treatment process; 2. Metrics of research investment by arenas within the production of medical knowledge within the specific treatment sector; 3. Metrics of contributions to scientific knowledge; 4. Metrics of network gaps in the production of innovative treatment protocols; 5. Metrics of economic and social benefits of medical research. The key starting point is the treatment process, which is defined by the differences in the nature of the illness, injury or health care problem that is being treated. The metrics for this are groups according to the four stages: prevention, intake and assessment, treatment, and post-treatment including long term care. Carefully specifying the stages in the treatment process associated with a particular morbidity allows for a fine-grained set of health care impact metrics or indicators. One could make additional distinctions within these four stages. For example, one might want to distinguish between diagnosis and prognosis. In addition to the four stages of the treatment process, we have added a category, knowledge about the health care problem, because a major part of biomedical and population research focuses on the development of understanding about the health care problem that eventually can lead to either prevention or treatment. Two to three indicators are suggested for each stage. The problem of actual vs. potential benefits--an issue that plagues many evaluations--is also discussed. To place the evaluation in its proper context, the kinds of investments made in medical research, both human and capital, are classified according to the specific stages of the treatment process. This highlights gaps. Another set of measures deals with detecting gaps in the idea innovation network (Hage and Hollingsworth, 2000) associated with a specific treatment sector. This is particularly important given the presence of a valley of death between medical research and the development of industrial innovations perceived to exist, again illustrating the advantages of selecting the sector level in the health care delivery system. A special section on metrics for knowledge contributions is suggested as well, given the importance of this for most academics. In this, a special emphasis is placed on the international impact of these contributions. Metrics for economic benefits flow naturally from the specific indicators for each stages of the treatment process. Examples include value of illness days saved from decline in morbidity incidence, reduction in the costs of tests for diagnosis, reduction in the patient's costs of waiting, value of days saved in hospitalization, value of days saved in rehabilitation and after care, etc. Surprisingly, focusing on the stages in the treatment process, which would seem to involve more work, simplifies the task of specifying the specific benefits of a particular kind of research finding. Finally a number of societal benefits are indicated as well including such things as increased equality in health care and duration of life by class and gender.