Network Evaluation – Diabetes Action Canada
Co-Investigators: Drs. Ava John-Baptiste, Janet Martin, Mathieu Ouimet, Jean-Eric Tarride, Patricia Trbovich, Sunita Vohra, William Wong, Homer Yang.
The State of Network Evaluation framework will be used to evaluate Diabetes Action Canada with real time feedback built in, and monetary value attached to each outcome. [13-15] The evaluation will be shared with the Network members to promote continuous improvement of the network performance including risk assessment (risk stratification) and risk mitigation strategies for various network’s projects/initiatives. The framework evaluates a Network based on three key pillars:
Network connectivity – an essential attribute for all networks. It is important to understand if a network’s efforts to weave its members among each other result not only in open and effective channels for active knowledge exchange but also for action and impact, across two dimensions: i) membership (individuals and organizations); and, ii) structure (how connections between members are structured and what flows through those connections);
Network health – is another crucial attribute for the network impact and its ability to sustain continuous enthusiasm, commitment and engagement of its members to work together as a network to achieve shared vision, mission and goals, across three dimensions: i) resources (external funding to sustain itself); ii) infrastructure (internal systems and structures that support the network – e.g. communication, processes, regulations); and, iii) advantages (capacity for joint value creation); and,
Network results – are usually expressed as overarching goals for achieving a particular change with significant societal impact, across two dimensions: i) interim outcomes that capture network performance and progress; and, ii) intended goal/impact. 
Our evaluation, questions and methods will differ based on the stage of Diabetes Action Canada development/evolution. In general we will use a mixed-methods approach with social network analysis (in collaboration with Drs. M. Ouimet and M.J. Dogba Laval University), semi-structured interviews, observations, surveys, focus groups, results/impacts data collection and analysis and review of relevant documentary sources. [13, 16]
Return on Investment (ROI) and Internal Rate of Return (IRR) Analyses
Guided by the framework developed by the Canadian Academy of Health Sciences (CAHS), the return of investment and internal rate of return for the Diabetes Action Canada will be analyzed. The CAHS framework captures impacts in multiple domains, at multiple levels and for a wide range of audiences. It tracks and assesses the impact of research activities under the following categories: 1) advancing knowledge; 2) capacity building; 3) informing decision-making (policy-makers and individual clinicians); 4) health impacts; and, 5) broad economic and social impacts.  This framework fully captures where health research impacts can be found including the health industry, other industries, government, public information groups and it recognizes that the impacts, e.g. health and well-being, can be accomplished in many ways; through healthcare access, prevention, treatment, the determinants of health, etc.  To evaluate the specific impact of the Diabetes Action Canada SPOR Network, the CAHS indicators will be further modified to reflect specific impacts on a particular population (e.g. Indigenous, other ethnically diverse communities, women) or domain/category or audience.
Lehoux, P. and S. Blume, Technology assessment and the sociopolitics of health technologies. J Health Polit Policy Law, 2000. 25(6): p. 1083-120.
Lehoux, P., et al., Redefining health technology assessment in Canada: diversification of products and contextualization of findings. Int J Technol Assess Health Care, 2004. 20(3): p. 325-36.
Fronsdal, K.B., et al., Health technology assessment to optimize health technology utilization: using implementation initiatives and monitoring processes. Int J Technol Assess Health Care, 2010. 26(3): p. 309-16.
Fronsdal, K., et al., Interaction initiatives between regulatory, health technology assessment and coverage bodies, and industry. Int J Technol Assess Health Care, 2012. 28(4): p. 374-81.
Giacomini, M.K., et al., Using practice guidelines to allocate medical technologies. An ethics framework. Int J Technol Assess Health Care, 2000. 16(4): p. 987-1002.
Howlett, M., Governance modes, policy regimes and operational plans: A multi-level nested model of policy instrument choice and policy design. Policy Sciences, 2009. 42(1): p. 73-89.
Diabetes Canada. One third of Ontario adults with diabetes not getting needed regular eye exams. 2015 [cited 2017; Available from: https://www.diabetes.ca/newsroom/search-news/ontario-adults-with-diabetes-not-getting-eye-exams.
Jones, S. and R.T. Edwards, Diabetic retinopathy screening: a systematic review of the economic evidence. Diabet Med, 2010. 27(3): p. 249-56.
Coronado, A.C., et al., Diabetic retinopathy screening with pharmacy-based teleophthalmology in a semiurban setting: a cost-effectiveness analysis. CMAJ Open, 2016. 4(1): p. E95-E102.
Johnston K, et al., The cost-effectiveness of technology transfer using telemedicine. Health Policy Plan, 2004. 19(5): p. 302-9.
Whited, J.D., et al., A modeled economic analysis of a digital tele-ophthalmology system as used by three federal health care agencies for detecting proliferative diabetic retinopathy. Telemed J E Health, 2005. 11(6): p. 641-51.
Brent, M., R. Merrit, and K. Foley, Improving Screening Rates In At-Risk Communities: Teleopthalmology Program. 2015.
Network Impact and Center for Evaluation Innovation Framing Paper: The State of Network Evaluation. 2014. 1-24.
Fagen, M.C., et al., Developmental evaluation: building innovations in complex environments. Health Promot Pract, 2011. 12(5): p. 645-50.
Patton, M.Q., The CEFP as a model for integrating evaluation within organizations. Cancer Pract, 2001. 9(Suppl 1): p. S11-6.
Stages of Network Development. Catalyzing Networks for Social Change: A Funder’s Guide 2011, Monitor Institute and Grantmakers for Effective Organizations
Panel on Return on Investment in Health Research, Making an Impact: A Preferred Framework and Indicators to Measure Returns on Investment in Health Research. Canadian Academy of Health Sciences. 2009: Ottawa, ON, Canada. p. 134.