AUDIT OF THE NEW SAR INITIATIVES FUND

JUNE 2005

Table of Contents

• 1.0 - Introduction
• 2.0 - Objectives and Scope of Evaluation
• 3.0 - Approach
• 4.0 - Findings and Analysis of NIF Issues
• 5.0 - Conclusions and Recommendations
• Appendix A: List of Documents Reviewed
• Appendix B: List Of Individuals Interviewed From Sponsoring Federal Departments, Provinces And Territories
• Appendix C: Interview Guide: Evaluation Of The New Sar Initiatives Fund (NIF)
• Appendix D: NIF Projects Funded In 2003/04 By Sponsor And Total Contribution
• Appendix E: NIF Projects Funded In 2004/05 By Sponsor And Total Contribution

SAR Vision Statement

“A Canada where the critical importance of Search and Rescue is reflected in a multi-jurisdictional approach to promoting individual, collective and organizational behaviours that minimizes the risk of injury or loss of life while maintaining timely and effective response services.”

Interdepartmental Committee on Search and Rescue, June 2002

1.0 - Introduction

The National Search and Rescue Secretariat (NSS) has requested an evaluation of the New SAR Initiative Fund (NIF), with particular emphasis on the contribution component of NIF and how it fits within Canada’s National Search and Rescue Program (NSP). This report explains how the evaluation was conducted and presents the findings of the evaluation. The evaluation is based on the Evaluation Matrix presented in Evaluation Framework for the New SAR Initiatives Fund, January 20, 2005, prepared by Consulting and Audit Canada (CAC).

1.1 - Background to NSP AND NIF

Search and rescue (SAR) is a critical service to Canada and Canadians given the country’s size, climate, topography and population. Consequently, the resources, capabilities and cooperation of all levels of government, the private sector and volunteers are critical for an effective and economically viable SAR service in Canada. Based on 2003 data, there were approximately 8900 SAR incidents in Canada, 84 percent of which were federal SAR incidents (marine, air and humanitarian) and 16 percent provincial/territorial (Source: NSS Briefing to PCO ICSAR Member, December 2004). The latter are primarily Ground SAR incidents for which the provinces and territories have responsibility.

The focal point for all SAR activities in Canada is the NSP, which comprises the SAR activities of a variety of autonomous organizations across Canada. These organizations include government, volunteer, military, emergency services, and emergency measures agencies. Basically, the NSP encompasses the efforts and activities of all levels of government, corporate and volunteer sectors, and a vast array of organizations and programs related to providing information, applying technology, conducting research, preventing SAR incidents from occurring or saving lives of people at risk.

Representatives from federal sponsor organizations comprise the Interdepartmental Committee on Search and Rescue (ICSAR). Within the non-federal jurisdiction, the NSS works directly with provincial and territorial SAR authorities and police services to develop and standardize the quantity and quality of SAR service available to the Provinces and Territories. Supporting these major federal and non-federal activities, the NSS provides a facility for conducting major studies and reviews, maintains a client communications organization which, amongst its many activities, publishes the quarterly electronic SARSCENE magazine, stages the annual SARSCENE workshops and produces SAR prevention information. The NSS also coordinates the Canadian contribution to the COSPAS-SARSAT satellite alerting system, and it monitors the standards and maintains the Canadian registry for personal locator beacons (PLB). In general, the NSS acts as a centre for SAR coordination and promotion in Canada.

In 2002, the ICSAR approved the following NSP objectives and Program strategies, which serve as the basis for the NIF categories (described later). These objectives and strategies were subsequently endorsed by the Lead Minister for Search and Rescue (LMSAR) – the Minister of National Defence:

NSP Objectives:

NSP Strategies:

1. Partnerships: Develop and strengthen partnerships to facilitate and enhance SAR prevention and response activities.
2. Multi-jurisdictional Exercises: Participate in a program of multi-jurisdictional exercises with a focus on areas of historical problems or future concern, and/or to confirm plans and procedures for SAR response activities.
3. Volunteers: Involve and maximize, as appropriate, the use of volunteers.
4. Technology and Innovation: Encourage the identification and development of best practices, innovation and/or new technologies to support SAR prevention and response activities.
5. Interoperability: Identify types and characteristics of SAR equipment that should be compatible and/or interoperable.
6. Data and Information: Promote the collection and free exchange of information and data applicable to SAR prevention and response.

Two years after the NSS was established in 1986, the NIF was created by the Federal Government to provide funding for new search and rescue initiatives that will improve the NSP in Canada. Since then, the NIF has become a cornerstone of the NSP and contributes significantly to advancing technology and improving Canada's ability to provide economical and effective SAR response in all areas and jurisdictions. The NSS supports and promotes the activities of the NSP as a means to achieve highly effective and economically responsible search and rescue programs throughout Canada, of which the New SAR Initiatives Fund forms one element, and provides direct, independent support to the LMSAR.

According to the NIF Program Guide, the main objectives of the NIF are to:

• Enhance SAR activities by federal and provincial/territorial organizations with specific jurisdictional responsibilities.
• Promote and support other projects designed to further the objectives of the NSP.
• Communicate SAR "best practices" to all parties involved in search and rescue in Canada.

The NIF program is accessible to all Canadians with an eligible idea, through a NIF-approved sponsor. Proponents must submit their proposals through one of the following recognized NIF sponsors:

• Environment Canada -- Meteorological Service of Canada (MSC)
• Parks Canada Agency
• Department of Fisheries and Oceans (Canadian Coast Guard)
• Department of National Defence (Canadian Forces)
• Royal Canadian Mounted Police (RCMP)
• Transport Canada
• Provincial and territorial governments (operations are coordinated by the police service of the jurisdiction, e.g., the RCMP, the Ontario Provincial Police (OPP), the Sûreté du Québec (SQ) and the Royal Newfoundland Constabulary (RNC)).

With an annual budget allocation of $8.1 million, NIF provides annual funding for new projects (or initiatives). By 2004, the NIF had invested over $130 million in over 700 projects. NIF projects fall under six general categories that are based on the NSP strategies. Since 2004, two of the NIF categories/priority areas – Prevention and Response – have been re-categorized to reflect a greater degree of specificity. The NIF categories prior to and after 2004 are shown in Table 1.1, as are the NSP strategies / priority areas that appear to correspond to them:

Table 1.1 NIF Categories and NSP Strategy/Priority Areas

^* Source: NIF Results –Based Audit Framework, NSS, December 2004
^** Source: NIF Program Guide, NSS, 2004

Until 2004, projects that fell under the Response, Prevention and R&D categories were the most heavily funded, with approximately 84 percent of all funds supporting these three categories (see Table 1.2). Similarly, of the 735 projects funded between 1988 and 2004, 566 or 77 percent were projects that fell under these same three categories. At the end of each fiscal year, a Final Report communicates the results to the public, and shares lessons learned.

The NIF requires that certain reporting requirements, guidelines and policies be met in order to receive funding approval. Provinces and territories and any Canadian organization or individual from the volunteer, non-profit, academic or other non-public sector are eligible for NIF contribution funding for projects that could enhance the effectiveness of the NSP. ICSAR departments also receive NIF Funding to carry out projects to enhance their own SAR programs.

1.2 - The NIF Projects

Applications to NIF must demonstrate that they address at least one of the NIF categories. On average, NIF accepts and provides funding for between 40 and 45 new projects each year. In addition, there are usually about the same number of multi-year projects that are carried over from previous years. As mentioned in the previous section, there were 735 projects funded by NIF between 1998 and 2004. Of these, 455 projects or 62 percent fell under the Prevention or Response category and represented 64 percent of all NIF project funding during this period. The category that had the fewest projects was the Integration of the NSP, with only two percent of the projects and the funds.

On an average cost per project basis, R&D and Response projects were the largest, at $231,000 and $225,000 per individual project, respectively. During the same timeframe, the least costly projects on average were those related to Volunteers, at $112,000 per individual project. Table 1.2 provides an overview of the NIF projects that were funded between 1998 and 2004.

Table 1.2 NIF Funding By Category Between 1998 and 2004^*

^* Source: NIF Results-Based Audit Framework, NSS, December 2004

During the period under review, 48 proposals were submitted in 2003/04 and 46 in 2004/05. Of these, 43 were approved for funding in 2003/04 and 42 in 2004/05. A breakdown of the proposals and projects for these two years by sponsor appears in Table 1.3. The subsequent tables (Table 1.4 and Table 1.5) provide a further breakdown of the projects that were approved by NIF category, and the total amount of the contributions per sponsor.

On a category-by-category basis, projects related to R&D were the most numerous (11 projects) in 2003/04 and represented the largest amount of funding received from NIF at $4.5 million. Significantly, federal departments sponsored all of the R&D projects. Similarly, the category to receive the second largest allocation of funding – Prevention – primarily involved federal sponsors; only one of the seven Prevention projects in 2003/04 (and a very small one at that) was sponsored by a province. Federal-sponsored Prevention projects averaged $533,000 each; by comparison, the British Columbia-sponsored Prevention project was valued at $3,300.

Table 1.3 NIF Proposals and Approved Projects: 2003/04 and 2004/05

As illustrated in Table 1.3, in each of the years examined, the majority of all of the projects that were submitted and approved for new NIF funding were federally sponsored. In 2003/04 and 2004/05, 86 percent and 78 percent of the total NIF contributions supported federally sponsored projects compared to the 14 percent and 22 percent that supported provincial and territorial sponsored ones. On a per capita basis, the average value of each provincial-sponsored NIF project over the two-year period, represented less than one-half the average value of its federal counterpart, i.e., $127K compared to $316K.

Among federal sponsors, DFO, Environment Canada and DND received the most funds. DFO alone received 52 percent of all funds distributed to federal sponsors over the two-year period. In 2004/05, DFO’s share of NIF funds was 62 percent of the total amount approved for federal sponsors. Similarly, the value of all DFO sponsored projects represented over 40 percent of the total NIF contributions.

Generally, the average size of federally sponsored NIF projects is similar among most departments, ranging from $313K to $391K for Parks Canada, DND, DFO and Environment Canada. For the NSS, RCMP and Transport Canada, projects averaged between $92K and $247K. Projects sponsored by the NSS represented the least expensive NIF funded projects on average.

At the provincial/territorial level, fewer than half of the provinces/territories received NIF contributions in 2003/04 and 2004/05. By contrast, all federal sponsors had received some NIF funding during this period. Except for the one project sponsored by Newfoundland and Labrador, no other Atlantic province received NIF funding in 2003/04 or 2004/05. Among provincial/territorial sponsors, British Columbia represented almost one-half (13) of the provincial NIF projects that were approved, followed by Ontario (8). With the exception of the one high value project in Alberta, projects sponsored by the provinces were relatively small when compared to those sponsored by federal departments and agencies. In fact, the average cost of each Ontario sponsored project was only $17K in 2004/05.

Table 1.4 Number of Projects Approved by NIF Category and Amount of Contributions: 2003/04

Table 1.5 Number of Projects Approved by NIF Category and Amount of Contributions: 2004/05

1.3 - Report Outline

Section 2 of the Evaluation Report presents the objective and scope of the evaluation, and Section 3 presents the approach. The Findings and Analysis of NIF Issues are presented in Section 4, which elaborates on each of the issues listed in the Evaluation Matrix. The report concludes with a Conclusions and Recommendations section, highlighting the overall, general and specific conclusions that were reached by the evaluation team, as well as a number of recommendations for consideration by NSS management and other NIF stakeholders.

2.0 - Objectives and Scope of Evaluation

As part of the NSS’ ongoing need to ensure that the Government’s efforts to provide appropriate and effective funding for Canada’s SAR sector are meeting the NSP’s objectives, a need was identified to carry out a summative evaluation of the NIF. The objective of this assignment is to assist the NSS in conducting a summative evaluation of the NIF, focusing on assessing Relevance and Success issues as identified in the Evaluation Matrix in the Evaluation Framework for the New SAR Initiatives Fund, January 20, 2005.

Relevancy: Does NIF continue to be consistent with the objectives and strategies of the NSP strategies, and does it realistically address an actual need? For the purposes of this evaluation, the following question was asked of the respondents:

Is the NIF in its design and implementation consistent with the objectives and priorities of the NSP and its horizontal coordination role?

Success: Is NIF effective in achieving the SAR response, prevention and R&D objectives, within budget and without unwanted negative results? Is NIF the right approach to support Canada’s SAR stakeholders and developing innovative applications? For the purposes of this evaluation, the following four questions related to success were asked of the respondents:

1. In what way(s) and to what extent have NIF-funded contribution^* projects contributed to achieving the SAR prevention objective of influencing individuals and organizations on the assessment of risks and on the importance of acquiring and using appropriate knowledge, skills and equipment?
2. In what way(s) and to what extent have NIF-funded contribution^* projects contributed to achieving the SAR response objective of ensuring an effective SAR response (capability) in all areas of Canada?
3. Would the NIF contribution^* projects and benefits have occurred without NIF funding?
4. To what extent have NIF-funded contribution^* projects contributed to advances in SAR Research and Development (R&D), and to what extent has this R&D contributed to minimizing the risk of injury and/or loss of life and to maintaining timely and effective response services?

^* Technically, the term “NIF contribution projects” refers only to projects carried out through contributions, i.e., by non-federal entities. The NIF as a whole also includes projects that are carried out by Federal departments and funded directly, i.e., not through contributions. For purpose of this evaluation, the questions, responses and analysis relate to both types of project.

Design and delivery issues of the NIF were intended to be addressed in the audit report that is also being conducted by CAC. Nonetheless, a number of design and delivery issues have been included in this report given the results of the evaluation findings. The design and delivery issues came about as a result of the following questions: What aspects (if any) of the design and delivery of NIF should be changed to make it more successful and efficient? What have NIF sponsors in general and the NSS in particular learned from their experiences with NIF projects and the recipient organizations?

The issues identified for the evaluation and the associated performance indicators were described in the Evaluation Framework for the New SAR Initiatives Fund, January 20, 2005, and were determined in consultation with the NSS managers following an initial review of material and interviews with NSS staff and two Merit Board members. The scope of the evaluation included a document review and interviews with NIF sponsors. However, it precluded conducting any interviews with the ultimate recipients of NIF funding. The evaluation issues and the performance indicators are presented in detail in Section 4.0 of this report. The focus of this evaluation is on NIF projects that were funded during the 2003/04 and 2004/05 fiscal years.

3.0 - Approach

The approach undertaken for conducting the evaluation of the NIF comprised the following tasks:

1. Reviewing background material on the NSP, NSS and NIF. The material was either provided by the NSS, or found on the NSS or other SAR-related websites. A list of documents reviewed is presented in Appendix A.
2. Determining in consultation with NSS staff whom to interview from the 13 NIF funding sponsors and interviewing sponsor representatives. A complete list of the interviewees is presented in Appendix B.
3. Developing an interview guide for the NIF sponsors. The interview guide is presented in Appendix C.
4. Analysing the information received, determining the findings and reaching developing conclusions. Based on these findings and conclusions, providing recommendations.

The individuals interviewed represented five federal government departments and agencies, seven provinces and one territory and were pre-selected in consultation with officials from the NSS. These sponsors served as a representative sample and include both large and small entities, as well as federal departments/provinces that have had numerous NIF proposals approved and some that have received very little NIF funding. In three cases, more than one individual from the same department/province was interviewed for this evaluation. In one case in lieu of an interview, two officials of a federal department provided written responses to the interview-questions. Although no recipient organizations were consulted, at least one representative from a major funding recipient participated in an interview together with a provincial government representative. A complete list of the individuals interviewed appears in Appendix B. Interviews were conducted with the sponsors listed in Table 3.1.

Throughout this evaluation, the terms “some”, “a few”, “many”, “the majority”, and “most” are used to indicate the approximate number of sponsors – and not the number of respondents – interviewed, who support a particular point or finding. Considering that 13 of 18 sponsors were interviewed, the terms are defined as follows:

• Some/a few: Two to three sponsors.
• Many: Between three and six sponsors.
• The majority: Six or more sponsors. If provincial/territorial sponsors only are designated, four or more constitute “the majority”. For federal sponsors, three or more.
• Most: If not all 13 sponsors, very close to that number. For provincial/territorial respondents all or close to all eight provincial/territorial sponsors. For the five federal sponsors, all or close to all five.

Given that the statistics and results for NIF projects require time to be received and analyzed by the NSS, this evaluation examines NIF projects and proposals for the fiscal years 2003/04 and 2004/05. However, general information relating to earlier NIF experiences by the sponsors has also been taken into account.

Table 3.1 NIF Sponsor Organizations

^* Interview included a representative from a recipient organization.

4.0 - Findings and Analysis of NIF Issues

In this section, each of the main evaluation issues is described and the findings are presented and analyzed in terms of Relevance, Success and Design and Delivery/Lessons Learned.

4.1 - Relevance Issue

The analysis of this issue is based on the following question:

Is the NIF in its design and implementation consistent with the objectives and priorities of the National SAR Program (NSP) and its horizontal coordination role?

Findings with respect to NIF Relevance Issue

1. Regarding the relevance of NIF in terms of it being consistent with the NSP strategies and priorities, all respondents agreed that there is definite congruence of the NIF funded projects with the NSP objectives and priorities. First and foremost, the NIF promotes a national strategy for SAR across Canada. For the SAR community in general and the sponsors in particular, the NIF is an essential source of funds for SAR-related initiatives and is an important way to share information on new initiatives among SAR sponsors and proponents. Given Canada’s extensive geography, its population, the varied topography, numerous waterways, extensive coastline, and international responsibilities, a strong SAR organization in every region of the country that is both seamless and state of the art is critical.
2. Without doubt, the NIF appears to reflect the identified priorities of the NSP and, in particular, the “Technology and Innovation” priority for federal sponsors and the “Volunteers” priority for provincial sponsors. While most NIF funded projects easily fall under one of the NSP strategies/priority areas and/or address more than one NIF category, most sponsors feel that there is a need for some flexibility within NIF. Sometimes, proposals are submitted that deal with unique problems or issues that on the surface do not necessarily fit into the priorities. Over time and/or when circumstances change, these projects could have a very important impact on SAR. For example, the results of a DFO-sponsored tsunami-related project have proven to be extremely important to Canada and the world in the aftermath of the Asian tsunami disaster of late 2004.
3. Similarly, the majority of provincial sponsors raised the issue of the sustainability of SAR funding in general and felt that equipment, maintenance and training should also be eligible for funding by the NIF.
4. A view that was common among all of the sponsors (both federal and provincial) was that the NIF demonstrates the federal government’s strong commitment to SAR. The NIF provides the federal government with a national presence that it may not otherwise be capable of having, especially in the area of Ground SAR (GSAR), which is largely a provincial responsibility. The NIF allows the federal government to play a leadership role that is essential for a country with so many competing interests at the federal and provincial levels. This often helps inhibit the specific and sometimes narrow interests of individual departments and provinces when it comes to competing for limited SAR resources. Consequently, the NIF usually compels individual sponsors to consider the national implications of a project rather than “self interest”. While the NIF provides another avenue for ongoing federal-provincial communication and dealings, it also allows SAR resources to be distributed and coordinated nationally and in a relatively equitable manner. Although over three quarters (86 percent in 2003/04 and 77 percent in 2004/05) of the annual NIF funds goes to federally sponsored projects (as illustrated in Table 1.3), there is a common belief among provincial sponsors that the manner in which the NIF funds is distributed is fair and equitable.
5. The majority of sponsors agreed that the NIF leads to greater standardization of SAR-related technologies, processes and methods across different jurisdictions. For a diverse country like Canada this is essential. For there to be greater interoperability among SAR organizations and a truly seamless SAR across Canada, there is a clear understanding that communication, cooperation and knowledge transfer among sponsors (federal-federal, federal-provincial, and province to province) is essential. The NIF clearly encourages this.
6. There is a clear belief among most sponsors that the NIF has resulted in greater financial access for small recipient organizations (e.g., volunteer groups) across Canada. By having a program such as the NIF, these organizations can use the NIF contribution? to leverage funds from other sources, e.g., provinces, private sector and international organizations. In the same vain, the NIF strengthens the capability of SAR organizations and especially those small groups that have limited resources. Having a proposal approved for NIF funding and receiving the funds shows confidence in these organizations, and acts as a significant boost for the morale of the members of these groups.
7. Most sponsors understand the limitations of the NIF and the limited resources that are available. Yet, there is also a sentiment that for a relatively small investment, NIF funded projects can have far reaching SAR-related consequences nationally and internationally. Project outputs/deliverables can be shared with or potentially sold to other jurisdictions. Most sponsors recognize that investments in SAR innovation and technology are generally “high risk” and appreciate that the NIF encourages selective risk taking vis-à-vis commercial applications. Given the unusual nature of the NIF as one of the very few federal funding programs aimed at SAR and that has such direct, life-saving objectives, SAR stakeholders greatly value this initiative.
8. A number of sponsors (both federal and provincial) feel the NSS needs to be more proactive vis-à-vis volunteer groups in particular, and to develop a more direct relationship (through contact and better communication) in order to better explain and promote the NIF and consequently encourage more innovative proposals from these groups. Many sponsors (the provincial ones in particular) do not have the necessary resources to undertake this role.

4.2 - Success Issue

Before examining the various aspects of the success issue, each of the interviewed sponsors was asked how they defined “success” with regards to NIF, as well as the barriers. The definition of success for a NIF project varied from sponsor to sponsor. Here is a sample of some of the responses received from the sponsors:

1. While some definitions were quantitative, the majority were qualitative given the difficulty in collecting actual statistics on the number of Canadians who participate in outdoor activities and the number of prevented accidents / injuries/ deaths that can be directly attributable to NIF projects. Parks Canada, for example, does collect data on the number of people who enter Canada’s National Parks; Transport Canada, police services and the provinces, on the other hand, have absolutely no way of determining how many individuals are involved in water-related activities at any given time and can only make general assumptions. Similarly, there was a general acknowledgement that success related to prevention initiatives, as well as those related to R&D, are especially difficult to measure, as they require much longer periods of observation and data collection.
2. A very clear indication of success is whether lives have been saved due to a NIF project; for example, preventing problems/accidents through a change in end users’ awareness, knowledge, attitudes and behaviour, and how well they are responded to. Once again, it is very difficult, if not impossible, to determine how many peoples’ lives may have been saved or injuries prevented directly or indirectly due to a NIF project. Based on Parks Canada data for the five years ending in 2003/04, the overall number of incidents (i.e., injuries and deaths) has remained constant, although the number of visitors to National Parks has continued to increase. However, the number of potentially life threatening injuries has declined by 34 percent during the same five-year period (Source: Parks Canada Agency Annual Report: 2003-2004). While NIF projects have surely helped (and there are some individual stories that attest to this), it is impossible to confidently point to NIF projects as the sole reason.
3. A simple measure of success relates to the management of the NIF project itself. Basically, this involves determining whether the project objectives have been met and, if not, why not. Related to this is whether the project deliverables have been produced, whether they were achieved on time and within budget. If all of these occur, many sponsors would consider a project to be successful irrespective of the usefulness of the outputs or tangible benefits received by a targeted population.
4. Another measure of success that was mentioned by a few sponsors relates to the project’s reporting system. The belief is that if recipient organizations fully comply with the reporting requirements – such as the timely submission of invoices with explanations of how funds were expended – that is success enough.
5. For some sponsors, success can be measured by the extent to which NIF project results are requested by and disseminated to other jurisdictions, i.e., the lessons learned. If the results/achievements of a project are embraced by other jurisdictions (e.g., adopting/ adapting a SAR best practice), that would be considered to be a significant success.
6. An obvious and important measure of NIF success involves the level to which the NIF projects have contributed to increasing capacity in a specific SAR sector, increasing or enhancing SAR capacity in an individual jurisdiction and, most importantly, across Canada.
7. Related to the previous item, another important measure of success is whether the NIF project deliverables/outputs increase the opportunity for interoperability and standardization within the same provincial/territorial jurisdiction, regionally, nationally and possibly internationally. The more standardized SAR technology, communications and methods become due to NIF projects, the greater the success level.
8. Given the difficulty in quantifying NIF successes in such SAR categories as prevention and R&D, an indirect but very valuable measure of success is the extent of any positive feedback that is received from recipients and other SAR stakeholders, particularly end-users of NIF project outputs. Anecdotal evidence of success provided by these groups is usually collected by the sponsors and is taken into account when determining how effective a NIF project outcome has been on end-users.
9. Sustainability could be considered as a potential measure of success. For example, when there is self-financed, on-going training of SAR trainers as a result of a specific NIF project output, that would be considered to be a significant success.

Barriers to success: As with any initiatives of this nature and despite the many successes of NIF, obstacles/issues arise that sometimes prevent the initiative from reaching its full potential. Based on the interviews conducted with the sponsors and CAC’s observations, the following appear to be barriers to greater success of NIF:

1. Given the variety and level of SAR knowledge of the participants involved in the different SAR activities, there is often a “disconnect” between the SAR professionals and the non-professionals with whom they work, and their understanding of the requirements for a NIF proposal. Similarly, whenever there are a variety of stakeholders involved in a proposal (especially the larger ones), the interests and priorities of each stakeholder may not be the same.
2. Since so many of the NIF proposals are expected from and generated by SAR volunteer organizations, there may not be as much appreciation or understanding for the administrative burden that the application process causes to this sector. Similarly, the volunteer groups themselves may not be well versed in how to properly prepare NIF proposals. The amount of time volunteers need to dedicate for preparing a NIF proposal is often quite considerable when that time could be spent doing actual SAR work.
3. Related to the previous barrier, is the general paper burden placed on both the recipients and the sponsors when preparing NIF proposals as well as reporting requirements, such as quarterly reports. There is evidence that some small voluntary groups do not submit proposals due to the amount of paperwork involved and the inexperience that they have in doing the paperwork. Similarly, and directly linked to their limited resources, some sponsors admitted that they might not be as demanding as they could in pushing recipients to report more fully on details of projects and the outcomes resulting from those projects. Together, these act as potential barriers to NIF projects as they may (a) inhibit receiving well thought-out NIF proposals from potentially deserving volunteer groups involved in SAR, and (b) prevent sponsors from demanding and receiving well documented NIF project reports from recipients.
4. Many smaller sponsors (both federal and provincial) simply do not have the human or financial resources to dedicate to assisting NIF proponents develop and prepare proposals. This may preclude a larger number of proposals (and better written ones) from being submitted for NIF funding from within some jurisdictions.
5. Some recipient organizations and sponsors have deficient project management practices. This is largely due to either a lack of training of the volunteer groups and/or limited resources.
6. With the exception of the annual meeting, there does not appear to be any formalized mechanism to regularly share in-depth information on results of NIF projects among sponsors as required by NIF. The NSS website provides a limited amount of information related to results.
7. Some concern was expressed by several sponsors (federal and provincial) that a potentially harmful issue is the diversion of limited NIF funds for proposals that do not meet basic criteria, e.g., equipment for urban SAR.
8. A few federal departments raised the issue of the reluctance that sometimes exists among members of the senior management cadre in some sponsor departments towards SAR Technology and Innovation.
9. In a few instances, there have been cases of recipient organizations trying to financially profit from NIF project deliverables that they have produced when the project deliverables were meant to be freely distributed to other SAR groups. One example is of an organization that sold copies of a software it developed with NIF funds, rather than provide the material to voluntary groups free of charge. Some sponsors believe that not-for-profit recipient groups should not be permitted to profit from their deliverables in this manner.

Findings with respect to NIF Success Issue # 1:

In what way(s) and to what extent have NIF-funded contribution^* projects contributed to achieving the SAR prevention objective of influencing individuals and organizations on the assessment of risks and on the importance of acquiring and using appropriate knowledge, skills and equipment?

1. With the SAR prevention projects that were reviewed, there appears to be congruence of the reported project results with the original proposal objectives. However, most sponsors also stated that the success of “prevention” projects is difficult to measure, as the results are less tangible than those for response projects and they may take a long period of time (perhaps up to 5 years) to determine if there has been an impact. For example, there is no realistic way to measure how many accidents or deaths may have been prevented due to NIF projects compared to the frequency of incidents. Only a few sponsors/recipients are able to track these kinds of statistics. There does not appear to be any significant monitoring of project results beyond the actual outputs and, as one respondent described how the results of prevention projects compare, "it is possible to see the output, but not the outcome".
2. A couple of sponsors did give some examples that may be attributable to NIF prevention projects. DFO for example, noted that the number of maritime incidents over the past 10 years has remained the same, but the number of lives lost has decreased. This is a success but could be due to a variety of factors, including some NIF prevention projects. Similarly, according to Parks Canada, the number of visitors to National Parks has continued to increase over the past 10 years, yet the numbers of incidents that occur are essentially stable.
   
   It was also noted that there has also been anecdotal evidence of success thanks to the feedback received from recipient organizations that deal with the public. For example, the Canadian Avalanche Association has provided numerous examples of positive feedback from users in British Columbia that clearly shows that NIF-sponsored prevention projects do work. The question of attribution remains.
3. Generally, sponsors (both federal and provincial) that received a greater portion of NIF funds were found to have more resources dedicated to NIF, including FTEs, dollars and resources to work with recipient organizations. Consequently, a greater number of proposals are submitted to NIF through these sponsors and a greater number are approved for funding. Thanks to the editorial/administrative support of the “NIF-equipped” sponsors, these proposals are more likely to be better-written and well thought out in terms of budgets and timeframes. Many smaller sponsors (both federal and provincial) simply do not have the human or financial resources to dedicate to NIF that could be used to assist proponents develop and prepare proposals.
   
   Although several provincial sponsors stated that they would welcome additional NIF funding for supporting proponent education and administration of NIF proposals (at either the provincial or NSS level), they understand that NIF simply cannot address all of the SAR gaps that exist between provinces. Some provincial sponsors noted that they sometimes try to fill the SAR disparity gap by directly consulting with provincial counterparts for information and advice on SAR best practices and sign province-to-province agreements on SAR-related knowledge transfer.
4. Some examples of successes in the SAR “prevention” area largely thanks to NIF are:
   
   
   • Avalanche projects that were once the domain of western Canada now exist in Newfoundland and Québec. This has resulted in significant program involvement and awareness. A number of European countries are now looking more closely at Canada’s extensive avalanche SAR experience and learning from it. As prevention initiatives and SAR measures increase and incidents are reduced, Canada as a tourist destination is enhanced. Canada’s SAR prevention capacity is an excellent marketing tool for high-risk mountain sports and can help boost related tourism.
   • There is a widespread sentiment that prevention-related material and information that is being produced is being broadly disseminated to individuals and organizations that might be exposed to related risks. For example, several sponsors cited the “Hug a Tree” program as a very successful program that is getting out to young people across Canada, through schools and youth organizations. Similarly, individuals who are involved in new high-risk sports appear to be taking precautions (e.g., wearing of life jackets) thanks to on-going prevention education over the years – directly or indirectly funded by NIF.
   • There has been ongoing development and expansion of weather dissemination systems such as Environment Canada’s Weatheradio network in the North where delivery of weather information is now provided in Inuktitut. The fact that this information is broadcast in a language that locals can now understand is a major success. Given the very harsh weather conditions in which they live and the vast distances between communities, one can only reflect on the positive consequences.
   • The Ice Floe Edge Break-up Warning Service that the Canadian Ice Service developed with NIF funding has been very popular with the Inuit communities where it was implemented. The MSC has received numerous requests to expand the service to other Arctic communities. No objective performance measures are in place but it is assumed that the interest in the service is causing Inuit to be more aware and more informed of the dangers at floe edge.

^* Technically, the term “NIF contribution projects” refers only to projects carried out through contributions, i.e., by non-federal entities. The NIF Program as a whole also includes projects that are carried out by Federal departments and funded directly, i.e. not through contributions. For purpose of this evaluation, the questions, responses and analysis relate to both types of project.

Findings with respect to NIF Success Issue # 2:

In what way(s) and to what extent have NIF-funded contribution^* projects contributed to achieving the SAR response objective of ensuring an effective SAR response (capability) in all areas of Canada?

1. With respect to the NIF projects that focus on the SAR response element, there appears to be congruence of the reported project results with the original proposal objectives. Although there is no “hard” evidence, all sponsors felt that in most cases, things were done appropriately. Overall, NIF has increased the knowledge, ability and capacity of SAR responders. Projects appear to be producing project deliverables/outputs that increase the opportunity for interoperability and standardization within the same jurisdiction, nationally and, in some cases, internationally. Better equipment, whether in the form of night goggles or new radios, has definitely improved the capability of responders.
2. In some provinces, the administrative capacity of volunteer organizations that are involved in SAR response, like those involved in SAR prevention, seems too weak and requires some support. While these volunteer organizations may do excellent work as SAR responders, they have little training and understanding of the administrative requirements related to NIF applications and follow-up.
3. Most provincial sponsors and some federal ones raised the issue of the sustainability of volunteer organizations involved in SAR response and the difficulty – due to costs – associated with providing ongoing training. There appears to be a need for greater emphasis on training in volunteer organizations across the country. There is also the spectre of volunteer organizations receiving state of the art SAR equipment, but insufficient funding (NIF or other) to maintain and replace the equipment as required, and training the volunteers on how to use this equipment.
4. Several sponsors noted that there is some evidence of economic spin-offs thanks to NIF contributions? and especially from the technology and innovation projects. In one case, a project that did not result in a useful technology for SAR has become successful for boating safety in general ?UHF radar radio communications?and is being purchased by the United States from a firm in Newfoundland and Labrador. Other examples are described in the “Findings with respect to NIF Success Issue # 4” section below.
5. Some examples of successful response projects funded by NIF are:
   
   
   • Parks Canada has been able to export its multi-jurisdictional ‘Table Top’ simulation exercise project to remote areas of the country.
   • Greater standardization and interoperability. For example, the Saskatchewan “Train the Trainers” used a national package developed by ERI Canada, which adheres to national standards. Similarly, the need for common communications standards (radio frequency) in New Brunswick, as well as in the NWT.
   • To enable SAR organizations to make well-informed decisions during response, better forecasting tools and models have been developed and research activities related to severe weather have been conducted.

^* Technically, the term “NIF contribution projects” refers only to projects carried out through contributions, i.e., by non-federal entities. The NIF Program as a whole also includes projects that are carried out by Federal departments and funded directly, i.e. not through contributions. For purpose of this evaluation, the questions, responses and analysis relate to both types of project.

Findings with respect to NIF Success Issue # 3:

Would the NIF contribution^* projects and benefits have occurred without NIF funding?

1. In the majority of cases that were examined, the NIF contribution?* projects would simply not have occurred without the NIF funding. Although most sponsors would continue to be involved in some aspect of SAR, their funding of projects would be greatly diminished; this is especially true in the provinces/territories and in at least one federal department. NIF funding is critical for SAR in some provincial/territorial jurisdictions and they feel that the federal government does not realize how crucial NIF funding is or the broad impact that it has on the SAR community within the jurisdiction. Similarly, some larger federal sponsors admitted that they might focus on other projects if NIF did not exist, e.g., solely large projects or projects that emphasize departmental interests only.
   
   
   • The first-responder training project in New Brunswick simply would not have happened.
   • The “Visitor Risk Management Process” project could not have happened. Now it is being looked at by the United States and New Zealand and has been taken-on as an internal Parks Canada activity.
2. Without NIF funding, volunteer organizations would need to do more fund-raising on their own. According to a few provincial respondents, there is a feeling that in some jurisdictions, NIF funding may appear to inhibit volunteer groups from seeking funding elsewhere. Some groups may find it simpler to focus on NIF proposals only, rather than spend time and effort searching for and applying to other sources of funding.
3. While the provinces are not likely to increase their funding for SAR, federal NIF funding helps leverage provincial funding. There is a feeling that NIF can more easily fund larger projects that individual sponsors cannot on their own. It is easier for some sponsors to provide in-kind contributions rather than cash.
4. Sponsors mentioned a number of direct benefits that have occurred thanks to the NIF contributions. For example, they noted that without NIF:
   
   
   • Volunteer search teams would not be as well trained to work with police departments.
   • DND would only fund military-related SAR. NIF binds everyone (civilian and military) together and civilian experiences benefit military SAR operations.
   • SAR best practices would not be shared among jurisdictions. However, there would be more of an effort to partner between/ among jurisdictions.

^* Technically, the term “NIF contribution projects” refers only to projects carried out through contributions, i.e., by non-federal entities. The NIF Program as a whole also includes projects that are carried out by Federal departments and funded directly, i.e. not through contributions. For purpose of this evaluation, the questions, responses and analysis relate to both types of project.

Findings with respect to NIF Success Issue # 4:

To what extent have NIF-funded contribution^* projects contributed to advances in SAR Research and Development (R&D), and to what extent has this R&D contributed to minimizing the risk of injury and/or loss of life and to maintaining timely and effective response services?

1. There is a general view among sponsors that it is difficult to assess results with R&D projects in the short and medium terms. They are usually quite expensive and there are numerous unforeseen variables. Respondents felt that there really needs to be some leeway when assessing these types of projects. Often, it is difficult for the Merit Board to properly assess the merits of an R&D proposal due to its technical complexity. The issue of whether R&D proposals can be fully assessed without independent scientific experts is of concern to several sponsors.
   
   Similarly, some sponsors do not like the idea that R&D projects use-up large chunks of limited NIF funds at the expense of smaller proposals. This reduces the availability of NIF funds for a greater number of smaller (and less scientific) projects that could produce important SAR benefits in the short-term.
2. While some larger sponsors have developed appropriate new SAR technologies and applied them, other sponsors steer clear of R&D projects due to problems in the past. According to DND, R&D represents approximately 95 percent of the NIF projects it sponsors. Given the high priority of R&D at DND and the presence of the Defence Research and Development Canada (DRDC), the high percentage of NIF funds that go to R&D is not surprising. What is somewhat of an issue based on the comments of several respondents is that significant amounts of NIF funds are allocated to R&D projects when those projects could conceivably be funded by the DRDC. That would leave additional NIF funds for non-R&D projects that are often difficult to fund. In addition to DND, other sponsors have R&D agreements with foreign government SAR agencies. For example, the CCG has R&D agreements with the United States and United Kingdom Coast Guards.
3. From the responses received, the most successful R&D projects have been those involved in communications. Communications (through common frequencies) among different SAR stakeholders and between civilian, police and military have been greatly improved. New radio equipment and associated technologies are leading toward greater standardization across Canada and are improving communications between SAR organizations within a province and between jurisdictions.
4. Examples of other successful R&D projects:
   
   
   • Although the Rapid Mount Airborne Sensor System (RMASS) project was cancelled due to a change in scope, the technology developed has been so successful it can/will be adapted by the United States Coast Guard for installation on their aircraft for Homeland Security purposes. This will help Canadian companies sell the technology and reap economic spin-offs.
   • Cold-water exposure testing was conducted by the DRDC. The United States Coast Guard, as well as other international organizations, will buy the software.
   • • The GPS based self-locating buoy that measures ocean drift was developed and tested by a small Nova Scotia firm. This small project was so successful that DND is now buying these buoys from the Halifax firm, as are some other countries.

^* Technically, the term “NIF contribution projects” refers only to projects carried out through contributions, i.e., by non-federal entities. The NIF Program as a whole also includes projects that are carried out by Federal departments and funded directly, i.e. not through contributions. For purpose of this evaluation, the questions, responses and analysis relate to both types of project.

4.3 - Design and Delivery/Lessons Learned Issue

Although the NIF audit report being conducted by CAC provides more in-depth analysis of the issues related to the design and delivery of the NIF, several issues were brought to the attention of the evaluation team that should be considered in conjunction with the issues raised in the audit report. Overall, most sponsors felt that NIF is well administered, that there is a clear business cycle, and the NSS is doing what it is mandated to do. While these issues are not addressed in any detail in this evaluation, other findings that relate to the design and delivery of NIF are highlighted, and include the following:

1. There is a question of whether the number and size of proposals being funded is appropriate. Some sponsors, including a major federal sponsor, feel that there is a need to have a clearer focus on the most urgent national priorities and only provide funding for those proposals that support these priorities. This focus could change from year to year. Any leftover funds could then be distributed to other deserving proposals that support less crucial priorities. Many sponsors also raised the issue of the time that is being spent by Merit Board members in reviewing lengthy proposals and proposals that do not necessarily comply with NIF proposal requirements. These sponsors indicated that the NSS should ensure that this does not happen by having better prescreening of proposals, either by the sponsors or the NSS.
   
   Sponsors were also divided on the issue of whether the NIF is funding too many or too few projects. While a few sponsors feel that the NIF funding is being spread too thinly, others (both provincial and federal) feel that most of the funds are going to a few sponsors that emphasize larger and more expensive R&D projects at the expense of fewer, smaller, and potentially more successful, SAR projects. All sponsors clearly recognize that NIF has limited funds to go around and that it has to be very selective in how these funds are distributed.
2. A majority of sponsors felt that the NIF project approval process is too slow and consequently often prevents projects from starting on time and reduces the actual annual duration of projects to 10-11 months. In future, some sponsors have said that they will encourage their proponents to submit multi-year project plans that would allow them to continue their work into a second year – i.e., the 12th month would actually be the first month of the second year of a project. This also makes it very difficult for sponsoring agencies to plan their resource commitments and assign NIF project-related duties to staff members in a timely manner.
3. Several sponsors noted that a weakness of the NIF is the lack of a built-in process that would provide regular annual funding for on-going activities initiated by NIF – i.e., post-project completion, operational funding. This would help prevent some projects from suddenly ending before achieving their full potential. However, as the CAC audit report states, on-going funding for the contribution portion of the NIF funding is not permitted. The NIF rules stated in the NIF Guide also do not allow funding of ongoing projects.
4. This lack of continuity in Merit Board members has resulted in less corporate knowledge being transferred, which, in turn, can reduce the effectiveness of the Merit Board and the quality of the information that can be shared.
5. Given the highly technical nature and value of most R&D submissions, a few sponsors (federal and provincial) lamented the fact that there is very little independent expertise brought-in to provide the Merit Board with unbiased scientific reviews of R&D proposals. Often, Merit Board members approve these large projects without fully understanding the technical aspects of the submissions.
6. A few federal sponsors criticized the MOUs that are signed between the NSS and provincial sponsors for being too weak. There is a feeling that the MOUs do not provide clear guidelines on roles and responsibilities, which could have very serious legal implications later on. The CAC audit report clearly states that MOUs for the contribution portion of NIF are not in accordance with the Treasury Board’s Transfer Payment Policy.
7. As mentioned previously, most sponsors (federal and provincial) feel that the administrative requirements by NIF on volunteer groups are especially burdensome. The application process, the forms, the formatting, and the cash flow structure may be too complicated for smaller volunteer organizations to handle. Some better-resourced sponsors assist proponents with their NIF applications and cash flow concerns. Sometimes, this burden is increased when the NIF and the provincial sponsors have different sets of reporting and financial rules (i.e., federal and provincial guidelines). Consequently, this frequently results in two sets of bureaucratic requirements.
8. Given the large numbers of volunteers involved in SAR across Canada, their high turnover and the central role they play in delivering SAR services, there is concern about the ongoing training of volunteers and the funding that is needed. Provinces, in particular, feel that this is an area that NIF should be more closely looking at and supporting. In British Columbia alone, there are 93 volunteer groups involved in SAR and 4,700 registered SAR volunteers.
9. When compared to the other federal program that provides some funding for SAR-related projects, the Joint Emergency Preparedness Program (JEPP), sponsors were of two opinions. While some felt that the NIF process was more complicated than the JEPP, others felt that the administrative requirements of NIF and the JEPP were very similar and that neither was better or worse in that regard. However, most sponsors preferred NIF because it does not require automatic cost sharing and accepts in-kind contributions, which the JEPP does not. The CAC audit report cautions that in-kind contributions need to be clarified and more closely monitored to more accurately reflect Treasury Board policies and guidelines.
10. Several sponsors questioned the lack of ongoing communication between the NSS and NIF proponents. Rather than wait for sponsors or others to communicate with potential NIF proponent groups, the feeling (especially among provincial sponsors) is that it would make more sense for the NSS to do so directly in order to have more timely information that is shared with all groups irrespective of where they are located or the resource capability of their sponsor.
11. In terms of how well NIF information is shared and communicated, all sponsors generally feel that the NSS is doing a relatively good job of providing project results and general information about the NIF through its website, the online SARSCENE magazine, and the annual SARSCENE workshops. All sponsors consider the project information listed on the website to be very useful, especially for identifying the projects that have already been funded. Several provincial sponsors noted that just knowing what is being funded in other jurisdictions acts as a significant learning opportunity for volunteer groups. This also ensures that identical proposals are not re-submitted by organizations in other jurisdictions.
    
    Although the information on the website is useful, there is a general feeling (by many provincial sponsors and several federal ones) that the NIF project information listed on the website is somewhat limited and “passive”. Federal and provincial sponsors both believe that, as Canada’s SAR coordination agency, the NSS could improve the manner and extent to which NIF project results are communicated to the SAR “audience” -- nationally and internationally. As one federal respondent noted: “the NSS needs to find out who uses and reads the results and make a better effort to reach them”. Another federal respondent stated that, “there needs to be more national exposure without being too passive”.
12. A few federal sponsors raised their concern about the increasing division between Canada’s urban and rural SAR requirements and were uneasy about the future implications on NIF and the availability of sufficient resources to satisfy non-urban SAR needs. The implication is that given the higher costs associated with the training of and provision of equipment for emergency responders in urban areas, a smaller share of the NIF financial pie would be available for their widely dispersed rural counterparts.
13. One federal respondent criticized NSS’ tendency to sometimes refer projects to other sponsors by asking them to sponsor a proposal that may not necessarily have 100 percent merit, let alone fit with the activities that the particular department undertakes. For example, DND was once asked to sponsor a non-military “software for radio” project to improve interoperability.

5.0 - Conclusions and Recommendations

As a result of the evaluation, five general categories of conclusions have been identified. For the purposes of this report, the conclusions are listed by category but not necessarily in order of importance.

5.1 - The Voluntary Sector

The SAR voluntary sector is a key component of Canada’s SAR capability. In some parts of the country, the administrative burden placed on them is high while their ability to comply may be questionable. This may require some additional tools (e.g., application guidelines, reporting guidelines) and financial resources. Generally, the voluntary sector lacks a strong administrative infrastructure. The sponsors that have staff dedicated to NIF are able to provide considerable support to NIF proponents; those that do not have the dedicated staff can only provide a bare minimum of support. This also raises the concern that proposals that are better written or groups that have better administrative abilities will receive more NIF funds than those that may have more innovative proposals.

Given that volunteers are the backbone of Canada’s SAR community and the demands placed on them, the NIF application process and subsequent reporting should not be as onerous as it appears to be at present. Additional NSS/NIF and/or sponsor support and communications may be needed to alleviate this deficiency. Due to the weak administrative infrastructure of many recipient organizations, they should continue to work through the existing sponsors; the financial accountability of sponsors remains key. Therefore, CAC recommends that:

1. NSS initiate discussions among all NIF sponsors (including direct dialogue with small recipient organizations) to determine how best to remedy the issues of administrative support and guidance, and ensure that small groups in particular are not prevented from applying for NIF funding due to perceived and actual administrative requirements.

5.2 - Research and Development

Given the large scale and technical/scientific complexity of most R&D proposals, there is some concern that the Merit Board and its members are not receiving sufficient independent scientific advice to allow them to make well thought-out decisions on the merits of these proposals. There is a feeling that there could be better provision of outside (and unbiased) expertise to support the Merit Board as required. There may also be room to have slightly different NIF funding guidelines and resources for R&D and non-R&D proposals. Therefore, CAC recommends that:

2. NSS ensure that a mechanism is in place to always provide, either prior to or during the Merit Board meeting, third-party reviews and expertise for any highly technical or scientific proposals that are received by the Merit Board. For example, separate Merit Board meetings could be held to review R&D proposals and non-R&D proposals so that third party experts are invited to the Merit Board session during which their expertise is required.

5.3 - Administration of NIF

There is concern that the time lag between the arrival of NIF proposal submissions, their approval and the official announcement is unnecessarily spread-out. The NIF project approval process appears to take much too long which, in turn, causes a chain reaction of considerable delays at the sponsor level and among the funding recipients. This prevents projects from starting on time and forces recipients to cut back on the originally stated deliverables and/or delivery dates. Therefore, CAC recommends that:

3. NSS immediately review the timeframes and approval processes that are in place to determine how best to reduce the NIF project approval period, from start to finish, and to be capable of announcing new approved proposals before the end of March.

5.4 - Knowledge Transfer

There is a need for regular and more focused communication between the NSS and all SAR stakeholders, and especially with potential NIF proponents: volunteers, private sector and educational institutions. SAR/NIF successes do not appear to be well publicized within the SAR community nor amongst Canadians in general. By not fully publicizing these successes, the federal government may not be getting as “big a bang” for its NIF investment (i.e., through greater awareness of Canadians across Canada and more support for SAR initiatives in the upper echelons of some federal departments). Having one SARSCENE conference per year does not appear to be enough. Similarly, if a project is not advancing as it should, waiting until the end of the year may be too late to make adjustments to the terms of reference or suggest other changes that could help set the project on course. Therefore, CAC recommends that:

4. NSS initiate additional forums throughout the year that bring together sponsors and NIF project proponents to discuss and share experiences and information on past, current and future NIF projects in an open manner.
5. NIF require mid-term project reports be submitted by all sponsors detailing the status of each project they have sponsored.
6. NSS endeavor to better publicize NIF successes to Canadians through articles in the mainstream media and in conjunction with other federal promotional material.

At the provincial/territorial sponsor level, there is a clear lack of continuity of representation on the Merit Board, and a lesser degree of involvement in NIF-related issues in general. These may be other causes for the lack of in-depth knowledge of the “inner workings” of NIF by some provincial and territorial sponsors. Consequently, some provincial/territorial sponsors may not be as effective representatives of the interests of their jurisdiction as they could be. Therefore, CAC recommends that:

7. NSS immediately review the timeframes and approval processes that are in place to determine how best to reduce the NIF project approval period, from start to finish, and to be capable of announcing new approved proposals before the end of March.

To relieve the increased burden on NIF and its limited funding base, there is a potential for the NSS to encourage more province-to-province communication and information sharing of best practices. Given that the NIF Guide implies that NIF project information will be shared and given the numerous projects that have been funded by NIF over the years and the associated SAR knowledge across Canada, this may also reduce the number of proposals that are submitted for similar undertakings and free up funds for other initiatives. Therefore, CAC recommends that:

8. NSS website provide more detailed SAR best practices (collected from the sponsors) based on NIF project results, and the contact details of the local experts.
9. NSS encourage sponsors to undertake more joint NIF projects that will encourage a greater number of joint efforts between and among different jurisdictions and promote the idea of limited direct funding from sponsors that can afford to do so, for small knowledge transfer projects.

5.5 - Policy “Creep”

Some provincial sponsors may not fully understand the parameters of their role and this is partly attributable to lack of clear guidance as exemplified in supporting administrative tools. There may be some justification for separate requirements for provincial and federal sponsors and clearer, more precise objectives for NIF. Given the potential for future legal or financial claims against the Crown, new guidelines and MOUs may need to be developed if existing documents are not considered to be satisfactory. Therefore, CAC recommends that:

10. NSS review all existing NIF administrative and management guidelines and MOUs and ensure that they are clearly worded and the roles and responsibilities of the NSS and the sponsor are clearly defined.

The growing separation between urban and rural SAR requirements is an issue that will continue to widen and requires a better understanding of how NIF contributions can be most effective.

The same is true for the differences between federal and provincial sponsor requirements. For there to be a truly seamless SAR across Canada, there may be a need to reconsider the design of NIF to better focus on the evolving needs of its clients/stakeholders. Therefore, CAC recommends that:

11. NSS invite sponsors to discuss how the present design of NIF can be improved to more effectively meet urban and rural SAR requirements and whether any significant design changes are needed, take the lead in determining federal SAR responsibilities in urban settings and introduce new policies that clearly explain what they are.
12. The Merit Board and NSS more closely monitor and examine those NIF submissions that focus on urban SAR projects to ensure that they truly fit the NIF guidelines and are not diverting funds away from more urgent projects.
13. NSS review all funding programs that support SAR initiatives in municipal/urban environments (e.g., Emergency Preparedness Canada, and provincial sources) and ensure that there is no overlap with NIF.

5.6 - Other Conclusions

A final conclusion is that while all of the sponsors feel comfortable with NIF and believe that it supports the NSP strategies, the evaluation has found no clear mechanism to link each of the NSP strategies to the NIF categories. The NSP strategies and NIF categories are not identical as illustrated in Table 1.1, yet the available data relates directly to the NIF categories only. Consequently, it is very difficult to measure how well each NIF-funded project and category fit on an NSP strategy by strategy basis. While the assumption is that NIF categories and projects are consistent with the NSP strategies, there is no way to do any accurate mapping.

14. NSS consider re-aligning the NIF categories so that they are more closely linked to the NSP strategies/priority areas.

Appendix A: - List of Documents Reviewed

• NSS Results-Based Audit Framework
• NIF Results-Based Audit Framework, National Search and Rescue Secretariat, December 2004
• New Search and Rescue Initiatives Fund Program Guide, National Search and Rescue Secretariat, 2004
• Charting the Future, National Search and Rescue Secretariat, 2003
• 1999/2000 Effectiveness Review
• National SAR Program Plan 2005
• NSP Accountability and Risk Management Model
• Strategic Directions: National Search and Rescue Program – 2004-2008, ICSAR presentation, December 2003
• NSS Level 1 Business Plan 2005-06, NSS, 08/12/2004
• National SAR Secretariat – Impact Assessment 2005-06
• New SAR Initiatives (NSS Web Document)
• NSS Briefing to PCO ICSAR member, December 2004
• A sample of project reports
• Lists of approved NIF projects for 2003/04 and 2004/05

Appendix B: - List of Individuals Interviewed from Sponsoring Federal Departments, Provinces and Territories

1. National Defence:
   Major Perry Kurzynski, Director, Air Requirements
2. Transport Canada:
   Peter Dudley, Safety Programs, Strategies and Coordination
3. Parks Canada:
   Perry Nilsen, Acting Chief, Ecosystems Protection, Ecological Integrity Branch
   Michel Villeneuve, National Public Safety Specialist
4. Environment Canada:
   Michael Crowe, Policy and Corporate Affairs, Meteorological Service of Canada (Collected written responses from several MSC Managers)
5. Fisheries and Oceans Canada:
   Joanne Clouatre, Acting Manager, Search and Rescue
6. Newfoundland and Labrador:
   Ralph Alcock, Assistant Deputy Minister, Public Protection & Support Services
   Harry Blackmore, President, Newfoundland & Labrador Ground SAR Association
7. Nova Scotia:
   Craig Maclaughland, Executive Director, Nova Scotia EMO
8. New Brunswick:
   Ernie MacGillivray, Director, New Brunswick EMO
9. Quebec:
   Capt. René Marchand, Responsable Des Spécialistes, Sûreté Du Québec
10. Ontario:
    Sgt. Don Webster, Emergency Response, Ontario Provincial Police
11. Saskatchewan:
    Kevin Roche, Director, Saskatchewan Emergency Management Organization
    Colin King, Corrections and Public Safety (CPS)
    Carl Fiske, CPS, Saskatoon Frontline Operations for NIF
    Richard Peach, Department of Justice, Regina
12. British Columbia:
    Jim McAllister, BC Provincial Emergency Program
13. Northwest Territories:
    Kevin Rowe, Emergency Measures Program Coordinator

Appendix C: - Interview Guide: Evaluation of the New SAR Initiatives Fund (NIF)

The priorities for the New National SAR Program (NSP) are guided by its approved vision, objectives and program strategies. The NSP vision and objectives are:

1. Vision Statement: A Canada where the critical importance of Search and Rescue is reflected in a multi-jurisdictional approach to: promoting individual, collective and organizational behaviours that minimize the risk of injury and/or loss of life, and maintaining timely and effective response services.
2. Response Objective: To ensure an effective SAR response (capability) in all areas of Canada.
3. Prevention Objective: To influence individuals and organizations on the assessment of risks and the importance of acquiring and using appropriate knowledge, skills and equipment.

To enhance and strengthen the NSP, the National Search and Rescue Secretariat (NSS) is conducting an evaluation of the New SAR Initiatives Fund (NIF), focusing on the assessment of its relevance and success. The underlying objectives of the NIF are to:

1. enhance SAR activities by federal and provincial/territorial organizations with special jurisdictional activities;
2. promote and support other programs designed to further the objectives of the NSP; and
3. communicate SAR best practices to all parties involved with SAR in Canada.

Consulting and Audit Canada (CAC), a Special Operating Agency of Public Works and Government Services Canada, has been engaged by the NSS to conduct this evaluation.

Relevance

1. Are you aware of the six NSP priority areas expressed as the six NSP Strategies?
2. What are the main NSP Strategies (i.e., Partnerships, Multi-Jurisdictional Exercises, Volunteers, Technology and Innovation, Interoperability, and Data and Information) that your organization focuses on? What are the main priority areas of SAR that your organization has sponsored for 2003-04 and 2004-05? Are they consistent with NSP strategies?
3. Should the NIF be funding projects outside the six priority areas identified by the NSP? What might these areas be?
4. Has the NSS/NSP been distributing the results from the NIF funded projects? Has this information been of help to you and your organization?
5. Why should the government have a program like NIF?

Success

6. How would you define success with your NIF-funded project(s)? In terms of technical success of deliverables? In terms of project management success, i.e., did the project managers deliver the deliverables on time and within budget, were reports provided to you on a regular basis in accordance with the terms and conditions of disbursing the funds?
7. For the NIF projects that you funded, to what extent have these projects contributed to achieving the SAR response objective? What evidence do you have that these projects have been ensuring an effective SAR response (capability) in all areas of Canada? Were there barriers to success?
8. For the NIF projects that you funded, to what extent have these projects contributed to achieving the SAR prevention objective? What evidence do you have that these projects have been successful in influencing individuals and organizations on the assessment of risks and on the importance of acquiring and using appropriate knowledge, skills and equipment? Were there barriers to success?
9. What types of benefits has your organization derived from NIF funding or the project itself? Were there any negative impacts from the NIF investment? Have you received any feedback from the NIF funding recipients, your SAR stakeholders or others within your organization?
10. If NIF did not exist, would you be as involved in SAR-related projects? What types of projects are you now undertaking that would not have been possible without NIF funding?

Design and Delivery/Lessons Learned

11. Based on your experience with NIF or with other programs, what suggestions do you have for improving the design, delivery, underlying processes and structure of NIF? How does NIF compare with other programs with which you have had experience?

Concluding Question

12. Is there any question that we did not ask and that you think that we should have asked? Is there anything that you would like to add?

Appendix D: - NIF Projects Funded in 2003/04 By Sponsor and Total Contribution

FEDERAL

NSS

1. PA Review Studies $80,000
2. 2003047 Consolidating and Implementing the National GSAR Information System $14,000
3. 2003048 Development of Linked Provincial/Territorial GSAR Certification Systems $90,000

Fisheries and Oceans

4. 2003027 SAR Access to Vessel Monitoring Information $151,800
5. 2003024 Safe Catch $388,352
6. 2003030 Bobbie the Safety Boat Program $142,890
7. 2003029 Finding Your Way: A Guide to Nautical Charts for Youth $54,120
8. 2003023 Personal Emergency Locator Device $515,684
9. 2003028 SLDMB Project Phase 2: Deployment Strategy $117,700
10. 2003022 Location-Based Risk Analysis of Recreational Boating Activity $674,300
11. 2003031 Translation of Medical First Responder Instructor Guide $41,965
12. 2003026 Cruise Ship Activity and Risk Analysis for Improved SAR Response Planning $371,800
13. 2003025 Remote Sensing Shoreline for Northern Labrador $908,850
14. 2003045 Maritime Survivor Locator System $566,280

Parks Canada

15. 2003043 Detecting and Locating Avalanche Victims using Advanced Radar Technology $132,426
16. 2003042 SnowSmart: Peers Helping Peers $334,802

National Defence

17. 2003016 Infra-red Eye Prototype Upgrade and Flight Testing $627,000
18. 2003021 406 MHz Traffic and Interface Monitoring System $99,000
19. 2003018 Prediction of Self-help Limits and Freezing Times $136,640
20. 2003019 Penetrating Radar Performance and Application Study $110,000
21. 2003017 Video Based Automatic Station Keeping (Phase 2) $257,950

RCMP

22. 2003034 Joint Police/Volunteer Training Initiative $22,000
23. 2003036 Mapping and Visualization Tools for Search Management $237,634
24. 2003033 SAR Interoperability in a Police Environment $15,016
25. 2003035 Methods and Technologies for Tracking at Night $65,709

Environment Canada

26. 2003013 Great Slave Lake Weatheradio and Repeater $51,256
27. 2003049 Fraser River Estuary Forecasts $226,600
28. 2003011 Surveying the GSAR Community for Gaps in Current Meteorological Information $115,500
29. 2003010 Impact of Hurricanes over the Canadian East Coast $800,000
30. 2003012 Nearshore High Resolution Wave Modelling $433,400
31. 2003009 Automation of the Volunteer Ships Observing Program (AVOS) $891,000

Transport Canada

32. 2003044 Feasibility Study - Replacing 121.5 MHz Homing Transmitter in 406 MHz Beacons $321,981

PROVINCIAL

Ontario

33. 2003008 Availability/Capability of SAR Dogs and Handlers – Database $71,500
34. 2003005 Forward Looking Infrared $684,882
35. 2003007 WESCAM Thermal Imager $294,400
36. 2003006 Lost Person Characteristics and Search Planning: Influence of Urban Environment $49,795

Saskatchewan

37. 2003041 Ground SAR: Train the Trainer $45,000

British Columbia

38. 2003040 Hug a Tree Presentations: Kettle River SAR $3,300
39. 2003037 SAR Equipment and Training: Westcoast Inland SAR $35,919

Alberta

40. 2003014 Canine Search Training Manuals for Volunteer Dog/Handler Teams $58,413

Northwest Territories

41. 2003001 SAR Information and Communications Systems $49,500
42. 2003002 Ground SAR Standards Update and Compliant Training $136,125

Appendix E: - NIF Projects Funded in 2004/05 By Sponsor and Total Contribution

FEDERAL

NSS

1. 2004053 NSS Audit and review $80,000
2. 2004052 NSS New SAR Initiatives Fund Information System (NIFIS) $ 198,424

Fisheries and Oceans

3. 2004044 DFO Education and Awareness on Major Ontario Waterways $55,000
4. 2004039 DFO Canoe and Small Craft Safety Program (Pilot) $378,917
5. 2004034 DFO Radio Channel Noise Reduction $167,970
6. 2004030 DFO Coming Back Alive $969,568
7. 2004033 DFO Investigation of Leeway and Drift for Ovatek Life Rafts $616,968
8. 2004041 DFO Marine Simulator Program - CCGA Pacific Region $346,726
9. 2004042 DFO Search and Rescue Crew Manual (French Version) - CCGA $90,200
10. 2004031 DFO Boating 101 $851,936
11. 2004040 DFO MAYDAY - Exhibit on Search and Rescue in British Columbia's fishing Industry $21,340
12. 2004032 DFO Alcohol and Boating Safety - A National Education Campaign Strategy $311,850
13. 2004038 DFO Bobbie the Safety Boat Program - CCGA $268,400
14. 2004036 DFO Liferaft Operational Performance - Evaluation of Technology, Human Factors and Training Elements $991,129

Parks Canada

15. 2004050 PC National Simulation Exercise Program for GSAR Based on Integrated Command $222,750
16. 2004051 PC Avalanche Decision Framework for Amateur Winter Recreationists $561,127

National Defence

17. 2004010 DND SPARTACUS - Advanced Sensor for the CC130 SARPAL Concept $880,000
18. 2004011 DND C130 SARPAL/RMAS Concept Human-Machine Interface $372,970

RCMP

19. 2004048 RCMP Distance Learning Package for SAR Manager Recurrency and SAR Commanders $160,430
20. 2004047 RCMP On-line Learning Module for On-Site First Response to Avalanche Incidents $243,737

Environment Canada

21. 2004045 EC Enhanced Forecast of Forms and Quantities of Precipitations $218,955

Transport Canada

22. 2004027 TC Development of a 406 MHz Beacon Locating Device $516,780
23. 2004028 TC Addition of Frequency Stability Measurement Capabilities to the 406 MHz Beacon Tester $172,854

PROVINCIAL

Newfoundland and Labrador

24. 2004029 NL Training and Proficiency Standards and Boating Safety Partnership - Newfoundland and Labrador SAR, NL $287,155

Quebec

25. 2004008 QC Quick and Efficient Response Unit, QC $41,695
26. 2004007 QC Online Path Monitoring System, QC $36,245

Ontario

27. 2004006 ON Volunteer Search and Rescue Training - West Grey Police Service, ON $4,793
28. 2004005 ON Preventative Search and Rescue (PSAR) - Thunder Bay District, ON $19,580
29. 2004001 ON Equipment and Training - Chatham-Kent, ON $27,845
30. 2004004 ON Surface Marine Search and Rescue - Owen Sound Fire Department, ON $17,512

Manitoba

31. 2004046 MB Enhancing Manitoba Search and Rescue Capabilities through Provincial Inter-Agency Training Programs, MB $116,208

British Columbia

32. 2004021 BC Communications Equipment Upgrading - Fraser Valley SAR, BC $16,616
33. 2004020 BC Search and Rescue Equipment, Training and Testing - BC $4,488
34. 2004016 BC Helicopter Hover Exit Training and Wilderness rescue exercise - Alberni Valley SAR, BC $8,118
35. 2004022 BC Swift Water Rescue Equipment Purchase - Mission SAR, BC $17,708
36. 2004019 BC Automatic Position Reporting System (APRS) for Shared Use - Northwest, BC $41,796
37. 2004023 BC Search and Rescue Training and Equipment - Princeton SAR, BC $12,311
38. 2004024 BC Computerized Mapping and Search Management Project - Sunshine Coast SAR, BC $9,020
39. 2004026 BC Development of a Multi-agency Incident Command Structure for GSAR - Central West Coast of Vancouver Island, BC $65,367
40. 2004025 BC Equipment, Training and Awareness Incentive - Vernon SAR, BC $54,985
41. 2004018 BC Mobile Command System and Enhanced Training Network - Campbell River SAR, BC $27,587
42. 2004015 BC AdventureSmart - Prepare and Be Aware, BC $981,200

Alberta

43. 2004009 AB YouthSafe Outdoors: Risk Management for Community-based Outdoor Recreation, AB $599,198