This archived Web page remains online for reference, research or recordkeeping purposes. This page will not be altered or updated. Web pages that are archived on the Internet are not subject to the Government of Canada Web Standards. As per the Communications Policy of the Government of Canada, you can request alternate formats of this page on the Contact Us page.
Library and Archives Canada
Prepared by the Audit and Evaluation Division
Library and Archives Canada
As stated in early documents 1 the stated vision was "to enhance the knowledge and appreciation of our past by providing access to the nation’s archival record via the information highway." CAIN was described as "a distributed, searchable network of networks which will link Canadians on the internet with every Canadian archives." In its key characteristics it would be flexible, bilingual, user-friendly and inclusive. Its main strategic framework elements were:
With regard to implementation it was anticipated that it would take place in three phases considering the uneven development of information networks and institutional databases among the various archives themselves (including provincial and territorial archives) who would be the key partners.2 These were:
The target audience (i.e. communities of interests or end users) identified in the planning stages included the following:4
In the early period efforts focused mainly on promoting the concept of CAIN and in securing a source of funding. Early estimates of the cost of CAIN were in the neighbourhood of $16 Million over a four year period and it was decided to approach the federal government for half of the funding required.5 After preliminary consideration of various options, the department of Canadian Heritage seemed to be the logical choice and presentations began as early as December 1998.6
Work was also done in developing terms and conditions for the program. These were approved by CCA in March 1999 and identified five funding streams:7
Thereafter efforts were made to develop the technical infrastructure to support CAIN. Progress in this area took a major step forward in January 2000 when the National Archives offered the CCA needed infrastructure in the areas of a web server, search engine and internet connection.
Although efforts had been made by the CCA to secure private sector sponsorship for the initiative, these were not successful. During the development period a significant effort was made by archives across Canada to make CAIN a reality. As well, the CCA spent over $200,000 of its own funds to get CAIN up and running.8
In February 2000 the federal budget allocated $20 Million for 2000-01 and $30 Million in the following year for the purpose of increasing Canadian cultural content on-line.9 Funding for the CCA to work on CAIN was finally secured from Canadian Heritage in October 2000 in the amount of $700,000 for the remainder of the 2000-01 fiscal year (a total of 20 projects were mounted). In the interim work continued on a web site prototype and the site itself was launched in October 2001. Funding for 2001-02 of $2.3 Million supported a total of 134 projects although there were delays in getting this funding approved and adjustments were required for some of the projects. Funding of $1.7 Million for 2002-03 was less than anticipated and required more adjustments including a contribution from CCA from its own regular funding to cushion the reduction.
Over the course of the three years in which funding was received from Canadian Heritage, the guidelines under which the funding was provided have been increasingly focused on making digital content available on-line. Canadian Heritage has become increasingly unwilling to fund projects involving other activities (infrastructure, descriptions, training, etc.). While the vision for CAIN included digital content, it was seen as a later stage in the work and not the major emphasis particularly in the first few years. Although there have been adjustments in the mix of projects proposed to meet the requirement for increasing digital content, the results have been that some projects have had to be adjusted, others have had to be dropped entirely, and funding levels have been lower than expected since CAIN is solely dependent on CCOP funding.
1.2 Objective of the Evaluation
From the outset funding for CAIN as provided by Canadian Heritage through Canadian Culture Online (CCOP) and its predecessor has not been a good fit due to different objectives. An evaluation of CAIN at this time would demonstrate whether or not the current approach is working and what lessons can be learned to date. On October 31, 2003 the Audit and Evaluation Committee of the Library and Archives Canada (LAC) approved the Terms and Conditions for the evaluation as presented in Appendix 1.
The evaluation examined the program as it existed on September 30, 2003 and was directed by the Audit and Evaluation division of LAC with the assistance of Consulting and Audit Canada. An Advisory Committee assisted in the development of the methodology that was used in the evaluation, assisted the evaluation team with documentation, facilitated contact with program participants, and reviewed the draft report (see Appendix 2 for the Terms of Reference for the Advisory Committee).
The evaluation itself used a number of methods including a review of documentation at LAC and at the Canadian Council of Archives. Interviews were conducted with staff at the Library and Archives, Canadian Council of Archives and National/Provincial/Territorial Archives Councils. Phone interviews were also conducted with a representative sample of program recipients across the program components and categories of recipients. An online client survey was also conducted using several listserves for which it was believed that the listserve members were possible users of CAIN (see Appendices 3A through 3D for more details).
Five case studies balanced across program components and categories of recipients were also completed. These case studies involved a file review as well as interviews with program recipients.
2.1 Achievement of Program objectives
2.1.1 Original targets and objectives of CAIN
The objectives of CAIN were not well aligned with the funding program CCOP.
The evaluation found that the original vision of CAIN included both online descriptions and digitized content as part of a sequence. The national catalogue of online descriptions was to be the first priority and once significant progress had been made and the technical infrastructure/network had been built, selected content was to be digitized and made available online.
All informants agreed that the primary objective of CAIN was to create a national online catalogue of descriptions of the holdings of Canadian archives. In terms of actual targets it appears that the goal was to have all fonds level descriptions for archives in Canada (up to 800 institutions) online and RAD compliant over a five year period.
While efforts were made to secure funding for this vision of CAIN, the only funding source that materialized was the Canadian Culture Online Program - CCOP. Over time CCOP placed greater emphasis on digitization and virtual exhibits which caused the CAIN projects to do likewise. The archival community has perceived this as a change in the direction of CAIN requiring it to move away from its original vision.
2.1.2 Extent to which CAIN has achieved its objectives
Some progress has been made in achieving the goal of a national online catalogue.
In terms of the objective of a national catalogue of descriptions it appears that CAIN is about 20% complete after 3 years of funding. The main reason it appears is that to date only about 250 out of the 800 archives have participated (those who have participated are about 50% complete but only 37% of this group felt that all descriptions would be completed in the next two years).
Although the funding devoted to the preparation of descriptions declined over the three year period, it still represented 44% of total expenditures. An additional 24% was spent on technical infrastructure and 12% on training and management. Only 20% was spent on digitization and virtual exhibits (see Appendix 5 for details). In total, CAIN received and used about 45% of the funding it had originally requested ($8.0 Million) towards the development of the national online catalogue and its supporting infrastructure.
Informants believed that significant progress was made in achieving a network and modest progress was made with regard to raising the profile of archives through an improved presence on the web. Those who participated in the program felt that the funding achieved its purpose (60-90% agreement) and that CCOP funding for CAIN was important in achieving program objectives (70-100%).
The use of the Archives Canada/CAIN website is not at a level where it could be said to be engaging Canadians but still appears to be primarily used by the archival community. It is not apparent that CAIN has improved access for Canadians. One of the possible reasons for this result is that CAIN has not been actively promoted to potential users. It should also be noted that traffic on CAIN is not reflective of the entire network of which CAIN is only one part. Data and time limitations did not allow for the gathering of statistics on provincial networks.
2.2 Program results and impacts
2.2.1 Impact of CAIN on the archival community
Positive benefits to the archival community have been apparent.
Informants believed that the impact of CAIN on the archival community had been positive particularly in the areas of standardized descriptions and improved communications among archives. Case study respondents also felt that some improvements in physical management of documents and space had occurred.
With regard to increases in requests from the public, participants responded that the greatest increases have been in online visits and emails which is reflective of their greater online presence which in turn was assisted by CAIN.
Although 44% of participants indicated that CAIN had improved their capacity to leverage funds, the funding profile of the institutions in the sample showed that their sources of funding are not any more diverse than before.
2.2.2 Satisfaction of clients with CAIN products
Desires of the public appear to be different from those of the archival community.
Participants, case study respondents and key informants from within the archival community all believed that clients were satisfied with CAIN products but admitted that few mechanisms were in place to assess client feedback in any systematic fashion.
The survey of the Archives Canada/CAIN website conducted as part of the evaluation was very revealing with regard to this finding. Firstly it would appear that the vast majority of those who visit the website are those who are already intensive users of archival resources (Appendix 4B). Those with less previous experience with archival resources were less enthusiastic about the website and its contents and when asked about future priorities were more likely to give a higher priority to digitization and genealogical resource. It would appear that there is a difference between the wants and desires of the archival community (i.e., the traditional users) and some segments of the public they would like to serve (i.e., the non-traditional users). Having a catalogue of online descriptions is a useful research tool and enables archivists to help both traditional and non-traditional users locate what they are looking for. Knowing where something is located (i.e., having a catalogue of online descriptions) might be the goal of archivists and those who are familiar with and sympathetic to the profession but it is only one of a number of priorities for the public.
2.2.3 Satisfaction of participants with program impacts
Recipients are satisfied with program outcomes.
Recipients were generally satisfied with the impacts of the program and found that funding was adequate to achieve its goals for the projects that were undertaken. Some 70% indicated that they would apply for CCOP funding again in the future.
2.3 Lessons Learned
2.3.1 What worked/didn’t work with design and implementation of CAIN
Views of what worked or did not work were seen only from the perspective of the needs of the archival community.
It was generally agreed by informants that the concepts underlying CAIN were sound (national network, standard descriptions, putting descriptions and content online) and that there had been positive impacts for those who had participated.
The issues most often cited with regard to what did not work were the adjudication process and what was perceived as CCOP’s changing objectives, changing program criteria and unstable funding from year to year, and untimely funding within each year. It was also felt that promotion efforts had been unsuccessful and public interest had not yet materialized.
Some within the archival community saw digitization of content and virtual exhibits as not being part of fundamental archival work. This led to the view that the community should have passed on the CCOP funding and waited for another source of funds. Others, however, pointed to what had been accomplished by CAIN and felt that the effort had been worthwhile.
2.3.2 What lessons from CAIN could be applied to similar initiatives
Agreement and understanding on program objectives by the funding authority, the program administrator and participants is crucial.
In terms of the experience of CAIN and CCOP it was apparent that misunderstandings occurred on both sides. Some informants felt that CCA was not effective in explaining to PCH what archives are all about and why a national catalogue of descriptions was an important first step towards an online presence for the archival community. They also felt that Canadian Heritage did not require CCOP criteria to be met from the outset thereby giving the archival community the impression that its vision had been accepted.
In terms of lessons learned from the CAIN/CCOP experience, it was clear that more careful consultations between the CCA and Canadian Heritage about the objectives and expected results of the two initiatives was needed to see if there in fact was a fit before deciding to go ahead. It may, in fact, have been wise to do a pilot or series of pilots to test the concept first. Once the program was up and running there should have been consultations with the archival community as changes were needed.
1. Raising CAIN: Building Canada’s Archival Information Network, November, 1997.
2. Raising CAIN: Exploring the Collective Memory of our Nation, October, 1998.
3. The timelines shown were based on work beginning immediately (1998). No further description of the work to be undertaken was provided at that time, however, the CCA was in the process of establishing the infrastructure needed to make CAIN a reality such as a CAIN Steering Committee and a Technical Sub-committee.
4. Canadian Archival Information Network, Business Case Framework, May 1999.
5. CCA Annual Report 1998-99, p. 19.
6. Ibid., p. 19.
7. Ibid., p. 20.
8. This included $71,111 in 1998-99, $115,521 in 1999-2000 and approximately $45,000 in 2000- 01 as noted in the Annual Reports for the respective years.
9. Budget 2000: Making Canada’s Economy More Innovative, Department of Finance, February 2000, p. 13.
10. A more complete version of the findings is presented in Appendix 4.