ARCHIVED - About Us

Audits and Evaluations

Review of Library and Archives Canada Conferences

Prepared by the Audit and Evaluation Division
Library and Archives Canada

March 2004

Final Report

Table of Contents

• 1.0 Introduction

• 2.0 Review Findings

• 3.0 Conclusions

• 4.0 Management Response

• List of Appendices

---

1.0 Introduction

1.1 Background

Library and Archives Canada (LAC) was involved in several major conferences during 2003 including the International Forum on Canadian Children's Literature, Symposium 2003 on Preservation of Electronic Records, Canadian Metadata Forum, and the Government of Canada Symposium on Information Management. LAC Management Board would like to capitalize on this collective experience by exploring opportunities for increased sharing of information and best practices, and by examining conference results and spin-off effects in the context of LAC's mandate. (Note: In this report, use of the word "conference" is intended to cover symposium, forum and other similar types of major events).

1.2 Purpose

The purpose of this review is to assess the overall success of the above conferences, focusing on lessons learned, strengths and weaknesses, and alignment of conference results with LAC's mandate.

1.3 Approach

The approach followed was to carry out a review of existing documents and media coverage, followed by interviews with a cross-section of key conference organizers, outside partners and sponsors, those at LAC responsible for the respective programs, participants from other departments and external organizations, LAC Corporate Communications and other conference specialists. The interview guide along with a list of those interviewed is included in the Annex 1. The field work for this review was carried out by the Corporate Research Group.

2.0 Review Findings

2.1 Objectives

Conference objectives were focused primarily on the short term need for community and awareness building but stopped short of considering the downstream implications with regard to potential outcomes and spin-off effects on the institution.

• The short term objectives of raising awareness and community building were clearly established and met as evidenced by the high levels of participation, conference satisfaction, and demand for a follow-up initiative.

• In most cases it was unclear what type of conference was actually intended, for example a one-time conference, a pilot to test the waters, first of a series, or other type.

• It was also unclear what potential downstream implications were anticipated in relation to spin-off effects, desirable/unanticipated outcomes, and impacts on the institution.

• Positive feedback was received with respect to achievement of objectives related to awareness and community building via participant satisfaction surveys and other direct feedback mechanisms.

• Measurement of tangible results, spin-off effects and impacts on the institution were hampered due to the incomplete consideration of potential outcomes at the objective setting phase.

2.2 Conference Logistics

In most cases conference organizers started from scratch with a limited understanding of what was required.

• Due to the relative inexperience of organizers and the absence of an internal conference infrastructure, or ready access to experienced resources, organizers often resorted to ad hoc approaches, last minute planning and scrambling for resources.

• As a result, a heavy workload was imposed on staff and additional expenses were incurred due to the need to hire outside expertise.

• Organizers were generally unfamiliar with government requirements with regard to partnering/sponsor arrangements, public events and advertising (Government of Canada Communications Policy).

2.3 Working with Partners

The emphasis on partnering and stakeholder engagement provided excellent opportunities for advancing the institution's agenda, and also provided new challenges with regard to selecting and managing relationships with conference partners.

• In total, the conferences attracted more than 40 partners/sponsors including federal departments and agencies, private sector companies, cultural institutions, academia and other levels of government (see Annex 5 for a list of partners/sponsors).

• In addition to financial support, synergies were created in some cases by connecting and working with like-minded organizations in pursuit of common interests - the Canadian Children's Literature event is a good example.

• Challenges include selection of the "right" partner, government rules/regulations pertaining to partnering arrangements, additional workload, balancing the interests of the commercial and public sectors, and managing expectations.

• Significant financial support by government and non-government organizations highlights the broad interest and potential for future partnering arrangements as a way of making the documentary heritage of Canada more accessible to Canadians and the world.

2.4 Outcomes and Spin-Off Effects

Positive spin-off benefits resulted, particularly in the case of the Canadian Children's Literature conference, however, expectations were created for follow-up events led by LAC that would take the field of interest to the next level.

• The Canadian Children's Literature conference provided the catalyst for forging partnership and acquiring financial support for several current programs with TD Bank, Hudson Bay, Harper Collins and the National Research Council.

• The Information Management Check Tool has generated numerous calls and resulted in the provision of advisory services to other departments with respect to the state of their information management.

• The Canadian Conservation Institute has had numerous training requests as a result of the conference.

• As a result of the Metadata conference, several Memoranda of Understanding have been negotiated with other departments for the provision of metadata services on a cost recovery basis.

• The Metadata and Information Management conferences generated an expectation for a follow-up event in the near future. Interest was also shown by participants of the Canadian Children's Literature and Preservation of Electronic Records conferences for some form of follow-up initiative.

2.5 Conference Resources and Workload

The underestimation of workload and resource requirements resulted in the imposition of a significant burden on internal staff and, in some cases, the need for additional resources.

• Due to organizers' relative inexperience and the absence of internal conference support, actual workload and resource requirements generally exceeded expectations. This was particularly the case in the weeks leading up to the conference when organizers lacked the required support related to registration, logistics, IT, internet, communications, promotion, outside contracting and other aspects.

• Organizers demonstrated resourcefulness in achieving significant cost offsets in the form of sponsorship funding and "in kind" contributions from government and non-government organizations.

• While there was some level of costing and budgeting at the planning phase, incomplete consideration was given to full costing of conference inputs, including the overall level of effort by internal staff and other indirect costs, in other words the total cost to the Crown.

3.0 Conclusions

1. When setting objectives, consideration should be given to the institution's strategic priorities, potential outcomes, impacts on the institution, and post-conference strategies.
2. The cost effectiveness of future conferences would be enhanced if a minimal level of internal resources and/or standing offer arrangements with private sector experts were available to provide guidance and assistance. If the institution intends to utilize conferences as strategic instruments to advance LAC and/or government priorities, consideration should be given to establishing a Conference Guide covering all aspects of organizing conferences including roles and responsibilities of various parts of LAC and government resources available. This guide should be updated regularly as organizational experience increases and best practices are discerned.
   
   
3. Conference partnering should be promoted wherever possible. In selecting partners, consideration should be given to their value added, linkages to the LAC mandate, and their capacity to work within government requirements.
4. Objective measures are required to assess the success of conferences in meeting objectives, value added, and impacts on the institution. These measures should be part of the initial approval process and subsequent reporting.
5. A more business-like conference approval and assessment process is necessary to maximize the return on conference investments with due regard for the institution's strategic priorities, full costing and benefits, and impacts on the institution.

4.0 Management Response

To be tabled at the Audit and Evaluation Committee meeting on April 23, 2004.

List of Appendices

1. Methodology and Interview Guide
2. Conference Highlights
3. Best Practices and Gaps
4. Conference Profiles
5. List of Partners/Sponsors

Appendix 1

Methodology

The approach included review of conference related documents and media coverage, followed by interviews with a cross-section of those responsible for various aspects of the respective conferences. An interview guide was developed to ensure consistencey in data collection and is shown below along with a list of those interviewed.

Interview Guide

Interview Guide - Conference Name: ___________________________

1. General Information

• Main Contact: Date: Length(days):
• Conference Host(s):
• Facilities/Equipment: Government, Private?
• Conference Type: Instructional, Commitment, Informative, Promotional, Technical, …
• Approximate Budget: Sponsor/Other Funding:
• Number and General Level Of Participants:
• Conference Organization & Management: Government, Private, and Shared…
• Workload Impact On Staff:
• Involvement of LAC Corporate Communications:
• Translation, Technical Aides And Special Needs Peculiar To this Conference:

2. Conference Planning And Execution

Please describe how successful the conference was in establishing and achieving a clear purpose, goals and expected outcomes. Please describe in terms of the role/contribution of the following conference elements and, where appropriate, make the link to government and/or Agency mandate and objectives:

(i) Conference Planning

Conference Vision/Primary Theme * Purpose Goals * Criteria For Success * Intended Outcomes
* Promotion/Marketing * Target Audience * Number of Participants

(ii) Conference Execution

* Governance Structure * Stakeholder Involvement (Committees, Partners, Co-Hosts, Sponsors)
* Conference Program/Agenda (Keynote Speaker, Trade Show, Interactive Activities, Tours…)
* Public Outreach Activities (where applicable)
* Conference Organization/Administration
* LAC Corporate Communications
* Location/Facilities/Equipment

(iii) Post Conference Review And Feedback

Assessment * Evaluation * Informal Participant Feedback * Other Stakeholder Comments* Comments From The Public * Media Articles * Other

3. Conference Results and Follow up Initiatives

4. Other Comments/Recommendations

Please identify other issues or factors, which have not been covered above.

5. Other Key Contacts

Please identify a representative cross-section of 3 or 4 participant and stakeholder contacts that we could approach for their views on this conference. If possible, we would like to interview 4 people for each conference including a key organizer, the program responsibility, a participant from another department, and an outside stakeholder from another organization.

6. Pertinent Documentation

Please provide relevant documents including any assessment, evaluation, participant feedback, testimonials, newspaper coverage, articles in industry journals, other.

Conference Interviewees

Conference	Key Organizer	Program Responsibility	Participant	Stakeholder or Partner
Canadian Children's Literature	Francine Proulx, Co-ordinator Litaracy Program TD Bank	Céline Gendron, Director Can-Literature Research Services, LAC Josiane Polidari, LAC	Jane Venus, Manager of Children's Programs, Ottawa Public Library	Jane Kilburn Boyle, Corporate & Public Affairs, TD Bank
Preservation of Electronic Records	Jane Down, Senior Conservation Scientist, CCL	René Paquet, Electronic Archives Preservation, LAC Joe Iraci, Senior Conservation Scientist, CCI	Richard Green, A/Director, Music Division, LAC	Paul Lima, Liaison/ Marketing, CHIN
Information Management	Gillian Cantello, Manager Internal Audit & Risk Management, LAC	Julia Ginley, Director Government Information Management Division, LAC		Ed Fine, Exec Director, Organizational Readiness, TBS
Metadata	Deane Zeeman, Metadata Co-ordinator, LAC	Ingrid Parent, Director General, Acquisitions & Bibliographic Services, LAC	Sheila Carey, Product Manager, Professional Programs, CHIN	Stephen Downes, Senior Research Officer, Online Learning/Metadata NRC

In addition to the above several key participants from LAC Corporate Communication were also interviewed.

Bob Ferris, Manager, Public Programs
François Gagnon, Program Support Services
Ghislaine Roy, Director Communications & Public Programs Division

Appendix 2

Conference Highlights

Canadian Children's Literature

• This conference provided the catalyst for establishing partnership and financial support for new programming initiatives which are now managed by the institution: Summer Reading Program (TD Bank), Children's Translation Awards (Hudson Bay Company), Broadbent Virtual Discussion Group (National Research Council), and the Harper Collins Lecture Series.

• The Summer Reading Program which resulted includes a 2 year funding commitment by Toronto Dominion Bank ($500,000) for LAC to coordinate program content and participation of local libraries across Canada. TD Bank officials indicated they were very impressed with the professionalism of LAC staff and would be prepared to partner in future if the fit were right.

• The significant contribution made by the conference champion (M. Roch Carrier) was recognized as being instrumental to the success of the conference.

• Information sessions and interactive children's activities were also included in the conference.

Preservation of Electronic Records

• This conference was co-chaired by National Archives, National Library, Canadian Heritage Information Network (CHIN) and the Canadian Conservation Institute (CCI) who provided the driving force.

• CCI already had the machinery and tools in place due to experience in running numerous large conferences, including the ability to draw on internal conference specialists spread across the Institute.

• Public Information Day (Preservation Quest) was a tremendous success, providing an opportunity for approximately 500 public citizens to direct questions to experts on how to preserve home movies, CDs, videos, and more.

• Post-conference revenues have been generated from the sale of the Post-Prints, i.e. Speakers Papers which were amended after the conference to reflect participants' inputs.

• The final report prepared by CCI provides an excellent model to follow with respect to the organization and delivery of large conferences.

Metadata

• Excellent leadership was demonstrated by LAC in hosting this

• Introductory" conference which successfully attracted a broad-based community of metadata practitioners - highlights the importance of this new field.

• Direct conference costs were recovered thanks to federal and private sponsorship.

• Approximately 90 % of participants (200) indicated their satisfaction and the need for a follow-up conference in 2004 to take this field to the next level.

• The conference was described by one of the foremost specialists in the field as "relentlessly positive".

Information Management(IM)

• This conference was successful in attracting a large number of senior government executives (245) to share progress and address IM challenges facing the Government of Canada.

• Introduction of LAC's Information Management Capacity Check Tool generated considerable enthusiasm from other departments interested in using this tool, as well as numerous post-conference requests for assistance.

• The evaluation Voting Technology provided useful feedback on the state of Information Management in federal departments (relatively low), and the value of holding another symposium which was quite high (80% range).

Appendix 3

Best Practices and Gaps

Below are general observations on best practices and areas of weakness demonstrated by the conferences with respect to conference organization and delivery.

Best Practices

• Use of high profile Conference Champions
• CCI's conference organizaton and delivery model
• Extensive use of partnering and sponsorship arrangements
• Conference governance structure, i.e. steering and other organizing committees
• Effective use of special events such as Public Forums, Trade Shows and other mechanisms such as a Decision-Making Model, Awards Ceremony, Information Management Capacity Check Tool, Children's Activities
• Preparation of Final Conference Reports
• Stimulating Keynote Speakers
• Appropriate facilities and social events.

Gaps

• Late planning and involvement of LAC Corporate Communications officials and private sector conference experts
• Underestimation of resources and workload requirements particularly for events involving the public
• Lack of familiarity with the government's communications policy requirements
• Lack of linguistic balance
• More extensive planning and an earlier start required for effective marketing and promotion
• Insufficient provision of technical back-up, logistical support, volunteers
• Marginal use of the internet
• Outsourcing of services to outside professionals did not allow sufficient time to take full advantage of their expertise, for example related to attracting sponsors and trade-show participation.

Appendix 4

Conference Profiles

Appendix 5

List of Partners

• Toronto Dominion Bank
• Verity Software Solutions
• VIA Rail
• Air Canada
• Amazon
• Coca Cola
• Harper Collins
• Hudson Bay
• Ottawa Citizen
• Power Corporation
• National Research Council
• Le Droit
• Statistics Canada
• National Defence
• Treasury Board Secretariat
• Canadian Museum of Civilization
• Communications Research Centre
• Canadian Museum of Nature
• Canadian Heritage Information Network
• Government Online Task Force
• Canadian Council of the Arts
• Canadian Conservation Institute
• Foreign Affairs
• Canadian Initiative on Digital Laboratories
• Friends of the National Library
• Multi-Cultural Artists
• National Arts Centre
• National Film Board
• National Gallery of Canada
• Bibliothèque Municipale de Gatineau
• Ottawa Public Library
• La Télévision de Radio-Canada
• BFB Sales Ltd.
• Canadian Institute for Historical Microreproduction
• Carr McLean
• ROMifications Publishing Inc.
• Crowley Micrographics
• Promomedia Group Inc.
• Tunstall and Tunstall Inc.
• Zomax Canada
• Several book companies
• CBC

December 17, 2004

Management Response for Evaluation of LAC participation in Conferences (Fora)

1. We agree with the conclusions stated in this report.
   
   
2. As part of the creation of the new institution, it is imperative that the organization of conferences by LAC be aligned with its strategic directions and priorities. The Programs and Services Sector, in collaboration with the Strategic Office and the Communications Office, will develop a framework, including processes and criteria, that will help assess proposals and make decisions on the appropriateness of organizing a conference. This framework, to be approved by Management Board, will enable Management Board to set clear objectives on areas of focus and investment over a multi-year period.