ࡱ> ` bjbjss .Dl<<<<<<<P777889dPJt:t:":::q;q;q;JJJJJJJ$KhM7J<<q;q;<<7J<<::LJeAeAeA<$<:<:JeA<JeAeA<<eA:h: @KNN<eAHq;0;"eA;;q;q;q;7J7JAdq;q;q;J<<<<PPP$/t5dPPPt5PPP<<<<<< The Merit Review Process for ƽ岻 Librarians: A Comparison of 2004 and 2005 Prepared for the MAUT Librarians' Section Executive by the MAUT-LS Professional Issues Committee Introduction In September 2005, a new process for determining the merit award for librarians was implemented by the Director of Libraries. This new process has raised a number of serious concerns among librarians, both in terms of how it was introduced and of its operational details. The Executive of the MAUT Librarians' Section has requested the MAUT-LS Professional Issues Committee (PIC) to examine these concerns. The following is PIC's report to the Executive on this matter. Terminology In this report, the following terminology will be used: 1998 Process: This phrase will refer to the library merit award process which was introduced in 1998 and which, with slight adjustments over time, remained in use up to and including the merit exercise conducted in 2004 to cover the 2003-2004 evaluation period. 2005 Process: This phrase will refer to the library merit award process which was introduced in 2005 and which was first used for the merit exercise conducted in 2005 to cover the 2004-2005 evaluation period. Development of the 1998 Process Prior to 1998, each administrative unit within the ƽ岻 Library System carried out its own separate merit evaluation. This approach had a number of disadvantages. Unit administrators rated their staff without the benefit of uniform guidelines, and without being able to compare the activities of their staff with those of librarians in other units. Because each unit was allocated its own narrow portion of the total Library merit pool, unit administrators had to fit their staff members as best they could within the cramped range available to them. This range was further limited by the parameters of the merit policy: the requirement that each unit not exceed the amount allocated to it, and the requirement that no more than a certain percentage of staff be placed in each merit category. If the application of these requirements was inconsistent from one unit to another, these inconsistencies would then have to be reconciled by the Director of Libraries when he or she received each unit's merit ranking recommendations. In 1997, as part of a larger initiative by the Academic Salary Policy Subcommittee to improve the merit process at ƽ岻, it was proposed that a new library-wide method of recommending the merit award be developed. The goal was to make the merit award process more objective and transparent than had been the case hitherto. The Office of the Director of Libraries and the MAUT Librarians' Section began consultations to develop this improved method. Two features of the new process would be the allocation of a point value to each work activity element listed by librarians in a report of their activities for the past year, and the use of a mathematical formula to compute a final score from these numbers. This approach was based on a system which was used by John Hobbins, then Acting McLennan Librarian, to determine merit in the Humanities and Social Sciences Library in 1997, and which was later discussed by the Director's Senior Management Group. Another feature would be the evaluation of librarians as a single group by a committee using one set of criteria. This method would allow work activities to be compared more broadly and uniformly between librarians. It would also have the advantage of keeping the Library's merit funds in a single broad pool, and of producing a single ranked list of librarians which could then be fitted more easily into the merit categories. The draft methodology developed from these consultations was distributed by e-mail to all librarians in November 1997 for information and comments, as well as for discussion at the MAUT Librarians' Section membership meeting later that month. Work was also done to establish the terms of reference of the Merit Implementation Standing Committee, which were finalized by June 1998. The details of these procedures (i.e., the 1998 Process) were announced to all librarians by the Director of Libraries in a letter dated 8 July, 1998 (a copy of which is appended to the present report). Accompanying this letter were Vice-Principal Chan's memo describing the academic salary policy for 1998/99, the Merit Increase Methodology for the libraries, the Merit Implementation Standing Committee's terms of reference, and the Librarian's Review of Activities form (see appended copies). The 1998 Process was well-received by ƽ岻 librarians, as illustrated by the December 17, 1998 letter (appended) sent by the Director of Libraries to the MAUT Librarians' Section Executive. In this letter, the Director mentions her pleasure at reading in the December 1998 MAUT Newsletter about the positive response by librarians to the new merit procedure. She goes on to thank the librarians who were involved in developing and implementing the process, and indicates that she would welcome input for fine-tuning it in the future. As previously mentioned, a number of adjustments were made to the process over the years. Some adjustments involved the way in which the Merit Implementation Standing Committee functioned; these are described in the next section of this report ("Operational Details of the 1998 Process"). Some adjustments involved the Review of Activities form, later called the Librarians Annual Report form (a copy of which is appended). This form was used to report the work done by librarians each year in their three areas of academic duties (position responsibilities; research and other scholarly and professional activities; contributions to the University and scholarly communities). The basic categories remained similar as the form evolved, though their order changed somewhat; explanatory notes were added to some items to clarify what they covered, and subsections were introduced under some categories to group the details more finely. Operational Details of the 1998 Process Initially, the Merit Implementation Standing Committee consisted of the Staff Development Librarian, three other selected members of the Director of Libraries' Senior Management Group, and an MAUT Librarians' Section Observer jointly selected by the Director and by the Chair of the MAUT Librarians' Section. The group was later expanded to include all SMG members who were unit heads, as well as the Library Personnel Officer. By including all SMG unit heads, the expanded Committee could count on having at least one member who could speak with direct, detailed knowledge about the work activities of every librarian being evaluated. The Director of Libraries did not serve on the Committee. The Committee was initially chaired by the Staff Development Librarian. When the Staff Development Librarian eventually retired, various members of the Committee were appointed by the Director from year to year to serve as Chair. The documents considered by the Committee were the Review of Activities form filled out by each librarian, and the Supervisor's Evaluation of each librarian written by his or her immediate supervisor. Every year, as the end of the academic year approached, the Director of Libraries would send out these report and evaluation forms to all librarians, with instructions that they be completed and submitted in May or June. This would allow the Merit Implementation Standing Committee several weeks to review this documentation prior to the late summer or early fall meetings at which the merit rankings would be conducted. In reviewing the documentation prior to these meetings, each member of the Committee was given or chose a staffing area for which he or she would be primarily responsible. (This practice was brought in later to save time.) For example, in the interest of objectivity, members might be asked to rank staff from library units other than their own. Members would then read and comparatively evaluate the files for which they were responsible, and would report their assessments to the Committee when it met to conduct the rankings. At the start of its deliberations, the Committee would discuss the methodology to be followed, covering such details as the activities to be counted and the way in which these activities would be scored. The Committee would evaluate the activities of every librarian (except members of the Director's Senior Management Group) in all three areas of academic duties. The Director evaluated the work performance of those reporting directly to her, while the Committee evaluated SMG members in the second and third areas of responsibility. The Committee endeavoured to work from written guidelines to ensure that different types of activities were scored as consistently as possible from person to person, while at the same time being flexible in situations where it would be fairer to make adjustments. For example, when rating performance in position responsibilities, the Committee would start by allocating five points to everyone. Five points represented the notion that the individual was performing all position responsibilities in a superior fashion. Points were added for going above and beyond or for some special circumstance. Points were deducted if not all the job was done or done well based on the supervisors evaluation. The assessments presented by each member would then be further compared to make sure that, for example, the librarians committee work was being reported in the same way by everyone, and that committees were being rated in a way which fairly reflected how much work each committee actually involved. The weighting prescribed by the Merit Increase Methodology was 70%-80% for position responsibilities, and 20%-30% for the other areas of responsibility (i.e. 10%-15% for research and other scholarly and professional activities, and 10%-15% for contributions to the University and scholarly communities). The percentage distribution to be used in a given year was decided by the Committee. In nearly all instances, the distribution used was 75% position responsibilities and 25% other areas; in only one year was a distribution of 80% and 20% used. Once the point totals were computed for each person, the Committee would produce a ranked list of librarians. As was done in the case of the ƽ岻 Pay Equity exercise, the Committee would then review this list to make any final minor adjustments that were required. The Committee would then submit the list to the Director of Libraries. At this stage, the Director would determine where members of SMG would fit into this ranked list, and would add their names to the list at the appropriate point levels. The general understanding was that the Director would not otherwise alter the rankings recommended by the Committee. The budget officer then took the ordered list to determine where the cut-off lines would be between the various merit award categories prescribed by that year's academic salary policy. The cut-off lines between categories would be placed to make the awards fit the money allocated to the Libraries for that year. It was understood that the Director would try to establish the cut-off lines in such a way that two librarians of equal ranking would not end up with different merit awards. As mentioned earlier, a non-voting MAUT Librarians' Section Observer was appointed to the Committee every year. Until 2002, the Observer was present for the evaluation of all three categories of academic duties. After 2002, the Observer was not included in discussions of work performance, but was present for discussion of the other two categories. When the merit award process was completed for a given year, the Observer would report back to the MAUT Librarians' Section Membership. For reasons of confidentiality, the Observer's report would not discuss the specifics of any particular case; it would simply consist of a statement as to whether, in his or her opinion as an independent observer, the work of the Committee in that year had seemed fair and objective and had followed proper procedures. In some years, this statement was supplemented by a statistical report from the Director to the MAUT Librarian's Section indicating, under each merit category, the number of librarians who were awarded that category. The 1998 Process, with its occasional revisions, was on the whole seen as fair, objective and transparent by ƽ岻 librarians. Problems with the process, where they existed, were few in number and minor in scale. For instance, not everyone may have been comfortable with the fact that the membership of the Committee was entirely appointed, nor with the fact that part of the merit rating for SMG members was done by the Director although, in fairness, it would have been a delicate matter for the Committee members (since they were drawn from SMG) to evaluate themselves on job performance. The occasional lack of consistency, clarity or completeness on the part of librarians filling in the annual reports and of supervisors writing performance evaluations sometimes complicated the work of the Committee. In an effort to be fair during the first few years, the Committee contacted librarians who had forgotten to add a crucial point to their annual report. This situation improved as more guidelines were developed for filling out reports and evaluations; for instance, the MAUT Librarians' Section once arranged a group workshop to advise librarians on how best to prepare their annual reports. The annual report form, while retaining its open design of having broad category headings followed by blank spaces below, gradually evolved so that it could be filled in with more consistency. More subcategories were introduced under each area of activity, and in later years the form was accompanied by a detailed instruction sheet for librarians and supervisors. The form's open-ended design was appreciated for working well, for being flexible and for allowing the opportunity to provide explanations where these could be useful. A list of the main features of the 1998 Process can be found in point form in the tables provided later in this report. Introduction of the 2005 Process In the months which immediately followed the arrival in early February 2005 of ƽ岻's new Director of Libraries, the Librarians Annual Report form normally sent out by the Office of the Director of Libraries in April or May was not issued. On June 30, the customary memorandum summarizing the Academic Salary Policy for the current year was e-mailed to all academic staff by (then Interim) Provost Anthony Masi. In September 2005, one month prior to the Payroll Office's October 3 deadline for receiving notification of merit award decisions for 2005, a new library merit procedure was introduced by the Director and announced to librarians by e-mail. This procedure, which was introduced without consultation with the MAUT Librarians' Section, included a new merit committee structure, a new merit evaluation process and a new Annual Performance Recognition and Development Report form. The introduction of the new procedure took place in the following steps: On September 6, the Director announced by e-mail that she was putting together a small working group to advise her on merit: The full committee will consist of: Trenholme Director of Libraries Associate Director, Systems & Technology Senior Management Representative Library Professional Representative Manager, Administration & Personnel I am now calling for nominations from the librarian tenure stream (excluding members of Senior Management). The Director's e-mail solicited nominations for these elected representatives, specifying that no SMG members would be eligible, and giving a nomination deadline of September 7 with a voting deadline of September 9. Customary library election procedures were not followed; for example, electors had to devise their own ballots. No mention was made of the Merit Implementation Standing Committee created in 1998, of the established procedures for determining its composition and chair, or of the MAUT Librarians' Section Observer. On September 9, the MAUT Librarians' Section Executive met with the Director to discuss merit and other concerns. The Director did not agree to have an MAUT Librarians' Section Observer present on the Merit Committee being formed. The Director indicated that she believed that a new merit process was necessary, but did not communicate the reasons for her decision in a memo to all librarians. On September 14, in an e-mail to all librarians, the Director proposed that the members of the committee advising her on merit could, if they were willing to do so, also meet with her to work on the ongoing matter of revising Chapter 2 of the Handbook of Regulations and Policies for Academic and Librarian Staff. The Director indicated that, if the members of the Merit Committee did not wish to become involved in revising the Regulations, a different committee would be formed to work on the Regulations. This second option is the one which was ultimately followed, with the exception of the Senior Management Representative who was automatically put on the Regulations Committee. On September 19, the Director distributed by e-mail a new Annual Performance Recognition and Development Report form (a copy of which is appended). This new form differed substantially from the earlier Librarians Annual Report form: it introduced new evaluation criteria and a new rating scale, which it applied retroactively to the already-completed 2004-2005 year. The new form also introduced sections dealing with future career advancement targets and knowledge/skills to be developed. The Director's e-mail set a deadline of September 26 for returning the forms, leaving librarians with only five business days to write their annual reports and for supervisors to write performance evaluations for each of the librarians reporting to them. On September 22, in response to concerns expressed regarding the short time frame for submitting the Performance Recognition and Development Report form, the Director extended the submission deadline to September 30 for those librarians who, because of absences, were not able to meet the September 26 deadline. Operational Details of the 2005 Process The methodology followed by the Merit Committee during its evaluation meetings one on the morning of Wednesday, September 28 and one during the afternoon and evening of Sunday, October 2, in addition to one or two preliminary meetings to discuss the Annual Performance Recognition and Development Report form is not known. The merit award letters sent to each librarian indicated that the weight allocated this year to position responsibilities was 85%, while the weight allocated to the second and third duty categories combined was 15%. One member of the Merit Committee was authorized by the Director to present at a meeting of the MAUT Librarians' Section (held on October 14) his general impressions of the process, without going into the specifics of how the Committee conducted its work. No other details have been released. A comparison of the main features of the 1998 Process with the known details of the 2005 Process is provided in the tables below. Comparison of 1998 Process and 2005 Process Table 1: Merit Process Development Merit Process Development (1998) Merit Process Development (2005) Collegially developed by the Director of Libraries and the MAUT Librarians' Section.Unilaterally introduced by the Director of Libraries.Developed in 1997-1998 over the space of approximately one year.Announced in September 2005 and used that same month to conduct the merit award exercise for the 2004-2005 fiscal year.Proposed methodology distributed to librarian staff for feedback during the course of the development process.Methodology to be used by Committee not released; no feedback on methodology sought from librarian staff.Full details of the Merit Increase Methodology, the Merit Implementation Standing Committee's terms of reference, and the Librarian's Review of Activities form mailed to all librarians once finalized.Announcements from the Director of Libraries covered only the election procedure and the details of the new Annual Performance Recognition and Development Report form.Report forms and process gradually refined in the years following implementation as experience is gained with them; details of revised methodology mailed to staff as needed.No revisions to process or to the form have as yet been formally communicated through e-mail memo or other documentation to the librarian staff. Table 2: Annual Reports Annual Reports (1998-2004)Annual Reports (2005)Titles: Review of Activities form; Librarians Annual Report form.Title: Annual Performance Recognition and Development Report form.Annual report component of the form is distinct from the supervisor's evaluation form.Combines annual report form and supervisor's evaluation form into a single document having parallel columns for each component. Categories correspond to the position responsibilities and other academic duties of librarians, providing a detailed breakdown of scholarly activities, teaching and service to the profession and to the university.Categories emphasize tasks, accountabilities and projects related to position responsibilities; details of other academic duties are almost entirely absent and are for the most part reduced to a single general category; no reference to teaching.Format is open, allowing detailed written descriptions under every category.Form prescribes a five-point letter-grade scale to evaluate various activities.No equivalent in the form used from 1998 to 2004.Form introduces evaluation categories for oral communication, written communication, interpersonal effectiveness, client service, organizational ability, judgment, application of knowledge / problem solving, leadership, resource management, flexibility, IT skills, diplomacy / tact and teamwork / cooperation. The categories do not state how these terms are to be interpreted for evaluation purposes.No equivalent in the form used from 1998 to 2004. Form includes sections asking about the librarian's career advancement targets for the next five years, knowledge or skills to be developed, and areas for improvement with target dates. Table 3: Merit Committee Merit Committee (1998-2004)Merit Committee (2005)Name: Merit Implementation Standing Committee.Name: Merit Committee.Composed of Senior Management Group (SMG) members (number has varied over the years), the Manager, Administration & Personnel, and an MAUT Librarians' Section Observer. The inclusion of all SMG members ensures that at least one person is familiar with the type of work performed by each librarian. Composed of the Trenholme Director of Libraries, the Associate Director, Systems & Technology, a Senior Management Representative, a Librarian Tenure Stream Representative, a Library Professional Representative, and the Manager, Administration & Personnel.All members appointed.Three members elected.Director of Libraries is not a member of the Committee.Director of Libraries is a participating member of the Committee.Committee chaired by the Staff Development Librarian (initially) and by another SMG member (in later years). The Chair worked with the MAUT Librarians Section Professional Issues Committee to improve the annual report form.Committee chaired by the Director of Libraries.Librarians have access to the annual report form before the year begins so that they know how they will be assessed. They have months to fill in the form.A new 2004-2005 form is issued Sept. 19, 2005. Librarians and supervisors have five business days to complete the annual reports and subsequent performance evaluations. (Table continued on next page) Table 3: Merit Committee (continued) Merit Committee (1998-2004)Merit Committee (2005)MAUT Librarians' Section Observer present in all years for evaluation of second and third categories of duties, and present in some years for evaluation of position responsibilities.No MAUT Librarians' Section Observer permitted to be present at any point. Committee work typically done over several weeks. Members see all of the librarians files in advance of the meeting so that they can familiarize themselves with achievements. Committee work done in a few days.Assessment of the fairness of the process followed by the Committee issued by MAUT Librarians' Section Observer at the end of the merit exercise.No assessment issued since MAUT Librarians' Section Observer was not allowed to participate in the Committee's work.In some years, a statistical summary of the number of librarians in each category is issued by the Director of Libraries at the end of the merit exercise. Statistical summary requested by the MAUT Librarians' Section from the Director of Libraries. No statistical summary issued. Table 4: Merit Evaluation Method Merit Evaluation Method (1998-2004)Merit Evaluation Method (2005) Evaluation methodology discussed by Committee members at the beginning of each year's merit exercise.Process not disclosed.Written evaluation guidelines used as much as possible.Process not disclosed.Methodology stresses counting activities as uniformly as possible for all librarians.Process not disclosed.Point system used to evaluate activities by librarians in all three areas of responsibility.Process not disclosed.In six years out of seven, position responsibilities count for 75% while the other two areas of responsibility count for a combined total of 25%. For the 2002-2003 merit year, the ratio is 80% to 20%.Position responsibilities count for 85% while the other two areas of responsibility count for a combined total of 15%.Committee evaluations yield a ranked list of all librarians, for submission to the Director of Libraries.Process not disclosed.Director of Libraries conducts merit ranking of SMG members for all three areas of responsibility or, in some years, only for position responsibilities.Process not disclosed.Director of Libraries divides the complete ranked list into categories that will best fit the merit categories and the amount of merit money allocated that year.Process not disclosed.  Concerns Arising from the Introduction of the 2005 Merit Process As previously noted, the 1998 Process was generally regarded by ƽ岻 librarians as being fair, objective and transparent. What few problems existed with the process were minor in scale, and in some cases were rectified over time for instance, as more detailed subcategories for the various types of academic duties were introduced in the Librarians Annual Report form. The 2005 Process, when it was introduced, replaced the Merit Implementation Standing Committee with a new Merit Committee which included three elected representative positions. In principle, this new arrangement has helped address the concerns of any librarians who may have been uncomfortable with the fact that the Merit Implementation Standing Committee had consisted only of appointed members. Likewise, the establishment of the new Merit Committee has potentially resulted in procedures whereby the SMG members are now being evaluated by the same committee for all three of their areas of responsibility though, in the absence of information on what evaluation procedures the Merit Committee followed, it is not possible to know if this is indeed the case. These two improvements, however, must be weighed against the serious concerns raised by the other changes summarized in the tables above. These concerns mainly revolve around the following issues: 1) The uncollegial and hurried manner in which the 2005 Process was put into place. As described earlier, the 1998 Process was collegially developed by the Director of Libraries and the MAUT Librarians' Section. The 2005 Process, by contrast, was unilaterally introduced by the Director of Libraries and immediately used to conduct the merit award exercise for the 2004-2005 fiscal year, all in the space of less than one month. During that month, concerns regarding this new process were brought to the attention of the Director of Libraries by the Executive of the MAUT Librarians' Section and through postings by librarians on both the ALLIBRARIANS (ƽ岻 librarian staff) and AMUL-L (MAUT Librarians' Section) discussion lists. Aside from the extension of the original submission deadline for the annual reports and supervisor evaluations from September 26 to September 30, no alteration to the 2005 Process appears to have been made as a result of the concerns expressed. 2) The introduction of a problematic annual report and evaluation form. Table 2 above outlines the main differences between the annual report forms used under the 1998 Process and the annual report and evaluation form introduced under the 2005 Process. One of the most striking differences between the two is the extent to which the new Annual Performance Recognition and Development Report form downplays the responsibility which librarians have under article 1.3.2 of Chapter 2 of the Handbook of Regulations and Policies for Academic and Librarian Staff to engage in research and other scholarly and professional activities and to contribute to the University and to scholarly communities. Academic duties are almost entirely absent and are for the most part reduced to a single general category. The form no longer mentions the University's reporting guidelines on publications, an important detail which prescribes that only publications which appeared the previous calendar year (rather than fiscal year) should be listed. Librarians who teach credit courses, or engage in other types of teaching activities, no longer find any reference to this activity in the form. At the same time, the new form introduces more than a dozen new evaluative criteria. These criteria are expressed in vague terms like "judgment" with no guidance provided on how a supervisor is supposed to interpret the meaning of these concepts. The form also includes new sections asking about the librarian's career advancement targets for the next five years, knowledge or skills to be developed, and areas for improvement with target dates. These new sections contradict the principle set out in the Academic Salary Policy that merit must be awarded to individuals strictly on the basis of their academic performance during the fiscal year just completed. If the Library administration wishes to request career-planning information of this nature, it would be preferable to do so through an entirely separate document so there is no possibility that it might inadvertently influence the deliberations of the Merit Committee. 3) The elimination of various mechanisms which the 1998 Process had used to promote the fairness and transparency of the merit evaluation process. One of the reasons which allowed librarians to have confidence in the 1998 Process is that its operational details were communicated in print to every librarian staff member. By contrast, virtually no information has been released about the operating procedures used by the new Merit Committee under the 2005 Process to evaluate librarians for merit. The only detail which is known about the Merit Committee's working method the figures were stated in the merit award letter received by each librarian at the end of the process is that position responsibilities were counted for 85% of the evaluation total while the other two areas of responsibility were counted for a combined total of 15%. These two figures deviate by 10% from the normal ratio of 75% / 25 % used in six of the past seven years, and by 5% from the ratio used exceptionally in 2002-2003, as shown below: 1997-1998 = 75% / 25% 1998-1999 = 75% / 25% 1999-2000 = 75% / 25% 2000-2001 = 75% / 25% 2001-2002 = 75% / 25% 2002-2003 = 80% / 20% 2003-2004 = 75% / 25% 2004-2005 = 85% / 15% Under the usual weighting ratio of 75% : 25%, position responsibilities counted for three times as much as the other two areas of responsibility combined. Under the weighting ratio of 85% : 15% introduced by the 2005 Process, position responsibilities now count for 5.67 times as much as the other two areas of responsibility combined. This change was made without consulting the MAUT Librarians Section Executive, which during the first few years of implementation of the 1998 Process was consulted by the Merit Committee through the Observer for input on the weighting of responsibilities. The Merit Committee's composition, unlike its operating procedures, was communicated to all librarians; indeed, three of the positions on the Committee were filled by librarians who were nominated and elected by their peers. Under the 2005 Process, the Merit Committee is chaired by the Director of Libraries, whereas under the 1998 Process the Merit Implementation Standing Committee operated at arm's length from the Director of Libraries, since the Director neither chaired nor sat on this Committee. Although librarians were evaluated by the entire Merit Committee, they were entitled to vote only for one of the three elected members of the committee (the member corresponding to the staff category non-tenure track, tenured/tenure-track, SMG members to which they belonged). The very short time (two days) allowed for voting, and the requirement that voters bring their ballot in person to the Office of the Director of Libraries, may have made it difficult for some voters (notably the ones who work at ƽ岻's Macdonald Campus) to exercise their right to vote. A certain amount of confusion about the mandate of the Merit Committee was introduced in mid-September when the Director of Libraries proposed that the Committee also be tasked to work with her on a revision of Chapter 2 of the Handbook of Regulations and Policies for Academic and Librarian Staff an idea which was ultimately rejected. The elimination from the new Merit Committee of the MAUT Librarians' Section Observer is a regrettable element of the 2005 Process, since it removes an accountability feature which fostered confidence in the process among librarians. The release by the Director of Libraries of a statistical summary of the number of librarians who were allocated to each merit category, while it did not occur every year, was another mechanism which helped make the 1998 Process transparent. It is especially unfortunate that the Director of Libraries has refused to release such a summary this year, given the many changes brought about by the introduction of the 2005 Process. 4) Retroactivity of the 2005 Process. The new Annual Performance Recognition and Development Report form introduced in September 2005 changed the reporting and evaluation categories for the 2004-2005 fiscal year already completed. This has caused great concern to the librarian staff. Having set their goals for the 2004-2005 academic year under the assumption that they would be evaluated on the same basis as in the previous year, they were confronted in September 2005 with the retroactive imposition of new performance measures. The absence of information on the merit evaluation process used by the new Merit Committee likewise prevents librarians from knowing if their activities were being rated for merit according to the expectations which were in place during the period under review. The abovementioned lowering of the value of the second and third areas of responsibility from 25% to 15%, for example, may have penalized those librarians who were more active in these areas than their colleagues during the year in question. It should be noted that while the 2004-2005 fiscal year was in progress, librarians were not informed that any changes were being contemplated to the annual report form or to the merit implementation procedures. Indeed, for most of the year in question (from June 1, 2004, to January 31, 2005) the present Director of Libraries had not yet taken office. After the Director took office on February 1, 2005, no announcement regarding annual report and merit procedure changes was made until September 2005, three months after the end of the 2004-2005 year. 5) The implementation difficulties encountered when the 2005 Process was used in the merit award exercise for the 2004-2005 fiscal year. The great haste with which the 2005 Process was introduced and used to conduct a merit exercise at one of the busiest times of the year caused numerous difficulties. Only a few days were allowed for librarians to complete their Annual Performance Recognition and Development Report, for their supervisors to write a performance evaluation, and for librarians and supervisors to meet in order to discuss and sign the forms. Some units which had already taken the initiative to write their annual reports and performance evaluations in the spring using the previous year's forms had to spend time transferring their work to the new, different form as best they could. Units which had not taken this initiative, since the call for reports and evaluations customarily made by the Office of the Director of Libraries in the spring was not made until September, had to scramble to complete the entire process in the very short period of time allowed. At least three librarians did not have a chance to see their own evaluations before the Merit Committee met to consider them. One unit was also hampered by the fact that its Administrative Librarian was out of the country at the time, and thus unable to write evaluations by the initial submission deadline. The short amount of time allocated to the Merit Committee to do its work raises additional concerns. The Merit Implementation Standing Committee for the 1998 Process, it will be recalled, would typically devote several weeks to reviewing the librarian annual reports and the supervisor evaluations prior to holding its meetings. As already noted, the Merit Committee under the 2005 Process devoted one or two preliminary meetings to discussing the Annual Performance Recognition and Development Report form, and devoted two meetings one on the morning of Wednesday, September 28 and one during the afternoon and evening of Sunday, October 2 to the merit evaluations themselves. Given that some fifty-seven librarians were eligible for merit for the year under consideration, the Merit Committee may only have been able to spend a limited amount of time on each file, and may have had less opportunity than under the 1998 Process to compare the files to each other to ensure uniformity of scoring. The merit exercise conducted in September 2005 resulted in some twenty-five librarians (out of about fifty-seven eligible for merit that year) filing appeals of their resulting merit awards. The appeals filed tended to involve cases in which librarians who had been given high-level or even top-level evaluations by their supervisors had nevertheless received low or even no merit, and/or cases in which the content of evaluations by supervisors was questioned. Librarians who filed appeals received a letter from the Director which communicated the Director's decision on their appeal and which included the Director's reasons, e.g., activities such as publication and conference presentations were discounted because not enough people did them; librarians who did not file appeals did not receive any sort of feedback on the reasons for the results of the merit process. Approximately seven of the appeals not resolved at the level of the Director of Libraries were further appealed to (then Interim) Provost Anthony Masi; of these, four have subsequently proceeded to some stage of the grievance process. Recommendations In light of the difficulties outlined above, it is to be hoped that these problems can be addressed and resolved through discussions between the Director of Libraries and the MAUT Librarians' Section. This partnership developed the 1998 Process which, over the course of seven merit exercises, was regarded by the librarian staff as being fair and transparent. That confidence was disrupted by the advent of the 2005 Process and by the merit exercise which immediately followed its introduction. It is imperative to restore this confidence in the next merit exercise. To that end, we recommend that the following principles be applied in revising without delay the library merit process currently in place: 1) Librarian staff should know in writing by the beginning of any given fiscal year the reporting mechanisms, evaluative standards and committee procedures by which they will be evaluated for merit at the end of that fiscal year. 2) All details pertaining to the merit procedure should be collegially developed and applied. 3) Adequate time should be allowed for the merit evaluation committee to assess the annual reports and supervisor evaluations of all librarians, both individually and comparatively, to ensure that all activities are weighed consistently according to well-defined guidelines. 4) The evaluation procedures should accurately reflect the three areas of responsibility specified for librarians as they exist in the Handbook of Regulations and Policies for Academic and Librarian Staff. 5) Librarians should be provided with feedback which clearly indicates why they were given the merit award they received in any given year. Respectfully submitted, The Professional Issues Committee of the MAUT Librarians' Section Louisa Piatti (Interim Chair) Maryvon Ct Lorie Kloda Marc Richard Phyllis Rudin Natalie Waters Elaine Yarosky April 20, 2006 Attachment: PDF file of appended documents     PAGE  PAGE 1 The Merit Review Process for ƽ岻 Librarians: A Comparison of 2004 and 2005 (April 20, 2006) 8@EHO\    G K ^ ̾xqcxq\UNUqGqG htR+h_r hh{)A hhc hh,hZhn5>*CJaJ htR+hc htR+hnh<hn5B*ph h<hk85B*CJaJphh<5CJaJhZhk85CJaJhZhn5CJaJhh|D#57CJaJhhk85CJaJhh<5CJaJhh<5>*CJ aJ hh\5>*CJ aJ 0NO 7gd>l & ' m w ( . 5 S W Ⱦүҡҡș҄rd]V htR+h[ htR+h}7hZhn5>*CJaJhZh;35>*CJaJhg? htR+h;3 htR+hE( hh ZhtR+h Z5 htR+h Z hhg? hhFH$htR+hg?5htR+hg?56htR+h Z56 htR+hg?hZhg?5>*CJaJ htR+h_r htR+hwh;3 htR+hc mt'wxFOPfjWXf%Ⱥݗݗݗ֗֓֌֌ֈց֌ htR+hhY htR+hbh` htR+h7 htR+hL9 h1h/& h1h> h1h;3 h1h, htR+h> htR+h;3 htR+h[ htR+h7 htR+h?t htR+hS htR+hf0 htR+h{' htR+hF|078`aE"F"h#i#%%''* *--00gdfgdzgd;3gd>%*1<hlEL_`aKLU+/Shp -TĽֲֲIJIJīī֫h`htR+h >* htR+h htR+h htR+hbQYhY htR+h;3 htR+h/&hY htR+hfb htR+hL9 htR+hN htR+hb htR+h{' htR+h htR+h]><Ac0IzWXqrAF u!!!" "D"i##$򠙠} htR+h htR+h0 htR+hb htR+h;e htR+hpl htR+hzhZhd# 5>*CJaJ htR+hwhtR+hd htR+h htR+hd h` htR+hs; htR+hm htR+h[x htR+h+z htR+h;3 htR+h 0$0$%%i%%%%%.&@&[&n&&&'1''''$(,(0(1(2(z(((((((( )3)4)G)[)\)])_)`)))Ϻȥϗȗϐ{{ h1hP#! h1h@;u htR+h%) htR+hwx htR+h htR+hSv htR+hY h1hz) h1hf h1h\p7 htR+hb htR+h`H htR+hf htR+hz htR+h; h1h h1h$ htR+h htR+h[+)))*** ***"*&***+++++,, ----<-F-g----Y.[.ɾɬɡzszlze^eW htR+hN htR+hN htR+h  h1h3O h1h9_ h1hP9n h1h+^P h1hzC h1h l htR+hzCh9_h l htR+hP9n htR+h+^Ph`h+^PmH sH h`hwxmH sH  htR+hwxh@;uh%)7B*phh@;u htR+h%) h1h%) h1h9_CJOJQJ^JaJ[.....0000000T11222222M3N3Z3_333333333333 4444b4v4444zs h1hE h1h [ h1hY h1hN h1hb h1h9_ h1h@<\ h1hD h1hN h1hmK h1h(9 h1hLhL htR+h(9 htR+hN htR+h`H htR+h htR+hNh` htR+h%) htR+hO+0s1t1445588[?\???????AACCDDEDkDDDDgd#gd}7gdzgd/gdf444444*5+5V5X5e555556667*797i7777&808[8f88888888 9>9^9륞랗{t{m{ft htR+hi htR+h" htR+hI( htR+h htR+hb htR+h[ htR+h| htR+hS) h1hS) h1hb h1hnp h1hFH$ htR+h [ htR+h@ h1h9_ h1h` h1h/ h1h@<\ h1h| h1h [ h1hE h1h0&^9c9o9::;:f:w::::;;W;Z;n;{;;;;;;;;;;;;O<P<Q<<<<<<<=1=8=C=Y===>>>>>>>>> ? ??˽˨ҋ htR+hDwhz)hc htR+h h1h h1ht h1h h1hc h1h@<\ h1h9_ h1hL@ htR+hI( htR+hz htR+hL@ htR+h/h2 htR+hY htR+h6??&?(?/?1?Z?[??????????.@k@@@@@@AAIAAAAAAAB&B(BBBBZCyC~CCCCCCCCCC˽˽˶}h3O htR+hi htR+hEh htR+hN htR+hTU htR+hC htR+h# htR+hk+ htR+h}7hZh#5>*CJaJhZh}75>*CJaJ htR+hwhDw htR+hz htR+hL@ htR+hDw htR+h/1CCDD$D@DADCDFDEEEEEEEEF1F:FIFPFgFqFFsGGGGGG/HCHDHHHaHjHrHzHHHHHHHHHHHHHIѼʵʵ h1hku: htR+hN htR+h1i htR+hth htR+hvb h1h h1hvb h1h3O h1hx h1h# h1hi h1hhhB*ph htR+hx htR+hi htR+h#3DE'E(EEEGG;I* htR+h; h1h; h1h h1hx h1h# h1hku: h1h1i8P>P?P@PPPwQxQRRSS TTTTTTHTITJTlTmTnTTTT|pdYNCNhhCJaJhhk8CJaJhhZCJaJhZh7{>*CJaJhZhZ>*CJaJhh5>*CJaJhZhQ15>*CJaJhZh7{5>*CJaJ htR+hQ1 htR+hwh>Y< htR+hY< htR+hC htR+hNh2 htR+h3 htR+hthZhtCJaJhZhJ5>*CJaJTTTTTTTTTTITJTmTnTTT$Ifgdk8l $Ifgdk8l gdk8gd}7gd#TTU>UjWW$Ifgdk8lkd$$Ifl0,"LL  t0644 lapTUUUU?UUUyff$Ifgdk8lkdo$$Ifl0,"LL t0644 lapUUhVVyff$Ifgdk8lkd$$Ifl0,"LL t0644 lapVVWDXyff$Ifgdk8lkd$$Ifl0,"LL t0644 lapDXEXXYyff$Ifgdk8lkdb$$Ifl0,"LL t0644 lapXXY+YlYYYYYYYYYYYYYYYYYZZZ`ZaZcZZxmxbWbWxWxLhhCJaJhhU CJaJhh#bCJaJhhCJaJhhk8CJaJhhFGlCJaJhZCJaJhZh>*CJaJhZhZ>*CJaJhhwCJaJhwCJaJhtR+CJaJhhd%CJaJh1hku:CJaJh1h;CJaJhh;CJaJhh^OCJaJYYYYYYYYYYYYYYyttttttttttttgdk8kd$$Ifl0,"LL t0644 lap YYYYZbZD11$Ifgdk8lkd$$Ifl0,"LL  t0644 lap$Ifgdk8l $Ifgdk8l bZcZZ;[yff$Ifgdk8lkds$$Ifl0,"LL t0644 lapZZ<[[[[[\\\,\?\\\] ]T]U]V]]]]]]]]]_j_l___X`Z`[`s`t`u````ģϘϘϣϘϘwlhhCJaJhZh>*䴳>*䴳>*䴳^䴳Y䴳峦䴳*䴳8䴳ѷ%䴳%䴳3䴳䴳`䴳(;<[\]yff$Ifgdk8lkd$$Ifl0,"LL t0644 lap] ]V]]yff$Ifgdk8lkd$$Ifl0,"LL t0644 lap]]]k_yff$Ifgdk8lkdf$$Ifl0,"LL t0644 lapk_l__Y`yff$Ifgdk8lkd$$Ifl0,"LL t0644 lapY`Z`[`t`u```wrrr_L$Ifgdk8l $Ifgdk8l gd}7kd$$IflF0,"LL t0644 lap````jWW$Ifgdk8lkd]$$Ifl0,"LL  t0644 lap`````!a#aCaaaaaaabbbccc3c4c5c9c:cJcLcbceciccccccccd3dBdDddɾɾɾɳꝳwԳllh1hoCJaJh1h8~CJaJho<CJaJhCJaJhh:CJaJh1h:CJaJhh 9CJaJh1hk8CJaJh1h{)ACJaJh1h&_CJaJh1h 9CJaJh1hgg CJaJhhk8CJaJhhgg CJaJ)``bcyff$Ifgdk8lkd$$Ifl0,"LL t0644 lapcc4cKcyff$Ifgdk8lkd$$Ifl0,"LL t0644 lapKcLcccyff$Ifgdk8lkdn$$Ifl0,"LL t0644 lapccddyff$Ifgdk8lkd$$Ifl0,"LL t0644 lapdddddddde e0e1e=e@eMeUeWeseveyezeeeeeeefff=f>fEf]fifjffĹ|ppehh>GCJaJhZh>G>*CJaJh>Gh>GCJaJh>GCJaJh>G>*CJaJh1hO=CJaJh1h&_CJaJh1h{)ACJaJh1hoCJaJhh^OCJaJh1h^OCJaJh1h:CJaJh1h 9CJaJhoh^OB*CJaJph$ddtefyff$Ifgdk8lkd$$Ifl0,"LL t0644 lapfff>f?f@fAfBfCfDfEfjfkfytttttttttoogd>Ggdkda$$Ifl0,"LL t0644 lap kffffVggD11$Ifgd, lkd$$Ifl0,"LL  t0644 lap$Ifgd, l $Ifgd, l fggThxh ijjjjjjjjjjk k kkvkwkkkkkkk3l4l5lKlMlllll-m.mmmmnʿ}rrrrrrjrjrh`CJaJhh]CJaJhhp(CJaJhheCJaJhh}7CJaJhhCJaJhhtR+CJaJhhk8CJaJhhCJaJhh8CJaJhZhZ>*CJaJh>G>*CJaJhh>GCJaJh1h>GCJaJ)ggThwhyff$Ifgd, lkd!$$Ifl0,"LL t0644 lapwhxh iiyff$Ifgd, lkdr$$Ifl0,"LL t0644 lapiijjyff$Ifgd, lkd$$Ifl0,"LL t0644 lapjjjjjjjjjjjjjjyttttttttttttgdkd$$Ifl0,"LL t0644 lap jjjjjjjk$Ifgdk8l $Ifgdk8l gdkkwkkjWW$Ifgdk8lkde$$Ifl0,"LL  t0644 lapkkkkyff$Ifgdk8lkd$$Ifl0,"LL t0644 lapkk5lLlyff$Ifgdk8lkd% $$Ifl0,"LL t0644 lapLlMlllyff$Ifgdk8lkdv $$Ifl0,"LL t0644 lapllmnyff$Ifgdk8lkd $$Ifl0,"LL t0644 lapnnnnnyff$Ifgdk8lkd $$Ifl0,"LL t0644 lapmnnnnnoo5o7oooooooo5p6p7papjppqq qqq9r=rQrarprrrsssstttttHuzszssssssl hVh L hVh=E hVhH h|h h|h|h2h| hVh_B hVhw hVhI1 hVhhh5>*CJaJhhT5>*CJaJhh7mCJaJh>GCJaJhCJaJhh]CJaJhh^OCJaJ*nno6oyff$Ifgdk8lkdi $$Ifl0,"LL t0644 lap6o7oooyff$Ifgdk8lkd $$Ifl0,"LL t0644 lapooooo6p7pqqtttuuuvuyttttttttttttgdkd $$Ifl0,"LL t0644 lap HuOubusutuuuvu}uuuuuuvvvv^wwx x$x,xDxExxxxxx&yMyOyPyRySyTyyyyyyQzž뷰돇wp hVhqdhVhw5hVh L6hVhw6 hV6 hVh hVho< h1h~? h1hFH$ h1hk hVhk h`h~? h`hwh`h L6h`hw6 hVhVhVhw hVhw hVh L hVh~?*vuuuNyOyPyyy}}12х)?UVgd+gdQzz;{{{{{{ |||||\}}}}r~s~~~~~~~ˁ-.2ü{sksk{hVh6hVh L6 hVhwhVhw6 hVhV hnphd hnphnph~@ hVh;\ hVhd hVhpBah2 hVh> hVh'O hVh% hVh hVh^ hVho<h`hVhmy>* hVhmy hVhg*2ER^`$.ci߃.HIlq„ńʄTUV} -fmnƾ٤䤝 hVh% hVhQh;45ΐϐАyzѤҤӤԤgdlgd#gdgdeztgdpō 45܎&͐ΐАWX‘(DEҒڒےCDLoqƿ}vov hVh%z hVh hVhx{w hVh+ hVh hVhm hVh# hVhv.hVhw6hVhv.6hVh hVh hVh h` hVhMxO hVhO Xh2hVhi*>* hVho< hVhi* hVhUk*,Eiou|ߗ({tmmfmf_ hVhx{w hVh>6W hVhB! h1hB! h1h h1h L h1hwh1hw6h1hFH$6h1h6hVh6hVhw6hVhv.6hVhO hVh^< hVhO hVhl hVh# hVh+ hVhm hVh%z hVhx%,EFITZ~"'=ϚU*+؝xyƠǠѠҠڠޠߠ 9:m|ɡ:<HJμ缵缮箧îîhqB hVhi* hVhj hVh hVh` hVhz6sh2 hVh4D) hVhx{wh` hVhUk hVh] h~@ hVhB! hVh>6W?efФѤҤӤԤ @ئQŖקڧ&04>Ihبۨyryryrkdrdkrrd hVh M hVhH hVhy hVhHhh` hVh0:V hVhmh"( hVh|x hVh8hhr5>*CJaJhh0:V5>*CJaJhVhx5>*ht hVhj hVh hVhi* h1hi* h1h h1he h1hqB%ժ֪bcd|ݫ /0?@AlgdygdHgd Ӫ&1;AFO{|Ϋիܫݫ/09>@lmoprsuvxyƻ䷳|hY0JmHnHuht ht 0Jjht 0JUhcBajhcBaUhghnhh9_hyhVhVmH sH hVhymH sH  hVh}7h"(hVhy>* hVhy hVh M hVhH hVhyh/lnoqrtuwxܬgdyh]hgdih]hgd? &`#$gd0Q^ܬݬhghcBaht hcht 6 ht 6hiht 6 ,1h/ =!"#$% m$$If!vh5L5L#vL:Vl  t65LO$$If!vh5L5L#vL:Vl t65LO$$If!vh5L5L#vL:Vl t65LO$$If!vh5L5L#vL:Vl t65LO$$If!vh5L5L#vL:Vl t65LO$$If!vh5L5L#vL:Vl t65Lm$$If!vh5L5L#vL:Vl  t65LO$$If!vh5L5L#vL:Vl t65LO$$If!vh5L5L#vL:Vl t65LO$$If!vh5L5L#vL:Vl t65LO$$If!vh5L5L#vL:Vl t65LO$$If!vh5L5L#vL:Vl t65LS$$If!vh5L5L#vL:VlF t65Lm$$If!vh5L5L#vL:Vl  t65LO$$If!vh5L5L#vL:Vl t65LO$$If!vh5L5L#vL:Vl t65LO$$If!vh5L5L#vL:Vl t65LO$$If!vh5L5L#vL:Vl t65LO$$If!vh5L5L#vL:Vl t65LO$$If!vh5L5L#vL:Vl t65Lm$$If!vh5L5L#vL:Vl  t65LO$$If!vh5L5L#vL:Vl t65LO$$If!vh5L5L#vL:Vl t65LO$$If!vh5L5L#vL:Vl t65LO$$If!vh5L5L#vL:Vl t65Lm$$If!vh5L5L#vL:Vl  t65LO$$If!vh5L5L#vL:Vl t65LO$$If!vh5L5L#vL:Vl t65LO$$If!vh5L5L#vL:Vl t65LO$$If!vh5L5L#vL:Vl t65LO$$If!vh5L5L#vL:Vl t65LO$$If!vh5L5L#vL:Vl t65LO$$If!vh5L5L#vL:Vl t65LO$$If!vh5L5L#vL:Vl t65L@@@ NormalCJ_HaJmH sH tH DA@D Default Paragraph FontRi@R  Table Normal4 l4a (k(No Listjj Q1 Table Grid7:V0@ Table Professionall:V0j%  5B*\`Jph  Table List 4w:V0    j0   5B*\`Jph4@"4 cHeader  !4 @24 cFooter  !.)@A. ? Page NumberD0NO7 8 `aEFhi" "%%((s)t),,--00[7\777777799;;D<E<k<<<<='=(===??;A?IJMMNNOٱPQQQQQQQQQQQQQQQQQR;^?^@^A^B^C^D^E^j^k^^^^V___T`w`x` aaabbbbbbbbbbbbbbbbbbbbbccwcccccc5dLdMddddeffnfffg6g7ggggggg6h7hiilltmumvmmmNqOqPqqquuyyy1z2z{{}}}}}}~)~?~U~V~45ΈψЈyzќҜӜԜբ֢bcd|ݣ /0?@Alnoqrtuwxܤ0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0000000000000 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0000 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0000000000@0@0 @0 @0 0 0 0 0 0 0 @0 @0 @0 @0 @0 @0 000000000000000000 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000@0h00@0h00@0h00@0h00@0@0@0@0@0@0@0h00^^V___w`x`aabbb~)~?~U~V~j00Ph00j00h00h00h00UUVDXYYbZ;[]]k_Y```cKccdfkfgwhijjkkkLllnn6oovulX\bgjkmnoprstvwxyz{}~Y  !!8@0(  B S  ? E?-i]ݤu]8*urn:schemas-microsoft-com:office:smarttagsdate  q 19199820200520064789DayMonthYear        W X KL^^^=^>^D^^_bbbbbbgg}ќҜkllnnooqrtuwxaU( (+-2-345555MMsV{V^^meqenn{~~~"&llnnooqrtuwx333333333333333aS"""m<<<<<<<=='=JLlLnLLQQ[XXXX[L[N]t]E^^{bbbcwcccc5dMdddnffg7ggg9msmЈԜd|ݣ.0AllnnooqrtuwxܤHLllnnooqrtuwxLK+myCmHhSQ| >nG ;\ ; U x] d# * : fb ] m q l t [iUkJU>G L.;_c%)+B!j1I(F28 gg P#!##|D#$$FH$d%~%/&w6&{'E(p(4D)w)h*i*k+tR+;-//f0I1Q1OX23;3_3R4p6\p7}7k8(9L9Y9ku:s;<Y<^<o<]>??g?)?Դ,Y;6WO XIXYbQYZ@<\?^0Q^&_ acBapBa3HbQbvbJce;e;+gt_gyh1iklFGlP9npnpxnpsz6s?tezt@;uwx{woxwx+zF|r~_B:;gU!9=HN}7 []E%zx(jiKQ7{x7SvBEH 7;fHYg8~=+, cnK`_r} ^xd z)(R9_%|xNDCL;t8[x\|,"( O=> @Dw@vmr^jrmn JS)'OmKY 9Ou~?yCjQ[oyrftje MOcdvww1I)2T` Z#bbzC#qB~@plTU ;iDv.R\SqdDPS ~_Qw04@i "7mgRt=Yb8>ԳLLL>?IJMMNNOٱPQQQQQQR;?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~Root Entry F0:R