Bibliography of Bias In Evaluation: Evaluation Procedure

  1. Getman, Julius. (1992). In the Company of Scholars: The Struggle for the Soul of Higher Education. Austin, TX: University of Texas Press.
    Keywords: tenure, peer review, justice and injustice
    Summary: Addresses on pages 109-129 “Tenure, Peer Review, Excellence, and Injustice.” Discusses the basic concepts behind peer review and the ways in which they may falter- with implications for the “soul of the academy.
     
  2. Barron, Dennis. (2002, September). "Getting Promoted." The Chronicle of Higher Education: Career Network, 1-5.
    Keywords: promotion and tenure, preparing dossiers, teaching and research, external reviewers
    Summary: Offers subjective observations and interpretations of how promotions occur using the University of Illinois at Urbana-Champaign as a case study. Notes the emotional toll the process takes on junior faculty members.
     
  3. Nieva, Veronica and Gutek, Barbara.(1980). "Sex Differences in Evaluation." Academy of Management Review, 5, 267-276.
    Keywords: gender, performance, evaluation, bias
    Summary: Reviews research on evaluation of qualifications and performance of men and women, and research on explanations/causes of performance. Most research shows "pro-male bias" in evaluation.
     

  4. Bergquist, William H. and Phillips, Steven R. (March 1975). “Components of an Effective Faculty Development Program.” The Journal of Higher Education 46: pp. 177-211. Full Article: https://www.jstor.org/stable/1980880
    Keywords: faculty development, institutional climate
    Summary: Since piecemeal efforts to improve college and university teaching have generally proven ineffective, we must turn to a comprehensive approach to faculty development, through which we can develop new methods of evaluation and diagnosis, find viable ways of introducing new technology and curricula, and explore new approaches to instructional improvement. Faculty development must give serious attention to the impact of change on the faculty member himself and on his institution. Organizational and personal development thus become essential to faculty development. It is only through such a comprehensive approach that efforts toward improvement can have lasting impact.

  5. Goodman, Madeleine J. (July 1990). “The Review of Tenured Faculty: A Collegial Model.” The Journal of Higher Education 61:pp. 408-424. Full Article: https://www.jstor.org/stable/1982078
    Keywords: union, peer review, compensation
    Summary: The University of Hawaii post-tenure review focuses on collegial professional standards rather than salary issues. By administration-union agreement chairpersons apply departmental expectations to assay deficiencies subject to final adjudication by a peer committee. Individual remediation plans emphasize faculty. Outcomes of the first review cycle are positive.

  6. Glasman, Naftaly S. (May 1976). “Evaluation of Instructors in Higher Education: An Administrative Function.” The Journal of Higher Education 47: pp. 309-326. Full Article: https://www.jstor.org/stable/1981424
    Keywords: evaluation instruments
    Summary: A conceptual framework containing three domains for an administrative perspective on faculty evaluation is presented. The first domain deals with faculty need satisfaction; the second centers on the instructors' work environment; and the third relates to the appropriateness of evaluation instruments.

  7. Kuh George D; Ransdell, Gary A. (May 1980). “Evaluation by Discussion: An Evaluation Design for Postsecondary Programs.” The Journal of Higher Education 51: pp. 301-313. Full Article: https://www.jstor.org/stable/1981011
    Keywords: models
    Summary: A formative evaluation model designed specifically for use with postsecondary programs is described. Evaluation by Discussion (EBD) is a relatively simple process that can be conducted without external evaluators. Previous experience with EBD suggests that the process is inexpensive, comprehensible, and responsive to needs of various program audiences.

  8. Rood, Harold J. (March 1977). “Legal Issues in Faculty Termination: An Analysis Based on Recent Court Cases.” The Journal of Higher Education 48: pp. 453-471. Full Article: https://www.jstor.org/stable/1979103
    Keywords: bias, legal cases
    Summary: Recent court cases involving faculty termination spell out in fairly clear and sometimes surprising ways the legal issues associated with faculty termination. The interpretations one finds in court of issues such as expectancy of reemployment, freedom of speech, and from suit are at great variance with those voiced within the academic community. An understanding of this shift in rules will help both faculty members and administrators avoid decisions that result in expensive and needless litigation.

  9. Lee, Barbara A. (January 1985). “Federal Court Involvement in Academic Personnel Decisions: Impact on Peer Review.” The Journal of Higher Education 56: pp. 38-54. Full Article: https://www.jstor.org/stable/1981721
    Keywords: bias, legal cases"
    Summary: It is widely believed that the courts have intruded on the prerogatives of academics to use peer evaluations in personnel decisions. This article's analysis of academic discrimination cases in the federal courts demonstrates that judicial review is deferential to peer judgments and appears to have little impact on colleges' faculty personnel policies.

  10. Harvey, Thomas R. (November 1974). “A Heretical Approach to Evaluation.” The Journal of Higher Education 45: pp. 628-634. Full Article: https://www.jstor.org/stable/1980730
    Keywords: faculty evaluation, valuation of criteria
    Summary: This paper suggests that, in situations of fiscal crisis, inadequate prior evaluation and time constraints, faculty evaluation cannot merely follow the traditional models of systematic evaluation. An alternative evaluation approach, called here "Aggregate Perception Evaluation Procedure" (APEP), is suggested and its usefulness for limiting the extent and scope of analysis is described.

  11. Dowdall, Jean. (May 2004). “Haunted by the Past.” The Chronicle of Higher Education Career Network.
    Keywords: references, hiring decisions
    Summary: Discusses how references play a key role in hiring decisions and how a candidate can increase their chances by strategically choosing their references and bring aware of references that may not be as positive as originally thought. Also discusses strategies for overcoming weaknesses in a job application
     
  12. Dowdall, Jean. (June 2004). “Haunted by the Past, Part 2.” The Chronicle of Higher Education Career Network.
    Keywords: references, hiring decisions, search committees
    Summary: Addresses how to use references from a search committee’s standpoint. Gives recommendations for how to evaluate the validity of negative references and under what circumstances certain negative attributes of a candidate can be overlooked
     
  13. Fogg, Piper. (June 2004). “Hello… I Must Be Going.” The Chronicle of Higher Education 50: pg A10.
    Keywords: tenure system, tenure reform, ivy league schools, faculty turnover
    Summary: Discusses the opportunities for tenure at many of the top Ivy League universities and the effects on the faculty and the university of the difficulty to obtain tenure. Uses individual cases to show frustration with the tenure system and suggests that reform of the tenure system may be appropriate at some universities.

    Accountancy for Qualitative Judgement of Scholarship

    1. Barron, Dennis. (2003, January). "External Reviewers." The Chronicle of Higher Education: Career Network, 1-5.
      Keywords: promotion and tenure process, letters of reference, external reviewers, evaluation of scholarship, tenure committees
      Summary: Illustrates the tenure process from a department’s point of view with references to the recent tenuring of a female faculty member. Focuses upon the impact of external reviewers and particularly on how tenure committees respond to external review letters when assessing a candidate’s scholarship.
       
    2. Allen, Henry Lee. (2001). "Workload and Tenure Policies In an Era of Organizational Change." The NEA Almanac of Higher Education.
      Keywords: workload policies and tenure, institutional forces, contextual factors, individual contours, structural contours, organizational contours, systemic contours
      Summary: States most studies of workload and productivity target items under the immediate control of professors while avoiding analysis of institutional forces affecting faculty decisions, priorities, or behaviors. Claims factors such as recruitment, search and selection patterns, mentoring, sponsorship, and social networking are ignored along with the effects environmental and organizational influences have upon academic careers. Suggests making valid inferences about the social behaviors of groups or individuals requires examining the individual, structural, organizational, and systemic parameters of an academic system.
       
    3. Letters to the Editor. (2002, April 5). "Collecting Letters of Recommendation: Can This Process Be Saved?" The Chronicle of Higher Education, 48, 1-4. Retrieved October 26, 2003, from http://chronicle.com/prm/weekly/v48/i30/30b00402.htm
      Keywords: promotion and tenure, letters of reference, integrity of process, corrupting forces, financial compensation
      Summary: "Examines the integrity of evaluation process for first appointments and for promotion or tenure. Highlights corrupt aspects of the process; for example how letters are made available to candidates, leaving the author open to retribution or legal action. Suggests improving the integrity of the system by financially compensating individuals asked to write recommendations.
       
    4. Fish, Stanley. (2002, September 13). "Somebody Back There Didn’t Like Me." The Chronicle of Higher Education: Career Network. Retrieved October 26, 2003
      Keywords: explicit procedure, explicit requirements, voting guidelines, annual review, mentoring, intellectual responsibility
      Summary: Discusses how the tenure process is open to opinion based on subjective, non-professional categories that can haunt the career of a failed tenure candidate. Suggests remedies for problems of personality encroaching on tenure process.
       

    5. Lamberti, Andrea. (1990, September 14). "Group studies bias case." The Tech. Retrieved October 26, 2003, from http://tech.mit.edu/V110/N34/kalonj.34n.html
      Keywords: gender bias, tenure process, tenure appeal, inexact procedure
      Summary: Details the appeal of tenure review committee decision on Gretchen Kolanji. Suggests the tenure process improperly took political activities and gender into account.

    6. Park, Shelley M. (January 1996). “Research, Teaching, and Service: Why Shouldn't Women's Work Count?” The Journal of Higher Education 67: pp. 46-84. Full Article: https://www.jstor.org/stable/2943903
      Keywords: gender bias, workload
      Summary: This article critically examines current university tenure and promotion criteria, suggesting that such criteria are both an effect and a source of gender bias in academia. It argues that current working assumptions regarding (1) what constitutes good research, teaching, and service and (2) the relative importance of each of these endeavors reflect and perpetuate masculine values and practices, thus preventing the professional advancement of female faculty, both individually and collectively.

    7. Menges, Robert J. and Exum, William H. (March 1983). “Barriers to the Progress of Women and Minority Faculty.” The Journal of Higher Education 54: pp. 123-144. Full Article: https://www.jstor.org/stable/1981567
      Keywords: gender bias, racial bias
      Summary: This article documents the underrepresentation of women minority faculty at senior ranks and discusses the contribution of promotion and tenure reviews to this situation. The analysis emphasizes potential conflicts between personal and professional values of women and minorities on the one hand and meritocratic values of academic culture on the other.

    8. Scott, Richard R. (Summer 1981). “Black Faculty Productivity and Interpersonal Academic Contacts.” The Journal of Negro Education: pp. 224-236. Full Article: https://www.jstor.org/stable/2295154
      Keywords: racial bias, affirmative action
      Summary: Describes how tenure and promotion criteria have an adverse effect on blacks. Racism can diminish scholarly achievements of black faculty directly or indirectly. Considers how interpersonal contact between black and white colleagues can inhibit black productivity.

    9. Miller, A. Carolyn and Serzan, Sharon L. (November 1984). “Criteria for Identifying a Refereed Journal.” The Journal of Higher Education 55: pp. 673-699. Full Article: https://www.jstor.org/stable/1981508
      Keywords: prestige of journals, perceptions of peers
      Summary: A survey of 349 editors reveals that few academic and professional journals follow recommended criteria for identifying a refereed journal. Practices vary across disciplines, at times significantly. This article discusses the effects of reviewing procedures on acceptance rates and their implications for promotion and tenure decisions. A standard for identifying refereed journals is proposed.

    10. Greenburg, Michael and Zenchelsky, Seymour. (Autumn 1990). “The Confrontation with Nazism at Rutgers: Academic Bureaucracy and Moral Failure.” History of Education Quarterly 30: pp. 325-349. Full Article: https://www.jstor.org/stable/368692
      Keywords: racial bias, political bias
      Summary: Describes in incident at Rutgers University involving an administrative cover-up of Nazi administrator. Involves bias in evaluating faculty member with different political views.

    Teaching Workload - Departmental Structural Bias

    1. Brehm, Denise. (2002, December 4). "Some faculty dissatisfied with quality of life at MIT." TechTalk. Retrieved October 26, 2003
      Keywords: workload, publish or perish, family, research, teaching, quality of life, parentin
      Summary: States that 41% of women faculty and 36% of men faculty are dissatisfied with MIT. Reports that 75% of respondents say that no matter how hard they work, they cannot get everything done. States 63% work more than 60 hours per week under intense pressure to publish.

    Service Workload - Committe, Service, Organizer

    1. Evans, Catherine. (2003, April 4). "Giving Up a Good Thing." The Chronicle of Higher Education: Career Network.
      Keywords: service workload, teaching workload, collaboration, family, parenting, publish or perish, qualitative research, teaching appraisal and tenure
      Summary: Discusses how pressures to research and publish conflict with family, service, and teaching. Rejects unrealistic demands of academia.

    Valuation for Criteria for Advancement- Teaching vs. Service vs. Research

    1. Linse, Angela R. (2002, November). Student Ratings of Women Faculty: Data and Strategies. Seattle, WA: Center for Engineering Learning and Teaching and Center for the Advancement of Engineering Education, 1-6.
      Keywords: student perspectives, faculty authority and credibility, student expectations, contact and accessibility, teaching effectiveness, tenure and promotion
      Summary: Lists concerns of women faculty from SEM disciplines about student ratings. Presents research findings about effects of instructor gender on student ratings of teaching. Suggests responses to suspicions of gender bias. Finds male and female faculty are concerned with how student ratings are interpreted and used by peers and administrators since at many institutions student evaluations are the only measure of teaching effectiveness included in tenure and promotion decisions.
       
    2. Tierney, William G. (1998). The Responsive University: Restructuring for High Performance. Baltimore, MD: The Johns Hopkins University Press.
      Keywords: promotion and tenure, evaluating faculty performance, definition of scholarship, reward structure, social partnerships, restructuring
      Summary: Examines the importance of widening the definition of scholarship. Shows that officials at many institutions do not readily find consensus on matters pertaining to reward structure. Suggests that campuses have a long way to go before they show that other forms of scholarship are accorded value equal to traditional research. Notes that institutions have begun enhancing rewards for teaching, which the author considers a notable beginning for institutions pursuing a more balanced definition of scholarship.
       
    3. Fox, Mary Frank.(1985). Publication, Performance, and Rewards in Science and Scholarship. In J. Smart (Ed.), Higher Education: Handbook of Theory and Research (pp. 255-282). New York, NY: Agathon Press, Inc.
      Keywords: performance; publication; rewards; equity and inequity.
      Summary: Addresses the reward structure of science and academia, with principles of reward based upon "achievement", variable performance levels in publication, and explanations of different levels of performance. Includes descriptions of implications and recommendations improving for equity and performance.
       

    4. Magnusen, Karl O. (September 1987). “Faculty Evaluation, Performance, and Pay: Application and Issues.” The Journal of Higher Education 58: pp. 516-529. Full Article: https://www.jstor.org/stable/1981785
      Keywords: performance, workload
      Summary: A job-related system for evaluating faculty performance in a college of business is examined. The method emphasizes the role of the department chair, links weighted faculty responsibilities in teaching, scholarship, and service, and uses results in annual evaluations and salary recommendations.

    5. Goodwin, Laura D. and Stevens, Ellen A. (March 1993). “The Influence of Gender on University Faculty Members’ Perceptions of ‘Good’ Teaching.” The Journal of Higher Education 64: pp. 166-185. Full Article: https://www.jstor.org/stable/2960028
      Keywords: gender differences, teaching style
      Summary: The responses of 250 faculty members to questions about "good" teaching were analyzed by gender as well as by rank and discipline. Interesting differences between the responses of the female and male professors emerged from the analyses, although overall there were relatively few gender-related ones.

    6. Linsky, Arnold S. and Straus, Murray A. (January 1975). “Student Evaluations, Research Productivity, and Eminence of College Faculty.” The Journal of Higher Education 46: pp: 89-102. Full Article: https://www.jstor.org/stable/1980925
      Keywords: relation between teaching and research
      Summary: The relationship of a professor's involvement in research to his classroom performance has been vigorously disputed for some time, but discussions of this problem have been largely polemical. This study determines empirically the relationship of these two dimensions of academic competence for a national sample of 16 colleges and universities. From this data it would appear that teacher ratings are only partly due to individual differences in teaching abilities but also vary with position within the social structure of the university. Some of the conflicting implications of the research findings for University policy are considered.

    7. Harry, Joseph and Goldner, Norman S. (Winter 1972). “The Null Relationship Between Teaching and Research.” Sociology of Education 45: pp. 47-60. https://www.jstor.org/stable/2111831
      Keywords: relation between teaching and research
      Summary: Analysis of the data from a survey of members of a university faculty and of student attitudes toward this faculty indicate no relationship between student-rated adequacy of teaching and the extent of a teacher's scholarly activity. It appeared that student criteria for determining an instructor's adequacy consisted largely of evaluations of the instructor's style. Research activity by faculty was related to content variables such as course difficulty and student study time. The commonly-held view of a conflict between teaching and research found no support, although there may be some conflict when longer-run effects are explored.

    Unspecified Evaluation Procedure - Committee Selection, External Refernces, Weight of Criteria

    1. Barron, Dennis. (2001, April). "What It’s Like to Be Denied Tenure." The Chronicle of Higher Education: Career Network, 1-4.
      Keywords: promotion and tenure; individual merit; ideology of meritocracy; structural factors; arbitrary factors.
      Summary: Illustrates the impact of being denied tenure. Emphasizes that most tenure cases fall into a gray area in which serious arguments can be made either way about the outcome of a case. Shows that there are structured and arbitrary factors that have little connection to individual merit influencing tenure outcomes.
       
    2. Campi, Esther. (2002, March 8). "Former IC prof sues for tenure denial." The Ithaca Journal. Retrieved October 26, 2003
      Keywords: Tenure process, teaching, student evaluation
      Summary: Tells the story of a former Ithaca College journalism professor denied tenure because of gender-biased student comments.
       
    3. Jackson, Allyn. (1994, March). "Fighting For Tenure: The Jenny Harrison Case Opens Pandora’s Box of Issues About Tenure." Notices 41(3), 187-194.
      Keywords: tenure review, gender bias, external review, dual appointment, sexism
      Summary: Documents Jenny Harrison’s tenure battle at UC-Berkeley including denial, appeal, lawsuit, and final external tenure review. Includes comments from colleagues.
       

    4. O’Hanlon, James and Mortensen, Lynn. (November 1980). “Making Teacher Evaluation Work.” The Journal of Higher Education 51: pp 664-672. Full Article: https://www.jstor.org/stable/1981171
      Keywords: pay, rank
      Summary: A well-designed plan for the evaluation of teaching can serve purposes of both improving instruction and making administrative decisions about pay, rank, and tenure. Five approaches to the evaluation of teaching are presented. These approaches are consistent with six principles derived from an analysis of current research and practice.

    5. Centra, John A. (September 1994). “The Use of the Teaching Portfolio and Student Evaluations for Summative Evaluation.” The Journal of Higher Education 65: pp. 555-570. Full Article: https://www.jstor.org/stable/2943778
      Keywords: student evaluations, evaluation criteria
      Summary: This study investigated the use of the teaching portfolio and student evaluations in evaluating ninety-seven faculty members for contract renewal. Two faculty peers and a dean evaluated the portfolios of each teacher. Student evaluations correlated reasonably well with portfolio evaluations by the deans and by one of the peers.

    6. Koon, Jeff and Murray, Henry G. (January 1995). “Using Multiple Outcomes to Validate Student Ratings of Overall Teacher Effectiveness.” The Journal of Higher Education 66: pp. 61-81. Full Article: https://www.jstor.org/stable/2943951
      Keywords: student evaluations
      Summary: Previous research has failed to validate student instructional ratings with respect to a full range of teaching outcomes. With students randomly assigned to instructors, two measures of student motivation explained 7 to 17 percent more variance in overall teacher effectiveness ratings than was explained by measures of student learning alone.

    7. Derry, J. O. (January 1979). “Can Students’ Ratings of Instruction Serve Rival Purposes?” The Journal of Higher Education 50: pp. 79-88. Full Article: https://www.jstor.org/stable/1981357
      Keywords: evaluation procedure, student evaluations
      Summary: Formative and summative uses of students' ratings place conflicting demands on the composition of a rating form, but these conflicts are minimized when faculty partially control the content of evaluation. Under those conditions, the purposes served by students' ratings complement one another.