Fruit of the Poisonous Tree (Guest Post by Dougal MacDonald, Elementary Education)

In this guest post, Dougal MacDonald, who is contract academic staff at both the University of Alberta and Athabasca University, offers his views on student opinion surveys. This is a reprinting, with Dougal’s permission, of an article that appeared in a recent newsletter from CUPE 3911 (Athabasca). 

“In teaching, you cannot see the fruit of a day’s work. It is invisible and remains so, maybe for twenty years.”  Jacques Barzun

Athabasca University, like a number of other universities, uses student opinion surveys to evaluate teaching competence. All academics are vulnerable to these kinds of evaluations but those who are part-time and/or on contract are especially vulnerable as their livelihood basically depends on getting good results on these instruments.  Those on contract usually do not have the luxury of also being evaluated based on their research or community service.

But, considering that student opinion surveys are being widely used in various academic institutions which are supposedly bastions of academic research, it is noteworthy that there is a distinct lack of research supporting the validity of their use.   Thus, rather than advancing research-based argument to support their use, their proponents tend to put forward mere assertions or else simply appeal to naked pragmatism, e.g., “they keep the students happy”, “it’s easy to crunch the results”.

In fact, the vast majority of the research that has been done on student opinion surveys opposes rather than supports their use.  See, for example, Braga et al’s article in Economics of Education Review, August 2014 (http://www.sciencedirect.com/science/article/pii/S0272775714000417). Based on the findings of such studies, numerous articles have been written critiquing their use. Some of the main arguments raised against using student opinion surveys to evaluate teaching include:

  • the surveys focus on student emotional disposition toward the teaching in the short term, long before the full impact of the teaching is actually known
  • using a single method to evaluate teaching, directly contradicting the well-established principle that good evaluation requires a number of methods
  • teaching effectiveness cannot be atomized into a checklist of specific behaviours
  • the response rate is low and even lower with online evaluations, skewing the validity of the results
  • in general, only very happy or very unhappy students are motivated to fill out the surveys
  • certain items that are commonly evaluated simply cannot be evaluated by students, e.g., the instructor’s course content knowledge
  • “averaging” the results is statistical nonsense because the choices (1, 2, 3, 4; not satisfied, somewhat satisfied, satisfied, fully satisfied ) are ordinal (counting) in nature rather than interval in nature (an equal distance apart)
  • results are influenced by variations in the mere quantity of student-instructor interaction
  • the very desirable teaching practice of challenging students may lead to lower evaluations
  • gender bias exists against female instructors
  • pressure is placed on instructors to do what is required to get a “good rating”
  • the surveys provide few useful insights to experienced instructors
  • there is no accountability for personal or even slanderous remarks
  • in the end, the opinion surveys are nothing more than a popularity contest

The above criticisms (and others) also highlight the gross unfairness of using the evaluations for what might be called disciplinary purposes.  If the opinion surveys themselves are flawed, then they cannot be validly used to assess teaching competence.  Further, how can that flawed assessment of competence be then validly used to “inform” such matters as which contract academic gets a letter of reprimand or is assigned to teach or reteach a particular course?  There is a saying in law about the “fruit of the poisoned tree”.  It refers to the fact that if the evidence is tainted, then anything gained from it is tainted also.  The same principle should apply in regard to any use of the results of student opinion surveys.

(Recommended Further Reading:  “Do student evaluations measure teaching effectiveness?”  Philip Stark, University of California Berkeley Professor of Statistics, October 2013.  http://blogs.berkeley.edu/2013/10/14/do-student-evaluations-measure-teaching-effectiveness/)

 

Posted in Uncategorized | Leave a comment

GSEC: Open Letter to ab-GPAC, GSAs, and Minister of Advanced Education (27 February 2017)

27 February 2017

An open letter to:

GSEC Addressees 5.png

Dear fellow stakeholders:

This open letter was written on behalf of the Graduate Students of English Collective (GSEC) at the University of Alberta (U of A). It is our formal statement of objection regarding the U of A Graduate Student Association’s (GSA) stance on the continued infringement upon our constitutional and human rights to legal organization by the Post-Secondary Learning Act (PSLA). The purpose of this letter is twofold: to inform stakeholders across the province that the pro-PSLA position taken by the lobby group ab-GPAC, which was also echoed in a letter from University Presidents to the Minister of Advanced Education, cannot be representative of graduate students; and to call for the GSA at the U of A to conduct transparent and robust consultation regarding the pursuit of union representation.

As it currently stands, the January 30, 2015 decision by the Supreme Court of Canada in the case of Saskatchewan Federation of Labour v. Saskatchewan sets a clear precedent for the GSA and other graduate students’ collectives or associations to pursue union representation. Yet, the President of the GSA at the U of A co-authored an official statement with ab-GPAC, claiming that graduate students across the province would prefer to remain under the PSLA and thereby forfeit their constitutional right to freedom of association. While we are aware that the GSA initiated a consultation with graduate students in its Special Bulletin of September 28, 2016, this email allowed less than a week for responses. This was insufficient consultation with the GSA membership at large. Moreover, the pursuit of union representation was discussed only once amongst GSA councillors in their September meeting. There was no clear consensus on that matter, and the GSA’s position regarding unionization was never put to a formal vote. Instead, ab-GPAC and the GSA Board expressed to the Minister of Advanced Education that maintaining the PSLA in its current form “is the only option that can allow graduate students to function in all of their roles without harsh and undue consequences”[1] because of their precarious relationship to the University as their employer. This position was taken after receiving only 62 responses from graduate students across the province. Ab-GPAC represents at least 15,000 graduate students in Alberta, 7,000 of whom are enrolled at the U of A. Therefore, the GSA Board members who sit on ab-GPAC should not have considered 62 respondents a representative sample of either the province or the U of A when it comes to the important decision of whether or not graduate students should be able to form or join unions.

GSEC is dismayed by the GSA Board’s unwillingness to inform the GSA Council or membership of its position on unionization. In not taking a position independent of ab-GPAC, the GSA has failed in its mandate to represent its constituents. Furthermore, neither of these groups has made any effort to be transparent. It took ab-GPAC three months to respond to GSEC’s requests for its official statement and the GSA has not disclosed to GSEC any documentation related to their decision-making process. The PSLA infringes upon the Canadian constitutional right to freedom of association and discriminates against all academic workers based on source of income — a protected ground under the Alberta Human Rights Act. The most direct consequence of this infringement for graduate students is that they remain under the false impression that they cannot pursue union representation. This is a critical issue that necessitates fair and accurate consideration of graduate students’ views and adequate discussion regarding unionization options. While the official consultation deadline on the issue of unionization passed on October 17, 2016, we believe there is no official end date for democratic engagement.

The GSA Board has acted undemocratically when constitutional and human rights are at stake. Whether for or against unionization, graduate students should have been given a fair chance to weigh in on this debate, which is changing the landscape of labour relations in Alberta. GSEC acknowledges that unionization is a complex issue. To understand the costs, benefits, risks, and rewards of unionization requires time and effort on individual and institutional levels. GSEC members, and all GSA members, deserve to enjoy the full range of constitutional and human rights. These rights should not have been surrendered.

In accordance with this position, we call on the GSA to:

  • Conduct transparent representation through fair and democratic consultation. Such consultation would include a town hall and also adequate time for conducting surveys, interviews, meetings, or any other means of allowing graduate students at this institution the opportunity to voice their opinions on the issue at hand.
  • Make publicly available any letters, minutes, or documentation between the GSA and ab-GPAC pertaining to the GSA’s position on unionization.

We trust that proper consultation and representation are the goals of the U of A GSA and other stakeholders. We look forward to working with the GSA to initiate and to facilitate a more formal and transparent consultation process because all 7,000 graduate students at the University of Alberta deserve full expression of their constitutional and human rights.

We, the undersigned,

Graduate Students of English Collective (GSEC)
Department of English & Film Studies

3-5 Humanities Centre

University of Alberta

Edmonton, AB   T6G 2E5

 

Points of Contact:

Gregory Blomquist, PhD Co-Chair

gblomqui@ualberta.ca

Keighlagh Donovan, MA Co-Chair

keighlag@ualberta.ca

Kevin Kvas, Labour and Equity Committee Chair

kvas@ualberta.ca

Shaina Humble, GSA Representative

shumble@ualberta.ca

 

[1]Sarah Ficko et al. “ab-GPAC Labour Relations Response —Graduate Students.” Oct. 17, 2016.

Posted in Uncategorized | Leave a comment

Open Letter from Professor Katherine Binhammer (EFS): Reject BA Core Proposal at Arts Faculty Council on Nov. 24th

Dear Colleagues,

I write to you today as an Arts colleague to strongly encourage you to attend Arts Faculty Council on November 24th, 2 pm (in Council Chambers, University Hall) and exercise your right to vote. As you know, on that day Council will be voting on the proposed new B.A. Core requirements and I am extremely concerned for the future of the Faculty and of our students if the proposal is passed. Thus, this email’s purpose is to rally the troops to show up and not allow a flawed proposal to pass.  Below I outline a few reasons why I will be voting against the proposal. Whether or not you agree with me on what the new B.A. core should look like does not matter; what matters is that we collectively reject the proposed model which would be a disservice to our students.

Here are my Concerns with the Current Proposal

1. What is the intellectual or pedagogical reasoning for the proposed new core requirements? Continue reading

Posted in faculties of arts canada, University of Alberta Compensation Negotiations 2012 | Tagged , , , | 1 Comment

Brad Bucknell (English & Film Studies) on the arguments at Arts Faculty Council on the transformation of the Centre for Writers

This is the fifth post on ArtsSquared by members of the Faculty of Arts on the Provost’s planned transformation of the Centre for Writers into an administrative unit run by Student Services. Further to a motion passed by Arts Faculty Council at its meeting of 25 May 2016 advising him to reconsider his decision, the Provost has agreed to delay any change to the Centre for Writers for one year, to allow for further review of the issue. Here Brad Bucknell (English & Film Studies) comments on the arguments of the Acting Vice-Dean of Arts, Stuart Landon, at the Arts Faculty Council meeting of 25 May 2016 as he spoke of his role in this decision.

It was, to say the very least, a curious and disheartening AFC meeting this past Wednesday. Near the end of the meeting, we began discussing the recent changes to the Centre for Writers. Specifically, we took up the removal of Professor Lucie Moussu as the Centre’s director. Many people there wanted to hear a more complete explanation of the replacement (indeed, the displacement) of Professor Moussu who is both a trusted colleague and a widely known and well-respected scholar. Nothing about her replacement made much sense, so we came seeking enlightenment. We left without it.

The explanation contained these “points.” We were told that there had been a year’s consultation on the matter, including conversations with Prof. Moussu. This may be true in some way; however, it is hard to know what the word “consultation” means here, since Prof. Moussu would hardly have agreed to her own removal from the directorship. Perhaps there was less consultation than plain old telling: this is how it will be. Continue reading

Posted in ualberta centre for writers | Tagged , , , | Leave a comment

Professor Moussu Speaks Out on Centre for Writers Changes Sought by Faculty of Arts (Open Letter)

Dear colleagues,

I would like to address two important points about the announced changes regarding the C4W: first, how it affects my career, and second, how it affects tutors in the C4W.

First, my career: I would like you to imagine a UofA Biologist, who works in her lab and conducts research projects with her research assistants and her participants. She teaches courses related to her research, gets grants for her research, mentors her graduate students through their studies, hires and train the best research assistants, recruits participants according to best practices and research ethics, presents and discusses her results at national and international conferences (often with her students), and publishes her results in peer reviewed journals (sometimes as co-author with her students). Every year, she writes an annual report about her work, gets evaluated by her peers at the UofA, and gets excellent feedback—one year, she even got tenure!

And then one day, she is told that she can no longer work in her lab. She can go inside her lab and watch what happens, but she can’t be the Principal Investigator anymore, can’t work with her research assistants, can’t recruit or interact with participants, and can’t decide what is or isn’t being done in the lab.

This is exactly what has happened to me and to my “lab.” Continue reading

Posted in Uncategorized | Tagged , | 1 Comment

Professor Sargent Responds to Provost’s Document on Situation with Centre for Wrtiers

The post below is the response of Professor Betsy Sargent (English & Film Studies) to the formal response that the Provost and Vice-President Academic Stephen Dew has offered to members of the General Faculties Council in the form of document 14.1.R for today’s meeting of GFC. The Provost’s document responds to a question posed by graduate student Shumaila Hemani which may be found here. The matter of what is happening to the Centre for Writers was first raised at the last meeting of the General Faculties Council, and GFC told that President Turpin would investigate.

Centre for Writers: Response to Statement from Provost and Vice-President Academic Steven Dew (Professor Betsy Sargent)

 

I want to respond not only as a Professor of English in the Faculty of Arts, but as a Professor in the disciplne of Writing Studies who has served as the Co-Chair of the University’s Writing Task Force, as Director of Writing Initiatives (2007-2012), and as the Founding Director of Writing Studies in Arts and the award-winning Writing Studies 101 (2008-2014).

I’ll respond to each paragraph in turn, starting with paragraph #2: Continue reading

Posted in Uncategorized | Tagged , , , | 1 Comment

Open Letter from Professor Garry Watson (English & Film Studies) on the Centre for Writers (To be discussed at the General Faculties Council meeting of 30 May 2016)

Dear Colleagues,

Many thanks to those of you who were able to attend Arts Faculty Council on Wednesday afternoon. As you know, the motion was passed to advise the Provost to delay by one year the decision to move the C4W out of Arts into Student Services and to allow the current director, Dr. Lucie Moussu, to complete her administrative term (which goes until July 1, 2017), said delay to allow for wider consultation and further research at least equivalent to the wide consultation and extensive research that created the C4W in the Faculty of Arts in the first place.

If you were unable to attend AFC on Wed and were so inclined, emailing a sentence to President Turpin <dturpin@ualberta.ca> today would be enormously helpful, since he’s chairing GFC tomorrow afternoon and this time-sensitive C4W issue is on the agenda. Something quite brief like the following would work well: “If I’d been able to attend AFC on May 25th, I would have added my vote to pass the motion advising the Provost to delay for one year any changes to the administration or directorship of the C4W.” Continue reading

Posted in ualberta centre for writers, university of alberta | Tagged , , , , | Leave a comment