19 Comments

My college is only just now doing the data and assessment craze. Don't get me wrong, I love assessment ... just not the bean-counting, disingenuous kind that I've heard about for so many years ... that has come to roost here.

Before I became an academic, I was a mathematician. I love data and numbers. The problem is ... I know mathematics too well and know just how bogus so much assessment data is.

Expand full comment
author

Perhaps you agree that Lucius Sherman’s “scientific” method for studying literature was similarly bogus?

Expand full comment

Absolutely.

Cather's poem is really, really good too.

Expand full comment

I gave up on the industry for the very reasons you mentioned here. Spent years trying to get poems published before they started charging at every publication: good timing, at least for the journals. It wasn't a worthless expenditure of effort, and there were some things I could have done better: but the cost/benefit ratio felt lopsided. I didn't feel like my poems were chosen to cultivate talent, but to fit an equality metric due to large submission numbers. (here we have numbers, once again) Nor did I feel that any but a tiny number of exceptions did anything different. That bit about the author bragging about 215 rejections is nuts. I prefer the story of Cormac McCarthy, who only sent The Orchard Keeper to Random House and got accepted the one time. (of course he was lucky that Faulkner's old editor was at the helm)

When I start my little publishing press, we'll see how that goes. I can do without NYC cocktail parties and bigger numbers mean less time writing. But I don't embrace the level of 'indie' either because the independent publishers are not independent ideologically or when it comes to the numbers game, even if they are independent business-wise. I don't think they would behave any differently from the big guys if they joined the club.

Perhaps a more Amish approach is needed: avoid the trains and other modern stuff in your "community," but on the rare occasion you go into the non-Amish world it's okay to ride a train here or there.

Expand full comment

Pro-Amish!!!

Expand full comment
author

Sadly, a young Cormac could not send anything to Random House today without representation. I like the Amish analogy. But anyone trying to publish commercially, which is how you get in bookstores, has to play the followers game. I get that the volume of submissions requires some kind of check, but one might wish for a literary culture where craft could stand alone. Maybe I’m wrong in thinking that craft once carried more weight. Cather complained that reading most short stories in magazines in her day was like sitting in a tepid bath.

Expand full comment

Well I definitely have to choose my sacrifices, just as warriors choose their battles. Social media and blurbs are already considerable sacrifices on my part. (basically, like Ted Gioia, I'm taking the Substack bet that today it's FB, tomorrow places like Substack) But I also think incremental long-term strategies can be effective: one just needs patience, that's all. As important as numbers are, another number also proves the pointlessness of it: the relatively small number of readers today. Fame and commercialism help get books into their hands, of course, but at the end of the day we're not trying to win over that many people.

Expand full comment
author

"at the end of the day we're not trying to win over that many people" -- how depressing, yet also undeniably true. I think I'm still trying to determine whether an independent writing life is a worthwhile investment of my time, and whether being too ambitious about it risks turning it into a vanity project, for the sheer lack of audience that you mention. As Adrian says below, Substack does seem to be a place where a smaller community can provide that sense of meaning and purpose. It's been stabilizing for me as I think about longform projects.

Expand full comment

Yeah, a vanity project is something to avoid. I try to avoid being too personal for that reason, but without looking like I'm trying to not be too personal. (come to think of it, Han Solo was right to tell Chewbacca to "fly casual") When reviewing literature, I focus on the conceptual stuff. Then add the personal touch to lighten up the topic and make it accessible. Hopefully I'm succeeding! The Lansdale novella I reviewed was a dark story.

Expand full comment

Found a shore here, Joshua. Staying true together.

Expand full comment
author

Indeed! I’m grateful for it.

Expand full comment

Of course I’m struck by the picture you paint of the Academy...an internally divided Collab House and so on. So much to think about. “Exclusivity” is its own whole notion and appeal. It’s what enables people to sign up for waitlists for designers’ $7500 purses/totes, released in caches of 2000 bags, distributed to Dubai, Tokyo, Paris, London, NYC, and LA. It’s what enables tens of thousands to apply to Harvard, hoping they can go on Tik Tok waving their acceptance letters or electronic “I’m in!” notices.

And then there’s the metrics of the numbers game on campuses like mine. “Low enrolled” courses are always threatened with cancellation. Language classes and upper levels in some majors (History, English) are uniformly small though (5, 6 students) and aren’t cancelled. Administriviators decide, apparently from metrics inside the caverns of their ‘expertise,’ what’s worthy of tutorial-style smallness and what can be lectured out to 90, 125, or more students who remain nameless. Professors shill for summer and winter classes, begging for enrollment and students, lest they fail to hit the small number of asses on seats needed to “run” the courses. If a semester class IS cancelled, the prof is either reassigned to a mass intro class, or forced to overload the next semester (teach five classes instead of four, etc).

And of course, majors, programs, units are disbanded and cancelled due to “numbers.”

Expand full comment
author

Thank you, Ree! I'm not sure what your discipline is, but you are spot on about the workload imbalance on many campuses, in terms of course sizes, advising loads, and more. And it is humiliating to be in the position of trying to market your own courses to students, in some cases competing against your own colleagues because the enrollment numbers can seem like zero sum contests. It leads to bad pedagogical decisions sometimes, like working "film" into the title. When I first began teaching, I was confident that my institution and my colleagues had my back and helped me promote my courses. Near the end of my tenure, the institution assumed no responsibility for promoting embattled majors, though it frequently featured other programs in marketing materials. This also led to some instances of advisors counseling students out of colleagues' courses -- a nasty business, indeed.

I'm not insensitive to the need for programs to be self-sustaining. But I fear that the numbers obsession, particularly regarding the arts and humanities, leads to self-fulfilling prophecies. Once institutions begin to see their own departments as losers and leave them to wither on the vine, there is virtually no way to turn that tide. What it will take, I believe, is a pendulum swing toward the applied arts and more technical programs, to the point that people notice the absence of the liberal arts. I would not be shocked if a generation of students rejected the jobs-based curriculum in favor of courses that tapped into older conversations and the age-old quest for meaning. But it is much harder to build a program that has been gutted than it is to sustain an existing one.

Expand full comment

Our new data and assessment office declared that they would start monitoring the DFW rates of classes, which refers to the proportion of passing to Ds and Fs as well as withdraws.

That has me very concerned, because as chair of philosophy and religion, I can tell you that philosophy courses tend to be very hard for students. I cannot tell you how many countless times I've been old that liberal arts courses are supposed to be "filler for real courses, like math." The idea of being held to logical rigor, e.g., give evidence for your claims, don't contradict yourself, etc. is foreign to almost all of them. My religion colleagues face a different problem; it's hard to teach religion when students won't read to seriously consider religions and civilizations alien to their own. I have yet another conference call tomorrow 'cause an instructor is freaking out about just how low his participation has become.

Expand full comment
author

Yes, there is a lot of buzz about the DFW rates now. I've heard from colleagues at other institutions about similar concerns, usually driven by a dean with an eye on retention.

There are legitimate concerns about grading, as we all discussed in the thread on labor-based grading a while back. And I'm of a mixed mind about things like "ungrading" (see this recent NPR story -- https://www.npr.org/2023/03/26/1164832694/to-help-new-students-adapt-some-colleges-are-eliminating-grades).

On the one hand, it's true that grades don't accurately measure how much learning is taking place. Someone could come in with a high degree of competence, achieve a high grade, and learn very little. Someone coming in with less aptitude or exposure to a subject might perform poorly, grade-wise, but learn a great deal. But I just don't think there is a broad consensus about what grades are trying to measure. Mastery of the actual material -- achieving competence in a subject -- is what I think grading ought to target. And if people are failing or withdrawing because they can't achieve a minimal level of competence, then grades are doing their jobs, regardless of how people might feel about it. I understand that grades can get in the way of learning, and that some grades are, themselves, unreliable measures of competence.

This is perhaps turning into a separate issue, but if someone were to focus less on the difference between a 90% (A-) and an 89% (B+) -- probably fairly arbitrary and subjective measures with huge consequences for some students -- and instead raise the bar for passing a course high enough to satisfy the competency question, then I might be in favor of ungrading or some such thing. In that case, there could be other, non-numerical bases for earning distinction or pursuing more self-directed learning beyond the minimal expectations for a course. But in that case, I think the bar for a pass ought to be closer to what the bar for a "B" is now. A can of worms, to be sure.

Bottom line: when admins are looking at DFW, they have different priorities than the professors who are giving the DFW grades, and there needs to be a set of shared values about what those things measures and why they matter.

Expand full comment

I read that NPR article and had a generally negative reaction to it, because it elided too many issues and overlooked what are, to educators, obvious red flags. For instance, the student that is going to school full time and working 40+ hours a week? Yeah, she's asking for trouble. She may not have known that, and deserves our support, but that's not a failure of grading. Alternative grading is no the answer to that kind of problem.

I agree with your post awhile back about how growth cannot come entirely at the expense of meeting standards. If a grade or college degree is to mean anything, it must indicate some level of objective achievement. So, I agree with your rearticulation here.

I think something like specifications grading is a better answer to these kind of problems then ungrading. Or, per the guided pathways model, unifying and synergizing curricula to align our learning objectives across courses (which smaller schools often do as a matter of course and professionalism).

Around here, the problem is that only 1 in 4 of my students read anywhere near grade level. Per studies on our own students, 1/3 of them are formally remedial in 3 subjects (reading, math, wriitng), but they still get funneled until the regular curriculum. The argument for that is that the evaluation exams don't reliably distinguish who is remedial and who is not (firmly supported by the research), bu then this institution like many just decides to do away with remedial as a solution to the problem.... and thus we get outrageously high DFW rates...

I'm going to share something that might get me in trouble, but i dont care at this point, because I care about students and education. We had a committee of 6 people do a review of sylalbi in half of the deparment, which incldues satellite campuses across central Iowa. Only 2 of 8 faculty minimally passed the review. Of that, half the courses taught in community colleges in central Iowa don't meet basic professional or regulatory standards. As in, a student has no idea how a grade is calculated, what assignments are due and when, and a that 1/2 to 2/3 of the course material was utterly off topic. As in, the course wasn't about what it was supposed to be about! It's a fight I've been having for 7 years, and it doesn't look to be much different at other Iowa community colleges.

So, I also really get why administrators are doing things like DFW rates in some sense, because I know that they know that we have quality control issues. But rather than let us do quality control, e.g., enforce professional requirements, we get bogus assessment procedures that are easy to, and often are, BS'd to provide cover.

Expand full comment
author

Whoa -- that is a whole other level of dysfunction. The medical school here at University Park, which I had hoped to join as part-time instructor, uses Pass/Fail. But the baseline expectations for passing are quite high, and students bring a strong level of engagement. So in that kind of environment, where students are fully bought in and committed to excellence, there's no need for parsing percentages or saddling everyone with a ton of informal writing to motivate preparation. The student who is working full-time while trying to take a five-course load is a very different situation, and professors can't be blamed for poor performance in that case.

The quality control issues, as we've discussed, are quite problematic when it comes to transfer credits. But that is a dead horse we don't need to flog further.

Expand full comment

With everything becoming 'data-driven' I wonder how much we're losing by ignoring what we can't, or don't yet know how to measure?

We've reduced human interaction to the most basic 'metrics': engagement, views, clicks.

What is the difference between an essay or story that affects a few hundred people deeply, maybe even alters the course of their lives, and a cat video that millions watch and then swipe past? From a metrics standpoint, the essay is an abject failure and the cat video is perfect content. (Here's a fascinating essay, The Bitter End of Content, on how content is optimized for meaningless by a reliance on metrics: https://freddiedeboer.substack.com/p/the-bitter-end-of-content )

When is depth and intensity of connection or experience more important? How do we quantify that?

There's a popular anecdote about a girl walking down the beach with her father. Thousands of starfish have washed up on the sand after a recent storm. As she walks, she picks them up at random and returns them to the sea.

"You know that what you're doing doesn't make any difference. It doesn't matter at all?" In a hurry to get home, her father admonishes impatiently.

She plucks the next starfish from the sand and holds it up, "It matters to this one." She tosses it back into its home.

How do we measure the meaning we have to one another? Why would we even want to mine that for success metrics? The wonder of life lies between and beyond these shallow measures.

Expand full comment
author

Sean, your point about mindless content is a really good one. Publishers are beginning to look not only for the raw number of social media followers, but also for signs of engagement that might predict book sales. But boxing ourselves in with these sure things, these low-risk wagers on ROI, really dampens creativity.

When Alfred Knopf was just starting out as a publisher, he was interested in attracting talent. There was a bottom line to consider, but he really wanted to recruit and retain writers over a long career. He had faith that if the craft was excellent, he could create a demand for it. The marketplace for books looked very different then, but Knopf continued to work with Willa Cather even when she turned down very lucrative movie rights proposals for her novels. The advantage of Knopf's system was that an artist could evolve throughout a career, not merely keep churning out the same kind of book that might satisfy a loyal readership. I think the numbers obsession in publishing now really hampers artists who want to try something new. Either their agent doesn't know how to sell it or the consolidated publishing industry doesn't see a lane for it in their streamlined product lines. Substack, of course, offers an alternative to this. But it is, as yet, not really a book publishing platform. Anyway, I'm straying a bit from your original point.

This is spot on: "How do we measure the meaning we have to one another? Why would we even want to mine that for success metrics? The wonder of life lies between and beyond these shallow measures." One of my soapboxes about academic assessment is that it is a crass and often unreliable measure of very complex human interactions. Teaching, like many of these meaning-based exchanges, is more like a faith journey than like continuous improvement in business. I guess mega churches use numbers to drive their product, but most devout people would take umbrage at a consultant using metrics to measure their spiritual growth.

Expand full comment