AI doesn't know what it doesn't know

And boy, does it show

I swear I never intended to spend this much time on AI, but here we are.

Before we get started, if you haven’t heard that we’ve now been blessed (and I use the word very loosely) with an hour-long AI George Carlin show, well… now you have. (Ed Zitron on BlueSky, where I found this link, said, “This headline should read “corpse violated by charlatans’,” and he’s not wrong. Carlin’s daughter is not at all pleased.)

I leave the precise language of Carlin’s probable, and thoroughly AI-free, response as an exercise for the reader—which I do hope you’ll grace us with in a comment!

Alrighty, then.

Earlier this week, I was poking around on LinkedIn, and saw that a connection had posted an answer to a question. I clicked through, and landed in LinkedIn’s new answer to Quora—a place where users can attempt to further establish (or create from whole cloth) their own expertise by answering questions about various professional topics. If you answer as few as three, LinkedIn will give you a badge declaring you a “top voice” in that subject for your free labor efforts, which I suppose tells you how many people (none) are clamoring for this feature.

Quora, as you may be aware, is a question-and-answer site, in the same vein as Yahoo!Answers, but somewhat more civilized. I say “somewhat” because, in recent years, it has descended into chaos much like Yahoo has, taken over with questions designed to generate outrage rather than actual questions that can elicit useful answers—to where people are now writing about the site’s precipitous decline.

(Full disclosure: I’ve written a decent number of Quora answers over the years, mostly about writing/creativity, and occasionally about Doctor Who. Even four or five years ago, I stopped doing as many because the quality of the questions started to seem dubious, at least to me.)

Why LinkedIn thinks mimicking Quora is a good idea is beyond me, but I suspect they think they can prevent the decline by pre-populating all the topics with questions written by… wait for it…you guessed it… AI.

AI may be able to write a perfectly reasonable question about manufacturing widgets or general leadership principles, but here’s the last in the series of questions it offers in the Creativity category. In the spirit of Dave Barry, I must emphasize that I am not making this up:

5. How can you measure your creativity?

Measuring your creativity can be beneficial in tracking your progress and evaluating your results. There are various approaches to measure creativity, depending on the purpose and criteria. For instance, self-assessment tools like quizzes, surveys, or journals can reflect on creative strengths, weaknesses, and preferences. Additionally, setting goals and indicators like the number of ideas generated or quality of solutions implemented is recommended. Peer assessment tools such as ratings, reviews, or testimonials can provide feedback and validation from others. Expert assessment tools such as tests, portfolios, or awards can offer an objective and professional evaluation of creativity. Seeking advice from experts can also help to improve and enhance creativity.

Yes, the AI not only provides the question but also some “background” that is actually trying to answer the question without answering the question… or something like that. The careful reader will note that absolutely none of that generated twaddle answers the actual question (and several sentences, notably the second and the last three, don’t even say anything that isn’t entirely self-evident)—probably because you can’t. More on that in a minute.

As tempted as I was to hop in and answer this question, thus moving me along toward my super-special “top voice” badge that means basically nothing, I refrained, because I could not think of an answer that was not this:

First you get out a ruler… and then you use it to whack the person who even suggested creativity is something you can measure quantitatively over the head repeatedly until they shut up and go away.

I had a feeling that might not go over so well on LinkedIn, even in a universe where LinkedIn thinks its site should be bribing its human users to answer ill-considered questions written by machines in an attempt to offer something of use to other humans. (It’s so circular it makes my head spin. Please, someone, make it stop.)

Granted, this is the sort of hare-brained question you also get out of the aforementioned management types who think everything should be quantifiable, and then drive the rest of the population crazy trying to meet some made-up “metric” that doesn’t actually mean anything. To them I can only ask,

• How do you measure love?

• How do you count rain?

• How do you measure happiness?

(I’d try to actually ask, but I’m afraid they’d try to come up with something and someone would get hurt in the process.)

You can measure creative output. You can count paintings, or songs, or books, or blog posts. But you can’t actually measure someone’s creativity any more than you can quantify your love for your children (or, I sincerely hope, try to figure out which one is the “best” or “favorite” child). It just doesn’t work that way, and to try is to bump up against so much subjectivity in the process of “assessment” as to prove the point all over again. It is, as I would have said to my ESL students many moons ago, “not a countable noun.”

AI doesn’t know this, of course. Any sane human, asked to write a set of questions for LinkedIn, should know this. But AI has no clue, because AI is not a person. AI is, at best, a milquetoast simulacrum of a person—and not a very bright one at that.

I’m not sure how we convince sites like LinkedIn that AI-generated questions are a bad idea. We could go post replies like the one I refrained from, but given the point of LinkedIn, that would probably just damage individual reputations rather than convincing them to stop. I’m not sure they’d care about the equivalent of a letter-writing campaign, either. Maybe they’ll have to learn like Quora has? (Am I brave enough to repost this piece there? Or even just link to it? Hmmm…)

As I said at the top, I never expected to end up writing this much about AI, but it does seem like you can’t turn around lately without AI suddenly rearing its head where you least expected it to be. (There’s more I haven’t even covered today, so I guess I’m STILL not done with AI.) It reminds me of this classic Saturday Night Live ad, which I leave you with because I think we all need a good laugh as we watch our society struggle with the temptation to outsource our brains to machines that haven’t got a clue what the heck they’re doing. Maybe Old Glory will save us. 😉