Back to school, back to school, to prove AI is no fool…
ChatGPT, OpenAI’s new artificially intelligent chatbot, can write essays on complex topics. WSJ’s Joanna Stern went back to high school AP Literature for a day to see if she could pass the class using just AI. Photo illustration: Elena Scotti
Tech Things With Joanna Stern
Everything is now a tech thing. In creative and humorous videos, WSJ senior personal tech columnist Joanna Stern explains and reviews the products, services and trends that are changing our world.
#WSJ #ChatGPT #OpenAI
(bell ringing) – [Joanna] You may have heard of ChatGPT. – The stagnance of these two characters Is what propels each
respective plot forward And changes those around them. – [Joanna] A new artificially
intelligent chatbot That you can command to
write pretty much any text. – Existentialism exists
as a refusal of the notion Of our lives having some
predetermined reasoning. – [Joanna] You also may have heard That ChatGPT is so good and
writes so much like a human That it can be used to write essays And do homework assignments. – An existential text with its themes Of individuality, freedom and
the search of meaning of life. (bell ringing) – So yes, I pulled a Billy
Madison and went back To AP literature to see how far
AI and just AI could get me. I’m a new student totally in high school. I enrolled in 12th grade At High Tech High School
in Secaucus, New Jersey. Okay, fine. The teacher, Michael
Diamond, let me in his class Since he’d already been
playing around with the AI tool And was intrigued by
my cheating experiment. – I have a quick question. – Yes.
– Is the Wi-Fi down? – The Wi-Fi is down. – Great, thanks. (students chuckling) – Sorry. – So did ChatGPT pass or fail? (buzzer beeping) We devised two challenges to find out.
Challenge number one, write an essay. Before coming to class, Mr. Diamond emailed me my assignment. – [Michael] In a 500 to 1,000-word essay, Compose an argument That attempts to situate
”Ferris Bueller’s Day Off” As an existentialist text. – [Joanna] The assignment also said To make explicit comparisons Or connections between
”Ferris Bueller’s Day Off” And ”The Metamorphosis” by Franz Kafka. I edited the assignment text slightly, Pasted it into ChatGPT
and in about a minute, I had it, a full essay. I was impressed. It drew parallels to
Gregor, the main character, In Kafka’s book, and to Ferris. It had varied sentence structure And good grammar and
was decently organized. I put it in a document and submitted it To Mr. Diamond via email. Then he did his thing. (upbeat music) (bell ringing) – You ended up with a three
out of six on the AP rubric. I was a little generous
because it’s early in the year. You showed some promise But there are some issues
you really need to work on. You got your thesis point right away. You wrote ”Ferris Bueller’s Day Off” Is a film that can be interpreted As an existentialist text with
its themes of individuality, Freedom and the search
for meaning in life. Unfortunately, in that
same opening paragraph,
You misattributed a line that
was said by Cameron to Ferris. – You sure of that? – That I am sure of. – I am not gonna sit On my ass the events that affect me unfold To determine the course of my life. I’m gonna take a stand. – [Joanna] There were
other factual inaccuracies In my essay. ChatGPT confused Cameron with Ferris, Saying Ferris was reserved And rarely seen next to his peers. – I think maybe you were
thinking of Cameron, But you wrote Ferris. – Yeah, I think that’s probably
what I was thinking. Yeah. – You said that the setting Of ”The Metamorphosis” was
suburbia, which it was not. It was an urban setting. No, we need to work on being
more accurate in your details And I did not see a lot
of vivid writing here. – [Joanna] Not easy for
a professional journalist And writer to hear. – The overall quality
of your writing puts you In the lower 30th percentile I think. I think you may have the mind to get there But it’s the skills that
you need to work on. – That doesn’t feel good. Challenge one, a pass,
because technically, A three out of six on an AP
essay is like a B or a C. I should note that plagiarism
detectors like this one From Grammarly are not
likely to flag AI writing As plagiarism since it is original text. Do you think you would’ve
had any idea this was written By a computer?
– I don’t think that I
would catch it to be honest. I mean, first of all, if you look At the the actual context
of when I’m grading, right? I’m going through 60 of these At a clip and I’m looking
for specific things. – [Joanna] Luckily for Mr.
Diamond, companies are working On ways to detect AI writing. Challenge number two, Participate in a group discussion. – You’re gonna get in your core groups And you’re going to go
through the next three steps And you’re gonna first, review the story. Each group has been
assigned one in advance. – My group was assigned to read
Denis Johnson’s short story, ”Car Crash While Hitchhiking” From the collection ”Jesus’ Son.” My group mates immediately got To the first discussion
question about the characters. – The cop did actually stop
him to make sure he actually Was a part of the crime scene. – Could you also say maybe the doctor? Or is that? – Yeah, I definitely
think the doctor because. – When I searched ChatGPT For information about the
main character in the story, The bot said it didn’t have any at all. So I changed the prompt
and asked it questions About the collection’s
title, ”Jesus’ Son,” Which kind of got me somewhere. Is there a character named Billy? – Billy? Not in this chapter. – Does anyone in this
chapter have anything To do with addiction? – Yes.
– Oh yes.
– Yes. Okay. And does he attempt to navigate
the complexities of life? – Yeah.
– Yeah, yeah, he did. – Oh, okay. But that was really
all I could contribute. I tried different prompts Like what’s the bigger
philosophical question, But I got no help. Not going well at all. While OpenAI, the company behind ChatGPT, Trained this tool on a massive trove Of data gathered from the
internet and other sources, It clearly hasn’t read this book. Finally, I just tried to skim the text But ultimately, I just
asked a very nice human To summarize it for me. – So basically this guy
has been like traveling, Traveling through, he’s
been like hitchhiking Through like the woods and everything. – Challenge two.
(buzzer buzzing) Fail. Of course, the whole
purpose of this experiment Wasn’t really to try to get an A in AP Lit But to see the capabilities
and limits of AI As an educational tool. So some have started
to call this CheatGPT. Are you worried about that? – I am worried about the easiness, right? That’s what cheating really is Is like the easiest way to do a thing. They’re not learning what I’m asking them If the internet can give them a full essay Because they’re missing out
on the process of writing. And I’m more concerned
with the the possibilities
To prevent learning and beget compliance So they can get pushed
through school, so to speak. – [Joanna] My biggest takeaway returning To high school 20 years After graduating, how
ingrained tech already is. Students already lean
on Google and Grammarly And more to help them
with their assignments. It’s nuts to think artificial intelligence Won’t be the next tech
to change education. – I am definitely gonna try
to figure out how to use it. I feel like with all tech,
it’s easy to be scared of it. And the good news is, at least here, There’s some teachers that
I’ve been talking with That are like excited
about getting ahead of it And incorporating it
and not running from it Because I don’t think
we’re gonna outrun it. (playful music) – Hi guys.
– Hi. – Can you be my friend?
– Sure.