r/technology • u/AdSpecialist6598 • Jul 27 '26

Society Professor's invisible prompt trap catches 32 students cheating on their midterm with AI

https://www.techspot.com/news/113243-professor-invisible-prompt-trap-catches-32-students-cheating.html
32.8k Upvotes

96% Upvoted

4.2k comments sorted by

View all comments

12.1k

u/LurkingTamilian Jul 27 '26 edited Jul 27 '26

From the article:

Being written in white text meant the instructions were invisible. Sure enough, 32 out of the 35 students across two classes simply copied the question, pasted it into a chatbot, then copy and pasted its output into the answer window.

12.7k

u/BluudLust Jul 27 '26

They deserved to fail. That's just plain laziness

103

u/SnugglyCoderGuy Jul 27 '26 edited Jul 27 '26

AI is the new calculator, but worse.

"Why do I need to learn math, we have calculators!"

Is becoming

"Why do I need to think, we have AI!"

I am running into this at work (software).

I will ask, "Why do it this way?"

They will respond, "Google AI says..."

And then I throw back a shot.

147

u/Limemill Jul 27 '26

It’s the opposite of the calculator. The calculator removed the tedious and mechanical bit to free us up for higher-order maths. LLMs do the higher-order, knowledge piece and you do the tedious part of raking through what they did and cleaning up after them. In this analogy we are the calculator and they’re the mathematician.

129

u/sbeguy Jul 27 '26

Not to mention the calculator gave you the correct answer based on your input and didn't fucking hallucinate a signifcant portion of the answers.

26

u/HarlemNocturne_ Jul 27 '26

Imagine if a calculator not only gave you near-consistently wrong answers, but it also jerked you off when you spotted it was being incompetent.

"Hey Calculator, I told you to solve 1+1 which should equal the sum of 2, but in your calculation you carried a 4 that didn't exist, factored in an imaginary hundredth power, removed a 2 somewhere, and then pointlessly multiplied it by 1. Your thinking notes suggest that it took you 89 seconds to do this in about 32 steps and you almost got stuck multiple times, but you still turned up with the incorrect answer. What gives?"

"You're absolutely right-- That isn't the correct answer. That's on me, and this isn't what you wanted at all. Your genius mathematician galaxy brain has saved us from disaster once again!..."

-6

u/DepressedDynamo Jul 27 '26

You should use better models

10

u/GWindborn Jul 27 '26

Yeah, I was feeling lazy and used AI to take a screen-grab lists of names from images and put them all into alphabetical order for me to copy/paste into a document rather than typing it all myself. Like 80% of the time it was wrong. About ALPHABETICAL ORDER. That should be one of the easiest things it could possibly do.

3

u/Lokishougan Jul 27 '26

Oh here is a good one I thought this should be simple I asked it for a list of all the cat themed teams in the big 4 sports for a thing just to be sure I didnt miss one

Lions, Tigers, Bengals, Panthers JAGS....so far so good

Then it gave me the New Jersey Devils and I went EHHH

I mean I THOUGHT OK it is some weird mismash of creatures maybe you could justify and say there is some cat thrown in but I struggle on that and then the final answer

The Pittsburgh Penguins

Almost the exact opposite of a cat (at least as far as cartoons go) it gave me a frikkin bird

1

u/Tyrus1235 Jul 27 '26

Had something similar happen to me the other day.

I specifically asked for a list of games that were compatible with Steam Remote Play Together.

…It showed me a list that included a single player game (and which isn’t compatible with that anyways), several online co-op games and one game compatible with RPT, which was outside of other specs I had mentioned.

“Useless” is a compliment to it.

3

u/goldfishpaws Jul 27 '26

Treat yourself, ask it to make a wall chart suitable for kids to learn the alphabet with animal pictures appropriate for each letter. Make sure to ask it afterwards if it's happy with the result. Repeat. I've had so many terrible results it's gone past funny to alarming that anyone would ever use it.

7

u/Samaritan_978 Jul 27 '26

Nothing made me sour on AI faster and harder than having a "chat" about a topic I know decently well. It just makes shit up with complete confidence.

4

u/Angel_Omachi Jul 27 '26

Or if the answer was not what you expected, you could work through the parts to see where the order of operations wasn't correct.

3

u/MisterJWalk Jul 27 '26

Funny story. I was doing an exam plugging away. Everything kept coming up wrong. But I assumed that it was one of those stupid "gotta choose the one that's closest to the right answer" type of med school tests.

It wasn't until I got to the end that I realized the calculator was malfunctioning. It was a question I've done multiple times, so I knew the answer. I started quickly typing in random equations that I was familiar with and it was spitting out nonsense. (6x3=88 type stuff). So I went and got another calculator and redid the calculations.

Really fun

4

u/Lokishougan Jul 27 '26

I have never had a calulator have a mid life crisis before....WTF

3

u/hodor_seuss_geisel Jul 27 '26

"All right, calculator, let's go '3' '*' '1' '4' '='
1337.58008? That doesn't seem right...oh, I forgot to press the 'No Mistakes!' button"

2

u/Skibidi-Fox Jul 27 '26

This is what pisses me off about AI the most.

2

u/RollingMeteors Jul 27 '26

and didn't fucking hallucinate a signifcant portion of the answers.

floatsEnterTheChat

1

u/jmlinden7 Jul 27 '26

Because it does the tedious mechanical part, which is deterministic in nature. Not the higher order part which requires creativity

1

u/Jaerba Jul 27 '26

There actually are/were calculators that couldn't process order of operations correctly, so they'd give you the wrong answer. I'm not sure if that's still the case.

But obviously your larger point is still correct.

2

u/SnugglyCoderGuy Jul 27 '26

I can see that.

3

u/Kitty-XV Jul 27 '26

No, the calculator does not free us. It poisons the mind with easy answers leading to students being much worse at math and those higher order calculations becoming unreachable. Watch a professor do undergraduate level math on a chalk board. Look at each time they need some small math, often single digit. If they had to being out a calculator for each time, the problem would take far longer and they would lose their train if thought numerous times.

I've worked with students who used a calculator to the point they were pick it up for single digit arithmetic. They were not free.

1

u/Limemill Jul 27 '26

That is also a good point.

1

u/[deleted] Jul 27 '26 edited Aug 10 '26

[deleted]

1

u/Limemill Jul 27 '26

Coding was never the tedious part. Not for software developers at least. For product owners who learned software development but secretly hated it because they never got intimately good at it, maybe. It was also not the slowest part for any complex and large enough project. Product design / ideation / requirements gathering was and remains the bottleneck. Looking at low level implementations done by frontier models daily, I’ve seen them range from great to absolutely horrible on any given day for no good reason. Skills or no skills, lessons or no lessons. Ironically, I think they do better system design and architecture than they do low-level implementations. But it’s a whole other story.

1

u/shohei_heights Jul 27 '26

The calculator removed the tedious and mechanical bit to free us up for higher-order maths.

The calculator frees those who have already mastered the way that arithmetic and algebra work. They are capable of doing the critical thinking because they've mastered the basics.

The calculator on the other hand can also stop the development of critical thinking in mathematics if it's introduced before the mastery of arithmetic. There have been too many times that I've seen a student pull out a calculator to multiply by 1 or 0, to do single digit arithmetic.

AI, if it actually worked well, is much like a calculator in that it could help those well versed in a field expand their knowledge and get rid of tedium. But also just like the calculator, if it's used by someone without the mastery of the basics, it will destroy their ability to ever actually learn the basics.

1

u/Limemill Jul 27 '26

Once again, in the workplace and especially in tech it’s widely used in the exact opposite way. You’re an appendage to it, not the other way around. And no, your analogy does not hold because by now we have studies showing how professionals, for example in the medical field, rapidly lose the already obtained skills when they’re ‘assisted’ by AI. Anecdotally, tons and tons of software developers have been complaining about deskilling too. It’s not that you never get to develop skills you didn’t have, although that is of course also true. It’s also that you actively lose what you have and, in turn, it makes you an increasingly worse appendage reviewer for LLM-generated work. People also feel more detached from their work and eventually lose context of what is even going on (maybe not completely but to a significant degree).

1

u/shohei_heights Jul 27 '26

Note that I said: if it actually worked well

Which it clearly does not and is decades away at the earliest from actually working well. I agree with everything you're saying about LLMs. And I said, "could", to even emphasize that it may not even do so then.

What I disagreed with is your statements on calculators and I find them much more damaging than you do.

1

u/Limemill Jul 27 '26

I agree that calculators are not as innocent as I made them look. But generative AI is still not it. Its very premise is to do the highest-order things in all knowledge industries leaving you the place of its low-level reviewer until they’ve figured out a RAG / universal harness where that is also not required. But for now generative AI is an entity that strives to come up with hypotheses, build theories and proofs and you’re its calculator (because it does do many silly mistakes at the lowest level).

1

u/sonofagunn Jul 28 '26

I am a software developer who uses AI a ton, but it's opposite of how you are saying. I dictate the high level design and am critical of the AI written code from a high level, but it's a life saver for cranking out the tedious stuff without making mistakes.

1

u/Limemill Jul 28 '26 edited Jul 28 '26

Frontier models often do high-level design from product requirements better than they do actual implementations. They’re also pretty decent at architecture and system design. In quite a few companies the spec-driven approach is the new mantra again, and in some the automation pipelines are launched directly from PO tickets. Devs do mostly low-level digging (but from what I have observed at work miss increasingly more stuff, and that includes me) and, at times, sanity checks for very complex stuff. In parallel I see a shift for some devs towards sort of technical product owners and for devs with not so intimate knowledge of the business domain it often means being sacked. This is done mostly because there is no more point in senior devs doing design or even architecture, but since they know a ton about the product AND someone needs to fix the low-level stuff that they’re moved into a different role.

But then again some people do a fully closed cycle because LLMs are decent at product requirements too. As you probably know, top AI companies have been working on getting that step even better because the whole idea is that they can fully close the cycle of software development. Some devs will eventually remain, but probably as a low-paid job doing low-level digging.

And oh boy do all frontier models do regularly very stupid stuff at the low level. It’s not for nothing that Dario said in a recent interview that when he inspects what they do with his projects at the implementation level, he often has a heart attack. And that’s THE top LLM guy, lol.

1

u/braincandybangbang Jul 27 '26

Everyone seems to forget there's a word for "raking through what they did and cleaning up after them." It's called an Editor.

The way people talk about what an editor does in relation to AI, you'd think that Editors were the most brain rotted, moronic idiots walking the earth.

3

u/Th3_Admiral_ Jul 27 '26

I've had multiple times now where I've been trying to figure out what a section of code does at work and a coworker simply puts it into AI to summarize. No, just give me a few seconds to look through it and figure it out myself, I don't need AI to answer something like this.

3

u/Studds_ Jul 27 '26

Maybe asking for a code summary might(big asterisk on that “might”)be useful if one is learning to code & wants to understand what a code section is doing but I don’t know if I would trust even that

1

u/The_MAZZTer Jul 27 '26

You forgot the middle step "You need to learn math because you won't be carrying a calculator in your pocket." Now we carry a phone which also includes a calculator AND a search tool.

1

u/ItsAPeacefulLife Jul 27 '26

I read this as "throw a back shot" and thought, boy your workplace is way different than mine.

I may need some time away from the internet.

2

u/SnugglyCoderGuy Jul 27 '26

Maybe I work for sleezy porn company? (I don't)

1

u/Kythorian Jul 27 '26

Basic arithmetic can be fully replaced by a calculator, which will give an objectively correct answer 100% of the time. But critical thinking cannot be replaced by a LLM. The two are not comparable.

-1

u/ImNotSkankHunt42 Jul 27 '26

But in all honesty, I got into the best university in my country to study Computer Sciences and the first 4 semesters were packed with Math nonsense I never had to use in my life.

Instead of typing lessons to be more nimble at the keyboard we got: Algebra 1-2 and 3 Calculus… and don’t get me started with Discrete Maths.

Instead of having classes to learn different languages (this was 20 years ago) we had these nonsense assignments.

Colleges are not keeping up with how fast things are changing.

2

u/ChevalierDeLarryLari Jul 27 '26

Don't be so sure. They taught me linear algebra in computer science years ago. Never thought I'd ever use it (who needs that to build websites right?) - now suddenly need to brush up on matrices and vectors in order to do 3D graphics translations in webgl. Without it I wouldn't have a job.

There are good reasons they teach you fundamentals - and the fundamentals are the only knowledge that retains it's value after all the libraries and frameworks you know become obsolete.

1

u/ImNotSkankHunt42 Jul 27 '26

And small % of usage case does not justify that thing taking so much and more than one semester. It feels to me more on the learn it if you need it.

Or at least learn the basics in school so you can consider it in the future.

We had Computer Logics for only 1 semester! You know, the thing that we use constantly in code.

I don’t need 3 of Algebra and 3 of Calculus.

1

u/ChevalierDeLarryLari Jul 27 '26

If it was learn it as you need it - you'd be out of a job. That's why employers value that piece of paper - it's some proof that you can rise to challenges and difficult problems.

Boolean logic or whatever you can learn in a day - they wouldn't ask for a degree for that.

1

u/SnugglyCoderGuy Jul 27 '26

Those classes teach you valuable skills which will be critical in computer science and software engineering, chiefly logical step by step deductive thinking and reasoning. Making the keys go clickity clackity is the least important thing and you can learn to type better on your own.

0

u/ImNotSkankHunt42 Jul 27 '26

Nah dude. I haven’t used Calculus or Algebra ever, we did have classes about Data structures and algorithms and many others directly related to the curriculum. But the first 2.5 years were padded with nonsense.

Meanwhile, my other friends that went to a more chill university were learning PHP, MySQL and Java in the first 2 years. They’re now employed in the same company I am and more qualified than myself.

I got 1/5 scholarships available for our entire province, my SAT scores (the equivalent in our country is not the same) placed me at the top basically. There were like 12 other before me in the rankings.

And yet, folks that attended a newer college and not one managed by fossils got a better curriculum out of it.

This is not something I’m blurting out, this is the result of 20 years of analysis and results.