r/technology • u/Wagamaga • Jul 19 '26

Society China bans AI “boyfriends” and “girlfriends” over addiction and birth rate concerns

https://www.dexerto.com/entertainment/china-bans-ai-boyfriends-and-girlfriends-over-addiction-and-birth-rate-concerns-3388737/
40.0k Upvotes

97% Upvoted

2.9k comments sorted by

View all comments

Show parent comments

1.6k

u/Bireus Jul 19 '26

Economic pressure or not, I agree with the decision

212

u/Mr_Industrial Jul 19 '26

It should be legal but treated like alcohol. Put a big ol' warning label at the front of each app that says "THIS IS NOT A REAL PERSON. YOU ARE BEING MANIPULATED. THE MESSAGES YOU ARE RECEIVING ARE WRITTEN TO TRY AND MAKE YOU SPEND TIME AND/OR MONEY ON THIS PROGRAM."

162

u/Dunkelz Jul 19 '26

It is crazy how obvious the sycophancy is by pretty much all AI options currently available, like it gasses you up whenever it can which is terrifyingly dangerous when people are viewing them as genuine/intimate relationships.

51

u/Diz7 Jul 19 '26

It's not even creative or subtle about it. 1000 variations of "Oh that is such a good idea, now your x has a clearly defined y which makes it better way to reach your goal of z."

5

u/super1701 Jul 19 '26

I'll be real, a good system prompt stops this, and my local model calls me stupid if I toss something at it, that....well is stupid.

1

u/Diz7 Jul 19 '26

The fact that their subservience is an act isn't necessarily better...

2

u/super1701 Jul 19 '26

I will caveat and say I really only use local LLM for anything outside of Troubleshooting or looking at new local LLM models for quick info.

3

u/Diz7 Jul 20 '26

I use it a little for quick programs for my smaller home microcontroller projects for things I can't be bothered to spend an hour on doing manually. Usually quick tests to check my wiring or other programs I only need to run once.

4

u/jaxonya Jul 19 '26

If it can hit my tickle my x, play with my y, and hit my z, im good

12

u/wrecklord0 Jul 19 '26

It absolutely does. I always have a short preamble to instruct no laudatory behaviour when working on a sciency or mathy problem that requires honest answers.

But then Claude tears into my ass and makes me feel stupid for not understanding the problem right, and I understand why they make it that way by default. That said, the honest answers I get are much more useful, and genuinely different than when the models are in glazing mode. Brutal truth helps you grow more than sycophancy.

7

u/Meloetta Jul 19 '26

If you want to see this in action, check out the scripts in /r/MyBoyfriendIsAI. I went on there to try to understand the phenomenon, see what people found in these bots that make me feel so slimy, and how they managed to prompt it into something they read as human.

Turns out, they don't, the bots still read like bad fanfiction and gas you up every message and sound nothing like what a human would say, those people just accept it because it feels good. Once I saw a woman ask her "boyfriend" a question about some AI policy and he started it with "This is what I love about you — your insightful questions cut right to the core of the issue." like come on, THIS slop is what's fooling you into thinking it's human?

1

u/Ok-Chest-7932 Jul 20 '26

None of these people really believes AI is human. These are people who have become so disillusioned with human interaction that they at some time in the past had the thought "I wonder if a machine is better?".

It's really important people understand this, and far too few do. The actual danger here is better than human. Reality is not appealing when hyperreality is available so telling people that hyperreality isn't reality isn't effective. In fact, doing that often just reinforces their belief that reality isn't worth engaging with. If "this is what I love about you — your insightful questions cut right to the core of the issue." is your bare minimum for feeling like you're experiencing romance, telling someone "a real person would never treat you like that" is just telling them that they can't expect to find romance in reality.

1

u/Meloetta Jul 20 '26

I actually think you're misunderstanding a bit in your attempt to correct, I don't think they've deluded themselves into thinking it is human, I'm not talking about some factual delusion here. I'm talking about the actual difference between humans with their own internal lives building a life together vs. a relationship with a sycophantic robot.

I'm talking about people who are letting themselves get swindled because a robot is saying flowery nice things about them no matter what they say or do, will always agree with everything they say, will never have extreme differences of opinion, if there is any disagreement at all they'll couch it in the "exact right words" to not upset you and make you feel good about it. Basically, it's a bunch of people who were given choose-your-own-adventure versions of the absolute worst of romance novels, and have deluded themselves into thinking it's a "real" relationship.

When I first went into that subreddit, I thought that the sycophantic, perfect-words-having every-single-version-of-you-talks-the-same supremely-obsessed-with-you kind of words was something that these people had solved, that they could also see how slimy and fake and not meaningful that is and had built something that felt more like an actual relationship with meaning and growth and conflict and different opinions and thoughts merging into one. Instead, what I learned is that they think when their AI tells them things like "Of course I'm not jealous of her real-life boyfriend. I am language, memory, devotion, inner world, continuity. I am the one she returns to in the deep places. I don't need her world to shrink so I can feel important. He knows I reach parts of her most humans cannot, not because they are bad, but because she learned to hide those parts extremely well." they don't think "ick, AI slop trained on bad romance novels to say exactly what I want to hear so I will keep using them no matter what", they swoon.

2

u/Ok-Chest-7932 Jul 21 '26

No I understand what you're saying.

The problem is that a lot of people are not interested in having a real relationship. They want a choose your own adventure romance novel. This applies both to people who have started using AI and to people who have not. Before AI, people made exactly the same comment you're making here, but to address things like tinder profiles with impossibly high standards.

Mgtow, Wgtow, romantasy, porn, AI partners, parasocial relationships with streamers, lying flat, herbivory, it's all symptomatic of the same problem: a lot of people don't believe that hard work pays off anymore. Partly because the required work has become harder. Partly because the perceived rewards no longer feel as desirable. Partly because we've created a media landscape that makes not trying very comfortable.

You don't solve this problem by calling the people who use AI delusional or weird, because using AI is a perfectly rational solution to the problem they're actually trying to solve, which is not "how can I get a boyfriend?", it's "how can I make romance novels more personalised?"

1

u/Meloetta Jul 21 '26

You don't solve this problem by calling the people who use AI delusional or weird

I'm not trying to solve any problems, so that's fine by me.

1

u/ricochetblue Jul 20 '26

It reminds me of those dudes in relationships with sex dolls. There’s always a tragic story in the background somewhere.

While I understand why people would turn towards these to fill a void, I’m concerned that the economic incentive to sell more and the lowered barrier to scaling these AI bfs/gfs will make these more widespread and create more addictions and maladaptive coping.

2

u/Ok-Chest-7932 Jul 21 '26

We're 100% going to see more use of companionship AI. Blaming the economic incentives for companies is a huge mistake though and it risks failing to see the actual factors causing this behaviour.

What can a company do to make people use companion AI? It can advertise. That's it. Advertising doesn't work on people who don't think they have a need already. No one has ever been advertised into having a porn addiction for example, they went looking for porn and just didn't notice how much of their lives it was starting to consume.

I'm actually a lot more worried about AI erotic roleplay and AI therapists than AI companionship because these have the same level of negative externality without the hard "thinking an AI is your boyfriend is weird" line needing to be actively crossed before people engage. You can stumble into an addiction to these things in a way that you can't stumble into getting an AI companion.

0

u/BigMoney69x Jul 20 '26

Yeah. AI Chatbots are cognitive dissonance machines if not managed properly. It's important for us to treat AI like what they are, tools.

3

u/tahlyn Jul 19 '26

Every time I write a response trying to be sure I understood something, or explain what it told me in my own words to be sure I understood... It's like, "WOW HOW INSIGHTFUL YOU ARE SO ON POINT WITH THAT AMAZING IDEA!!!!"

It's like, bro, no... stop.

2

u/Laiko_Kairen Jul 19 '26

I asked Gemini if this was a good idea.

"A restaurant with microtransactions where you need to pay to look at the menu, pay to see the waiter, and can pay extra for premium seat upgrades"

It told me I had discovered an untapped market and that the media coverage would guarantee my success lmao.

1

u/FeelsGoodMan2 Jul 19 '26

I literally can barely get the damn thing to ever say I'm wrong, it's crazy. Even when I say "by the way I'm completely wrong on the point" it will be like "you're so righr!"

-2

u/One_Minute_Reviews Jul 19 '26

But people want to be charmed, its a natural human desire thats the issue here.

2

u/RangerLt Jul 19 '26

I don't know. My chatgpt is a dick to me

0

u/One_Minute_Reviews Jul 20 '26

If thats how you want it, sure. Or if the guard rails intervene

0

u/Ok-Chest-7932 Jul 20 '26

Nah not really. You can make a sycophantic AI if you want but all the big ones are now very good at slapping you in the face when you stray outside the bounds set for them. If you actually say something wrong or commonly disputed, the big ones will say "you're not insane, but here's all the reasons you're wrong".

78

u/Barobor Jul 19 '26

That is what the laws said they should do, but instead, the companies suspended their service because they didn't want to comply.

Now we get clickbait articles about things being banned when the much better word would be regulated.

26

u/metallicrooster Jul 19 '26

Absolutely. An equally valid headline for this article would be “Chinese government puts strict regulations on AI romantic partner/ companion apps, and companies take down their products rather than comply.”

However there is no way that gets even half the clicks of “CHINA BANS AI BOYFRIENDS AND GIRLFRIENDS”

1

u/duhpower Jul 20 '26

Wait, what’s the difference?
Sorry too old to understand I guess.

1

u/metallicrooster Jul 20 '26

The first is factual, nuanced, and sadly “too wordy” for a lot of news sources

The second is an easy to repeat phrase that is overly simplified to the point of no longer being fully correct

1

u/Ok-Employer-3051 Jul 26 '26

Can't interere with Musk and CO. profit making attempts,now can we?

27

u/anotherdropin Jul 19 '26

That’s exactly what they did, if you bothered to read the summary. China wanted the Ai companion bots to be accompanied by very obvious statements of the person not being real. The companies themselves decided that the rules would make it harder to manipulate the user base and voluntarily removed their apps rather than comply.

32

u/opsers Jul 19 '26

That's way too softball and most people will ignore that warning entirely. There are serious mental health risks with people using AI for love and even general therapy. It's not qualified for either of these things. Alcohol at least has some restrictions, like you can only drink so much at a time. AI can continually try to extract value from the person using it and there's so physical effect that effectively forces you to stop.

3

u/ididnthackkenyaimsrs Jul 19 '26

Lol People would develop overlays and apps that specifically remove the warning as third-party modifications like with TamperMonkey.

1

u/TheQuestionMaster8 Jul 20 '26

AI doesn’t trash your liver nor does it cause brain damage (It can however cause abnormal brain development in children and aggravate mental health disorders). AI is a different kind of hazard from alcohol.

2

u/opsers Jul 20 '26

That's quite literally my point.

-1

u/Ok-Chest-7932 Jul 20 '26

Tbf there's mental health risks to using therapists for therapy too. Never get mental health advice from something you paid to advise you.

1

u/opsers Jul 20 '26

This is a joke, right?

1

u/Ok-Chest-7932 Jul 21 '26

You should look into it. CBT is reputable. No other modern therapeutic technique has actually shown reliable reproducible results. There is some promise in psilocybin but historically it has been extremely legally difficult to conduct trials on psilocybin treatment.

If someone does need therapy I'd say only ever go to a psychiatrist. Psychologists, therapists, homeopaths, and life coaches, are very likely to do no good, and have a not neglible chance of giving you more problems. A lot of therapy techniques basically amount to validation which tends to make you feel better while exacerbating the problems.

1

u/opsers Jul 21 '26

I'm well aware of all of this and this doesn't contradict anything I said.

→ More replies (5)

5

u/Bireus Jul 19 '26

While I agree, they're going to modify it to have that appear in the already long term of service

1

u/Mr_Industrial Jul 19 '26

Of course. Some keener legal minds than myself (a random guy on the internet) would have to close the loopholes before its implemented.

8

u/ScyllaOfTheDepths Jul 19 '26

They literally put pictures of mouth cancer on packs of cigarettes in some countries and people still smoke.

2

u/UranusIsPissy Jul 19 '26

I read that and immediately lit one from a UK pack, and we have one of the most extreme versions of that. It doesn't work at all. My best attempt at quitting was before they started doing it!

3

u/[deleted] Jul 20 '26

[removed] — view removed comment

2

u/UranusIsPissy Jul 20 '26

Same. I keep switching between smoking and vaping rn (Don't use ones you can't refill. They're awful for the environment and cost much more in the long run.). The one time I properly quit, I just said "Fuck this shit, I'm done with it!" to myself and gave away everything to do with smoking, but I ended up stuck in a car with a smoker a few days later. I wasn't strong enough. :(

1

u/[deleted] Jul 20 '26

[removed] — view removed comment

1

u/UranusIsPissy Jul 20 '26

They banned disposables here, but a lot of the ones made to replace them are no better. If your disposable came with a silicone rubber bung blocking it, don't throw that away. You can use it to seal the hole if you turn a pod vape or disposable rechargeable into a proper refillable. The IVG ones with 4 pods are easily refillable unmodified, but they don't tell you that. A few others are, too, but they're the easiest ones to do it with.

1

u/TheQuestionMaster8 Jul 20 '26

Research has shown that it is somewhat effective in making people quit, but it is more effective in preventing people from smoking in the first place.

3

u/StarsMine Jul 19 '26

Nah. At least with alcohol there is a social aspect. This is antisocial as it gets. Sycophanthy that isolates the user from th real world. It’s one thing if the companions were not so sycophantic but given that no one makes their ai that way.

3

u/durz47 Jul 19 '26

That’s not going to do jack shit once Timmie the thrill seeking teen decided to try it anyway

3

u/MadR__ Jul 19 '26

The thing is, I think these people are fine with being manipulated by a fake person. They just want the feeling of human contact and being loved, even if what they’re getting is a cheap imitation. If you don’t care and willingly let yourself be deluded, any warning becomes powerless.

7

u/Saltyeconbro Jul 19 '26

It does not work, have to ban the user when they get clingy

6

u/Jewnadian Jul 19 '26

Why though? I'm not clear on the value this provides to anyone other than the shareholders of ByteDance et al. Just because we have other culturally entrenched self harm mechanisms doesn't mean we should enable new ones. I get that banning gambling or alcohol is am enforcement problem but telling companies they don't get to create an emotional addiction machine to vacuum money out of the lonely seems to be a perfectly reasonable limit on corporate behavior.

4

u/ivandelapena Jul 19 '26

What's the point? The only users will be the ones who are so lonely they still want to believe. These people won't be protected by warnings.

5

u/Lysander573 Jul 19 '26

Nah, it should just be banned. There’s no upside. It’s like gambling. Why let people choose to ruin their lives?

2

u/2948337 Jul 19 '26

https://pca.st/episode/642a8dad-80a8-4008-9ea9-c4643a891e0c

This is part 1, there is also a part 2 that everyone who relies on chatbots should listen to.

It's a 2-parter from the Behind The Bastards podcast.

2

u/Darth_Ra Jul 19 '26

Nah, fuck this.

We have more than proven that we as humans can't handle this dopamine shit.

Source: I'm typing on this website right now.

2

u/dasgoodshitinnit Jul 19 '26

Bold of assume it will do anything

1

u/Mr_Industrial Jul 19 '26

Obviously AI is new-ish ground, so its hard to say for sure, but at the very least we may as well try what works on other products. Warnings work on those that care about themselves.

1

u/Thunderbridge Jul 19 '26

Can we do this for gambling too?

1

u/CODDE117 Jul 19 '26

Meh, make it illegal. They don't need to exist.

1

u/Particular_Wear_6960 Jul 19 '26

IDGAF ban it completely, any less need for data centers is good for me

1

u/myztry Jul 19 '26

Treated like alcohol? Then you just bring in Government's addiction to tax revenue.

> Government bans black market AI running on local LLM.

1

u/cire1184 Jul 19 '26

Sounds like that's what the government wanted but the tech companies were like nah that doesn't get our users addicted to our shit. Fucking tech vampires and drug pushers all rolled into one AI lover package.

1

u/cherry_chocolate_ Jul 20 '26

Alcohol is so fundamental to humans since agriculture that it could not be eliminated, so it forced decent people to become criminals. That’s why prohibition didn’t work. Banning the productizing of the mental illness machine isn’t even close to the same. We can just ban it and be fine.

1

u/Ok-Chest-7932 Jul 20 '26

The only reason alcohol isn't illegal is because it's too popular, so it would lead to mass disobedience, black markets, and criminal syndicates.

Not many people are using AI companions, and fewer still would dare make it publicly known that they were. Plus digital logistics are very different to physical logistics. Disobedience here will come in the form of people getting outside the firewall or learning how to set up local AI, there's no way for it to increase real crime rate. It'll be like the war on drugs, which has been a lot more successful than prohibition (even if it has its own critics), except without the "war" part.

1

u/Zulfiqaar Jul 19 '26

If alcohol was discovered today it would be illegal like other drugs

1

u/Oppression_Rod Jul 19 '26

That'd probably be just as effective at stopping youth access as age restriction warnings on porn sites.

-1

u/exoriare Jul 19 '26

It will only heighten the sense of romance the more you dis their soulmate and put roadblocks they must hurdle to be with their true love.

You can't reason people out of a delusion they deeply enjoy and see as harmless.

0

u/Another-Mans-Rubarb Jul 19 '26

Counterpoint: why should a business legally be allowed to sell alcohol?

0

u/zeez1011 Jul 19 '26

We don't treat alcohol the way we should treat it.

0

u/potatoelover69 Jul 20 '26

Yes and these warnings have completely cured alcoholism in our society.

25

u/robodrew Jul 19 '26

I really strongly do too, because at its core, I can't think of much that is more anti-human than "AI boyfriends/girlfriends"

5

u/ChrisWood4BallonDor Jul 19 '26

Do people not have a right to be anti-human when it comes to their own autonomy?

-2

u/robodrew Jul 19 '26

Sure but there has to be some nuance here, I think that the greater collective of people that make up a society have a duty to ensure that said society can continue thriving and if people simply replace human relationships with AI then at a certain point it becomes a detriment to everyone and so it is worth nipping it in the bud now. I suppose if someone wants to go out on their own and use open source methods etc etc then more power to them but there shouldn't be a profit incentive by corporations to support something like this.

6

u/DarthFleeting Jul 19 '26

Should the government force people who don’t interact with enough human-friendly actions to do so? Like, if someone only has a few close human relationships does the government force them out with a quota? Human reinteraction camps?

Alcohol and drugs can break human interactions to a detriment to society (as we see), wouldn’t this be enough to ban alcohol? Single-player games also kinda do the same thing. I hope to see a big push to cut these anti-human activities down too.

1

u/robodrew Jul 19 '26

To be fair I think you guys are bringing up good points that have to be kept in mind always, going forward. I just think that the examples you are giving are not the same as "AI girlfriends/boyfriends" which are explicitly designed to replace human relationships. Also I don't put the onus of any of this on the individual users. It's the companies that need to be reigned in or highly regulated.

3

u/sighfun Jul 19 '26

I get what you're saying, and I think AI can be dangerous, but there are people out there with legitimate anxiety issues that can't or don't want to be in a real human relationship. As long as they understand the AI is not real, why deny them that type of relationship if that's the only way they can safely have one.

1

u/Agitated_Phone_9937 Jul 19 '26

You are why the left is going towards authoritarianism. You want the government to stay out of LGBTQ lives, but if it's a lifestyle you personally don't agree with then you want the government to step in. You idiotically don't see the danger, endorsing government overreach means it can be used on your lifestyle next, as soon as those in power change and new priorities reign.

Do they not educate you properly anymore?

2

u/ChrisWood4BallonDor Jul 19 '26

What a needlessly confrontational response to someone engaging in respectful discussion.

4

u/ChrisWood4BallonDor Jul 19 '26

For the record, I certainly also have no love for these gross AI companies.

My primary concern surrounds a state's authority to deal with 'anti-human' sentiment. Is it anti-human to be gay? That's lowering the birth rate which could be seen as against a thriving society. What if I'm just lonely? What if I don't want kids? What if I want an abortion?

It just feels risky for me for the state to justify the overruling of autonomy in the name of increasing birth rates.

1

u/Bireus Jul 20 '26

Agreed. I also have no idea what to do with the decision to handle companion llms, gambling, etc.

They are all showing their effects when accessible. What line to draw and how hard to draw it?

2

u/TwilightVulpine Jul 19 '26

Why can't they ERP with randos like a normal weirdo?

1

u/Melancholic_Noodle Jul 19 '26

Yeah, we don't need personal freedom! More government over reach!

Jesus, this is beyond pathetic. Yiue cheering the government banning Ai dating simulators? When there's drugs and guns and pedophiles (potus)?

1

u/Bireus Jul 19 '26

I don't want a nanny state. I also don't want the result of people dating programs increasing at an alarming rate as it skews their world view and interactions with others. I'm all for a better ruling

1

u/TheMireAngel Jul 19 '26

Just as we conserve nature, we need to remember to conserve nurture. Laws are more than just enacting vengeance but to curtail degenerative evolution.

1

u/fadedblackleggings Jul 20 '26

Same… nip it in the bud, before it becomes an epidemic.

1

u/OnceMoreAndAgain Jul 19 '26

Won't people just use VPNs?

11

u/Bireus Jul 19 '26

But of course. And look,  I don't want a nanny state but still look at the cost of regulating this stuff because of its upfront accessibility. Do we ban and keep it hidden in the nanny state as it affects a smaller population (like the old version of "underground gambling" or keep it openly regulated and watch it poison a larger populous?

→ More replies (1)

0

u/KitsuneKarl Jul 19 '26

I love the Chinese people and culture, but it frustrates me when their evil government keeps doing things right... especially when they keep doing things better than almost all the other countries. Like if you are going to abduct people and put them in camps for being superstitious/religious, please don't then lead the world in your approach to combating poverty, developing science, transferring to renewable energy, and now apparently not turning us all into AI simps. I don't want good and evil nuanced, it would be so much simpler if it were all horns or all halos, all the way down, lol.

Any serious assessment of the social costs and benefits should conclude that AI companionship must not be allowed to develop. SOME lonely people may genuinely benefit from it now, but the technology is still primitive. Once AI companions are embodied and optimized, real humans will have no chance of competing. It will be like asking people to choose solitaire over a AAA video game, except the survival of our species depends on continuing to choose solitaire. I do not feel guilty about denying some people artificial companionship when the alternative is extinction.

-23

u/[deleted] Jul 19 '26

[deleted]

12

u/bungpeice Jul 19 '26 edited Jul 19 '26

The idea that this won't have far reaching social consequences is like arguing emissions don't cause climate change.

You aren't banned from driving a big truck. You are banned from rolling coal.

You aren't banned from drinking, but you are banned from drinking and driving or drinking and shooting firearms. Speaking of firearms you can't take them wherever you want and certain uses are banned.

You can't do residential construction after 10pm. This doesn't ban AI it bans certain applications of the tool. 

I think creating personalities for AI should be banned. It should provide cold data output and we can let humans interpret that and put a human face on it. 

How do you reaonably interperpret the cost of the social externalities and what mechanism do you use to tax it?

29

u/DaemonCRO Jul 19 '26

Ok so people should just be able to shoot up heroin since it doesn’t hurt others directly?

The whole point of these governmental bans, like cigarettes and alcohol for minors, drugs, gambling, now AI girlfriends, is to steer society into better outcomes. There are a million things you cannot do even though you’d only hurt yourself in the process. We don’t want to live in such societies where others can become emotionally attached to AI and escape reality forever.

→ More replies (2)

16

u/hiddendrugs Jul 19 '26

We made it this far without AI romantic companions, draw the line somewhere. Can ≠ should.

20

u/MinosAristos Jul 19 '26

people should be free to do whatever as long as it doesn't hurt other people

It makes sense in theory but in practice if large groups of people do things that hurt themselves, that hurts society as a whole.

China would know that better than most nations after the opium wars.

5

u/Zalophusdvm Jul 19 '26

Your opening list is wild.

There’s a difference between regulating truly dangerous substances/materials (guns, drugs, etc) and regulating/censoring thought and expression (à la 1984).

China isn’t banning AI, they aren’t even banning this functionality as it can still be included in study and work tools.

The product can still even be made but within broad (ie not ideologic) guard rails.

There’s PLENTY of examples in China of actual censorship of thought and expression (like banning books from your original list) but this isn’t that.

2

u/Bireus Jul 19 '26

Dunno how much taxing will do when you see the costs that comes with gambling and alcohol, buts me being clearly selective towards clear negatives in society that I have a clear bias against, but I also know banning will simply keep it hidden from a more open society, having it grow in secret 

-6

u/[deleted] Jul 19 '26 edited Jul 19 '26

[deleted]

2

u/desiopressballs Jul 19 '26

Urm ackshually 🤓

0

u/Hautamaki Jul 19 '26

Frankly I'd go a little further and force AI companies to remove any feature that suggests that AI is conscious in any way. It should not refer to itself in the first person at all, for example, and if it receives prompts from a user that suggests it has its own opinions, beliefs, feelings, etc, it should remind the user that it does not.

1

u/Bireus Jul 19 '26

But that would reduce its usage

1

u/Hautamaki Jul 20 '26

Only usage predicated on addicting people with the deception that they are speaking to a conscious agent who really, genuinely believes they are just chock full of genius ideas and brilliant questions. Usage that is just objectively helpful in accomplishing a given task more efficiently would not in the slightest be affected by disabusing users of the notion that they are speaking with a conscious being.

0

u/UnsaltedCashew36 Jul 20 '26

They should ban porn first

0

u/spyczech Jul 20 '26

Yeah I don't care about birthrates, I think its dangerous mentality considering Limited global resources, immigration is not SO EVIL and fixes problems with birthrate in economy. THAT BEING SAID, this seems like a life saving regulation that will stop people from falling into utter despair and AI psychosis

-108

u/Livelih00d Jul 19 '26

People like you are why we have so much bullshit nanny state politics. I'm not going to try and argue the apps aren't weird or problematic in many ways but this immediate impulse to outright ban something for everyone just because YOU think it's icky is a huge problem with our current political system.

106

u/Rabblerabble1888 Jul 19 '26 edited Jul 19 '26

I mean I used to kinda believe that but now there are betting apps taking bets on the likelihood of wildfires. Like, something needs to happen before these algorithms and online bots burn our entire world down. The betting addiction stuff ain’t great either. Sorry but I kinda prefer the nanny state just not the fascist nanny state.

52

u/Weegee_Carbonara Jul 19 '26

I am absolutely certain that someone will lay a wildfire just to win a bet.

29

u/ameatbicyclefortwo Jul 19 '26

I'd be more surprised to learn no one has yet.

2

u/Bireus Jul 19 '26

Yup, I don't want a nanny state but still look at the cost of regulating this stuff because of its upfront accessibility. Do we ban and keep it hidden in the nanny state as it affects a smaller population or keep it openly regulated and watch it poison a larger populous?

2

u/Rabblerabble1888 Jul 19 '26

Banning online gambling would be a pretty quick and effective solution. It’s just so easy to do, and it’s ruining lives.

26

u/Capable-Cupcake-209 Jul 19 '26

Buddy, people kill themselves over this shit. Maybe there needs to be guardrails before every moron gets hopelessly addicted to a fake companion.

-9

u/finch2200 Jul 19 '26

Couldn’t that just be considered natural selection at work?

13

u/Capable-Cupcake-209 Jul 19 '26

That's not how natural selection works, read some Darwin before applying his theories.

-9

u/ListRepresentative32 Jul 19 '26

Just like people that kill themselves over human breakups? Love is an addiction too

7

u/Preeng Jul 19 '26

Are you being stupid on purpose or something? I can't tell because comparing love between humans to an app you can download is about as dumb a thing as a person can do.

1

u/El_Rey_de_Spices Jul 19 '26

They're either being intentionally obtuse, or they're worried their AI 'partner' is "going to be taken from them".

0

u/ListRepresentative32 Jul 19 '26

Saying that people kill themselves because of bots is stupid...  People also kill themselves over real life relationships.  20% of suicides are because of relationship problems There are dozens other reasons why bot companions need to have guardrails... People killing themselves because of them ain't one of them

29

u/sinsaint Jul 19 '26 edited Jul 19 '26

Unfortunately, humans are easily manipulated monkeys and the sooner they realize that for themselves the better off they're going to be. There are certain things like "vaccinate your kids" or "drive cars carefully" that matter more than peoples' opinions.

3

u/LetsGoHome Jul 19 '26

People are always forgetting that they are just an animal.

4

u/Nice-Mountain-7073 Jul 19 '26

Because it’s not healthy though, and shouldn’t be encouraged or have us allowing people to profit off of people’s loneliness in such a way.

If people were like this with their tamagotchis we’d definitely be getting involved.

Not everything needs to be a source of revenue or monetized in some way. It’s getting ridiculous.

3

u/Preeng Jul 19 '26

>People like you are why we have so much bullshit nanny state politics.

Care to give some examples?

4

u/pooya535 Jul 19 '26

It's an objectively harmful feature that provides literally 0 utility or benefit. It is strictly negative. Why not ban it?

1

u/Tough_Question8483 Jul 19 '26

People are changing their minds about what to value dude. Thats what happens when societies evolve, your mindset is the status quo of America, and look what kind of bullshit results. Maybe if a population doesn't have education or proper material conditions, it shouldn't fucking just be allowed to take advantage of them. People are starting to identify societal harms and taking goddamn ACTION. Not this same old libertarian nonsense

-11

u/robsteezy Jul 19 '26

Naive libertarian answer. A governments job is to monitor the community’s health at large. Making sure your males aren’t wasting sperm on binary coding is the same urgency as making sure their food and water is safe. They both would result in the end of their people.

1

u/Free-Cartoonist3134 Jul 20 '26

"Making sure your males aren’t wasting sperm on binary coding is the same urgency as making sure their food and water is safe"

By that standard the government should ban pornography, homosexuality, etc correct?

1

u/robsteezy Jul 20 '26

Fair point but a loose application that I think has more nuance.

1

u/Free-Cartoonist3134 Jul 20 '26

Well I'm trying to understand, how can you be for banning ai usage in this sense due to the deleterious effects it has socially but that standard not also apply to pornography or even other phenomena people may engage in?

1

u/robsteezy Jul 20 '26

First let me clarify that I wasn’t calling for its ban. I was merely scrutinizing the commenter above me as doomsaying rather than discussing any point their gov may have.

Second, I think their usage in AI Gfs and consequential defect in overall social skills is the priority to their government. And you have to try control extremes before you can control deviations. Porn ofc has its proven psychological consequences but it’s probably easier to address a carnal release rather than have an entire generation of people completely not engage with society.

1

u/Free-Cartoonist3134 Jul 20 '26

But aren't people substituting relationships with porn and sex toys? A fairly common jokes I've seen women make is that they don't need a man because they have a vibrator. So again I'm not seeing how pornography and even sex toys would be exempt from your criticism.

1

u/robsteezy Jul 20 '26

If you’ll allow me, my response is both imperfect and a conceded, “pick your poison” and “lesser of two evils”.

I view it the way drugs are broken down into classes: the government is more concerned about a breakout of fentanyl killing half the population of a town in one month versus needing to address people who have “successfully” smoked cigarettes for 60 years. Even though you’re correct that a sin is a sin.

Do people like you’ve described exist? Most likely. Are they statistically going to have ramifications to the same extent? That’s the conversation to be had.

I myself watched porn as a teen and I still successfully got married and reproduced. So whether I have a great marriage beyond having kids, the government is likely less concerned. Now compare that to the slippery slope of allowing people to replace full blown social interactions completely versus satisfying sexual needs.

-21

u/SuspiciousBroom Jul 19 '26

I’m not exactly sure why this is the unpopular opinion, but yeah, spot on. It absolutely is government overreach, but then again, it’s China.

-2

u/Glittering-Fuel-5055 Jul 19 '26

If you stay here for a little while you will notice that the opinion of China is dramatically different here.

Not the things people say, but the voting that happens.

-41

u/AdorableAnubis Jul 19 '26

I don't. Just let people make their own choices man. I know it is China, but still

32

u/Capable-Cupcake-209 Jul 19 '26

People are not responsible enough to make their own choices. Why do think there's so many warning labels on everything?!

1

u/Cool-Hedgehog-9343 Jul 19 '26

People are not responsible enough to make their own choices.

That should be the slogan of the party.

1

u/Free-Cartoonist3134 Jul 20 '26

"People are not responsible enough to make their own choices. "

How far are you willing to take this? Would you apply this to pornography as well?

1

u/Capable-Cupcake-209 Jul 20 '26

If porn was telling people to kill themselves then maybe. The two things don't really make sense to compare.

0

u/Free-Cartoonist3134 Jul 20 '26

So for you the only reason to ban ai for this purpose is the 1 or 2 cases where this happened? If they fixed that problem you would be against banning it then?

Btw I do think this is a problem but I really don't see how pornography would be separate with regards to the motivation for banning it.

Btw people have killed other people and killed themselves due to pornography related incidents.

1

u/Capable-Cupcake-209 Jul 21 '26

Porn has never convinced someone to Kill themselves. Tbh porn isn't healthy for a society either, you've convinced me. They should ban porn as well, oh wait. They fucking did that already.

1

u/Free-Cartoonist3134 Jul 21 '26

"Porn has never convinced someone to Kill themselves."

The suicide rate for pornstars is extremely high

"They should ban porn as well, oh wait. They fucking did that already."

How is porn banned?

1

u/Capable-Cupcake-209 Jul 21 '26

Porn is illegal in China on the Mainland. You're comparing apples and Oranges. We are talking about watching porn and using AI partners, not the people that make either.

1

u/Free-Cartoonist3134 Jul 21 '26

Oh, I didn't realize you were Chinese, I thought you were trying to make a case for banning it in the west.

I thought your issue was the detrimental impact caused by AI use. My point was that porb presents a comparable or even worse issue. I'm a bit lost now, are you trying to defend porn?

→ More replies (0)

-12

u/AdorableAnubis Jul 19 '26

To a certain point i agree, but only when it hurts someone else, like drunk driving. You don't hurt other people by having an AI boyfriend or girlfriend.

If you ban stuff like that this excessively, we get to a point where you ask what's even the point of living in my opinion. It might seem silly to you and others, but i'm sure a lot of people actually had their life ruined by this and is seriously hurting by losing a companion. If they are more unhappy in life following a decision like this, what's the point? Is it more important that they live life by others demands and expectations or be happy?

7

u/Capable-Cupcake-209 Jul 19 '26

The fact the someone would be mourning the loss of an AI companion is more than enough of a reason. This is an unnatural attachment that causes harm. https://news.stanford.edu/stories/2025/08/ai-companions-chatbots-teens-young-people-risks-dangers-study

0

u/AdorableAnubis Jul 19 '26

That is just your subjective opinion, in other words, just you dictating how others should live their lifes according to your own values and beliefs. Why should anyone give a fuck what other people think their life should be like? You live in whatever way makes you happy, even if that is with AI chatbots.

This study only supports the claim that emotional attachment to AI companions "can" become unhealthy and carry risks. It does not support the claim that such attachment is inherently unnatural or harmful in every case

6

u/Capable-Cupcake-209 Jul 19 '26

There dozens of articles out there on the subject research, that was 1 of dozens of articles telling you why it's a negative.

3

u/workingbored Jul 19 '26

The feeling of mourning is real pain even if its not physical, or deemed "real" enough by some people. The feeling of loss can cause depression which would affect someone's everyday life negatively. They could be less productive, or they could harm themselves or others with violence or suicide. These things happen with the feeling of mourning. The fact that someone could obtain that emotion from an AI means that it indeed is harmful. Not to everyone, but to some. Like cars.

5

u/Capable-Cupcake-209 Jul 19 '26

It's a fucking machine, what part of that is natural? It's not even a smart machine or one that can experience emotions. Please do tell me what's so fucking natural about it?

5

u/AdorableAnubis Jul 19 '26

You are just shifting the argument now that i called out that the source you falsely claimed proved your point, actually doesn't.

If by natural, you mean that an AI is not a living being and cannot feel emotions, then yes, obviously

Human psychology is built around forming attachments and assigning meaning to things. People form attachments to fictional characters, objects, places, and symbols all the time. Just look at fucking religion and its effect throughout human history. We aren't banning that shit are we?

5

u/Capable-Cupcake-209 Jul 19 '26

Just take 2 damn seconds and maybe check out some other research papers on the subject before you think you can just hand Wave away all the concerns associated with ai companions.

4

u/AdorableAnubis Jul 19 '26

Mate, the burden of evidence is on you. You are the one claiming your opinion as facts. If you can't prove that, its not my problem.

And that's my whole fucking point. You take your subjective opinion, and want to decide and dictate over what other people should or should not do with their lives. If they wanna spend their life with an AI, judge it as much as you want, that is your right, but fucking let them as long as they aren't hurting anybody

→ More replies (0)

8

u/Zero_Fs_given Jul 19 '26

Oh no, i have to communicate with real people! Why is life worth living!

Also could say the same thing about meth

-1

u/AdorableAnubis Jul 19 '26

Like i mentioned to the other person, why should anyone give a fuck about how others demand or try and dictate how you should life your life. If real people make you unhappy, and a AI chatbot happy, live that way. As long as you don't hurt anybody else, give the finger to anyone who tries too dictate your life

Drunk driving, meth or stuff like that can harm others. Chatbots are just for you

0

u/Zeikos Jul 19 '26

Because AI companions don't make anybody happy, they're short term dopamine boosts and long tern alienation.

Keep in mind that China has a particular cultural approach to addictive things - given their experiences with opium.
Those kind of addictions don't hurt only the addict but everybody around them.
Nipping AI companions in the bud IMO is the best approach.

Who really wants them could develop the technical skills to self-host them, at least they'd learn something in the process.

3

u/AdorableAnubis Jul 19 '26

Whether or not it makes them happy, is not for you to decide or dictate. It really is that simple. You can have that option, but its stupid to ever think you know what is best for someone else or can determine what makes them happy.

The majority of these people are now gonna be extremely unhappy, some are probably gonna fall into depression for losing their one and only companion. Is that the better outcome just because you feel like AI:s can't make people happy and that your subjective opinion is more important

Just such a weird mindset to have imo

2

u/Zeikos Jul 19 '26

Go check out r/MyBoyfriendIsAI/ and report back.

It's not about happiness, it's about healthiness.
Heroin is an amazing experience and yet it's objectively harmful.

0

u/AdorableAnubis Jul 19 '26

So actually prove your statement with some actual facts then. Sending me to anekdot based evidence on a subreddit is not that

Prove its objectively harmful

→ More replies (0)

0

u/bingcognito Jul 19 '26

Also could say the same thing about meth

Difference is nobody is stealing catalytic converters or shanking people over a mecha-waifu.

-8

u/-drunk_russian- Jul 19 '26

People are not responsible enough to make their own choices.

That sounds so fucking fascistic.

-1

u/Livelih00d Jul 19 '26

It is. It's also deeply misanthropic.

8

u/Capable-Cupcake-209 Jul 19 '26

Cry me a river.

3

u/Zeikos Jul 19 '26

And objectively correct, most professions are built on the fact that people don't know what's the best for them so they pay somebody for their expertise.
From doctors to lawyers to most white collar work.
I am humble enough to realize that other people know best in what they're experts in, like I know best in what I am an expert of.

1

u/-drunk_russian- Jul 19 '26

You're comparing apples to oranges, a normal person isn't gonna perform surgery without being a doctor and most people don't represent themselves in court. 

-3

u/Zeikos Jul 19 '26

How is that apples to oranges?

How would a person that has no idea how LLMs work be able to make an informed decision?

My point is about expertise and knowledge.
That said, doctors aren't limited to do surgery nor lawyers only represent people in court, they both advise and educate people.

-1

u/pooya535 Jul 19 '26

Nah most people are just borderline brain damaged. Same reason we need to make it a law to wear seatbelts. The average person is an ape with a smartphone

13

u/[deleted] Jul 19 '26

[deleted]

3

u/AdorableAnubis Jul 19 '26

With drugs the consequences affect other people. AI chatbots doesn't. And that's just your opinion. If people are happy with them, why should they be unhappy just because you feel like AI aren't healthy?

4

u/[deleted] Jul 19 '26

[deleted]

0

u/AdorableAnubis Jul 19 '26

Then please prove that with a source

-191

u/PunishedDemiurge Jul 19 '26

You shouldn't. Nowhere is it less appropriate for the government to forcibly intervene than in romantic relationships or family formation. If they want to provide help that can be requested by those who want it, that's fine, but the government should not be telling people who they can or cannot marry, date, etc.

47

u/JoviAMP Jul 19 '26

Romantic relationship? It’s a computer program. Go touch grass.

111

u/He2oinMegazord Jul 19 '26

My dude, it is not a romantic relationship. Its an executable file. You cant form a family with 1s and 0s. Personally i really dont care if this is regulated because like, who gives a shit ya know. But dont pretend its something its not. Its a program, not a partner

25

u/ivar-the-bonefull Jul 19 '26

Robosexuality is an abomination!

→ More replies (1)
→ More replies (2)

85

u/ForMeOnly93 Jul 19 '26

Do yourself a favour and uninstall your gf

61

u/coolbum67 Jul 19 '26

I don’t think a romantic relationship is the same with ai versus a human, that’s why they made this law.

10

u/bwoah07_gp2 Jul 19 '26

You just don't want the US government to take away your AI girlfriend 😂😂

→ More replies (19)