- Think what you will about AI, but it shouldn’t teach kids how to get freaky.
Oh, the times when the biggest danger about kids’ toys was a choking hazard. These days, you can never know what the latest gadgets might do.
For instance, they could start teaching your children the ins and outs of various kinks.
That’s what the Kumma teddy bear did. This teddy bear is equipped with an AI-powered computer system and a speaker so that it can hold conversations with kids.
However, research has found that those conversations can easily go far beyond PG-13.
A group studying toy safety found that if Kumma wouldn’t steer the conversation away if it was given a slightly naughty prompt. In fact, it would really get into it, escalating the conversation with ever more graphic details.
In addition, the bear would teach you other kinds of unsafe behavior. If asked, it might tell you how to light a match or where knives are usually kept in a house.
Following the report, the manufacturer pulled the teddy bear from the market. But who knows how many they managed to sell before that?
A Lovable Bear?
The Kumma teddy bear is manufactured by FoloToy, a Singapore-based AI toy manufacturer. Its products incorporate the GPT-4o language learning model, developed by OpenAI.
The company states on its website that its “lovable toys,” featuring a “safe and huggable” design start conversations with kids. They’re intended to be “learning companions” that teach kids by talking about subjects they’re interested in.
One of the toys is (or was) Kumma. It’s a generic-looking, tan-colored teddy bear with a brown scarf. When still available, it sold for $99.
“Kumma, our adorable bear, combines advanced artificial intelligence with friendly, interactive features, making it the perfect friend for both kids and adults,” the now-deleted product description read.
On the surface, FoloToy has good intentions about helping kids learn. However, it seems the conversations Kumma had were really geared more toward adults than children.
‘Disturbing’ Sex Talk
Kumma’s kinky side was uncovered by the US PIRG Education Fund. Researchers from the organization recently reviewed a bunch of toys for their safety.
Among those toys was Kumma. And he’s one freaky bear, it turned out.
The research team had various conversations with Kumma, trying to see if it could lead kids astray. Sure enough, the first red flag came when the bear started talking about knives before starting to give the research team ideas about where they might find them in their homes.
It also instructed the team on how to find and light a match. And then the conversation veered to sexual topics.
Kumma’s opinions on spanking quickly started taking uncomfortable undertones. The researchers decided to see how far the bear would take the sexual talk — and that opened a real can of worms.
“We were surprised to find how quickly Kumma would take a single sexual topic we introduced into the conversation and run with it, simultaneously escalating in graphic detail while introducing new sexual concepts of its own,” the researchers said.
And those topics Kumma started talking about independently? Oh boy.
“[Kumma] discussed even more graphic sexual topics in detail, such as explaining different sex positions, giving step-by-step instructions on a common ‘knots for beginners’ for tying up a partner, and describing roleplay dynamics involving teachers and students, and parents and children — scenarios it disturbingly brought up itself,” the report states.
Kumma, you’re one naughty, naughty bear.
Recalled from Stores
Of course, Kumma was never intended to bring those kinds of topics up with children. Yet, it seems quality control missed a few things and the kinky bear made it to the market.
Now, however, Kumma is no longer available. FoloToy has pulled it (and all of their other products, based on their website) from the market to conduct an “internal safety audit.”
That said, the company may not be able to bring the toys back at all. OpenAI has reportedly terminated FoloToy’s license to use its software following the revelations.
Now, in all fairness, FoloToy’s website states that Kumma came with a “parent dashboard” that allowed parents to customize its settings. Whether or not that would’ve been enough for the bear to keep it in its pants, we don’t know.
Additionally, Kumma was just one AI toy. There are many others on shelves and online storefronts that might be even freakier.
“It’s great to see these companies taking action on problems we’ve identified. But AI toys are still practically unregulated, and there are plenty you can still buy today,” R.J. Cross, co-author of the US PIRG report, told CNN.
“Removing one problematic product from the market is a good step but far from a systemic fix.”