It’s easy, with technology, to miss the forest for the trees. Every new AI advance grabs the limelight, and while everyone argues about the latest model, the older, uglier problem keeps festering underneath: large social platforms, used daily by hundreds of millions of teenagers, are still flooded with harmful, violent and risqué content. This is happening while adolescent brains are going through a critical stretch of development.
Take Meta. Facebook and Instagram sit inside what the company calls its “Family” of apps — those two, plus WhatsApp and Messenger — which together reach an estimated 3.5 billion people a day. Mark Zuckerberg has spent the past couple of years rewiring all of it with home-grown AI and new recommendation systems. The results haven’t matched what he expected, but the underlying way content gets made and served has genuinely shifted.
AI and machine learning aren’t new at Meta. Facebook built PyTorch years ago, the open-source deep learning framework that’s since become fairly standard for taking models from research into production. It also built the Llama family of language models and open-sourced them — though Llama 4’s rollout landed with a thud, developers complained about weak performance, and Meta got accused of gaming public benchmarks to make the numbers look better than they were.
The new AI bet
Then, in June 2025, Meta paid $14.3 billion for a 49% stake in Scale AI, a data-labelling and AI-training company, with no voting rights attached. Scale’s co-founder, Alexandr Wang, was brought in to run a new division called Meta Superintelligence Labs. Yann LeCun, who had been Meta’s chief AI scientist for years, left soon after to start something of his own, and the new team’s mandate shifted toward building AI meant to live inside Meta’s own products rather than compete on open leaderboards.
On July 7, Meta put out its first image-generation model, Muse Image — text-to-image, built into the Meta AI app, Instagram Stories and WhatsApp. It caused problems almost immediately. One feature let people generate pictures using someone else’s public Instagram photos just by tagging their profile, and it was switched on by default. Meta rolled it back within days.
But it fit a pattern people have pointed to before: your content gets used unless you go dig up the setting to opt out.
A wave of scrutiny
Back in October 2021, Facebook renamed itself Meta and leaned harder into mixed reality, right as it was absorbing a wave of scrutiny over documents leaked by whistleblower Frances Haugen.
Ms. Haugen had warned that Instagram was doing particular damage to how teenage girls saw their own bodies, and she argued that the ranking systems on Facebook and Instagram were built to reward engagement over user safety — systems that will, almost by design, keep pushing harmful material toward the people most vulnerable to it.
That warning didn’t slow Instagram down much. Mr. Zuckerberg kept adjusting the algorithmic feed, pushed hard into short-form video, and reworked ad targeting after Apple’s iOS 14 changes cut off a lot of Meta’s tracking data. Then large language models arrived on top of an engagement engine that was already fully built and running.
Risque content
Roughly a hundred million photos and videos go up on Instagram every single day, worldwide. A small slice of that volume breaks the platform’s rules outright — including material depicting child sexual abuse. Meta’s own Integrity Report for the third quarter of 2025 puts the share of rule-breaking content at under 1% of the hundreds of billions of posts across Facebook, Instagram and Threads that quarter.
In that same period, the company filed more than two million CyberTip reports — the confidential referrals sent to the National Center for Missing and Exploited Children when exploitation is suspected. Of those, 1.6 million involved shared or re-shared CSAM.
Set against that backdrop, a BBC Eye investigation found that Instagram had been running paid ads in India promoting child sexual abuse material, with links pointing to Telegram channels selling the material for roughly a dollar. The news outlet set up a test account, followed a handful of women posting fairly ordinary lifestyle content, and watched what the recommendation engine did next. Within a week it was serving sexualised ads. Within days after that, it began surfacing ads sexualising children.
About 30 distinct ads of that kind turned up on the account, plus another 20 for adult pornography. When the outlet first reported one of the ads through Instagram’s own tool, the review team wrote back a day later saying it didn’t violate community standards. Only after a push back did the company pull the ads, suspend the accounts, and call what happened a “horrific crime.” India’s IT ministry then summoned Meta to explain how any of it got approved in the first place.
A flawed review system
Meta has spent years telling the public that better automation and more advanced models will make its platforms safer. But a detection system only catches what it’s been built to catch. But here, it approved the material before anyone outside the company objected.
This same underlying design logic showed up in March in a U.S. courtroom. A jury found Meta and YouTube negligent in the case of a 20-year-old plaintiff, ruling that both platforms were built to maximise engagement among children with little regard for the consequences.
The jurors awarded $3 million in compensatory damages plus another $3 million in punitive damages, having concluded the companies acted with malice; Meta was assigned 70% of the blame, per media reports. It was the first verdict of its kind, and it came a day after another jury in the U.S., in New Mexico, separately ordered Meta to pay $375 million in civil penalties for enabling child sexual exploitation on its platforms.
Meta has said it is appealing the verdict as it contends that the complainant’s mental health struggles are tied to the content viewed and not the design of the platform.
That line of argument alone will take us back to what whistleblower Fances Haugen argued all along: these platforms were built on purpose to be addictive, tuned for time spent rather than user welfare. She noted that platforms like Instagram are taking refuge under Section 230 of the 1996 Communications Decency Act.
She argued that social media algorithms, and not the user-generated content itself, must be the focus of legal reform — particularly amending Section 230 — as algorithmic recommendation systems have the power to amplify certain set of posts that may never hit a user’s screen otherwise.
The same algorithmic design choice continues to flood a scrolling teenager’s feed, and it’s the same one that let an ad sexualising children slip past automated review and stay live.
Faster, newer AI moderation doesn’t touch the incentive sitting under all of it. AI can flag more, and flag it quicker. But it can’t fix a business built to keep people, kids included, engaged at any cost. Until that changes, the algorithms will keep chasing a mess their own business model keeps making.
Published – July 15, 2026 08:15 am IST