Free website looks at user reviews to determine which apps are a danger to children

The New York Times today reported on a website called The App Danger Project that was created by a computer scientist named Brian Levine who is at the University of Massachusetts Amherst. The site uses AI to scan reviews of social networking sites on the App Store and Google Play Store to evaluate the context of certain words and phrases used in these reviews including "child porn" or "pedo." Levine says, "There are reviews out there that talk about the type of dangerous behavior that occurs, but those reviews are drowned out. You can't find them."

The website uses a Machine Learning algorithm to find apps that are flagged as being possibly dangerous to children because of the nature of the user reviews. The site is free and while Levine isn't looking to monetize it for personal profit, he does seek donations to the University of Massachusetts to cover the cost of running the project.

Levine and 12 other computer scientists did some digging and discovered that out of the 550 social networking apps offered by the App Store and the Google Play Store, 20% of them had two or more complaints in their reviews about content that was characterized as "child sexual abuse material." A whopping 81 apps had seven or more such complaints. Levine says that Apple and Google need to do a better job of giving parents information about such apps and policing their app storefronts to kick out such titles.

As you probably know by now, Apple and Google take as much as 30% of the value of an in-app transaction. The Times, citing data from app analytics firm Sensor Tower, says that last year the two major app storefronts helped three apps generate $30 million in sales even though the trio had multiple user reports of sexual abuse. The apps were: Hoop, MeetMe, and Whisper.

The Justice Department, in more than a dozen criminal cases in various states, described the three apps as "tools" used by subscribers to ask children to send them sexual images or to meet with them. And some apps that are a danger to children remain in the App Store and Google Play Store according to a computer scientist named Hany Farid who worked with Mr. Levine on the App Danger Project.

Farid says, "We’re not saying that every app with reviews that say child predators are on it should get kicked off, but if they have the technology to check this, why are some of these problematic apps still in the stores?" A Google spokesman says that the company investigated the apps listed by the App Danger Project and found no evidence of child sexual abuse material. The Google spokesman said, "While user reviews do play an important role as a signal to trigger further investigation, allegations from reviews are not reliable enough on their own."

Apple also investigated the App Store apps listed by the website and ended up kicking out 10 apps although it won't reveal their names. "Our App Review team works 24/7 to carefully review every new app and app update to ensure it meets Apple’s standards," said a company spokesman.

Hoop, one of the apps on the App Danger Project list as being a danger to kids, had 176 reviews out of the 32,000 posted since 2019 mention sexual abuse. One such review pulled from the App Store said, "There is an abundance of sexual predators on here who spam people with links to join dating sites, as well as people named 'Read my picture.' It has a picture of a little child and says to go to their site for child porn."

The app, now under new management, says it has a new content moderation system making the app safer. Liath Ariche, Hoop’s chief executive, noted that the app has learned how to deal with bots and malicious users. "The situation has drastically improved," he says. MeetMe parent The Meet Group told the Times that it doesn't tolerate abuse or exploitation of minors and Whisper did not respond to requests for comment.

It should be noted that apps like WhatsApp and Snapchat are also on the App Danger Project website, both listed as being "unsafe for children." One could argue that the App Danger Project is being too sensitive, but when it comes to children many would answer that zero tolerance is the only way to protect the children.