TechDigits

Tech news
Tuesday, Mar 19, 2024

A teenager on Tiktok disrupted thousands of scientific studies with a single video

Researchers were caught by surprise after a short video sent a flood of new users to a survey platform. Thousands of scientific studies had to toss out weeks of data because of a 56-second TikTok video by a teenager.

The July 23rd video is short and simple. It opens with recent Florida high school graduate and self-described “teen author” Sarah Frank sitting in her bedroom and smiling at the camera.

“Welcome to side hustles I recommend trying — part one,” she says in the video, pointing users to the website Prolific.co. “Basically, it’s a bunch of surveys for different amounts of money and different amounts of time.”


That video got 4.1 million views in the month after it was posted and sent tens of thousands of new users flooding to the Prolific platform. Prolific, a tool for scientists conducting behavioral research, had no screening tools in place to make sure that it delivered representative population samples to each study. Suddenly, scientists used to getting a wide mix of subjects for their Prolific studies saw their surveys flooded with responses from young women around Frank’s age.

For researchers who rely on representative samples of the US population, that demographic shift was a major problem with no obvious cause and no immediately clear way to fix.

Doing science on the internet


Though not particularly well known, Prolific is part of a small collection of online tools that have transformed the way corporations and scientists study the way people think and act. The first and largest of these research platforms is Amazon-owned Mechanical Turk, which was released in 2005 as a general-purpose platform for crowdsourcing work on repetitive tasks. Soon after it was released, behavioral scientists realized its potential value for their research, and it quickly revolutionized several research fields.

“Before Mechanical Turk existed, all social science research had to happen in the laboratory. You’d need to bring in college sophomores and put them through questionnaires and surveys and whatnot,” said Nicholas Hall, director of the Behavioral Lab at the Stanford School of Business.

“That’s an enormous time- and labor-intensive endeavor. Online research makes it so much easier. You program a survey... you put it online, and within a day, you have 1,000 responses,” Hall told The Verge. “That changed the face of social science.”

The Behavioral Lab at Stanford mainly uses the newer, smaller Prolific platform for online studies these days, Hall said. While many Mechanical Turk customers are big businesses conducting corporate research, Prolific gears its product to scientists.

The smaller platform offers more transparency, promises to treat survey participants more ethically, and promises higher-quality research subjects than alternative platforms like Mechanical Turk.

Scientists doing this sort of research in the United States generally want a pool of subjects who speak English as a first language, are not too practiced at taking psychological surveys, and together make up a reasonably representative demographic sample of the American population.

Prolific, most agreed, did a good job providing high-quality subjects. The sudden change in the platform’s demographics threatened to upend that reputation.

TikTok aftermath


In the days and weeks after Frank posted her video, researchers scrambled to figure out what was happening to their studies.


A member of the Stanford Behavioral Laboratory posted on a Prolific forum, “We have noticed a huge leap in the number of participants on the platform in the US Pool, from 40k to 80k. Which is great, however, now a lot of our studies have a gender skew where maybe 85% of participants are women. Plus the age has been averaging around 21.”

Wayne State psychologist Hannah Schechter seems to have been the first person to crack the case.

“This may be far-fetched,” she tweeted, linking to Frank’s video, “but given the timing, virality of the video, and the user’s follower demographics....”

Long-standing Prolific survey-takers complained on Reddit that Frank had made it difficult to find paid surveys to take on the overrun platform.

“Now it’s just another bullshit site to spend hours and make pennies on,” wrote one user, who said they had previously made $30 a week on the platform.

Frank, who “guesstimated” she had made a total of about $80 taking surveys on Prolific before her video, told The Verge she also noticed a difference on the platform.

“Less studies have been available for me and everyone else,” she told The Verge. “I’ve received some really mean comments accusing me of single-handedly ruining the site and being selfish — even though I received no compensation for that video.”

She added that she hoped Prolific would be able to set up a system to deal with its changed demographics.

“I also predict that a lot of people who signed up after seeing my video will forget about it, and the surge will die down,” she said.

Prolific co-founder and CTO Phelim Bradley told The Verge that many of the new users do seem to be dropping off.

“Prior to Tiktok, about 50% of the responses on our platform came from women,” he wrote in an email. “The surge knocked this up as high as 75% for a few days, but since then, this number has been trending down, and we’re currently back to ~60% of responses being from women.”

A chart showing the increase in female users over time.


According to Bradley, about 4,600 studies were disrupted by Frank’s TikTok, around a third of the total that were active on the platform during the surge. Of those, he said, the vast majority should be salvageable.

Prolific has refunded researchers whose studies were significantly impacted by the surge in women survey takers and introduced a new suite of demographic screening tools. The company announced these steps a month after Frank posted her video. The company has now re-organized, putting a team in charge of demographic balancing in order to more quickly recognize and respond to this sort of problem in the future.

“Honestly, we were somewhat caught by surprise, and we didn’t predict how large the impact was going to be,” Bradley said.

The surge isn’t all bad. Refreshing the pool of survey takers probably has long-term benefits, says Vlad Chituk, a Yale graduate student in psychology who was running several pilot studies on Prolific when the surge hit. When subjects take lots of psychological surveys, they learn the tricks scientists use to gather data, and that can impact the way they respond to future survey questions. Fresh subjects provide higher-quality data.

“Young women who enjoy TikTok are people too,” he said.

As for Frank, she says that her side hustle video is now the most popular TikTok she’s ever posted.

“It definitely didn’t occur to me that the video would blow up. I just posted it for my friends and followers, not for the reach it ended up getting,” she said. “I think it blew up because the site is genuinely so cool, and people love efficient ways to make money.”

For the time being, Frank has paused most of her own side hustles as she settles into her freshman year at Brown.

Newsletter

Related Articles

TechDigits
0:00
0:00
Close
FTX's Bankman-Fried headed for jail after judge revokes bail
America's First New Nuclear Reactor in Nearly Seven Years Begins Operations
Southeast Asia moves closer to economic unity with new regional payments system
Today Hunter Biden’s best friend and business associate, Devon Archer, testified that Joe Biden met in Georgetown with Russian Moscow Mayor's Wife Yelena Baturina who later paid Hunter Biden $3.5 million in so called “consulting fees”
Google testing journalism AI. We are doing it already 2 years, and without Google biased propoganda and manipulated censorship
Musk announces Twitter name and logo change to X.com
The future of sports
TikTok Takes On Spotify And Apple, Launches Own Music Service
Hacktivist Collective Anonymous Launches 'Project Disclosure' to Unearth Information on UFOs and ETIs
Typo sends millions of US military emails to Russian ally Mali
Server Arrested For Theft After Refusing To Pay A Table's $100 Restaurant Bill When They Dined & Dashed
Democracy not: EU's Digital Commissioner Considers Shutting Down Social Media Platforms Amid Social Unrest
Sarah Silverman and Renowned Authors Lodge Copyright Infringement Case Against OpenAI and Meta
Why Do Tech Executives Support Kennedy Jr.?
The New York Times Announces Closure of its Sports Section in Favor of The Athletic
Florida Attorney General requests Meta CEO's testimony on company's platforms' alleged facilitation of illicit activities
The Poor Man With Money, Mark Zuckerberg, Unveils Twitter Replica with Heavy-Handed Censorship: A New Low in Innovation?
The Double-Edged Sword of AI: AI is linked to layoffs in industry that created it
US Sanctions on China's Chip Industry Backfire, Prompting Self-Inflicted Blowback
Meta Copy Twitter with New App, Threads
BlackRock Bitcoin ETF Application Refiled, Naming Coinbase as ‘Surveillance-Sharing’ Partner
UK Crypto and Stablecoin Regulations Become Law as Royal Assent is Granted
A Delaware city wants to let businesses vote in its elections
Alef Aeronautics Achieves Historic Milestone with Flight Certification for World's First Flying Car
Google Blocked Access to Canadian News in Response to New Legislation
French Politicians Advocate for Pan-European Regulation on Social Media Influencers
Melinda French Gates Advocates for Increased Female Representation in AI to Prevent Bias
Snapchat+ gains 4 million paying subscribers in its first year
Apple Makes History as the First Public Company Valued at $3 Trillion
Elon Musk Implements Twitter Limits to Tackle Data Scraping, but Faces Criticism for Technical Misunderstanding
EU and UK's Slow Electric Vehicle Adoption Raises Questions About the Transition to Green Mobility
Top Companies Express Concerns Over Europe's Proposed AI Law, Citing Competitiveness and Investment Risks
Meta Unveils Insights on AI Usage in Facebook and Instagram, Amid Growing Calls for Transparency
Crypto Scams Against Seniors Soar by 78% in 2022, Experts Urge Vigilance
The End of an Era: National Geographic Dismisses Last of Its Staff Writers
Shield Your Wallet: The Perils of Wireless Credit Card Theft
Harvard Scientist Who Studies Honesty Accused Of Data Fraud, Put On Leave
Putting an End to the Subscription Snare: The Battle Against Unwitting Commitments
The Legal Perils of AI: Lawyer Faces Sanctions for Relying on Fictional Cases Generated by Chatbot
ChatGPT’s "Grandma Exploit": Ingenious Hack Exposes Loophole in AI, Generates Free Software Codes
The Disney Downturn: A Near Billion-Dollar Box Office Blow for the House of Mouse
A Digital Showdown: Canada Challenges Tech Giants with The Online News Act, Meta Strikes Back
Distress in the Depths: Submersible and Passengers Missing in Titanic Wreckage Expedition
Mark Zuckerberg stealing another idea: Twitter
European Union's AI Regulations Risk Self-Sabotage, Cautions smart and brave Venture Capitalist Joe Lonsdale
Nvidia GPUs are so hard to get that rich venture capitalists are buying them for the startups they invest in
Chinese car exports surge
Reddit Blackout: Thousands of Communities Protest "Ludicrous" Pricing Changes
Nvidia Joins Tech Giants as First Chipmaker to Reach $1 Trillion Valuation
AI ‘extinction’ should be same priority as nuclear war – experts
×