For his Ph.D. in astrophysics, Chris Farrell spent five years mining data from a giant particle accelerator. Now, he spends his days analyzing ratings for Yelp Inc. YELP +1.85% 's online business-review site.
Mr. Farrell, 28 years old, is a data scientist, a job title that barely existed three years ago but since has become one of the hottest corners of the high-tech labor market. Retailers, banks, heavy-equipment makers and matchmakers all want specialists to extract and interpret the explosion of data from Internet clicks, machines and smartphones, setting off a scramble to find or train them.
"People call them unicorns" because the combination of skills required is so rare, said Jonathan Goldman, who ran LinkedIn Corp.'s LNKD +2.07% data-science team that in 2007 developed the "People You May Know" button, which five years later drove more than half of the invitations on the professional-networking platform.
Employers say the ideal candidate must have more than traditional market-research skills: the ability to find patterns in millions of pieces of data streaming in from different sources, to infer from those patterns how customers behave and to write statistical models that pinpoint behavioral triggers.
At e-commerce site operator Etsy Inc., for instance, a biostatistics Ph.D. who spent years mining medical records for early signs of breast cancer now writes statistical models to figure out the terms people use when they search Etsy for a new fashion they saw on the street.
At mobile-payments startup Square Inc., a Ph.D. in cognitive psychology who wrote statistical models to examine how people change their political beliefs now looks for behavioral patterns that would identify which merchants are more likely to have clients demand their money back.
Another 28-year-old at Yelp, with a Ph.D. in applied mathematics, turned his dissertation research on genome mapping into a product used by the company's advertising team. The same genome-mapping algorithm is now used to measure the effect on consumers when multiple small changes are made to online ads.
"Academia is slow and only a few people see your work," said Scott Clark, who designed the genome-mapping algorithm. "At Yelp, I can be pushing out experiments that affect hundreds of millions of people. When I make a small change to the Yelp website, I have a bigger impact."
Some such experiments have raised alarms. Facebook Inc. FB +0.23% was recently in the spotlight for an experiment in which its data-science team sought to manipulate people's emotions by altering the content of their news feeds.
Christian Rudder, president of IAC/InterActiveCorp.'s OkCupid dating site, recently disclosed in a blog post that the site manipulated its feeds by inflating the likelihood that any two people were a match, to encourage them to use the service more.
Mr. Goldman, who now heads a new data-science group at Intuit Inc., INTU +0.53...See full post