Integrative, HIGH-THROUGHPUT EXPERIMENTS

The traditional lab-experiment methodology has become widely criticized for failing to replicate and generalize beyond the narrow conditions of the study. Although these criticisms are widely regarded as rooted in methodological or statistical shortcomings, I believe the problem is more fundamental. Within the existing experimental paradigm, each experiment is designed to test a single theoretically motivated hypothesis, and the choice of which variables to manipulate (and hold fixed) is largely up to the experiment designer. In this project, we explore an alternative approach, “integrative” and “high throughput” experimentation, wherein researchers would run experimental conditions that systematically cover the complete parameter space—the union of existing theories—associated with a given experimental design.

Tools: Empirica (Virtual Lab Platform).

Scientific writings:

• Almaatouq, A., Griffiths, T. L., Suchow, J., Whiting, M. E., Evans, J., & Watts, D. J. (2024). Beyond Playing 20 Questions with Nature: Integrative Experiment Design in the Social and Behavioral Sciences. Behavioral and Brain Sciences.

• Almaatouq, A., Becker, J., Houghton, J. P., Paton, N., Watts, D. J., & Whiting, M. E. (2021). Empirica: a virtual lab for high-throughput macro-level experiments. Behavior Research Methods, 1-14.

• Almaatouq, A., Becker, J., Bernstein, M. S., Botto, R., Bradlow, E. T., Damer, E., Duckworth, A., Griffiths, T., Hartshorne J. K., Lazer D., Law, E., Liu, M., Matias, J. N., Rand, D., Salganik, M., Satlof-Bedrick, E., Schweitzer, M., Shirado, H., Suchow, J. W.; Suri, S., Tsvetkova, M., Watts, D. J., Whiting, M. E., & Yin., M. (2021). Scaling up experimental social, behavioral, and economic science. Technical Report.

IN SEARCH OF SYNERGY IN THE TASK SPACE

Are two heads better than one, or do too many cooks spoil the broth? I argue that one of the keys to solving the puzzle is to better understand the underlying nature of the tasks being performed. But because no clear language exists to describe tasks in a way that allows for straightforward comparisons across studies, the role of task characteristics remains poorly understood. This project is about developing a comprehensive, empirically grounded theory of group tasks.

Scientific writings:

• Almaatouq, A., Alsobay, M., Yin, M., & Watts, D. J. (2021). Task complexity moderates group synergy. Proceedings of the National Academy of Sciences, 118(36)

  • [Selected Media: Press Release, Quartz (Op-ed), World Economic Forum]

• Almaatouq, A., Alsobay, M., Yin, M., & Watts, D. J. (2024). The effects of group composition and dynamics on collective performance. Topics in cognitive science, 16(2), 302-321.

COLLABORATIVE INTELLIGENCE: HUMANS AND AI JOINING FORCES

Many people have already speculated about the possible rise of superintelligent computers. But few people have thought seriously about another -- we believe much more likely -- possibility: the creation of superintelligent groups that may include people, computers, and/or other kinds of individuals. That is the focus of this project.

Scientific writings:

• Vaccaro, M., Almaatouq, A., & Malone, T. (2024). When combinations of humans and AI are useful: A systematic review and meta-analysis. Nature Human Behaviour, 1-11.

• Ahn, D., Almaatouq, A., Gulabani, M., & Hosanagar, K. (2024). Impact of Model Interpretability and Outcome Feedback on Trust in AI. In Proceedings of the CHI Conference on Human Factors in Computing Systems (pp. 1-25).

• Burton, J. W., Hahn, U., Almaatouq, A., & Rahimian, M. A. (2024). Algorithmically Mediating Communication to Enhance Collective Decision-Making in Online Social Networks. ACM Collective Intelligence.

• Campero, A., Vaccaro, M., Song, J., Wen, H., Almaatouq, A., & Malone, T. W. (2022). A Test for Evaluating Performance in Human-Computer Systems. arXiv preprint arXiv:2206.12390.

• Burton, J. W., Almaatouq, A., Rahimian, M. A., & Hahn, U. (2021). Rewiring the Wisdom of the Crowd. In Proceedings of the Annual Meeting of the Cognitive Science Society (Vol. 43, No. 43).

• Noriega, A., Meizner, D., Camacho, D., Enciso, J., Quiroz-Mercado, H., Morales-Canton, V., ... & Pentland, A. (2021). Screening Diabetic Retinopathy Using an Automated Retinal Image Analysis System in Independent and Assistive Use Cases in Mexico: Randomized Controlled Trial. JMIR Formative Research, 5(8), e25290.

COLLECTIVE INTELLIGENCE UNDER AN ENVIRONMENT-DEPENDENT FRAMEWORK

Numerical estimation is one common form of tasks critical to many kinds of decisions. One simple but effective strategy to improve the accuracy of numeric estimates is to use the aggregate of multiple estimates, taking advantage of a statistical phenomenon popularly known as the "wisdom of crowds." In this project, we explore the conditions under which groups exhibit "crowd wisdom" (versus "crowd madness") and focus on the role of social influence and features of the tasks being performed.

Scientific writings:

• Almaatouq, A., Rahimian, M. A., Burton, J. W., & Alhajri, A. (2022). The distribution of initial estimates moderates the effect of social influence on the wisdom of the crowd. Scientific reports, 12(1), 1-8.

• Becker, J., Almaatouq, A., & Horvát, E. Á. (2020). Network structures of collective intelligence: The contingent benefits of group discussion. arXiv preprint arXiv:2009.07202.

• Almaatouq, A., Noriega-Campero, A., Alotaibi, A., Krafft, P. M., Moussaid, M., & Pentland, A. (2020). Adaptive social networks promote the wisdom of crowds. Proceedings of the National Academy of Sciences, 117(21), 11379-11386.

  • [Selected Media: Press Release, MIT-SMR (Op-ed), Advanced Science News]

• Almaatouq, A. M. (2019). Towards stable principles of collective intelligence under an environment-dependent framework (Doctoral dissertation, Massachusetts Institute of Technology).

• Moussaïd, M., Noriega Campero, A., & Almaatouq, A. (2018). Dynamical networks of influence in small group discussions. PLOS ONE, 13(1), e0190541.

Fragile Families Challenge

The Challenge is a Kaggle-like competition based on the Fragile Families and Child Wellbeing Study, which followed thousands of American families for more than 15 years, collecting information about the children, their parents, their schools, and their overall environments. 

As participants in the Challenge, we were asked to use the background data from birth to age nine (approximately 12,000 features), and known outcomes at age 15 for a small portion of the children as training data, to predict outcomes in the following six key categories: (1) Grade point average (academic achievement) of the children; (2) Grit (passion and perseverance) of the children; (3) Material hardship of the household (a measure of extreme poverty); (4) Eviction of the families (for not paying the rent or mortgage); (5) Layoff of the caregiver; (6) Job training (if the primary caregiver would participate in a job skills program).

More than 150 teams from around the world submitted over 3,000 predictive models. Our submission was ranked first in predicting GPA, grit, and layoffs, and was ranked 3rd for job training, 8th for material hardship, and 11th for eviction.

Our team consisted of myself, Eaman Jahani, Daniel Rigobon, Yoshihiko Suhara, Khaled Al-Ghoneim, and Abdulaziz Alghunaim.

Scientific writings:

• Salganik, M. J., Lundberg, I., Kindel, A. T., Ahearn, C. E., Al-Ghoneim, K., Almaatouq, A., ... & McLanahan, S. (2020). Measuring the predictability of life outcomes with a scientific mass collaboration. Proceedings of the National Academy of Sciences, 117(15), 8398-8403.

  • [Media: MIT Technology Review, phys.org]

• Rigobon, D. E., Jahani, E., Suhara, Y., AlGhoneim, K., Alghunaim, A., Pentland, A. S., & Almaatouq, A. (2019). Winning Models for Grade Point Average, Grit, and Layoff in the Fragile Families Challenge. Socius, 5, 2378023118820418.

  • [Media: MIT News]

Friendship Reciprocity & behavioral change

As a fan of all things Tolkien, I thought it would be appropriate to begin the description of this project with one of my favorite quotes from the realms of Middle Earth.

The hobbits at Bilbo's farewell party found this unexpected statement rather difficult. As Tolkien explains, there was some scattered clapping, but most of the assembled party-goers were trying to work it out and see if it came to a compliment. The difficulty of understanding the statement stems from the linguistic style and language Bilbo used in his speech. But why were those who did understand it disappointed?

I guess for hobbits, just like us, reciprocity is one of the expectations of affectionate relationships. For instance, we assume that when we consider another person a “friend,” that person also thinks of us as a friend. I mean, we like them, they must like us, right? 

In this project, we analyzed self-reported relationship surveys from several experiments around the world (from human subjects, not hobbits!), and found that while most people assume friendships to be two-way, only about half of the friendships are indeed reciprocal. In itself, this may seem like an interesting but minor finding, but this large proportion of asymmetric friendships translates to a major effect on the ability of individuals to persuade others to cooperate or change their behavior.

Scientific writings:

• Almaatouq, A., Radaelli, L., Pentland, A. and Shmueli, E., 2016. Are You Your Friends’ Friend? Poor Perception of Friendship Ties Limits the Ability to Promote Behavioral Change. PLOS ONE, 11(3), p.e0151588.

  • [Media: The New York Times (2016, 2018), Psychology Today (part 1, part 2, part 3), Business Insider, Phys.org, The Independent, The Guardian]

• Almaatouq, A., Radaelli, L., Pentland, A. and Shmueli, E., 2016 The Role of Reciprocity and Directionality of Friendship Ties in Promoting Behavioral Change Social, Cultural, and Behavioral Modeling, pp.33-41

Social Physics of Unemployment

Much of our knowledge about how mobility, social networks, communication, and education affect the economic status of individuals and cities has been obtained through complex and costly surveys, with an update rate ranging from fortnights to decades. However, recent studies have shown the value of mobile phone data as an enabling methodology for demographic modeling and measurement.

Many of our daily routines are driven by activities either afforded by our economic status or related to maintaining or improving it, from our movements around the city, to our daily schedules, to our communication with others. As such, we expect to be able to measure passive patterns and behavioral indicators, using mobile phone data, that could describe local unemployment rates. 

To investigate this question, we examined anonymized mobile phone metadata combined with beneficiaries’ records from an unemployment benefit program. We found that aggregated activity, social, and mobility patterns strongly correlate with unemployment. Furthermore, we constructed a simple model to produce accurate reconstructions of district-level unemployment from mobile communication patterns alone.

Scientific writings:

• Almaatouq, A., Prieto-Castrillo, F., & Pentland, A. (2016, November). Mobile communication signatures of unemployment. In International conference on social informatics (pp. 407-418). Springer, Cham.

• Almaatouq, A. M. (2016). Complex systems and a computational social science perspective on the labor market (Master’s dissertation, Massachusetts Institute of Technology).

THE ECOSYSTEM of SOCIAL SPAM

Spam in Online Social Networks (OSNs) is a systemic problem that imposes a threat to these services in terms of undermining their value to advertisers and potential investors, as well as negatively affecting users’ engagement. While the well-studied email spam is (almost!) a solved problem, OSNs spam is a very different and interesting problem. 

In this work, we compare normal and malicious users on Twitter in terms of their behavioral properties. We find that there exist two behaviorally distinct categories of spammers, just like viruses: 1) naive, short-lived & aggressive; 2) sophisticated, stealthy that embeds itself first. We then analyze the detectability of these spam accounts with respect to three categories of features, namely, content attributes (linguistic cue ), social interactions (dimensions of information diffusion patterns ), and profile properties (metadata related to the account). Our biggest finding was that that malicious accounts can easily generate chatter that is indistinguishable from benign (human) users, while it is much harder for these malicious bots to mimic the social interactions of human users. 

Scientific writings:

• Almaatouq, A., Shmueli, E., Nouh, M., Alabdulkareem, A., Singh, V.K., Alsaleh, M., Alarifi, A. and Alfaris, A., 2016. If it looks like a spammer and behaves like a spammer, it must be a spammer: analysis and detection of microblogging spam accounts. International Journal of Information Security, pp.1-17.

• Almaatouq, A., Alabdulkareem, A., Nouh, M., Shmueli, E., Alsaleh, M., Singh, V.K., Alarifi, A., Alfaris, A. and Pentland, A.S., 2014, June. Twitter: who gets caught? observed trends in social micro-blogging spam. In Proceedings of the 2014 ACM conference on Web science (pp. 33-41). ACM.