CogArena benchmark tests multimodal agents on browser-based behavioral experiments

[arXiv] A preprint paper introduces CogArena, a benchmark of ten experimental paradigms from the social and behavioral sciences in which agents complete experiments through a standard web browser, using the same visual stimuli and response interfaces as online participants. A three-level scoring procedure evaluates whether the agent completed the task, how accurately it performed, and whether its data show the behavioral signatures each paradigm reliably produces in people. According to the paper, across four multimodal models, each run ten times per task, finishing an experiment and reproducing its signatures are different things: many sessions did not finish, and those that did reproduced some signatures reliably and others not at all. The paper is authored by Kianté Fernandez, Caitlin Chen, Jialu Zhou, Bartek Sadowski, Anthony C. Miceli and Ian Krajbich; it is a preprint and has not been peer-reviewed. CogArena provides the tasks, scoring code and a public leaderboard for new submissions, according to the authors.