Google Cloud launches Physical-Engineering's Last Exam benchmark; all frontier models score below 50%

[Google Cloud] Google's AI & Infrastructure team said on Nov. 3 that it has launched Physical-Engineering's Last Exam (PLE), a benchmark and open competition developed with Mercor to test frontier models on mechanical, electrical, civil and structural engineering tasks. According to Google Cloud, it released 29 public tasks, grading tools and a leaderboard. PLE comprises 85 long-horizon tasks: 30 mechanical, 27 electrical, 18 civil and 10 structural, with each task averaging 61 files and 5,170 files in total, including PDFs, spreadsheets, emails, 2D CAD drawings, 3D BIM models and document revisions. Google Cloud said 44 tasks were expert-authored by practicing engineers, while 41 were designed by engineers and assembled by an agent pipeline, then validated and made harder by engineers. The team tested 14 frontier models using seven public agent harnesses. According to Google Cloud, all frontier models scored below 50% on the private test set, with the top model on its native harness reaching a mean score of 48.3%; on co-authored tasks, the highest-scoring agent configuration satisfied 91% of individual constraints, but only 41.3% of its deliverables passed all hard safety checks.