Most AI evaluation focuses on individual models \u2014 one system, one benchmark. But AI systems increasingly operate alongside other agents and alongside us. How do you evaluate an AI that cooperates, competes, deceives, or negotiates? That’s a fundamentally different problem, and one that becomes more urgent as AI agents are deployed in the real world.<\/p>\n
Prof. Leibo pioneered this question at Google DeepMind, creating Melting Pot \u2014 a benchmark suite of 50+ environments and 256 test scenarios for evaluating social generalization in AI agents \u2014 and Concordia, a platform for simulating social interactions between language model-based agents. This talk explores how we evaluate AI not as isolated tools, but as social actors.<\/p>\n[\/vc_column_text][\/vc_column][\/vc_row][vc_row type=”in_container” full_screen_row_position=”middle” column_margin=”default” column_direction=”default” column_direction_tablet=”default” column_direction_phone=”default” scene_position=”center” text_color=”dark” text_align=”left” row_border_radius=”none” row_border_radius_applies=”bg” row_position_desktop=”default” row_position_tablet=”inherit” row_position_phone=”inherit” overflow=”visible” overlay_strength=”0.3″ gradient_direction=”left_to_right” shape_divider_position=”bottom” bg_image_animation=”none”][vc_column column_padding=”no-extra-padding” column_padding_tablet=”inherit” column_padding_phone=”inherit” column_padding_position=”all” flex_gap_desktop=”10px” column_element_direction_desktop=”default” column_element_spacing=”default” desktop_text_alignment=”default” tablet_text_alignment=”default” phone_text_alignment=”default” background_color_opacity=”1″ background_hover_color_opacity=”1″ column_backdrop_filter=”none” column_shadow=”none” column_border_radius=”none” column_link_target=”_self” column_position=”default” gradient_direction=”left_to_right” overlay_strength=”0.3″ width=”1\/2″ tablet_width_inherit=”default” animation_type=”default” bg_image_animation=”none” border_type=”simple” column_border_width=”none” column_border_style=”solid”][image_with_animation image_url=”9590″ image_size=”full” max_width=”100%” max_width_mobile=”default” animation_type=”entrance” animation=”None” animation_movement_type=”transform_y” hover_animation=”none” alignment=”” border_radius=”none” box_shadow=”none” image_loading=”default”][\/vc_column][vc_column column_padding=”no-extra-padding” column_padding_tablet=”inherit” column_padding_phone=”inherit” column_padding_position=”all” flex_gap_desktop=”10px” column_element_direction_desktop=”default” column_element_spacing=”default” desktop_text_alignment=”default” tablet_text_alignment=”default” phone_text_alignment=”default” background_color_opacity=”1″ background_hover_color_opacity=”1″ column_backdrop_filter=”none” column_shadow=”none” column_border_radius=”none” column_link_target=”_self” column_position=”default” gradient_direction=”left_to_right” overlay_strength=”0.3″ width=”1\/2″ tablet_width_inherit=”default” animation_type=”default” bg_image_animation=”none” border_type=”simple” column_border_width=”none” column_border_style=”solid”][vc_column_text css=”” text_direction=”default”]\n