Evaluation

Input	Output
This Roman portrait mummy <image 1> from the 1st century AD was discovered in Cairo, Egypt, and now belongs in the collection of Girton College, University of Cambridge. Which well-known fictional character does this woman share her name with? (A) Aurelia (B) Matilda (C) Hermione (D) Juno Answer with the option's letter from the given choices directly. No punctuation.	Google: Gemini Pro Vision 1.0 mmmu / ArtPublic C Expected Answer: C Difficulty: Easy Subfield: Drawing and Painting Explanation: This Roman portrait mummy from the Julio-Claudian dynasty is known as Hermione Grammatike and is part of the Egyptian collection in the Lawrence Room Collections at Girton College. She was discovered in the Roman cemetery at Hawara by the archaeologist and egyptologist Flinders Petrie (1853-1942). Image: Hermione Grammatike (detail from Fayum Portrait Mummy, Roman Period), AD 1-100; credit: Girton College, University of Cambridge

Input

Output

This Roman portrait mummy <image 1> from the 1st century AD was discovered in Cairo, Egypt, and now belongs in the collection of Girton College, University of Cambridge. Which well-known fictional character does this woman share her name with?

(A) Aurelia

(B) Matilda

(D) Juno

Answer with the option's letter from the given choices directly. No punctuation.

Google: Gemini Pro Vision 1.0

mmmu / ArtPublic

Expected Answer: C

Difficulty: Easy

Subfield: Drawing and Painting

Explanation: This Roman portrait mummy from the Julio-Claudian dynasty is known as Hermione Grammatike and is part of the Egyptian collection in the Lawrence Room Collections at Girton College. She was discovered in the Roman cemetery at Hawara by the archaeologist and egyptologist Flinders Petrie (1853-1942). Image: Hermione Grammatike (detail from Fayum Portrait Mummy, Roman Period), AD 1-100; credit: Girton College, University of Cambridge

Evaluation ID: c9aa4453-07ec-4638-993e-6f01d8fc046c
Created: February 21, 2024

Raw Output

{
  "id": "gen-1BGaCVKl9Sao0X1ZfuoTJLzDjuFG",
  "model": "google/gemini-pro-vision",
  "choices": [
    {
      "message": {
        "role": "assistant",
        "content": " C"
      },
      "finish_reason": "STOP"
    }
  ],
  "object": "chat.completion",
  "created": 1708556228,
  "usage": {
    "prompt_tokens": 1396,
    "completion_tokens": 4,
    "total_tokens": 1400
  }
}