Login
Input
Output
Evaluation Input Image

True or False: The name of <image 1> is 'This is not a Pipe' by René Magritte



(A) True


(B) False


(C) No enough evidence


Answer with the option's letter from the given choices directly. No punctuation.


Google: Gemini Pro Vision 1.0
mmmu / ArtPublic

A


Expected Answer: B

Difficulty: Easy

Subfield: Drawing and Painting


Explanation: False: The title of this painting is 'The Treachery of Images' by René Magritte 1928-1929. On the painting, in French, it says 'This is not a pipe'

Evaluation

Evaluation ID
7014d833-b900-49e1-9bee-c32ec539c7c5
Created
February 21, 2024
Raw Output
{
  "id": "gen-MNwwwOAjlrxRdMrzCxSJ2xJHgCda",
  "model": "google/gemini-pro-vision",
  "choices": [
    {
      "message": {
        "role": "assistant",
        "content": " A"
      },
      "finish_reason": "STOP"
    }
  ],
  "object": "chat.completion",
  "created": 1708556439,
  "usage": {
    "prompt_tokens": 1268,
    "completion_tokens": 4,
    "total_tokens": 1272
  }
}