Login
Input
Output
Evaluation Input Image

You can see a grid of 9 boxes, one of which is empty (marked as ?). You


have to choose which of the 6 alternative shapes (A-F) to be placed in the


empty box in order to complete the pattern that connects the shapes.


Provide your prediction as a single letter. Your output should be a single letter!


OpenAI: GPT-4 Vision
Visual deductive reasoningPublicFail

E


Evaluation

Evaluation ID
f682adff-d1d7-45fa-810c-14c699bdbc25
Created
March 21, 2024

Evaluation Rating

Binary Pass/Fail
Fail
Raw Output
{
  "choices": [
    {
      "message": {
        "role": "assistant",
        "content": "E"
      },
      "finish_reason": "stop"
    }
  ],
  "model": "openai/gpt-4-vision-preview",
  "id": "gen-teG1qFUe7414Q3MyljO66MveKfcC",
  "object": "chat.completion",
  "created": 1711063271,
  "usage": {
    "prompt_tokens": 1183,
    "completion_tokens": 1,
    "total_tokens": 1184,
    "total_cost": 0.01186
  }
}
Raw Rating Output
{
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": null,
        "tool_calls": [
          {
            "id": "call_HPxJdtXTaOkjavto43dUqIqs",
            "type": "function",
            "function": {
              "name": "get_eval_rating",
              "arguments": "{\"binary_pass_fail\": false}"
            }
          }
        ]
      },
      "logprobs": null,
      "finish_reason": "tool_calls"
    }
  ],
  "model": "openai/gpt-4-turbo-preview",
  "id": "gen-RhU2LUQBuLxBgFtwfb9NVPVWJBRJ",
  "object": "chat.completion",
  "created": 1711063276,
  "usage": {
    "prompt_tokens": 199,
    "completion_tokens": 32,
    "total_tokens": 231,
    "total_cost": 0.00295
  }
}