Row 5377

Row ID: 5377 | Dataset Entry | Axioma AXP Content Repository

Content Data

This page contains data entry 5377 from the Axioma AXP content repository. The structured data below represents the complete record for this entry.

If both RoPE and ALiBi work under the assumption that we should assign increasingly lower scores the further apart two tokens are, wouldn't the score be so penalized at some point that even if there is an interesting fact 1 million tokens away, we couldn't retrieve it because the positional encoding would force it to have such a low score?

FieldValue
text If both RoPE and ALiBi work under the assumption that we should assign increasingly lower scores the further apart two tokens are, wouldn't the score be so penalized at some point that even if there is an interesting fact 1 million tokens away, we couldn't retrieve it because the positional encoding would force it to have such a low score?
label r/machinelearning
dataType post
communityName r/MachineLearning
datetime 2024-04-29
username_encoded Z0FBQUFBQm5LakwyeDg4VE5yRXM1MVZCcHp5ZEEteTZUbU50azhEeE00VnFzZE0tYmNucnpPVXpqNEo0ZjBZVGRBeXI3R3BHRlBGN280blpLX2I2S3FQbVg3NFdaczZxSlE9PQ==
url_encoded Z0FBQUFBQm5Lak9GamJiTWFHRnRTR3FtYWdENmxPVFRpc005M1Z4c0ZURUsyNU9GWmx5SmdtVWlmcGs2TzYtUmNaZVRLUS1faWhodVJ6MWJ4aEhud3dmTVFEUUlFd3VZYkV5VlBjQ0NRdHBFc0ZqWkhrbHVUZkc3ZWxMbGN6RFA2cmIyZkR6TFYycVlWVDBoTXF5alhubDlLYzJKdWxfekZWUi1XT0JWQ1l2XzBLTEplcXRGaGg3NVhtQlY4YUk0WkZKTEtnUnZ0YlpLQnpFOFZlSXZDNW5nOXhxWWhXcHk5dz09

Raw Record

{
  "text": "If both RoPE and ALiBi work under the assumption that we should assign increasingly lower scores the further apart two tokens are, wouldn't the score be so penalized at some point that even if there is an interesting fact 1 million tokens away, we couldn't retrieve it because the positional encoding would force it to have such a low score?",
  "label": "r/machinelearning",
  "dataType": "post",
  "communityName": "r/MachineLearning",
  "datetime": "2024-04-29",
  "username_encoded": "Z0FBQUFBQm5LakwyeDg4VE5yRXM1MVZCcHp5ZEEteTZUbU50azhEeE00VnFzZE0tYmNucnpPVXpqNEo0ZjBZVGRBeXI3R3BHRlBGN280blpLX2I2S3FQbVg3NFdaczZxSlE9PQ==",
  "url_encoded": "Z0FBQUFBQm5Lak9GamJiTWFHRnRTR3FtYWdENmxPVFRpc005M1Z4c0ZURUsyNU9GWmx5SmdtVWlmcGs2TzYtUmNaZVRLUS1faWhodVJ6MWJ4aEhud3dmTVFEUUlFd3VZYkV5VlBjQ0NRdHBFc0ZqWkhrbHVUZkc3ZWxMbGN6RFA2cmIyZkR6TFYycVlWVDBoTXF5alhubDlLYzJKdWxfekZWUi1XT0JWQ1l2XzBLTEplcXRGaGg3NVhtQlY4YUk0WkZKTEtnUnZ0YlpLQnpFOFZlSXZDNW5nOXhxWWhXcHk5dz09"
}
Explore Dataset Explore Row

Entry Information