Row 36151

Row ID: 36151 | Dataset Entry | Axioma AXP Content Repository

Content Data

This page contains data entry 36151 from the Axioma AXP content repository. The structured data below represents the complete record for this entry.

Isn't it true that the current models we have access to have been almost entirely trained on text only? Now that we're moving to multi-modality, it seems like there are a lot more tokens that can be sourced from non-text formats like audio and video and used as training data. A lot of leading AI researchers seem to agree that there is still significant headroom for current model improvement given more training data and compute. Of course, architectural improvements like the theorized move from strict next token prediction to an "energy-based" model for response generation will come, and they will push the tech forward. But from what I've seen, I wouldn't bet against more significant advancements coming before the paradigm behind model creation fundamentally changes.

FieldValue
text Isn't it true that the current models we have access to have been almost entirely trained on text only? Now that we're moving to multi-modality, it seems like there are a lot more tokens that can be sourced from non-text formats like audio and video and used as training data. A lot of leading AI researchers seem to agree that there is still significant headroom for current model improvement given more training data and compute. Of course, architectural improvements like the theorized move from s…
label r/openai
dataType comment
communityName r/OpenAI
datetime 2024-05-21
username_encoded Z0FBQUFBQm5Lak1KV1E3OEJfeHBrY0RQRThLVkU2eFR6YWlTSElUMV9NQm9zbVZiU2pYcEFPSmtJa2dhTzBWNHAyVUxYQk1aVXR1b0xGVURJNVZtekFQS0twVm4xRzZQb1E9PQ==
url_encoded Z0FBQUFBQm5Lak9ZYmw0ZDVDR1lLRTc1T1ZENV9xdVJPNkZzM0tLaWxSZ2QzZ3JYTlppQ3MyT3V6b0lWaDU2aG5pRjZodnVzNGxOYmp2T244bXJjQWtZMmswTUVCNzBXbHE5NnB3M01GMDZjd09aU3NPaTNjbmNvM21HUnZZX0prVHNmUk45SnhwS29KTEFlYVZwZ3N3MHBVRmtXMXlVTFdjR3hvSGhiRVhyUnpkQTNaWWgxVHA5S3M5T25hWlRrXzYxLTV0QTVMajRr

Raw Record

{
  "text": "Isn't it true that the current models we have access to have been almost entirely trained on text only? Now that we're moving to multi-modality, it seems like there are a lot more tokens that can be sourced from non-text formats like audio and video and used as training data. A lot of leading AI researchers seem to agree that there is still significant headroom for current model improvement given more training data and compute. Of course, architectural improvements like the theorized move from strict next token prediction to an \"energy-based\" model for response generation will come, and they will push the tech forward. But from what I've seen, I wouldn't bet against more significant advancements coming before the paradigm behind model creation fundamentally changes.",
  "label": "r/openai",
  "dataType": "comment",
  "communityName": "r/OpenAI",
  "datetime": "2024-05-21",
  "username_encoded": "Z0FBQUFBQm5Lak1KV1E3OEJfeHBrY0RQRThLVkU2eFR6YWlTSElUMV9NQm9zbVZiU2pYcEFPSmtJa2dhTzBWNHAyVUxYQk1aVXR1b0xGVURJNVZtekFQS0twVm4xRzZQb1E9PQ==",
  "url_encoded": "Z0FBQUFBQm5Lak9ZYmw0ZDVDR1lLRTc1T1ZENV9xdVJPNkZzM0tLaWxSZ2QzZ3JYTlppQ3MyT3V6b0lWaDU2aG5pRjZodnVzNGxOYmp2T244bXJjQWtZMmswTUVCNzBXbHE5NnB3M01GMDZjd09aU3NPaTNjbmNvM21HUnZZX0prVHNmUk45SnhwS29KTEFlYVZwZ3N3MHBVRmtXMXlVTFdjR3hvSGhiRVhyUnpkQTNaWWgxVHA5S3M5T25hWlRrXzYxLTV0QTVMajRr"
}
Explore Dataset Explore Row

Entry Information