Row 23141
Content Data
This page contains data entry 23141 from the Axioma AXP content repository. The structured data below represents the complete record for this entry.
>GPT3 - added an auto regressive layer
GPT-1 and GPT-2 are both autoregressive.
>This was the last GPT release to come with a publication.
The InstructGPT paper from 2022 was notable.
>cherry picked examples to make it more “human.”
Instruction tuning, general supervised fine-tuning, and RLHF are not "cherry picked" trivialities. It's a fundamental change to the usefulness of LLMs and how people interact with them.
>Note: This is around the time Altman came back.
"text-davinci-002" and "code-davinci-002" were first made available in March 2022. ChatGPT was first made available in late November 2022. The whole debacle with Altman's removal happened in November 2023.
>Brockman quit.
Again in November 2023. And Brockman was an ally of Altman and returned the same day as Altman, so it's unclear what you're trying to say.
>GPT4 - used all the money from the Microsoft deal to buy more data to train ChatGPT
Triple wrong. GPT-4 finished pre-training before the release of ChatGPT. Most of the cost was compute, with the total cost being around 100 million. This is a year after Microsoft's 1 billion dollar investment and half a year before their 10 billion dollar investment.
>GPT 1 and 2 laid the ground work for generalized pretraining (Generalized Pretrained Transformer)
GPT does not stand for "Generalized Pretrained Transformer".
| Field | Value |
|---|---|
| text | >GPT3 - added an auto regressive layer GPT-1 and GPT-2 are both autoregressive. >This was the last GPT release to come with a publication. The InstructGPT paper from 2022 was notable. >cherry picked examples to make it more “human.” Instruction tuning, general supervised fine-tuning, and RLHF are not "cherry picked" trivialities. It's a fundamental change to the usefulness of LLMs and how people interact with them. >Note: This is around the time Altman came back. "text-davinci-002" and "c… |
| label | r/machinelearning |
| dataType | comment |
| communityName | r/MachineLearning |
| datetime | 2024-05-21 |
| username_encoded | Z0FBQUFBQm5Lak1CQVcwdmVrell5QVpMNV9vWlFIeWJIZmU1OC02M0hwRkNGdF85QmZHU1NHeE1mczhHNlF5bm5qd1ZRQ1NyRllTMC1nME4zd3FMcDAwQkVGRjUweTR2d3c9PQ== |
| url_encoded | Z0FBQUFBQm5Lak9RNlZfclpNTEZhTnhFdzJiU014RmU4UzQyRGlkOWpZVl9XcnRSOFlFMF9Wbnl1NTA5RE9BQS1jamhCRkZnaXY4OUxlZ1k1WnJxVE1fd1hWemU4dV83VmkxYkZmOTE3R2tFYWFBUG5iWlQ5TDkxZy14QzIySUhzS2FIaEZKNkxDd2plOHRMZE5Pa19KZFR4c2JkU1c4bUlBZnVTQVB5aU1DWklmcnhndVRyVkw3Tm5jZElqdG5WcGNjc0VaUy0tbmpIbDdyOHlZbWNkejlYaHBrQTc4bzY3dGFfRzNLMVRTQzVIUklFT1htQTIzWT0= |
Raw Record
{
"text": ">GPT3 - added an auto regressive layer\n\nGPT-1 and GPT-2 are both autoregressive.\n\n>This was the last GPT release to come with a publication.\n\nThe InstructGPT paper from 2022 was notable.\n\n>cherry picked examples to make it more “human.”\n\nInstruction tuning, general supervised fine-tuning, and RLHF are not \"cherry picked\" trivialities. It's a fundamental change to the usefulness of LLMs and how people interact with them.\n\n>Note: This is around the time Altman came back.\n\n\"text-davinci-002\" and \"code-davinci-002\" were first made available in March 2022. ChatGPT was first made available in late November 2022. The whole debacle with Altman's removal happened in November 2023.\n\n>Brockman quit.\n\nAgain in November 2023. And Brockman was an ally of Altman and returned the same day as Altman, so it's unclear what you're trying to say.\n\n>GPT4 - used all the money from the Microsoft deal to buy more data to train ChatGPT\n\nTriple wrong. GPT-4 finished pre-training before the release of ChatGPT. Most of the cost was compute, with the total cost being around 100 million. This is a year after Microsoft's 1 billion dollar investment and half a year before their 10 billion dollar investment.\n\n>GPT 1 and 2 laid the ground work for generalized pretraining (Generalized Pretrained Transformer)\n\nGPT does not stand for \"Generalized Pretrained Transformer\".",
"label": "r/machinelearning",
"dataType": "comment",
"communityName": "r/MachineLearning",
"datetime": "2024-05-21",
"username_encoded": "Z0FBQUFBQm5Lak1CQVcwdmVrell5QVpMNV9vWlFIeWJIZmU1OC02M0hwRkNGdF85QmZHU1NHeE1mczhHNlF5bm5qd1ZRQ1NyRllTMC1nME4zd3FMcDAwQkVGRjUweTR2d3c9PQ==",
"url_encoded": "Z0FBQUFBQm5Lak9RNlZfclpNTEZhTnhFdzJiU014RmU4UzQyRGlkOWpZVl9XcnRSOFlFMF9Wbnl1NTA5RE9BQS1jamhCRkZnaXY4OUxlZ1k1WnJxVE1fd1hWemU4dV83VmkxYkZmOTE3R2tFYWFBUG5iWlQ5TDkxZy14QzIySUhzS2FIaEZKNkxDd2plOHRMZE5Pa19KZFR4c2JkU1c4bUlBZnVTQVB5aU1DWklmcnhndVRyVkw3Tm5jZElqdG5WcGNjc0VaUy0tbmpIbDdyOHlZbWNkejlYaHBrQTc4bzY3dGFfRzNLMVRTQzVIUklFT1htQTIzWT0="
}
Entry Information
- Entry ID: 23141
- Repository: Axioma AXP
- Dataset: arrmlet/reddit_dataset_36
- Total Entries: 100,000