OpenAI released ChatGPT on 30 November 2022 as a free research preview. The conversational system lets people ask questions and continue with follow-ups, while the company gathers feedback on how it behaves.
OpenAI says ChatGPT can acknowledge mistakes and challenge incorrect assumptions. The model is fine-tuned from GPT-3.5, with human trainers supplying conversations and ranking alternative responses to help shape its behaviour.
How human feedback shapes replies
OpenAI described that training approach in its January 2022 InstructGPT research. People first wrote examples of desired answers and compared model responses. Their rankings trained a reward model, which then guided further training. The aim was to make a language model follow a person’s instructions more closely, rather than simply continue a passage of text.
The preview remains experimental. OpenAI warns that replies can sound convincing while being incorrect, and that rephrasing a question can change the answer. Feedback can be submitted through the chat interface.
Illustration: Sulat. Logo: OpenAI / AREA 17.
Discover more from Sulat AI & Tech News
Subscribe to get the latest posts sent to your email.
