WebGPT

OpenAI trained a research version of GPT-3 that can search the web, synthesize information, and cite its sources to provide more accurate answers to questions. Language models like GPT-3 are useful for many different tasks, but have a tendency to “hallucinate” information when performing tasks requiring obscure real-world knowledge.23 To address this, OpenAI taught GPT-3 to use a text-based web-browser. The model is provided with an open-ended question and a summary of the browser state, and must issue commands such as “Search ...”, “Find in page: ...” or “Quote: …”. In this way, the model collects passages from web pages, and then uses these to compose an answer. The model is fine-tuned from GPT-3 using the same general methods they used previously. OpenAI improved the helpfulness and accuracy of the model’s answers, by training a reward model to predict human preferences, and optimizing against it using either reinforcement learning or rejection sampling. Read the full story at https://openai.com/blog/improving-factual-accuracy/