ChatGPT works scarily well, but can we trust it?

OpenAI is nothing if not a master of grabbing headlines with its head-turning research on artificial intelligence. Products resulting from its research, such as AI Gym, MuseNet, and Dall-E, have always made an impact. The company’s latest – ChatGPT, released last week – has delivered yet again.

ChatGPT is a conversational AI model based on the Generative Pre-trained Transformer (GPT) family of AI models. If you’ve spent any time on Twitter over the past few days, you’ve seen countless screenshots of ChatGPT in action.

Created with Midjourney / Image credit: Sau Sheong Chang

How does it work? According to OpenAI, the model uses reinforcement learning from Human Feedback (RLHF), adopting the same methods as InstructGPT (an earlier sibling model), but with slight differences in the data collection setup. An initial model was trained using supervised fine-tuning by human AI trainers who provided conversations in which they played both sides: the human user and an AI assistant.

To create a reward model for reinforcement learning, OpenAI collected two or more model responses ranked by quality, from conversations that AI trainers had with the AI assistant. Using these responses, OpenAI fine-tuned the model using Proximal Policy Optimization.

How well does the final model work? Scarily well.

ChatGPT is built to interact in a conversational way, and it keeps track of the input received and its responses. Let’s start simply by speaking to it like you would to any normal person.

[Sau Sheong (SS)] What is your name?

[ChatGPT] I’m Assistant, a large language model trained by OpenAl. I’m not a person, I’m a program designed to answer questions to the best of my ability based on the information I’ve been trained on.

[SS] Who trained you?

[ChatGPT] I was trained by a team at OpenAl. OpenAl is a research institute that focuses on developing artificial intelligence in a responsible and safe way. The team at OpenAl trained me to be able to understand and generate human-like text, so that I can answer questions and have conversations with people like you.

[SS] How were you trained?

[ChatGPT] I was trained using a technique called deep learning, which involves training a large neural network on a large amount of data. In my case, the data used to train me consisted of many millions of examples of human-generated text, which allowed me to learn the patterns and structures of human language. This process took a lot of time and computational power, but it allows me to process and understand natural language with a high degree of accuracy.

A programmer and a poet?

Can we trust ChatGPT?