The workings of ChatGPT, the latest natural language processing tool

The ChatGPT an artificial intelligence tool developed by OpenAI has created a buzz among computer scientists and programmers due to its creative capabilities.

About ChatGPT:

• ChatGPT is a ‘conversational’ AI and will answer queries just like a human would.
  • It can answer follow-up questions.
  • It can also “admit its mistakes
  • It can challenge incorrect premises and reject inappropriate requests.
  • It is being seen as a replacement for much of the daily mundane writing, from an email to even college-style essays.
• Developed by: OpenAI, a research, and development firm, was founded as a nonprofit in 2015.
• Language Used: It has used the GPT 3.5 series of language learning models (LLM).
  • GPT stands for Generative Pre-trained Transformer 3.
  • It relies on deep learning techniques to produce human-like text based on inputs.’

Why the ChatGPT is called a Language Model and not a chatbot?

• A language model is software that:
  • prints out a sequence of words as output that are related to some words given as input with appropriate semantic relation.
  • It is often used in natural language processing (NLP) applications, such as speech recognition, automatic translation, and text generation.
  • ChatGPT fits more appropriately in this definition and hence differs from chatbots.

ChatGPT as a neural network:

• Neural network: It can be considered as a large network of computers that can fine-tune its output of words based on the feedback given to it during stages of training.
  • ChatGPT has undergone this training process.
  • This training process and the technology together are called Reinforcement Learning.
  • All these technologies are part of artificial intelligence (also called Machine Learning).

How ChatGPT looks at words?

• “Word embedding”: It represents words as a matrix of numbers that can be manipulated inside computers.
  • The context in which a word has been used is differentiated by the neural network mentioned above. Example; shoot appears with the ‘gun’ or appears with the ‘camera’.
• “Transformer”: It is a refining technique that is later on deployed to help a neural network to accurately “understand” the context of a sentence or a paragraph.
  • The output “comprehension” can be used for multiple purposes like answering a question etc.

Development of ChatGPT:

Use of language models:

• ChatGPT follows a generation of language models that were released by OpenAI in 2018.
• In 2018, OpenAI released the Generative Pre-Training (GPT) language, model.
  • Here, generative means that it is a type of neural network that can create new content based on input content, called Training Data.
  • This technology makes it suitable for creative tasks like writing a new story.

Transformer technique:

• GPT was improved and “Generative Pre-trained Transformer 2” or GPT-2 was released in 2019. GPT-3 with even more sophisticated neural networks was launched in 2020.
• In early 2022, GPT3.5 was released and ChatGPT is the successor to GPT3.5.

Accuracy of ChatGPT:

• The accuracy of ChatGPT or any language model can be measured using standard techniques like:
  • ROUGE metric: It is Recall-Oriented Understudy for Gisting Evaluation”. It compares ChatGPT’s output of content against a standard expected content and measures the overlap as a success percentage.

Is ChatGPT the most powerful NLP tool?

• For general purposes, ChatGPT can be considered the most powerful for now.
• However, it may not be equally powerful in specialized contexts, like an automatic conversational assistant for medical queries.