Context

The ChatGPT an artificial intelligence tool developed by OpenAI has created a buzz among computer scientists and programmers due to its creative capabilities.z

About

About ChatGPT:

• ChatGPT is a ‘conversational’ AI and will answer queries just like a human would.
• It can answer follow-up questions.
• It can also “admit its mistakes
• It can challenge incorrect premises and reject inappropriate requests.
• It is being seen as a replacement for much of the daily mundane writing, from an email to even college-style essays.
• Developed by: OpenAI, a research, and development firm, was founded as a nonprofit in 2015.
• Language Used: It has used the GPT 3.5 series of language learning models (LLM).
• GPT stands for Generative Pre-trained Transformer 3.
• It relies on deep learning techniques to produce human-like text based on inputs.’

Why the ChatGPT is called a Language Model and not a chatbot?

• A language model is software that:
• prints out a sequence of words as output that are related to some words given as input with appropriate semantic relation.
• It is often used in natural language processing (NLP) applications, such as speech recognition, automatic translation, and text generation.
• ChatGPT fits more appropriately in this definition and hence differs from chatbots.

ChatGPT as a neural network:

• Neural network: It can be considered as a large network of computers that can fine-tune its output of words based on the feedback given to it during stages of training.
• ChatGPT has undergone this training process.
• This training process and the technology together are called Reinforcement Learning.
• All these technologies are part of artificial intelligence (also called Machine Learning).

How ChatGPT looks at words?

• “Word embedding”: It represents words as a matrix of numbers that can be manipulated inside computers.
• The context in which a word has been used is differentiated by the neural network mentioned above. Example; shoot appears with the ‘gun’ or appears with the ‘camera’.
• “Transformer”: It is a refining technique that is later on deployed to help a neural network to accurately “understand” the context of a sentence or a paragraph.
• The output “comprehension” can be used for multiple purposes like answering a question etc.

Development of ChatGPT:

Use of language models:

• ChatGPT follows a generation of language models that were released by OpenAI in 2018.
• In 2018, OpenAI released the Generative Pre-Training (GPT) language, model.
• Here, generative means that it is a type of neural network that can create new content based on input content, called Training Data.
• This technology makes it suitable for creative tasks like writing a new story.

Transformer technique:

• GPT was improved and “Generative Pre-trained Transformer 2” or GPT-2 was released in 2019. GPT-3 with even more sophisticated neural networks was launched in 2020.
• In early 2022, GPT3.5 was released and ChatGPT is the successor to GPT3.5.

width="648">

Other language models: BERT (Bidirectional Encoder Representations from Transformers) from Google. Language Model for Dialogue Applications (LaMDA) from Google

Accuracy of ChatGPT:

• The accuracy of ChatGPT or any language model can be measured using standard techniques like:
• ROUGE metric: It is Recall-Oriented Understudy for Gisting Evaluation”. It compares ChatGPT’s output of content against a standard expected content and measures the overlap as a success percentage.