ChatGPT, the latest natural language processing tool
- Category
Science & Technology
- Published
9th Dec, 2022
-
Context
The ChatGPT an artificial intelligence tool developed by OpenAI has created a buzz among computer scientists and programmers due to its creative capabilities.z
About
About ChatGPT:
- ChatGPT is a ‘conversational’ AI and will answer queries just like a human would.
- It can answer follow-up questions.
- It can also “admit its mistakes
- It can challenge incorrect premises and reject inappropriate requests.
- It is being seen as a replacement for much of the daily mundane writing, from an email to even college-style essays.
- Developed by: OpenAI, a research, and development firm, was founded as a nonprofit in 2015.
- Language Used: It has used the GPT 3.5 series of language learning models (LLM).
- GPT stands for Generative Pre-trained Transformer 3.
- It relies on deep learning techniques to produce human-like text based on inputs.’
Why the ChatGPT is called a Language Model and not a chatbot?
- A language model is software that:
- prints out a sequence of words as output that are related to some words given as input with appropriate semantic relation.
- It is often used in natural language processing (NLP) applications, such as speech recognition, automatic translation, and text generation.
- ChatGPT fits more appropriately in this definition and hence differs from chatbots.
ChatGPT as a neural network:
- Neural network: It can be considered as a large network of computers that can fine-tune its output of words based on the feedback given to it during stages of training.
- ChatGPT has undergone this training process.
- This training process and the technology together are called Reinforcement Learning.
- All these technologies are part of artificial intelligence (also called Machine Learning).
How ChatGPT looks at words?
- “Word embedding”: It represents words as a matrix of numbers that can be manipulated inside computers.
- The context in which a word has been used is differentiated by the neural network mentioned above. Example; shoot appears with the ‘gun’ or appears with the ‘camera’.
- “Transformer”: It is a refining technique that is later on deployed to help a neural network to accurately “understand” the context of a sentence or a paragraph.
- The output “comprehension” can be used for multiple purposes like answering a question etc.
Development of ChatGPT:
Use of language models:
- ChatGPT follows a generation of language models that were released by OpenAI in 2018.
- In 2018, OpenAI released the Generative Pre-Training (GPT) language, model.
- Here, generative means that it is a type of neural network that can create new content based on input content, called Training Data.
- This technology makes it suitable for creative tasks like writing a new story.
Transformer technique:
- GPT was improved and “Generative Pre-trained Transformer 2” or GPT-2 was released in 2019. GPT-3 with even more sophisticated neural networks was launched in 2020.
- In early 2022, GPT3.5 was released and ChatGPT is the successor to GPT3.5.
width="648">
Other language models:
- BERT (Bidirectional Encoder Representations from Transformers) from Google.
- Language Model for Dialogue Applications (LaMDA) from Google
|
Accuracy of ChatGPT:
- The accuracy of ChatGPT or any language model can be measured using standard techniques like:
- ROUGE metric: It is Recall-Oriented Understudy for Gisting Evaluation”. It compares ChatGPT’s output of content against a standard expected content and measures the overlap as a success percentage.