Additionally, data collection needs to comply with copyright laws and ethical considerations to respect privacy and intellectual property rights.

After data collection Transformers are trained on this massive amount of data which creates a base model for the gpt which posses the ability to understand the context in the language and this base model can be easily fine tuned to multiple tasks like language translation , text summarisation ,text generation , sentiment analysis etc.

STAGE-2( SUPERVISED FINE TUNING):

At this stage, the role of the prompt engineer becomes crucial. The prompt engineer’s responsibility is to fine-tune the pre-trained GPT model on specific tasks or domains using smaller, labeled datasets. This fine-tuning process allows the prompt engineer to tailor the model’s capabilities to particular applications, such as text generation, translation, sentiment analysis, and more. By fine-tuning the model, the prompt engineer can harness the power of GPT’s language understanding while customizing its responses to suit the desired context and achieve optimal performance in targeted tasks.

The model is fine-tuned to create an agent which is capable of talking like human so a dataset is created as where a human being asks questions and another human being who acts like bot generates response to these questions as shown in the figure below. This dialogue based data set question answer/ request response data set is prepared on a very huge scale of millions of records here is an example how records would look like.

Input: How are you?

Target: I am good thank you. How are you?

Now the model is fine tuned on this dataset and during this step in the research paper it was mentioned that they have been using Stochastic gradient descent optimiser

Now after this training stage we have our model supervised fine tuned model which is capable of carrying conversations like humans, But there is a limitation the SFT-GPT model is capabale of answering requests related to the requests present in the data set only and if anything is asked which is not related to dataset then the SFT based generative model would generate some random response that would dosen’t even make sense. Thus to overcome this phase the next stages were created and This stage played a major role in creation of chat-gpt.

STAGE-3 (REINFORCEMENT LEARNING THROUGH HUMAN FEEDBACK):

Here the prompt engineer queries the model and the model gives multiple responses as shown in the image and the prompt engineer will rank all responses and the response with top rank will be picked as the response from the model this process is called ranking through human feed back.