chat4dChatGPT is a hugely successful conversational AI chatbot released in November 2022 by OpenAI. It has advanced capabilities, such as performing writing tasks well, includingChatGPT sometimes writes plausible-sounding but incorrect or nonsensical answers. Fixing this issue is challenging, as: (1) during RL training, there’s currently no source