ProCoT: Stimulating Critical Thinking and Writing of Students through
  Engagement with Large Language Models (LLMs)
Tosin Adewumi Lama Alkhaled Claudia Buck Sergio Hernandez Saga Brilioth Mkpe Kekung Yelvin Ragimov Elisa Barney
We introduce a novel writing method called Probing Chain of Thought (ProCoT),
which prevents students from cheating using a Large Language Model (LLM), such
as ChatGPT, while enhancing their active learning through such models. LLMs
have disrupted education and many other feilds. For fear of students cheating,
many educationists have resorted to banning their use, as their outputs can be
human-like and hard to detect in some cases. These LLMs are also known for
hallucinations (i.e. fake facts). We conduct studies with ProCoT in two
different courses with a combined total of about 66 students. The students in
each course were asked to prompt an LLM of their choice with one question from
a set of four and required to affirm or refute statements in the LLM output by
using peer reviewed references. The results show two things: (1) ProCoT
stimulates creative/critical thinking and writing of students through
engagement with LLMs when we compare the LLM solely output to ProCoT output and
(2) ProCoT can prevent cheating because of clear limitations in existing LLMs
when we compare students ProCoT output to LLM ProCoT output. We also discover
that most students prefer to give answers in fewer words than LLMs, which are
typically verbose. The average word counts for students, ChatGPT (v3.5) and
Phind (v8) are 208, 391 and 383, respectively.