What Makes Us Human? What makes us human? The Future of Learning and Generative Artificial Intelligence (GOLCAT) project at UNESCO
There are still issues to be worked out. Questions remain about whether LLMs can be made accurate and reliable enough to be trusted as learning assistants. More institutions need to explore their advantages and disadvantages and share what they are learning if they want their students to benefit from the tool.
The Future of Learning and Generative Artificial intelligence is a project at the University of Nashville. Students who need to use ChatGPT, for courses such as computer science, get access to a paid version. This variant of the chatbot can use other programs to execute computer code, augmenting the bot’s mathematical capabilities.
In a February preprint, researchers described how in a benchmark set of relatively simple mathematical problems answered by students aged 12–17, almost half of the questions were answered correctly. If the problems were more complex — requiring ChatGPT to do four or more additions or subtractions in the same calculation — it was particularly likely to fail.
Teachers were spooked when ChatGPT was launched a year ago. The artificial-intelligence (AI) chatbot can write lucid, apparently well-researched essays in response to assignment questions, forcing educators around the world to rethink their evaluation methods. pen-and-paper exams were brought back by some countries. And some schools are ‘flipping’ the classroom model: students do their assignments at school, after learning about a subject at home.
Tawil, who has worked in education at UNESCO for more than two decades, says that understanding AI’s limitations is crucial. He says it is essential to rethink how to teach and assess learning because of how bound up they are in human endeavors. What makes us human is being redefined.
He likens the attention that they’re attracting to that previously lavished on massively online open courses and educational uses of the 3D virtual worlds known as the metaverse. Neither have the transformative power that some once predicted, but both have their uses. “In a sense, this is going to be the same. It’s not bad. It is not perfect. It’s not everything. It’s a new thing,” he says.
An important question around the use of AI in education is who will have access to it, and whether paid services such as Khanmigo will exacerbate existing inequalities in educational resources. DiCerbo says Khan Academy is now looking for philanthropists and grants to help to pay for computing power and to provide access for under-resourced schools, having prioritized such schools in the pilot phase. It is important that the digital divide doesn’t happen.
More researchers like Beghetto will be able to use the tools to create chatbots for their students. After his initial workshop, Beghetto plans to use the bots in a course that he is developing. ASU hosts secure versions of the LLMs in its private cloud to minimize privacy concerns, says Elizabeth Reilley, ASU’s executive director of AI acceleration, who is based in Phoenix.
Using a general LLM combined with RAG differs from previous machine-learning approaches, which sought to train an AI system to simulate a science expert, says Danielle McNamara, executive director of ASU’s learning engineering institute in Tempe. Those tools lacked generalized capabilities, such as the capacity to incorporate baseball into chemical concepts, that could help students. McNamara and her colleagues will look at how effective the tools are.
But unlike ChatGPT, when the LLM answers a query, it does not rely just on what it has learnt in its training. Instead, it also refers to a specific corpus of information, which minimizes hallucinations and other errors, says Satya Nitta, chief executive of the company. Nitta says that Merlyn Mind fine-tunes its LLMs to confession when they do not have a high-quality response and work on producing a better answer.
“One-on-one tutoring is the single most effective intervention for teaching, but it’s very expensive and not scalable,” says Theodore Gray, co-founder of Wolfram Research, a technology company in Champaign, Illinois. “People have tried software, and it generally doesn’t work very well. There is a chance that educational software could work. Gray told Nature that Wolfram is working on a tutor but gave no further details.
More generally, Lynch stresses that it’s crucial that any chatbot used in education is carefully checked for its tone, as well as accuracy — and that it does not insult or belittle students, or make them feel lost. Emotions are key to learning. You can legitimately destroy somebody’s interest in learning by helping them in a bad way,” Lynch says.
But whether Khanmigo can truly revolutionize education is still unclear. LLMs are taught not to check the facts because only the next most likely word is included. They therefore sometimes get things wrong. To improve its accuracy, the prompt that Khanmigo sends to GPT-4 now includes the right answers for guidance, says DiCerbo. Khan Academy wants users to let the organization know when it makes a mistake.
According to Khan Academy, more than 28,000 teachers and students are using Khanmigo in the US this school year. Users include private subscribers as well as more than 30 school districts. The cost of computing is covered by people and school districts pay for access to students. OpenAI has agreed to stop using Khanmigo data for training.
Khanmigo works differently from another one. It appears as a pop-up chatbot on a student’s computer screen. There is a problem that students are working on. The tool automatically adds a prompt before it sends the student’s query to GPT-4, instructing the bot not to give away answers and instead to ask lots of questions.
She says PyrEval scores help students reflect on their work and it could indicate that the idea needs to be more clearly explained or that they made small conceptual errors. The team is comparing the results of the tasks done by the team and other LLMs.
With help from educational psychologist Sadhana Puntambekar at the University of Wisconsin–Madison, PyrEval has scored physics essays5 written during science classes by around 2,000 middle-school students a year for the past three years. The essays are not given conventional grades, but PyrEval enables teachers to quickly check whether assignments include key themes and to provide feedback during the class itself, something that would otherwise be impossible, says Puntambekar.
Companies are marketing commercial assistants, such as MagicSchool and Eduaide, that are based on OpenAI’s LLM technology and help schoolteachers to plan lesson activities and assess students’ work. The PyrEval4 tool was created by a team of computer scientists at Pennsylvania State University and was meant to be used to read essays and find key ideas.
“Are there positive uses?” asks Collin Lynch, a computer scientist at North Carolina State University in Raleigh who specializes in educational systems. Absolutely. Is there any risk? There are a lot of concerns and risks. But I think there are ways to mitigate those.”
Privacy is another hurdle: students might be put off working regularly with LLMs once they realize that everything they type into them is being stored by OpenAI and might be used to train the models.
There has been a lot of attention given to its use in education since it was introduced by Openai. LLMs work by learning how words and phrases relate to each other from training data containing billions of examples. They produce sentences, including the answer to an assignment question, when they respond to user prompt.
Many instructors fear that the increase of ChatGPMT will make it easier for students to cheat. Yet Beghetto, who is based in Tempe, and others are exploring the potential of large language models (LLMs), such as ChatGPT, as tools to enhance education.
A group of graduate students and teaching professionals last month were asked by an educational psychologist if they would be interested in discussing their work with him. As well as talking to each other and conversing with a collection of creativity-focused chatbots that Beghtetto had designed, they are going to host a platform on which to showcase the creations of his institute, Arizona State University.
