Harvard & DEC Surveys of AI Use by Students Yield Startling Disclosures

Two new surveys of artificial intelligence utilization by college students have recently appeared, and their results mirror each other in significant respects.

Unlike several previous studies, both surveys suggest that students are now moderately to highly engaged with using generative AI in their coursework. However, both surveys also suggest much lower confidence among students that they possess sufficient AI knowledge and skills. In one of the polls, almost half of the students said that they don’t feel adequately “prepared for a future workforce that heavily utilizes AI.”

The first study surveyed students at Harvard College, the undergraduate division of Harvard University. In that poll, 326 undergraduates filled out questionnaires about how artificial intelligence had impacted their experiences, and the researchers studied the influence of AI on their study habits, course choices and career prospects. Fieldwork was conducted during April 2024, and the researchers published the study the following June.

The second poll studied about 4,000 undergraduates, master’s and PhD students from 16 countries around the world, with fieldwork conducted during July 2024 and the results published in August. This survey was conducted by the Digital Education Council (DEC), a new association launched in April 2024 by twelve high-profile universities outside the United States.

Founding members include Bocconi University in Milan, the Hong Kong University of Science and Technology, the Melbourne Business School, Singapore Management University, Tec de Monterrey (Mexico), Esade (Spain), and the University of the Witwatersrand (South Africa). The alliance was organized by the world’s largest educational technology accelerator, London-based SuperCharger Ventures.

Unlike some recent studies that reported less engagement, the DEC and Harvard studies report that almost 90 percent of students use AI in their studies. The reports display similar engagement results between the two samples. The DEC study reports 86 percent of respondents use AI, and the Harvard study reports 87.5 percent.

The Harvard students apparently use AI more frequently. In that study, about half of the students use AI daily or every other day, whereas in the DEC sample, about half use AI daily or weekly. However, that engagement drops substantially when the researchers ask about daily use; both studies reported that only about a quarter use an AI platform every day.

At first it might seem like ChatGPT is the most used AI tool among both samples. That’s because two-thirds of the Digital Education Council sample reported using ChatGPT for coursework, along with 25 percent adoption rates for both Grammarly and Microsoft Copilot. The DEC sample also reported a smattering of other apps like Claude Al, Blackbox, DeepL for translations, and Canva for images. The average DEC respondent uses more than two AI tools, and 22 percent use more than three.

At Harvard, an overwhelming 95 percent of students use ChatGPT, and in addition, Anthropic’s Claude and the programming AI assistant GitHub Copilot are each used by roughly a fifth of the sample. But not all of the Harvard students are using the same version of ChatGPT. This is a crucial and—until now—underreported distinction.

The paid subscription version of ChatGPT is branded as ChatGPT Plus, and sells for $240 per year. About a third of the Harvard students report buying premium subscriptions, and most of them are probably buying ChatGPT Plus.

Why would they do that? The researchers write that “paying money makes a difference: students who spend money on Al report getting more benefit from it, and rely less on University resources and traditional search engines.”

So how much would a paid subscription plan boost the value that Harvard students receive from AI? From the report:

The answer seems to be: quite a lot. . .Compared to students who only use free Al products, students who pay for Al subscriptions are over twice as likely to use generative Al products instead of Wikipedia or Google search. They are also almost three times as likely to report decreased utilization of office hours because they can consult Al instead.

The Harvard researchers then report the results of chi-squared statistical tests. Those results verify that the differences between the free and subscription student product groups are indeed statistically significant.

Certainly this result could raise a disturbing equity issue that might affect students from lower socioeconomic backgrounds, who at most colleges, are typically those from underrepresented and disadvantaged groups such as first-generation and minority students. The Harvard researchers found that 40 percent of students not receiving financial aid pay for AI subscriptions, but only 20 percent of students on financial aid do so.

In other words, students who don’t receive financial aid are twice as likely to pay for AI subscriptions as financial aid recipients. Assuming that these differences are due to the $240 yearly cost of ChatGPT Plus as the prohibitive factor instead of any other factors, and given that paid AI plans substantially improve user experiences as the Harvard students report, an equity issue would clearly exist.

Other things equal, the students on the paid plans appear to be saving significant amounts of time and effort compared to students on the free plans, and those savings could certainly translate into better grades and more time to take advantage of Harvard’s opportunities—like more interviews with on-campus job recruiters. About 35 percent of the sample expressed fears that their classmates would use AI to gain such unfair academic advantages.

Currently, Harvard has implemented clear rules around student AI use, but the researchers point out that they’re not enforceable. They advocate changes that would enable that enforceability, but it’s not apparent from the report what specific changes they have in mind.

The Digital Education Council study asks, “Is Gen Al the new Google?” Based on the surprising results from both surveys, it might be. Both studies reported information searching as the most frequent use case, and both also report much higher proportions of students who are searching for information using AI than disclosed by the many previous surveys of college students we’ve covered here at OnlineEducation.com.

The DEC study asked respondents, “What do you usually use Al tools for?” Searching for information was the top-ranked result, selected by a whopping 69 percent of the sample.

The Harvard study asked respondents about their primary purpose for using generative AI. Their most common use case is to answer general questions, such as “How does a 401k plan work?” In a result that again closely mirrors the Digital Education Council’s results, about 72 percent of the Harvard sample selected this option. Already, for about 33 percent of the Harvard sample, AI is now replacing the usual sources of information most of us have relied upon during the past 25 years, such as Google search and Wikipedia.

The second-ranked use case in both surveys involved help with writing assignments, like writing student papers. In the Digital Education Council study, that use case primarily means using AI to check grammar, selected by 42 percent of the sample. In the Harvard study, respondents said that use case not only means proofreading but also means coming up with ideas and developing first drafts. About 54 percent of the Harvard students selected this option.

Beyond the first two rankings, the purposes reported by the two samples markedly diverge. In declining order of priority, the Digital Education Council study’s respondents reported that they used AI to summarize and paraphrase documents and create first drafts.

Harvard’s students reported that they next used AI to write their email messages; other priorities included using AI with programming assignments, data processing, creative assignments like developing graphics and translating foreign languages.

The lowest priority in the Harvard study—selected by only about 5 percent—involved a curious entry labeled “entertainment or companionship.” Now, what sorts of companionship could a student obtain through ChatGPT that wouldn’t be available in the midst of 25,000 Harvard students on campus in Cambridge, Massachusetts, or 250,000 college students among 50 colleges right next door in the world’s largest college town, Boston?

The headline news out of the Digital Education Council study appears to be that about half of the students don’t feel AI ready. In that survey, 58 percent disagreed with the statement, “I have sufficient AI knowledge and skills,” and 48 percent disagreed with the statement, “I feel prepared for a future workforce that heavily utilizes Al.”

Students in that survey were harshly critical of their universities. For example, 80 percent of the students reported that their school’s AI integration doesn’t fully meet their expectations. And an overwhelming proportion—usually greater than 70 percent in each of the following cases—expected their schools to take four steps. Students wanted their colleges to increase the use of AI in teaching and learning, offer more AI literacy courses, provide AI training for students and faculty on the effective use of AI tools, and involve students in their decision-making processes about which AI tools are implemented.

Harvard students were also critical of their university’s artificial intelligence efforts, citing a lack of key classes and student services. This result might also seem surprising, given that Harvard is arguably the best-resourced university in the United States, with by far the biggest endowment and largest budgets in many academic departments and student support facilities.

For example, in 2023 the National Association of College and University Business Officers (NACUBO) reported that Harvard University controls a staggering $49.495 billion endowment—and no other American college or university even comes close. That endowment is $8.748 billion or 21 percent greater than the nearest competitor, Yale University, and $13 billion or 36 percent more than third-ranked Stanford University.

Harvard students wish their university offered more classes covering the future impact of generative artificial intelligence. The researchers suggested that although several computer science classes on their campus cover AI topics—including the most popular online college course in the world, Harvard’s CS 50—there were very few classes exploring the implications of “increasingly powerful Al systems on society, the economy, and the pace of technological progress.”

Over half of the survey’s respondents (55 percent) wish that Harvard offered more of these classes. The researchers also recommended that Harvard’s career development and placement center offer AI-aware career planning services to help students “more thoughtfully consider the negative impacts of recent and future automation on careers.”