Can Augmented Reality Help Students Learn Design by Seeing How Structures Work?

“Traditional lectures, textbook diagrams, and static physical models struggle to convey complex structural mechanics because forces are invisible and dynamic.”

Seungoh Paek and Daniel Hoffman, University of Hawaiʻi at Mānoa

In architecture and engineering education, students are often asked to understand forces they cannot see. They study diagrams, formulas, and physical models meant to explain how structures carry weight, transfer loads, bend, compress, or resist pressure. But for many undergraduate design students, the movement from abstract structural mechanics to practical design judgment remains difficult.

At the University of Hawaiʻi at Mānoa, a multidisciplinary team is developing an augmented reality learning system that aims to make those relationships more visible. Supported by a $400,000 grant, the three-year project brings together architecture, computer science, and education researchers to create a more interactive learning environment for undergraduate design students. 

The goal is not to replace hands-on learning with screens. It is to make physical models more responsive, so students can push, pull, bend, revise, and immediately see how invisible forces move through a structure. Rather than treating structural analysis as something students encounter through formulas alone, the project tests whether augmented physical models can help students build what design educators often call structural intuition.

Architecture, engineering, and construction fields require students to move between creative design, technical reasoning, and digital tools. Yet the core educational problem remains practical: students need to understand how structures behave well enough to make better design decisions. Augmented reality is useful only if it helps students develop that judgment.

To understand what this project could reveal about immersive learning, we spoke with University of Hawaiʻi at Mānoa professors, including assistant professor in the Department of Information and Computer Sciences Dr. Nurit Kirshenbaum, whose work examines tangible interfaces and spatial augmentation, as well as Dr. Seungoh Paek and Dr. Daniel Hoffman from the College of Education’s Department of Learning Design & Technology, who are part of the project team and focus on developing and testing technology-enhanced learning.

Daniel Hoffman is an associate professor in the University of Hawaiʻi at Mānoa College of Education’s Department of Learning Design & Technology. He earned an EdD and EdM in instructional technology & media from Teachers College, Columbia University, an MS in the science of teaching from Pace University, and a BA in English literature and Drama from St. Michael’s College. 

Dr. Hoffman’s interests focus on digital learning, interactive multimedia, games and simulations, media literacy, and computer science education.

Nurit Kirshenbaum is an assistant professor in the Department of Information and Computer Sciences at the University of Hawaiʻi at Mānoa. She earned a PhD and an MS in computer science from the University of Hawaiʻi, an MS in interactive media from Quinnipiac University, an associate degree in math from Tel Aviv University, and a BSc in electrical engineering from Technion. 

Dr. Kirshenbaum is interested in human-centered design, tangible user interfaces, cyberinfrastructure, shape-changing interfaces, learning technologies, games, interactive media, authoring, and visualization tools.

Seungoh Paek is a professor in the University of Hawaiʻi at Mānoa College of Education’s Department of Learning Design & Technology. She holds multiple degrees from Teachers College, Columbia University, and Sungshin Women’s University. 

Dr. Paek’s research interests focus on interactive multimedia design, instructional technology, cognitive science, developmental psychology, math education, and evaluation.

The UH Mānoa grant supports SpARC Learning, a project designed to help architecture students understand structural behavior through augmented, hands-on learning. The system, called SpARC Learning, combines Spatial Augmented Reality, Tangible User Interfaces, motion capture, and AI-supported feedback so students can work with physical models while digital information is projected directly onto them.

From a learning design perspective, Dr. Paek says the project responds to “a critical gap in Architecture, Engineering, and Construction (AEC) higher education, specifically within Statics & Structures curricula.”

That gap is not simply that structures courses are difficult. It is that many architecture students enter the field through design, creativity, and visual thinking, while statics and structures courses often depend on abstract math and physics formulas. Students may learn to solve equations or memorize procedures for exams without developing a practical sense of how physical systems behave.

“Traditional teaching yields graduates who memorize formulas to pass exams but fail to develop a practical, intuitive understanding of structural principles needed in the industry,” Dr. Paek explains.

The difficulty is partly sensory. Forces, load paths, stresses, compression, and tension are not directly visible in a static diagram or a wooden model. Students can be told what is happening inside a beam or frame, but the behavior itself remains abstract.

Dr. Hoffman describes SpARC Learning as a shift away from passive instruction toward a more responsive learning environment. “Traditional lectures, textbook diagrams, and static physical models struggle to convey complex structural mechanics because forces are invisible and dynamic,” he says.

Meanwhile, Dr. Kirshenbaum approaches the same teaching problem from the interface side. In addition to the difficulty of teaching structural behavior through formulas and static models, how do students encounter visual information? Digital models can be powerful, especially for experts who already know what to look for. But students who are still developing spatial reasoning may not know how to interpret or explore complex visualizations on their own.

“In general, we have used terrains with data layers projected on them to convey the deep connection between geospatial data and the topography of places,” Dr. Kirshenbaum shares, noting that projected information can show how agricultural lands or rainfall data correspond directly to terrain.

For novices, that connection can be harder to see on a screen. Dr. Kirshenbaum points out that experts can often “get” more from digital models they can manipulate in two or three dimensions, but students who are new to a subject may not understand the bigger context or know how to explore advanced software on their own.

SpARC brings those two concerns together. It starts with physical demonstrations that instructors already use, then adds real-time calculations and feedback so students can connect what they do with what the model shows.

“In the project, we are working more directly with the architecture curriculum,” Dr. Kirshenbaum says. “We then augment these manipulatives with real-time calculations and feedback, so we combine the benefits of the tangible manipulation with the screen benefits.”

The result is a learning environment that treats physical models as interfaces for computation, visualization, and inquiry. In that sense, augmented reality is used to increase learning ability beyond traditional learning practices.

The project also reflects a particular view of immersive learning. In many education technology settings, immersion is associated with individual screens or headsets. The UH Mānoa project moves in another direction, using augmented reality to bring students together around a shared physical task.

For Dr. Paek and Dr. Hoffman, that shared environment is part of the learning design. Spatial Augmented Reality projects information directly onto physical models rather than requiring students to use head-mounted displays or individual screens. That keeps students oriented toward the same object, where they can use their hands, talk with one another, and respond to the same visual feedback.

Dr. Kirshenbaum gives the example of a force-balance activity in which three students hold strings attached to a washer and pull in different directions. As students pull, they feel the tension in the strings, observe how the washer moves, and see how forces interact.

“The force on the washer balances out, i.e., forces on one side equal forces on the other,” Dr. Kirshenbaum says.

The augmented system adds feedback to that physical experience. It can visualize the forces being applied in real time, giving students a way to connect what they feel in their hands with what they see projected in front of them. It can also add game-like goals, such as asking students to move the washer to a highlighted point.

Dr. Paek and Dr. Hoffman describe similar examples using “smart rubber bars or sensor-embedded beams.” When students push, pull, or bend those objects, the system overlays digital graphics showing force magnitudes, directions, and internal states, including color coding such as red for compression and blue for tension.

The purpose is to make structural behavior visible at the moment students are acting on the model. Dr. Hoffman explains that this helps shift the pedagogy from “mathematical memorization” toward “structural intuition,” giving design students a more durable, qualitative understanding of how physical systems behave in practice. That kind of intuition has practical implications for design work. 

Dr. Paek expounds that a student who develops structural intuition can visually evaluate a structural problem, sketch a plausible design, understand the implications of design revisions, and ask engineering consultants better questions. Structural thinking becomes part of the design process earlier, rather than a constraint added after the fact.

Dr. Kirshenbaum also emphasizes the importance of what happens when that learning becomes social and embodied. “These kinds of activities are highly collaborative and cannot be achieved with screens or something isolating like VR,” she shares. “The students are all working together around a single table with a joint objective and share the embodied experience.”

By producing the conditions themselves, feeling the system respond, and negotiating the task with other students in real time, there is deeper learning value wrought from the connection between action and understanding.

However, the promise of augmented learning depends on more than whether students find the system engaging. A classroom can become more interactive without necessarily improving comprehension. For the UH Mānoa team, the next step is to study whether augmented physical models help students understand concepts more deeply than existing classroom methods.

Dr. Kirshenbaum says the project is still moving toward classroom evaluation. “In the next two years, we hope to introduce the system in a classroom setting and evaluate learning outcomes,” she says.

Dr. Hoffman describes the evaluation design in more specific terms. He says the project will conduct a three-year study using Design-Based Research, comparing control groups taught with a standard curriculum against experimental groups using the SpARC system. Researchers will use formative and summative tests, along with pre- and post-attitude surveys, to evaluate learning gains, motivation, and structural comprehension.

The project will also study how students use the system. Dr. Paek explains that the researchers will analyze system logs, media recordings, and post-reflection assignments to evaluate student engagement, kinesthetic interaction, and group collaboration dynamics.

Dr. Kirshenbaum describes a related comparison between ordinary tangible activities and augmented ones. “We hope to see more physical engagement (making the classroom more interactive is still a worthwhile goal) in the way people interact with the tangibles and other teammates, but we also intend to compare quiz results about the modules we are developing between the simple tangible manipulatives and the augmented ones,” she says.

Together, those measures give the project a concrete test. The question is not whether hands-on activity helps students. It is whether the added layer of projection, calculation, feedback, and interaction helps students form a clearer mental model of the concept being taught.

If students using the augmented version can explain forces more clearly, apply concepts more accurately, or transfer what they learn to later design tasks, the system has a stronger case as a teaching tool. If the results are limited, that finding is still useful. It helps instructors understand when real-time visualization adds value and when a simple physical model is enough. And from that will be seen what students can do with the concept after the end.

The project’s research design also shows why technology-enhanced learning systems are difficult to develop quickly. SpARC is not only a classroom tool. It is a combination of curriculum design, hardware, software, sensors, projected visualization, physical materials, and evaluation.

Dr. Paek and Dr. Hoffman both say policymakers and grantmakers need to understand that systems like SpARC require “sustained, multi-year funding” because the work depends on iterative design-based research. That includes hardware configuration, software development, implementation, and repeated evaluation.

They also point to the need for UI and UX testing, along with external interdisciplinary panels that can help refine both the curriculum and the technology before a system is scaled. The physical side of the project matters as well. Sensors, specialized equipment, materials, and classroom setup can affect whether an approach is practical beyond a single research environment.

The broader goal is not simply to build one augmented reality classroom activity. Dr. Paek and Dr. Hoffman say support should extend beyond initial development to include open-source repositories and public platforms that help other institutions adopt or adapt the tools.

Dr. Kirshenbaum connects that scaling question back to the underlying learning approach. For her, the value of tangible learning is tied to how people build understanding through physical exploration. 

“I am a great believer that fiddling with ‘things’ is a helpful way of learning in the constructivist philosophy that it can help people build their own understanding about processes,” she says.

That belief also shapes how she thinks about collaboration. Physical objects do not only give students something to manipulate. They give groups a shared point of attention. “I also love how physical ‘things’ help bring many people together; it makes you think of playing, and playing is a social activity,” Dr. Kirshenbaum notes.

The same logic can apply to other STEM subjects that involve spatial relationships or invisible processes. Dr. Kirshenbaum says it is increasingly possible to add a dynamic computational layer to physical models. 

“It is not too hard these days to add the computational, dynamic layer that can capture the scene and add visualization feedback,” she considers, adding that she can imagine similar approaches being used to enhance “flow in diagrams,” including “electricity flow in a circuit or water flow on terrain.”

That does not mean every diagram or model needs augmentation. The point is more specific. When a subject depends on forces, flows, spatial relationships, or changing conditions, tangible augmented systems may help students connect what they do with what they need to understand.

Ultimately, for the UH Mānoa team, the promise of augmented reality is not that it simply makes the classroom look more advanced. It is that it will likely help students see relationships that are otherwise difficult to connect: force and movement, form and feedback, individual action and shared understanding. As humans, some concepts become clearer when students can work with them as things, not just as images on a screen.