Gemstone members use tablets to teach blind students math

Mat Jennings, a junior computer engineering major, demonstrates how the Haptic screen technology works to provide texture on a touch screen.

Andrea Bajcsy experienced life without eyesight — but not for long.

In fact, she lost her eyesight at the start of class and regained it by the time class finished as part of an experiment to work toward making technology to help the blind. 

Bajcsy, now a junior computer science major, experimented with this during a design course two years ago, putting on a blindfold while another student led her around the campus.

“I was the blind person, and I was traumatized,” Bajcsy said. “My buddy took me to the computer lab in the business building and she said, ‘Use the computer now,’ and I realized there’s no way for me to use this computer — I don’t even know where to start.”

After this, Bajcsy began to think about how she could get through college, especially her STEM classes, without her eyesight.

This is all part of the Gemstone honors program, in which students work on longterm projects for their four-year college careers. Bajcsy’s team proposed using technology to help blind students learn about concepts often taught visually, which soon became the basis for Team Haptic, a Gemstone team planning to use touch-screen tablets to teach math to blind students.

They used a rejiggered Microsoft Surface tablet they outfitted with a capacitive layer, or feel-screen, by Tangible Haptics, an independent company they have a partnership with, in the hope of using the Haptic technology to help blind students by allowing them to feel textured shapes and graphs.

Giving a tablet’s flat surface a textured feeling is no easy task, but it can be accomplished with Haptic technology — the same kind that makes controllers vibrate or become touch-screen ready. It reads the pressure from a finger, sends out a voltage signal and makes it feel as though one is touching a rough, grainy surface.

“By applying these voltage signals to the capacitive layer, it creates micro-contortions that when you move your finger, it creates these bumps and it creates this perception of texture,” said Mat Jennings, a junior computer engineering major and a member of Team Haptic.

Right now, the team has demos of how the technology works. But they soon want to adapt it so texture can exist over shapes and graphs and students can feel them, Bajcsy said.

For example, Jennings said, if the team wanted to draw an exponential graph, they could apply a Haptic effect to the graph pixels and students could then scan it with their hands. 

Team Haptic members are in their third year of the Gemstone program and recently began a Launch UMD kickstarter campaign to raise money for another Surface tablet and more software to be able to test the experiment, said Amelia Bateman, a junior Haptics member and math and computer science major.

The team has overwhelming support, she said, but the team’s 14 members were not drawn to the project purely for cool technology.

“It’s also the helping people,” Bateman said. “It’s just a fact of research that a lot of the time, if you’re doing research, you’re just doing the grunt work in the background, and someone later down the road will put it into something cool for society.”

Team Haptic also plans to try to teach math to a group of 8- to 10-year-old blind students with the Haptic technology, while another group uses traditional methods.

Marcio Oliveira, a kinesiology professor and the project’s faculty adviser, said he thinks the students learning from Haptic technology will be able to learn the material better than those without it.

“Because of the technology that we have and the mobility … learning is going to be very personalized,” said Oliveira, also the public health school’s educational innovation and technology assistant dean. “So the technology is secondary in the learning process, but it enables you to learn.”

Carolyn Fink, a professor in the counseling, higher education and special education department, said the visually impaired are an underserved group and technology could make a huge difference for them.

“The great step forward that smaller electronics have had is to be able to allow the students to be in the general education environment but still get helped,” Fink said. “If a visually impaired student can have a small device that gives them access to the general ed curriculum, then that’s huge for them to be a part of the larger school environment.”