Computer Science Students Work for Social Good

It was finally time for Mengting Xia to sign up for her capstone senior design class and she scrolled through the class options. “I searched the professors’ websites to see what they were currently doing research on, and I heard that Professor Zhu was doing assistive technology and I was interested in that—in that assistive technology that helps people in need,” Xia said.

Maybe Xia had an inkling that she was about to step into a new world where computer science students have the chance to become agents for social good. That’s what Professor Zhigang Zhu hopes will happen. The City College teaching and research professor created this capstone computer science class for students to develop applications that have beneficial use in the real world. Zhu had taught the senior design class relatively traditionally. But about twelve years ago, he began to think about how the students’ projects and his research could make an impact and help people.

He thought about the many students who received extended time on tests because of learning or processing issues. He realized that people all around him needed an assist and he wondered if his students could help. He eventually changed his own research from surveillance and robotics to assistive technologies with a goal to improve the lives of people with disabilities. His next step was to implement an entrepreneurship focus with assistive technology into his existing senior design class. 

His students come to him with engineering and computing backgrounds, and the thought of using their skills for the good of others is often foreign at first. “It’s a very nice opportunity for our students to understand, particularly in big cities like New York City, that there are a lot of people who need help. We can do a lot of things, in addition to just finding a job and earning a life,” Zhu said. 

With the new focus came a new name: Branding and Entrepreneurship of Assistive Technology for Social Good (BEAT). Through the years, the class has focused on creating helpful resources for the physically impaired. As Zhu learned about autism, he made Autism Spectrum Disorder (ASD) a focus for some of his classes.

Goodwill NY/NJ, the non-profit that runs job training programs and thrift shops, became a partner in Zhu’s project. Zhu also brought in the New York State Industries for the Disabled, which helps with funding. Both groups work with students every semester advising them about the realities of what people with autism face in the workplace. 

Carlos Saravia completed Zhu’s senior design class during the 2019-2020 semesters. He and his team consulted with Goodwill as they developed their project. Saravia didn’t know the class was going to focus on assistive technology when he registered for it. But he was good with the focus when he found out.

“I wanted to do something interesting and useful because a lot of other senior design classes end up just being another technical lecture class, so this was a good change of pace,” Saravia said. 

He and his team went to Goodwill and learned that the staff uses role playing with ASD job trainees. A staffer pretends to be an interviewer or a boss and simulates scenarios the person with ASD may face in the workplace. The Goodwill trainers offer feedback so that the prospective job applicants can build on what they learn. 

Saravia’s team decided to create a virtual reality role-playing game, called Undaunted, to simulate situations that are often emotionally stressful to people with ASD.

Through a first-person virtual experience, the user assumes the position of an employee on their first day and practices interactions with coworkers. An iPad-type device provides discussion options and lets the user know if they responded in an unfriendly or disrespectful way and teaches the best response. 

Saravia’s team sees the app working in tandem with Goodwill staff, providing additional practice for the person with ASD. The app creates a report detailing areas that need attention, which allows those running the program at Goodwill to focus on those areas of weakness in their in-person role playing. 

Goodwill also offered helpful direction to Mengting Xia and her team. They found that people with ASD have trouble grocery shopping, so they came up with a solution.

They designed a shopping app they call Parashop, a mobile application that helps the user create grocery lists and gives instruction to aid in shopping. Parashop provides images that match a grocery list so that when someone shops they can compare the items to their list. When the item has a UPC code, a quick scan does the confirmation, but when it comes to fruits and vegetables, typically there are no codes.

Xia and her team found pictures of fruits and vegetables and then drew their own to create models and build data sets. It took them a very long time, but when their app correctly identified an apple, they were thrilled. 

The course stimulates the students’ imagination and often forces them to learn new skills not covered by their classes. 

Saravia’s team didn’t know how to design virtual reality, so they taught themselves through reading and YouTube videos.

“None of us knew what we were getting ourselves into and things seemed very easy at the beginning, and then you run into one problem and then another and then another. Sometimes just finding the right resource to solve it is difficult, Saravia said. After having been taught by teachers for so many years, he was encouraged to see that he can learn on his own.

While the partnership with Goodwill is an asset, not every team has to use it. Jin Chen and her team got inspiration from a more personal source. Chen’s grandmother lives alone in China and finds technology challenging. Chen wanted to create something that would help her grandmother. Their app is called Share & Care. 

Chen’s team surveyed adults young and old and discovered many young people only contact older family members a few times a year. The older adults often forget passwords and find navigating mobile applications confusing.

With Share & Care the user commands their photos to scroll down and up and finds and likes social posts just by using their voice—in English or Chinese. The user can also assign a family member to manage their contacts. 

Chen’s team had difficulty with the voice control aspect of the app at first. But after a lot of research, they figured it out. They also got feedback and suggestions from Professor Zhu through their weekly presentations.

Funding provided another big challenge. “It costs a lot of money to host the app on the server and the database. We definitely hope to make it open source because I think it’s a very good app that people would like to use. But right now we definitely do not have any financing to post or host our app,” Chen said. She plans to stay at City College and earn her master’s degree and hopes to figure out the funding issue with Professor Zhu.

Zhu emphasizes innovation, and he wants his students to take ownership in an entrepreneurial way. He encourages them to participate in the Zahn Innovation Center competitions at City College.

The Zahn incubator, funded by the Zahn family and the Moxie Foundation, nurtures entrepreneurial initiatives at City College and gives students, faculty and staff the tools they need to transform their ideas into into viable businesses. But Zhu finds very few of his students want to start their own business. “Sometimes they just want to do coding, find a job, and support their families. We have some work to do here,” Zhu said. 

With the support of the Zahn Innovation Center, this year he required all of his students to enter the Zahn competition. He understood that this required a lot of extra effort. But the results pleased him. “In the past, I have had maybe one team, maybe no teams. We actually had three out of eight teams get into the selection, which is very good,” Zhu said.

The classes got an extra boost when Zhu teamed up with the Integrated + Branding (BIC) program. A grant from the Campus Engagement Network (CEN), funded by the Moxie Foundation, made it possible for BIC’s Professor Gerardo Blumenkrantz to collaborate with Zhu and his students on building branding and marketing into their projects.

Remote learning complicated the last part of the spring 2020 semester. But the student innovators pressed on and found the process rewarding.

Saravia thinks his virtual reality project taught him a lot. He loved translating the needs of non-technical people into something tangible. He’s glad to know that as he finished four years of very hard work, he can actually create useful tools.

“Knowing the fact that I didn’t go through my whole entire computer science curriculum—like it didn’t all go to waste—I saw through working through this project that I could actually apply my skills to a real life problem. It was kind of encouraging to me in the future,” Saravia said. 

Developing the shopping app gave Xia and her group a sense of fulfillment. Xia said that often in other classes she designed apps that weren’t actually going to be used by anyone. But this class was different.

“For this we actually received much feedback to think more on the user side, to develop an app that fits their needs – like not what we think they need but what they think they need,” Xia said. As a future software engineer, Xia now thinks about that and how she could engineer things that will help underserved populations. 

Chen enjoyed the challenges that came with designing the messaging app. “I really wanted to do something meaningful because this is a final project. [Professor Zhu] really inspired us to see the outside of life.”

As a professor, Zhu has one overarching goal. “My number one priority is education. I want the students to have an awareness of the social need. And also that they get a bigger picture rather than just doing a project to earn money. So that’s my goal. It’s not my goal for the students to just find jobs. But I do want them to really see and shape a bright future for all [and see] that they have more opportunities than they thought,” Zhu said.