Having an employee command a responsive robot to move blocks around a table would seem to be a relatively simple task.

In reality, it’s not.

For instance, which of three types of colored blocks does the robot pick up? Better yet, which of four different type of blue blocks is needing to be moved? And, finally, where does the particular block need to be moved?

That’s the challenging task four computer science, software engineering and mathematics students have undertaken this year in a project that will be featured among papers presented at the 2019 Association for the Advancement of Artificial Intelligence’s Spring Symposium on Autonomous Machines and Human Awareness, taking place March 25-27 at Stanford University.

Project faculty mentor Michael Wollowski, associate professor of computer science and software engineering, points out that the students’ work is being showcased alongside technology that’s on the cutting edge of artificial intelligence, commonly known as AI.

“We were invited, and gladly accepted, to turn our paper into a book chapter, to be published,” says the professor. “I’m delighted with the students’ work on this project. When I inform colleagues about the work that’s been done, they’re amazed that this has been accomplished by students at the undergraduate level. Most of the work in AI is being done by students in graduate- or doctorate-degree programs.”

Finally, Wollowski states, “My weekly meetings with these students are the highlight of my week on campus. They’re smart and highly creative. This was a unique opportunity for them. They eagerly accepted the challenge and came up with some really innovative things.”

The current approach to industrial automation is to rely entirely on an autonomous robot to perform repetitive tasks. However, an automated system makes it difficult to adapt to challenging situations that emerge during the course of production. A human employee has the ingenuity to adapt to these challenges, but can’t necessarily match the endurance or precision of a robot in accomplishing these tasks.

The students have taken steps to develop a human-robot collaboration system within a shared space, combining the best features of human reasoning and knowledge with the robot’s lack of physical fatigue.

Team member Michael Crowell demonstrated a prototype of the system during the Department of Computer Science & Software Engineering’s 2019 Senior Projects Expo in the final week of the winter academic quarter.

“If a worker wants the robot to pick up a particular orange block, the human points at it and provides a command. Then, the robot, through our system reads where the worker is pointing and decides which block to pick up and move,” says Crowell, who is majoring in computer science, software engineering and mathematics. “If the robot decides to pick up the wrong block, because there are two orange blocks in the same space, the worker just gives another command with more precise information. It’s the worker and robot working together.”

The students’ system utilizes a series of mathematical algorithms to determine a shared understanding between the worker’s instructions, from simply pointing a finger, and where the block is located on a table.

Then, more information is gathered in order to find out where to move the block to its desired location. “It has been a complicated process,” admits computer science and mathematics senior Adit Suvarna. “Research goals are ever-changing. Every time we thought we had something done, we found something else. We’re still finding things to do in order to improve things. It’s a never-ending process.”

Other members of the team are computer science, mathematics and software engineering students Sheng Dong and Walter Panfil.