What were the challenges to be solved and what specific benefits were achieved?

The challenge was to develop an artificial intelligence (AI) system to assist operations and prevent human error in manufacturing.

In line with initiatives such as Industrie 4.0 in Germany and Society 5.0 in Japan, the manufacturing industry is accelerating steps towards innovating production using AI and robotics, and the automation of menial tasks.

How can the Industrie 4.0 approach be described?

The basic idea is to automate simple tasks or delegate them to robots, while more complex tasks remain in the hands of the worker. New technological processes must therefore recognize different activities of the human actors in order to support them and to prevent them from possible mistakes. DFKI and Hitachi researchers have succeeded in developing AI systems that meet these requirements.

What could be achieved?

The special strength lies in:

1. a technology that captures the objects viewed by the worker using eye tracking glasses and learns to distinguish them through the process of machine learning. Objects such as "screw" or "screwdrivers" are thus reliably detected, without impairment by background movements or other objects in the field of view.
2. a method that measures human activities using portable sensors. The resulting data is evaluated with DeepLearning and allows the recognition of single actions such as "grasping" or "twisting".
3. a “hierarchical activity-recognition model” which, on the basis of the combination of the human actions and the objects involved, points to certain workers' activities. By linking the technologies mentioned in 1 and 2, the model can recognize an activity such as "twisting a screw". This framework allows a simple recognition of the different working activities - provided that actions and objects can be trained in advance.

What can others learn from it?

With the joint research project, DFKI and Hitachi would like to promote the development of technologies that lead to an improvement in the assistance systems at the production sites, to improved guidelines for work processes and to avoidance of operator errors.