Enlarge

What challenges had to be solved?

Making new, demanding customer requirements such as customized products available in a short time is what drives the technological developments of the smart factory.
In manufacturing, the trend is clearly towards lot size 1, which means that workers need assembly instructions more often and must document work steps for quality assurance.
The search is therefore on for methods, systems and technologies that support the worker in producing individual series quickly and without errors according to customer requirements. Meanwhile, gaze and gesture control is finding its way into the hall floor, as it is a robust technology that supports natural movement sequences and can be learned quickly. Ergonomic operation is important for the acceptance of assistance systems.

What concrete benefits have been achieved?

Since the operation by means of glances and gestures does not require a computer mouse to be moved or pressed onto a touch screen, the actual work process is not interrupted, the worker remains at the workpiece, can keep gloves on and has less walking distances.
Image recognition and machine learning support the quality assurance process.
By logging the digitized process, a digital twin is automatically generated and enriched.

Highlight of the application

"Gaze and gesture control in the process of worker guidance in conjunction with artificial intelligence can lead to efficiency gains here - in the office area these applications already show an increase of around 12 percent, why should this not be feasible on the hall floor?
Steffen Himstedt, Trebing + Himstedt.

How can the Industrie 4.0 approach be described?

All in all, the application shows a paperless assembly process in production, which is electronically guided by work instructions and supported in execution by start-up technologies in order to document work progress without contact. The process is digitally monitored in the background and automatically locked in case of errors. All work processes are electronically documented and the information on the digital twin's CV file is stored. This Digital Twin can be exchanged with customers of the physical product.
In times of increasingly individualized products (lot size 1), methods, systems and technologies that support the worker in producing individual series quickly and without errors according to customer requirements could be demonstrated in an assembly process.
The digital twin is automatically enriched in the background with the "as built" information and made available in the cloud for further use.

Artificial intelligence using machine learning supports the human eye in quality control and thus helps to overlook errors.
What has been achieved in concrete terms?
Energy-autonomous data acquisition over a period of 5 - 10 years with derived safe preventive maintenance. Avoidance of rejects, unplanned machine downtimes and reduction of tool costs.
Customers such as Continental or BMW now have part of their tool management of their worldwide locations in the eltimon cloud.

What measures were used to achieve the solution?

Only a standard eye tracker such as Tobii Eye-Tracker and the software Nuia Productivity+ from 4tiitoo is required for gaze control. The eye-tracker emits a weak infrared light which is reflected by the eyes of the user and calculates the direction of gaze. To get started, a short calibration of the user is sufficient, which takes less than 30 seconds. In principle, the software is placed transparently over the program to be controlled without changing it. The software recognizes where buttons and menu bars are arranged on the underlying user interface.
For gesture control, a wristband from Kinemic, which contains sensors for acceleration, position and rotation, provides feedback from the worker to the system and is acknowledged by the wristband using vibrations. The appropriate software now translates the movements into function controls of the program to be operated. The wristband with a battery life of approx. 10 to 12 hours communicates with the computer via the standard Bluetooth interface.
For quality assurance, an image of the end product is taken and, using machine learning algorithms of reference images, a decision is made as to whether the end product is OK or NOK.

What can others learn from this?

In production, the trend is clearly towards lot size 1, which means that workers often need assembly instructions and must document work steps for quality assurance. Gaze and gesture control in the worker guidance process can lead to efficiency gains here - in the office area, applications show an increase of around 12 percent.