A human-machine interface

More than 16 million tonnes of waste paper are processed each year in Germany,’ said Philip Klippel of Munich University at the biannual Sensor-Based Sorting congress in Aachen.  This article is part of a tech series, in this case highlighting the most promising breakthrough technologies presented at the 2016 edition of the two-day event. 

Klippel cited a ‘huge variety’ of visual features in this waste stream and had looked to near-infrared spectroscopy to ‘break the curse of dimensionality’. Trials successfully classified 10 different types of paper while managing a 78% recognition rate by taking 175 scans per second.

‘A realistic view’

Almost 4 million samples of dry paper and cardboard were analysed in the sensor-based sorting project overseen by Klippel. The material was sorted at an operational German recycling facility with a capacity of 130 000 tonnes per year in order to form ‘a realistic view’ of the results. The system used 172 near-infrared tracks and 1204 RGB (Red-Green-Blue) tracks at 175 scans per second and at a conveyor belt speed of 0.5 metres per second, covering a width of around 90 cm.

‘Our classifier implementation system is a classification and regression tree (CART) that allows a ranking of features by importance,’ said Klippel, who tested both coated and uncoated samples. Overall results were corrected for easy recognition of the background (conveyor belt) so that the figures would not be ‘too optimistic’. Klippel and his team now want to extend the method to other waste streams, especially plastics.

A matter of purity

‘I’m sure that all recyclers will agree that when you get 95% purity, you naturally want 96% or 97%, but getting the most out of your materials is tricky and, not least, expensive,’ Dr Aude Maitrot of resource management specialist Veolia told delegates in Aachen. ‘So Veolia wanted to investigate how to refine the results without having anyone touch the material.’

She detailed the success of the I-SORT3R project, which centres around a semi-automatic device that allows operators to sort waste without touching it, thus taking employees out of a ‘smelly, dusty and potentially dangerous environment’.

One such unit has been in operation at a material recovery facility in Amiens, France, since September 2013. The process is based on a vision system that provides an image displayed on a touchscreen on which ‘intruders’ from a predominant waste stream can be selected.

People vs. machine?

‘Air nozzles eject selected objects automatically,’ Maitrot explained. A line-scan camera with a specific lighting system was designed in order to ensure a ‘realistic’ visual rendering of the infeed. The material is treated according to Veolia’s patented sequential adaptive loop technology (SALTO), enabling the sorting of different streams on a single conveyor.

So what makes I-SORT3R special? For one thing, it differs from other sorting solutions such as near-infrared spectroscopy and multisensory technologies by virtue of the fact that it only has automated extraction. ‘This means it preserves the human ability to recognise objects that other sensors did not pick up on, restoring people as the missing link in the chain of quality control,’ said Maitrot. She labelled this a ‘human-machine interface’.

Touch screen control

At the facility in Amiens, an image of the materials is displayed on the touchscreen monitor for four seconds. Veolia opted for static images instead of a video stream as this allows for more freedom of spatial organisation. The duration was determined in relation to conveyor speed, width, length and material throughput. ‘Each image represents a portion of the belt,’ Maitrot noted.

She continued: ‘In our project, the customised lighting system ensures that transparent plastics are seen as such, not black – which is, of course, the colour of the belt underneath. We experimented to get the visual rendering just right, not underexposed or overexposed. The colour saturation displays the material purity.’

Picking precision

‘An image processing algorithm is imbedded in I-SORT3R software,’ she told delegates. It handles object segmentation plus detection of isolated and overlapping objects. The system executes these commands before touchscreen operation, giving the operator the opportunity to appreciate the scenario fully.

Picking precision is achieved thanks to image segmentation by computing the intersection of a disk centred on the touched point and the nearest object surface. ‘The average productivity of those working with the system was measured at 4230 clicks per hour,’ Maitrot pointed out.

A recent French norm sets the average for manual labour at roughly 2000 object selections per hour when factoring in prevention of musculoskeletal disorder. Ultimately, Veolia is confident that remotely operated sorting is ‘opening the way to a new generation of sorting centres’.

Best results with LIBS

‘Most recyclers in Japan have installed a variety of machines to recover iron and the mixed non-ferrous metals – however, this mixed scrap is not separated,’ lamented Professor Dr Shuji Owada of Waseda University. ‘No effective automated processes have been developed for this purpose, while labour costs have increased over the last few years.’

He identified laser-induced breakdown spectroscopy (LIBS) as the best analytical method to achieve mutual separation of mixed scrap metal.

In February, Waseda University and several research institutes installed LIBS sorters made by German manufacturer SECOPTA at Japanese recycling facilities as part of a national e-scrap programme. Efforts focused on recovering minor rare metals from printed circuit boards, such as tantalum, as well as high-purity copper plate.

The sorting machines adopt a Y-scan system; analyses were carried out on the copper samples by changing laser pulse energy from 3 to 160 μJ. ‘In order to evaluate the effect on analysis accuracy, we calculated electron density, average intensity and plasma temperature, and measured the peak to background (P/B) ratio of the copper obtained by 10 LIBS readings,’ Owada noted.

High detection probabilities

Setting an appropriate plasma detection delay time is deemed ‘vital’ in acquiring an accurate elemental analysis. ‘We saw that the maximum intensity was obtained at 0.5 μs of the delay time and that the maximum P/B ratio was reached at 8 μs,’ said Owada. The settings vary depending on the metals to be detected.

Even a slight shift of 1 to 2 mm in the laser focal position ‘considerably affected’ the analysis accuracy. Owada advised using a mechanism to fit the focal position onto the flat surface of the sample and at the respective height of the samples running on the conveyor belt.

etection probabilities for all the standard samples were as high as 100%. ‘The minimum length of sample material for optimum detection is 0.2 mm at 0.5 m/s belt speed,’ Owada stated. ‘As a next step, we are now focusing specifically on sintered tantalum from used electronics.’

Taking on black plastics

Black plastics or multi-layered products in the waste stream are ‘virtually unsortable’ with today’s sorting solutions, pointed out Christian Brandt of Germany’s Fraunhofer Institute. Three of the institute’s bodies have partnered in a bid to overcome this issue. ‘As a first step, we have developed a frequency domain line-scan camera working in the terahertz (THz) range with frequencies below 300 GHz,’ Brandt stated.

Using mainly plastics derived from end-of-life cars, plastic flakes on a conveyor belt have been exposed to a THz sensor as well as an RGB line camera and pneumatic ejection. The material yielded approximately 10 points measured per flake, and a total of 35 000 during the entire trial. This translates into a 97% precision rate in sorting ABS and a 96% rate for polypropylene and polyethylene.

Better performance expected

One problem encountered was that the shape of the samples analysed by the THz sensor alone was ‘too coarse to allow precise ejection’, Brandt admitted. ‘The more compact the material flow, the bigger the problem.’

Furthermore, the material may be only partially visible at times. ‘Luckily, the data fusion of the THz and the RGB image enabled us to recover the shape of the objects with high spatial resolution,’ Brandt noted. By using improved next-generation sensors with a ‘higher dynamic range’ as well as more different data for training and testing, the research team expects a ‘much better performance’ in harder separation tasks.

‘Mechanically articulated bridge’

‘Optical sorting systems may appear as the logical future technology for waste treatment, but analysis of possible ejection configurations proves that a one and only ejection technology does not exist,’ argued Winfrid Rauch of Manufacture à Besançon (MaB). While the various contemporary machines available have their own advantages and disadvantages, ‘a new ejection principle can contribute to better sorting performances’, Rauch stressed.

The new method developed by MaB can be described as a ‘mechanically articulated bridge’. The patented system is characterised by a downstream orientated bridge device; the bridge elements are opened when the optical sensor recognises a chosen item – for example, newspaper rather than plastic bottles.

This allows the machine to remain closed when it encounters what it is programmed to let pass – say, plastic bottles – so that they fall onto the conveyor belt directly below the screening platform. The desired materials – that is, newspapers in the test project – are re-routed via the now open arm of the system and fall onto a different conveyor belt.

Smooth path achieved

‘It is important that there is a seamless connection between the screening area and the bridge arms so that no material can get stuck in the cracks,’ Rauch noted. ‘We have achieved a smooth path to the conveyor belt in our pilot, so we know this is definitely possible.’

While a pneumatic ejection system has to ‘shoot’ on to a gravity centre to optimise the removal movement and while a robot hand has to ‘know’ the item form in order to grab it effectively, neither is required for the mechanically articulated bridge, Rauch observed.

This new solution would be a suitable alternative for sorting materials of a limited height and weight, and also materials that are relatively wet or have a rough surface, according to the speaker. 

The profitability and maximum efficiency of the new system would be proven, he added, once tests confirmed the time needed to move and retract the bridge arms, together with the average energy consumption. The latter, however, wouldn’t be too high as the only energy needed is the indirect help afforded by an electrical or hydraulic cylinder opening/closing the individual bridge sections.

‘Bag of visual words’

‘Classification is a core task when sorting bulk materials, leaving objects to be either accepted or rejected based on features as programmed by the engineer,’ said Matthias Richter of the Fraunhofer Institute. ‘While successful in the past, this approach is reaching its limits. More challenging tasks and complex objects call for automated methods to learn how to interpret the data intelligently via precise pattern recognition methods.’

Typically, the engineer has to manually input the pre-defined feature descriptors of each kind of material – such as colour, shape, length, etc. With less clearly defined classes combined with growing variety in the appearance of an object, the manual approach becomes ‘less and less feasible’.

A number of sorting approaches focus exclusively on colour such that broad application in different industrial settings is ‘questionable’, Richter stated. An automatic derivation of object descriptors and classification rules would therefore ‘clearly be an improvement’.

The suggested approach is based on the ‘bag of visual words’ (BOW) framework and is driven by the insight that each existing document carries words, of which some words give more valuable information than others.

‘Nearly perfect classification’

‘The key idea is to consider an image to be composed of visual words, in which the data clusters form a so-called vocabulary,’ Richter explained. ‘Then, images (material readings) can be classified by finding out which of and how often these descriptions occur.’

The imaging part of the sorting system consisted of an RGB line camera producing a resolution of 170 µm x 170 µm per pixel. The overall inspection width was 700 mm while halogen lamps were used for illumination.

During the first two rounds, two types of stone were fed through the system separately, according to ‘dense sampling’ which considers a key point to extract the low-level local descriptors, yielding a total of 5883 images.

The work produced ‘nearly perfect classification’, Richter told his audience in Aachen. Features could be extracted in less than 90 ms per sample once the vocabulary had been determined. ‘The bulk of the processing time was spent in computing every single object pixel,’ said the researcher. ‘Clearly, the results are far too slow for a real-time application, at least for the moment. Our system is a prototype that has not yet been optimised for speed.’

The main objective now is to balance the supply of discriminative information with a much more efficient computational load. The Fraunhofer team hopes that trial and error will further boost the results to fit the time slot of 15 ms for all objects between inspection line and ejection stage.

SECOPTA: quick and reliable sorting

In Aachen, Christian Bohling of Germany-based SECOPTA presented the new MopaLIBS elemental analyser, which determines the atomic composition of materials supplied via a multi-element analysis of the spectral information.

‘Robust’ measuring lasers with repetition rates of 20-100 kHz work with a constant laser power. Owing to the short measuring time of small objects on conveyor belts, speeds of up to 3 metres per second are possible. Scans can be made in a single line on the belt, over the whole width of the belt and with an extra pre-ablation laser to clean the material prior to separation.

‘By default, all individual measurements that are recorded will be averaged,’ Bohling pointed out to delegates. The average spectrum is then analysed in accordance with the determined concentrations for the desired items and assigned to a pre-defined type of material.

The sorting of aluminium in the alloy groups 1000 to 7000 ‘used to not be very practical’ whereas using the MopaLIBS system allows for quick and reliable sorting into ‘tight’ specification classes. Another optimised application is sorting low-alloy steel scrap.

Read other articles from this 2016 magazine right here!