Projects

ILIAD – Intra-Logistics with Integrated Automatic Deployment: safe and scalable fleets in shared spaces

Horizon 2020 Research and Innovation Action, 2017-20.

The ILIAD project is driven by the application needs of fleets of robots that operate in intralogistics applications with a high demand on flexibility in environments shared with humans. In particular, the project aims to enable automatic deployment of a fleet of autonomous forklift trucks, which will continuously optimise its performance over time by learning from collected data. The University of Lincoln’s contributions to the project will include ensuring long-term operation of the ILIAD system, maintaining its environment representations over time while learning and predicting activity patterns of human co-workers; developing qualitative models for human-robot spatial interaction; systems architecture development and systems integration; and managing experimental work at test facilities, including the University’s National Centre for Food Manufacturing. ILIAD is a collaborative project involving four academic institutes and five industrial partners across four European countries.

Contact:  Tom DuckettGrzegorz Cielniak & Marc Hanheide

STEP2DYNA: Spatial-temporal information processing for collision detection in dynamic environments

Horizon 2020 EU.1.3.3. – Stimulating innovation by means of cross-fertilisation of knowledge

In the real world, collisions happen at every second – often resulting in serious accidents and fatalities. Autonomous unmanned aerial vehicles (UAVs) have demonstrated great potential in serving human society such as delivering goods to households and precision farming, but are restricted due to a lack of collision detection capability. The current approaches for collision detection such as radar, laser-based Ladar and GPS are far from acceptable in terms of reliability, energy consumption and size. A new type of low cost, low energy consumption and miniaturized collision detection sensors is badly needed not only to save millions of people’s lives but also make autonomous UAVs and robots safe to serve human society. The STEP2DYNA consortium proposes an innovative bio-inspired solution for collision detection in dynamic environments. It takes the advantages of low cost spatial-temporal and parallel computing capacity of visual neural systems and will realize a new chip specifically for collision detection in dynamic environments.

Contact: Shigang Yue (PI)

ENRICHME: Enabling Robot and assisted living environment for Independent Care and Health Monitoring of the Elderly

EU Horizon 2020, Research & Innovation Action, 643691

enrichmeENRICHME is a collaborative project, involving academic institutes, industrial partners and charity organizations across six European countries. It tackles the progressive decline of cognitive capacity in the ageing population proposing an integrated platform for Ambient Assisted Living (AAL) with a mobile robot for long-term human monitoring and interaction, which helps the elderly to remain independent and active for longer. The system will contribute and build on recent advances in mobile robotics and AAL, exploiting new non-invasive techniques for physiological and activity monitoring, as well as adaptive human-robot interaction, to provide services in support to mental fitness and social inclusion. Our research contribution in this project focuses in the area of robot perception and ambient intelligence for human tracking and identity verification, as well as physiological and long-term activity monitoring of the elderly at home. Primary tasks include developing novel algorithms and approaches for enabling the acquisition, maintenance and refinement of models to describe human motion behaviors over extended periods. as well as integration of the algorithms with the AAL system.

Team: Nicola Bellotto (PI), Shigang Yue (Co-I), Tryphon Lambrou (Co-I) Serhan CosarFeng ZhaoManuel Fernandez-Carmona

FLOBOT: Floor Washing Robot for Professional Users

EU Horizon 2020, Innovation Action, 645376

flobotFLOBOT is a collaborative project, involving academic institutes and industrial partners across five European countries. The project will develop a floor washing robot for industrial, commercial, civil and service premises, such as supermarkets and airports. Floor washing tasks have many demanding aspects, including autonomy of operation, navigation and path optimization, safety with regards to humans and goods, interaction with human personnel, easy set-up and reprogramming. FLOBOT addresses these problems by integrating existing and new solutions to produce a professional floor washing robot for wide areas. Our research contribution in this project is focussed in the area of robot perception, based on laser range-finder and RGB-D sensors, for human detection, tracking and motion analysis in dynamic environments. Primary tasks include developing novel algorithms and approaches for enabling the acquisition, maintenance and refinement of multiple human motion trajectories for collision avoidance and path optimization, as well as integration of the algorithms with the robot navigation and on-board floor inspection system.

Team: Nicola Bellotto (PI), Tom Duckett (Co-I), Zhi Yan

STRANDS: Spatio-Temporal Representations and Activities for Cognitive Control in Long-Term Scenarios

EU FP7 Integrating Project, 600623

STRANDS will produce intelligent mobile robots that are able to run for months in dynamic human environments. We will provide robots with the longevity and behavioural robustness necessary to make them truly useful assistants in a wide range of domains. Such long-lived robots will be able to learn from a wider range of experiences than has previously been possible, creating a whole new generation of autonomous systems able to extract and exploit the structure in their worlds.

L-CAS scientists in STRANDS are part of a European team of researchers and companies and contributes with their unique expertise in long-term mapping and behaviour generation in integrated robotic systems. Our robot developed in STRANDS is called “Linda”.

Contact: Tom Duckett & Marc Hanheide

HAZCEPT: Towards zero road accidents – nature inspired hazard perception

EU FP7-PEOPLE-2012-IRSES

temporary image for hazcept The number of road traffic accident fatalities world wide has recently reached 1.3 million each year, with between 20 and 50 million injuries being caused by road accidents. In theory, all accidents can be avoided. Studies showed that more than 90% road accidents are caused by or related to human error. Developing an efficient system that can detect hazardous situations robustly is the key to reduce road accidents. This HAZCEPT consortium will focus on automatic hazard scene recognition for safe driving. HAZCEPT will address the hazard recognition from three aspects – lower visual level, cognitive level, and drivers’ factors in the safe driving loop.

Contact: Shigang Yue (PI)

Synthesis of remote sensing and novel ground truth sensors to develop high-resolution soil moisture forecasts in China and the UK

Science and Technology Facilities Council (STFC) – Newton Fund: Earth Observation, Modelling and Autonomous Systems For Agri-Tech In China, 2016-19.

The availability of water is a key driver of agricultural productivity, and the impact of water availability on global food production is seen as a key global risk and challenge. This project seeks to develop agri-tech solutions to help alleviate the issue of water availability in agriculture, and for producers to ultimately drive water use efficiency. Currently, there is no system to measure soil moisture distribution accurately across a field, and the resolution of remote sensing has not been sufficient for agricultural applications or for local water management to reduce flood risk. The project will deploy two new sensors (one static, one mobile) within China that measure soil moisture content as a function of the albedo of cosmically generated fast neutrons (Cosmos sensor, designed by Hydroinova, US). The project coordinates the expertise of four key groups, the University of Lincoln (robotics, mapping and deployment of autonomous vehicles); the Institute of Ecology and Agrometeorology (IEAM) of Chinese Academy of Meteorological Sciences, University of Information Science &Technology; the Centre for Ecology and Hydrology (Wallingford); and the School of Geography and Earth Sciences, The University of Aberystwyth.

Contact:  Simon Pearson & Grzegorz CielniakTom Duckett

Exploiting invisible cues for robot navigation in complex natural environments

EPSRC Standard Research Scheme

Outdoor navigation in natural environments remains a challenge for robotics. Recent breakthroughs in robot navigation have been dependent on specific sensor technologies, such as laser depth sensors and GPS, and advanced image processing. The ability of animals such as ants to navigate effectively without such power- and computation- hungry systems are a proof of principle that alternative cheaper approaches are viable.  We propose to build a sensory system that gathers the full range of light cues available to the ant, in its natural ecological situation, and to analyse the information contained in this signal. We will also analyse how the specific sensor layout (ommatidia array), peripheral receptor characteristics, and the motor behaviour of the ant may contribute to extracting salient information. The data will form a test-bed for comparison of algorithmic and neural models of the processing that underlies the navigation capabilities of the ant. The final outcome of this research will be a detailed design for a small scale and low power vision system for outdoor navigation.

Contact: Michael Mangan

Active Vision with Human-in-the-Loop for the Visually Impaired

Google Faculty Research Award, Winter 2015

The research proposed in this project is driven by the need of independent mobility for the visually impaired. It addresses the fundamental problem of active vision with human-in-the-loop, which allows for improved navigation experience, including real-time categorization of indoor environments with a handheld RGB-D camera. This is particularly challenging due to the unpredictability of human motion and sensor uncertainty. While visual-inertial systems can be used to estimate the position of a handheld camera, often the latter must also be pointed towards observable objects and features to facilitate particular navigation tasks, e.g. to enable place categorization. An attention mechanism for purposeful perception, which drives human actions to focus on surrounding points of interest, is therefore needed. This project proposes a novel active vision system with human-in-the-loop that anticipates, guides and adapts to the actions of a moving user, implemented and validated on a mobile device to aid the indoor navigation of the visually impaired.

Team: Nicola Bellotto (PI), Grzegorz Cielniak (Co-I), Jaycee Lock

3D Vision-based Crop-Weed Discrimination for Automated Weeding Operations

BBSRC and Innovate UK (Agri-Tech Catalyst, Early Stage Feasibility)

This project will investigate the technical foundations for the next generation of robotic weeding machinery, enabling selective and accurate treatment of specific weeds. The proposed technology is a novel combination of low-cost 3D sensing and learning software together with a suitable weed destruction technique. The proposed developments will lead to more efficient cultural weeding equipment resulting in better management of weeds and reduced input use, bringing several benefits to food producers, sellers and society. The technical objectives of the project include detection of plants using a low-cost 3D camera vision system, discrimination of target crop plants from weeds at different growth stages, providing an intuitive system training interface for rapid deployment, development of a proof-of-concept weed destruction technique and finally integration and evaluation of the developed technology in automated weeding products.

Contact:  Grzegorz Cielniak & Tom Duckett & Simon Pearson

3D Vision Assisted Robotic Harvesting of Broccoli

BBSRC and Innovate UK (Agri-Tech Catalyst, Early Stage Feasibility)

broccoli projectThere is an urgent need to reduce the costs of production of field brassica crops, in particular, broccoli. Labour costs are a significant proportion of overall production costs. High labour usage also drives complex management and potentially social issues. In this project, we will test whether 3D camera technology can be used to identify and select broccoli which is ready to harvest within commercial crops. This will provide a key underpinning step towards the development of a fully automatic and camera guided robotic harvesting system for broccoli. The commercial benefits are highly significant, as the broccoli crop is one of the worlds largest vegetable crops, and almost all of it is manually harvested.

Contact: Tom Duckett & Grzegorz Cielniak & Simon Pearson

Autonomous Field Rover for Agricultural Research

Research Investment Fund (RIF), University of Lincoln

This project targets the challenge of developing technology for autonomous soil sampling, extending the state of the art for soil quality assessment and correlating measured soil variability with crop yield. We therefore propose to develop a novel autonomous vehicle for the agri-industry which can amalgamate multiple sensors to develop highly precise soil fertility maps. This cross-disciplinary project is based within the newly formed Lincoln Institute for Agri-Food Technology and includes contributions from the Schools of Computer Science, Engineering, Life Sciences and the National Centre for Food Manufacturing at the University of Lincoln.

Contact:  Grzegorz Cielniak (PI)

FInCoR: Facilitating Individualised Collaboration with Robots

Research Investment Fund (RIF), University of Lincoln

The FInCoR project will investigate novel ways to facilitate individualised Human-Robot Collaboration through long-term adaptation on the level of joint tasks. This will enable robots to work with human more effectively in scenario such as high value manufacturing and assistive care. Imagine a robot helping to assemble a car’s dashboard more effectively, knowing that it is working with a left-handed person; or a robot assisting an elderly employee in a car factory who is skilled in fitting a speedometer, but requires a third-hand holding the heavy mounting frame in place. Despite significant progress in human-robot collaboration, today’s robotic systems still lack the ability to adjust to an individual’s needs. FInCoR will overcome this limitation by developing online, in-situ adaptation, putting the “human in the loop”. It will bring together flexible task representations (e.g. Markov Decision Processes), machine learning (e.g. reinforcement learning), advanced robot perception (e.g. body tracking), and robot control (e.g. reactive planning) to make progress from pre-scripted tasks to individualised models. These models account for preferences, abilities, and limitations of each individual human through long-term adaptation. Hence, FInCoR will enable processes known from human-human collaboration, such as two colleagues working together and learning more about each other’s strengths, preferences, and strategies, to take place in human-robot teams.

Contact: Marc Hanheide & Peter Lightbody

Mobile Robotics for Ambient Assisted Living

Research Investment Fund (RIF), University of Lincoln

The life span of ordinary people is increasing steadily and many developed countries, including UK, are facing the big challenge of dealing with an ageing population at greater risk of impairments and cognitive disorders, which hinder their quality of life. Early detection and monitoring of human activities of daily living (ADLs) is important in order to identify potential health problems and apply corrective strategies as soon as possible. In this context, the main aim of the current research is to monitor human activities in an ambient assisted living (AAL) environment, using a mobile robot for 3D perception, high-level reasoning and representation of such activities. The robot will enable constant but discrete monitoring of people in need of home care, complementing other fixed monitoring systems and proactively engaging in case of emergency. The goal of this research will be achieved by developing novel qualitative models of ADLs, including new techniques for 3D sensing of human motion and RFID-based object recognition. This research will be further extended by new solutions in long-term human monitoring for anomaly detection.

Contact: Nicola Bellotto (PI), Tom Duckett & Serhan Cosar (Co-Is), Claudio Coppola

RFID-based Object Localization with a Mobile Robot to Assist the Elderly with Mild Cognitive Impairments

Internally funded, University of Lincoln

Solutions for Active and Assisted Living (AAL) like smart sensors and assistive robots are being developed and tested to help the elderly at home. However, it is still very difficult for a robot to understand the context in which its services are supposed to be delivered, even for simple tasks like finding the keys left by a person somewhere in the house. These are practical problems affecting the life of many elderly people, in particular those suffering Mild Cognitive Impairments (MCI). Radio-frequency identification (RFID) technology is a viable option for locating objects in domestic environments. The main aim of this research is therefore to develop a system for a mobile robot to detect and map the position of RFID-tagged objects, exploiting its capability to explore the house and sense the environment from multiple locations. The system will be evaluated in the lab and in real elderly houses.

Contact: Nicola Bellotto (PI), Grzegorz Cielniak & Manuel Fernandez-Carmona (Co-Is), George Broughton

Autonomous Conversational Systems

BaxterPlayingGamesWe investigate interactive machines that perceive, act, communicate, and learn. This is motivated by the fact that communicating with multiple modalities such as speaking, touching and pointing is a natural and efficient form of interaction among humans. Our approach is three-fold. First, we treat (multimodal) perception, action and communication jointly rather than independently. Second, we aim to increase the autonomy of intelligent conversational systems by reducing the amount of human intervention (development-wise) in order to train machines from example interactions rather than contrived for the purpose of system training. Third, we use conversational systems as a test bed in order to challenge machine learning algorithms in their application to real-world systems that can operate in realistic scenarios — beyond lab environments. Example applications include robots playing games and personal assistants providing information in hands-free scenarios.

Contact: Heriberto Cuayahuitl (PI)

Cognitive Social Robotics

Internally funded, University of Lincoln  &  NVidia hardware grant

Cognitive Social Robots in the Real WorldTo be useful for the real world, robots will need to be social to effectively interact and help people. The robots should take into account the cognitive abilities and limitations of their human partners, just as people do when interacting with others. To do so involves the combined application of artificial intelligence, cognitive science, psychology, and robotics. This research develops autonomous, cognitively capable, and social robots to interact with people in real environments to achieve measurable benefits. Evaluation work is conducted in the real-world wherever possible, in the domains of education, healthcare, and other collaborative applications.

Contact: Paul Baxter (PI)

Completed Projects

LIVCODE: Life like information processing for robust collision detection

EU FP7-PEOPLE-2011-IRSES, Coordinator

Animals are especially good at collision avoidance even in a dense swarm. In the future, every kind of man made moving machine, such as ground vehicles, robots, UAVs aeroplanes, boats, even moving toys, should have the same ability to avoid collision with other things, if a robust collision detection sensor is available. The six partners of this EU FP7 project from UK, Germany, Japan and China will further look into insects visual pathways and take inspirations from animal vision systems to explore robust embedded solutions for vision based collision detection for future intelligent machines.

Contact: Shigang Yue (PI)

Trainable Vision‐based Anomaly Detection and Diagnosis (TADD)

Technology Strategy Board funded Technology Inspired CR&D – ICT Project

Market demand for automation of food processing and packaging is increasing, leading to a demand for increased automation of industrial quality control (QC) procedures. This project is developing a multi‐purpose intelligent software technology using computer vision and machine learning for automatic detection and diagnosis of faulty products, including raw, processed and packaged food products. The developed vision systems are user-trainable, requiring minimal set‐up to work with a wide variety of products and processes. The technology will be refined and evaluated by testing in automated QC equipment in the food industry.

Contact: Tom Duckett

EYE2E: Building visual brains for fast human machine interaction

In the real world, many animals pocess almost perfect sensory systems for fast and efficient interactions within dynamic environments. Vision, as an evolved organ, plays a significant role in the survival of many animal species. The mechanisms in biological visual pathways provide nice models for developing artificial vision systems. The four partners of this consortium will work together to explore biological visual systems in both lower and higher level by modelling, simulation, integration and realization in chips, to investigate fast image processing methodologies for human machine interaction through VLSI chip design and robotic experiments.

Contact: Shigang Yue

Autonomous Control of Agricultural Sprayers

Technology Strategy Board funded CR&D Feasibility Study

This project aims to improve the efficiency of agricultural spraying vehicles by developing a robust control system for the height of the spraying booms, using laser sensing to model the 3d structure of the crop canopy and terrain ahead of the vehicle. This new technology will enable greater autonomy in agricultural sprayers, enhance and simplify interaction between the driver and the vehicle, and result in an optimised spraying process.

Contact: Grzegorz Cielniak, Tom Duckett

Vision-based Identification of Beetle Damage in Field Beans

Technology Strategy Board funded CR&D Feasibility Study

The project aims to analyse field bean produce intended for human consumption for the larval damage caused by, and presence of adult Bruchus rufimanus (bean seed beetle). This will be achieved by employing state-of-the-art computer vision and machine learning techniques. The study will also consider the potential to develop hand-held instruments using the proposed technology that can be employed for rapid analysis of damage and insect presence.

Contact: Grzegorz CielniakTom Duckett