## Publications

### 2019

• N. Bellotto, S. Cosar, and Z. Yan, “Human detection and tracking,” in Encyclopedia of robotics, M. H. Ang, O. Khatib, and B. Siciliano, Eds., Springer, 2019.

In robotics, detecting and tracking moving objects is key to implementing useful and safe robot behaviours. Identifying which of the detected objects are humans is particularly important for domestic and public environments. Typically the robot is required to collect environmental data of the surrounding area using its on-board sensors, estimating where humans are and where they are going to. Moreover, robots should detect and track humans accurately and as early as possible in order to have enough time to react accordingly

@incollection{lirolem30916,
editor = {M. H. Ang and O. Khatib and B. Siciliano},
month = {July},
booktitle = {Encyclopedia of robotics},
year = {2019},
title = {Human detection and tracking},
publisher = {Springer},
author = {Nicola Bellotto and Serhan Cosar and Zhi Yan},
abstract = {In robotics, detecting and tracking moving objects is key to implementing useful and safe robot behaviours. Identifying which of the detected objects are humans is particularly important for domestic and public environments.
Typically the robot is required to collect environmental data of the surrounding area using its on-board sensors, estimating where humans are and where they are going to. Moreover, robots should detect and track humans accurately and as early as possible in order to have enough time to react accordingly},
keywords = {ARRAY(0x56147f6a8138)},
url = {http://eprints.lincoln.ac.uk/30916/}
}

### 2018

• R. Akrour, A. Abdolmaleki, H. Abdulsamad, J. Peters, and G. Neumann, “Model-Free Trajectory-based Policy Optimization with Monotonic Improvement,” Journal of Machine Learning Research (JMLR), 2018.

Many of the recent trajectory optimization algorithms alternate between linear approximation of the system dynamics around the mean trajectory and conservative policy update. One way of constraining the policy change is by bounding the Kullback-Leibler (KL) divergence between successive policies. These approaches already demonstrated great experimental success in challenging problems such as end-to-end control of physical systems. However, these approaches lack any improvement guarantee as the linear approximation of the system dynamics can introduce a bias in the policy update and prevent convergence to the optimal policy. In this article, we propose a new model-free trajectory-based policy optimization algorithm with guaranteed monotonic improvement. The algorithm backpropagates a local, quadratic and time-dependent Q-Function learned from trajectory data instead of a model of the system dynamics. Our policy update ensures exact KL-constraint satisfaction without simplifying assumptions on the system dynamics. We experimentally demonstrate on highly non-linear control tasks the improvement in performance of our algorithm in comparison to approaches linearizing the system dynamics. In order to show the monotonic improvement of our algorithm, we additionally conduct a theoretical analysis of our policy update scheme to derive a lower bound of the change in policy return between successive iterations.

@article{lirolem32457,
year = {2018},
title = {Model-Free Trajectory-based Policy Optimization with Monotonic Improvement},
journal = {Journal of Machine Learning Research (JMLR)},
author = {R. Akrour and A. Abdolmaleki and H. Abdulsamad and J. Peters and Gerhard Neumann},
publisher = {Journal of Machine Learning Research},
abstract = {Many of the recent trajectory optimization algorithms alternate between linear approximation
of the system dynamics around the mean trajectory and conservative policy update.
One way of constraining the policy change is by bounding the Kullback-Leibler (KL)
divergence between successive policies. These approaches already demonstrated great experimental
success in challenging problems such as end-to-end control of physical systems.
However, these approaches lack any improvement guarantee as the linear approximation of
the system dynamics can introduce a bias in the policy update and prevent convergence
to the optimal policy. In this article, we propose a new model-free trajectory-based policy
optimization algorithm with guaranteed monotonic improvement. The algorithm backpropagates
a local, quadratic and time-dependent Q-Function learned from trajectory data
instead of a model of the system dynamics. Our policy update ensures exact KL-constraint
satisfaction without simplifying assumptions on the system dynamics. We experimentally
demonstrate on highly non-linear control tasks the improvement in performance of our algorithm
in comparison to approaches linearizing the system dynamics. In order to show the
monotonic improvement of our algorithm, we additionally conduct a theoretical analysis of
our policy update scheme to derive a lower bound of the change in policy return between
successive iterations.},
keywords = {ARRAY(0x56147f78b3d8)},
url = {http://eprints.lincoln.ac.uk/32457/}
}
• A. Binch, N. Cooke, and C. Fox, “Rumex and Urtica detection in grassland by UAV,” in 14th International Conference on Precision Agriculture, 2018.

. Previous work (Binch & Fox, 2017) used autonomous ground robotic platforms to successfully detect Urtica (nettle) and Rumex (dock) weeds in grassland, to improve farm productivity and the environment through precision herbicide spraying. It assumed that ground robots swathe entire fields to both detect and spray weeds, but this is a slow process as the slow ground platform must drive over every square meter of the field even where there are no weeds. The present study examines a complimentary approach, using unmanned aerial vehicles (UAVs) to perform faster detections, in order to inform slower ground robots of weed location and direct them to spray them from the ground. In a controlled study, it finds that the existing state-of-the-art (Binch & Fox, 2017) ground detection algorithm based on local binary patterns and support vector machines is easily re-usable from a UAV with 4K camera despite large differences in camera type, distance, perspective and motion, without retraining. The algorithm achieves 83-95\% accuracy on ground platform data with 1-3 independent views, and improves to 90\% from single views on aerial data. However this is only attainable at low altitudes up to 8 feet, speeds below 0.3m/s, and a vertical view angle, suggesting that autonomous or manual UAV swathing is required to cover fields, rather than use of a single high-altitude photograph. This demonstrates for the first time that combined aerial detection with ground spraying system is feasible for Rumex and Urtica in grassland, using UAVs to replace the swathing and detection of weeds then dispatching ground platforms to spray them at the detection sites (as spraying by UAV is illegal in EU countries). This reduces total time requires to spray as the UAV performs the survey stage faster than a ground platform.

@inproceedings{lirolem31363,
publisher = {14th International Conference on Precision Agriculture},
author = {Adam Binch and Nigel Cooke and Charles Fox},
year = {2018},
title = {Rumex and Urtica detection in grassland by UAV},
month = {June},
booktitle = {14th International Conference on Precision Agriculture},
keywords = {ARRAY(0x56147f73abc0)},
url = {http://eprints.lincoln.ac.uk/31363/},
abstract = {. Previous work (Binch \& Fox, 2017) used autonomous ground robotic platforms to successfully detect Urtica (nettle) and Rumex (dock) weeds in grassland, to improve farm productivity and the environment through precision herbicide spraying. It assumed that ground robots swathe entire fields to both detect and spray weeds, but this is a slow process as the slow ground platform must drive over every square meter of the field even where there are no weeds. The present study examines a complimentary approach, using unmanned aerial vehicles (UAVs) to perform faster detections, in order to inform slower ground robots of weed location and direct them to spray them from the ground. In a controlled study, it finds that the existing state-of-the-art (Binch \& Fox, 2017) ground detection algorithm based on local binary patterns and support vector machines is easily re-usable from a UAV with 4K camera despite large differences in camera type, distance, perspective and motion, without retraining. The algorithm achieves 83-95\% accuracy on ground platform data with 1-3 independent views, and improves to 90\% from single views on aerial data. However this is only attainable at low altitudes up to 8 feet, speeds below 0.3m/s, and a vertical view angle, suggesting that autonomous or manual UAV swathing is required to cover fields, rather than use of a single high-altitude photograph. This demonstrates for the first time that combined aerial detection with ground spraying system is feasible for Rumex and Urtica in grassland, using UAVs to replace the swathing and detection of weeds then dispatching ground platforms to spray them at the detection sites (as spraying by UAV is illegal in EU countries). This reduces total time requires to spray as the UAV performs the survey stage faster than a ground platform.}
}
• P. Bosilj, T. Duckett, and G. Cielniak, “Connected attribute morphology for unified vegetation segmentation and classification in precision agriculture,” Computers in Industry, vol. 98, pp. 226-240, 2018.

Discriminating value crops from weeds is an important task in precision agriculture. In this paper, we propose a novel image processing pipeline based on attribute morphology for both the segmentation and classification tasks. The commonly used approaches for vegetation segmentation often rely on thresholding techniques which reach their decisions globally. By contrast, the proposed method works with connected components obtained by image threshold decomposition, which are naturally nested in a hierarchical structure called the max-tree, and various attributes calculated from these regions. Image segmentation is performed by attribute filtering, preserving or discarding the regions based on their attribute value and allowing for the decision to be reached locally. This segmentation method naturally selects a collection of foreground regions rather than pixels, and the same data structure used for segmentation can be further reused to provide the features for classification, which is realised in our experiments by a support vector machine (SVM). We apply our methods to normalised difference vegetation index (NDVI) images, and demonstrate the performance of the pipeline on a dataset collected by the authors in an onion field, as well as a publicly available dataset for sugar beets. The results show that the proposed segmentation approach can segment the fine details of plant regions locally, in contrast to the state-of-the-art thresholding methods, while providing discriminative features which enable efficient and competitive classification rates for crop/weed discrimination.

@article{lirolem31634,
pages = {226--240},
author = {Petra Bosilj and Tom Duckett and Grzegorz Cielniak},
publisher = {Elsevier},
journal = {Computers in Industry},
year = {2018},
volume = {98},
title = {Connected attribute morphology for unified vegetation segmentation and classification in precision agriculture},
abstract = {Discriminating value crops from weeds is an important task in precision agriculture. In this paper, we propose a novel image processing pipeline based on attribute morphology for both the segmentation and classification tasks. The commonly used approaches for vegetation segmentation often rely on thresholding techniques which reach their decisions globally. By contrast, the proposed method works with connected components obtained by image threshold decomposition, which are naturally nested in a hierarchical structure called the max-tree, and various attributes calculated from these regions. Image segmentation is performed by attribute filtering, preserving or discarding the regions based on their attribute value and allowing for the decision to be reached locally. This segmentation method naturally selects a collection of foreground regions rather than pixels, and the same data structure used for segmentation can be further reused to provide the features for classification, which is realised in our experiments by a support vector machine (SVM). We apply our methods to normalised difference vegetation index (NDVI) images, and demonstrate the performance of the pipeline on a dataset collected by the authors in an onion field, as well as a publicly available dataset for sugar beets. The results show that the proposed segmentation approach can segment the fine details of plant regions locally, in contrast to the state-of-the-art thresholding methods, while providing discriminative features which enable efficient and competitive classification rates for crop/weed discrimination.},
url = {http://eprints.lincoln.ac.uk/31634/},
keywords = {ARRAY(0x56147f5b7790)}
}
• S. Cosar, Z. Yan, F. Zhao, T. Lambrou, S. Yue, and N. Bellotto, “Thermal camera based physiological monitoring with an assistive robot,” in IEEE International Engineering in Medicine and Biology Conference, 2018.

This paper presents a physiological monitoring system for assistive robots using a thermal camera. It is based on the detection of subtle changes in temperature observed on different parts of the face. First, we segment and estimate these face regions on thermal images. Then, by applying Fourier analysis on temperature data, we estimate respiration and heartbeat rate. This physiological monitoring system has been integrated in an assistive robot for elderly people at home, as part of the ENRICHME project. Its performance has been evaluated on a new thermal dataset for physiological monitoring, which is made publicly available for research purposes.

@inproceedings{lirolem31779,
year = {2018},
title = {Thermal camera based physiological monitoring with an assistive robot},
publisher = {IEEE},
author = {Serhan Cosar and Zhi Yan and Feng Zhao and Tryphon Lambrou and Shigang Yue and Nicola Bellotto},
booktitle = {IEEE International Engineering in Medicine and Biology Conference},
month = {July},
abstract = {This paper presents a physiological monitoring system for assistive robots using a thermal camera. It is based on the detection of subtle changes in temperature observed on different parts of the face. First, we segment and estimate these face regions on thermal images. Then, by applying Fourier analysis on temperature data, we estimate respiration and heartbeat rate. This physiological monitoring system has been integrated in an assistive robot for elderly people at home, as part of the ENRICHME project. Its performance has been evaluated on a new thermal dataset for physiological monitoring, which is made publicly available for research purposes.},
url = {http://eprints.lincoln.ac.uk/31779/},
keywords = {ARRAY(0x56147f6e1568)}
}
• Q. Fu, C. Hu, J. Peng, and S. Yue, “Shaping the collision selectivity in a looming sensitive neuron model with parallel ON and OFF pathways and spike frequency adaptation,” Neural Networks, 2018.

Shaping the collision selectivity in vision-based artificial collision-detecting systems is still an open challenge. This paper presents a novel neuron model of a locust looming detector, i.e. the lobula giant movement detector (LGMD1), in order to provide effective solutions to enhance the collision selectivity of looming objects over other visual challenges. We propose an approach to model the biologically plausible mechanisms of ON and OFF pathways and a biophysical mechanism of spike frequency adaptation (SFA) in the proposed LGMD1 visual neural network. The ON and OFF pathways can separate both dark and light looming features for parallel spatiotemporal computations. This works effectively on perceiving a potential collision from dark or light objects that approach; such a bio-plausible structure can also separate LGMD1’s collision selectivity to its neighbouring looming detector — the LGMD2.The SFA mechanism can enhance the LGMD1’s collision selectivity to approaching objects rather than receding and translating stimuli, which is a significant improvement compared with similar LGMD1 neuron models. The proposed framework has been tested using off-line tests of synthetic and real-world stimuli, as well as on-line bio-robotic tests. The enhanced collision selectivity of the proposed model has been validated in systematic experiments. The computational simplicity and robustness of this work have also been verified by the bio-robotic tests, which demonstrates potential in building neuromorphic sensors for collision detection in both a fast and reliable manner.

@article{lirolem31536,
title = {Shaping the collision selectivity in a looming sensitive neuron model with parallel ON and OFF pathways and spike frequency adaptation},
year = {2018},
journal = {Neural Networks},
publisher = {Elsevier for European Neural Network Society (ENNS)},
author = {Qinbing Fu and Cheng Hu and Jigen Peng and Shigang Yue},
month = {December},
url = {http://eprints.lincoln.ac.uk/31536/},
keywords = {ARRAY(0x56147f766440)},
abstract = {Shaping the collision selectivity in vision-based artificial collision-detecting systems is still an open challenge. This paper presents a novel neuron model of a locust looming detector, i.e. the lobula giant movement detector (LGMD1), in order to provide effective solutions to enhance the collision selectivity of looming objects over other visual challenges. We propose an approach to model the biologically plausible mechanisms of ON and OFF pathways and a biophysical mechanism of spike frequency adaptation (SFA) in the proposed LGMD1 visual neural network. The ON and OFF pathways can separate both dark and light looming features for parallel spatiotemporal computations. This works effectively on perceiving a potential collision from dark or light objects that approach; such a bio-plausible structure can also separate LGMD1's collision selectivity to its neighbouring looming detector -- the LGMD2.The SFA mechanism can enhance the LGMD1's collision selectivity to approaching objects rather than receding and translating stimuli, which is a significant improvement compared with similar LGMD1 neuron models. The proposed framework has been tested using off-line tests of synthetic and real-world stimuli, as well as on-line bio-robotic tests. The enhanced collision selectivity of the proposed model has been validated in systematic experiments. The computational simplicity and robustness of this work have also been verified by the bio-robotic tests, which demonstrates potential in building neuromorphic sensors for collision detection in both a fast and reliable manner.}
}
• Q. Fu, C. Hu, P. Liu, and S. Yue, “Towards computational models of insect motion detectors for robot vision,” in 19th Towards Autonomous Robotic Systems (TAROS) Conference, 2018.

In this essay, we provide a brief survey of computational models of insect motion detectors, and bio-robotic solutions to build fast and reliable motion-sensing systems for robot vision. Vision is an important sensing modality for autonomous robots, since it can extract abundant useful features from visually cluttered and dynamic environments. Fast development of computer vision technology facilitates the modeling of dynamic vision systems for mobile robots.

@inproceedings{lirolem31671,
author = {Qinbing Fu and Cheng Hu and Pengcheng Liu and Shigang Yue},
title = {Towards computational models of insect motion detectors for robot vision},
year = {2018},
month = {July},
booktitle = {19th Towards Autonomous Robotic Systems (TAROS) Conference},
keywords = {ARRAY(0x56147f84f840)},
url = {http://eprints.lincoln.ac.uk/31671/},
abstract = {In this essay, we provide a brief survey of computational models of insect motion detectors, and bio-robotic solutions to build fast and reliable motion-sensing systems for robot vision. Vision is an important sensing modality for autonomous robots, since it can extract abundant useful features from visually cluttered and dynamic environments. Fast development of computer vision technology facilitates the modeling of dynamic vision systems for mobile robots.}
}
• G. H. W. Gebhardt, K. Daun, M. Schnaubelt, and G. Neumann, “Learning robust policies for object manipulation with robot swarms,” in IEEE International Conference on Robotics and Automation, 2018.

Swarm robotics investigates how a large population of robots with simple actuation and limited sensors can collectively solve complex tasks. One particular interesting application with robot swarms is autonomous object assembly. Such tasks have been solved successfully with robot swarms that are controlled by a human operator using a light source. In this paper, we present a method to solve such assembly tasks autonomously based on policy search methods. We split the assembly process in two subtasks: generating a high-level assembly plan and learning a low-level object movement policy. The assembly policy plans the trajectories for each object and the object movement policy controls the trajectory execution. Learning the object movement policy is challenging as it depends on the complex state of the swarm which consists of an individual state for each agent. To approach this problem, we introduce a representation of the swarm which is based on Hilbert space embeddings of distributions. This representation is invariant to the number of agents in the swarm as well as to the allocation of an agent to its position in the swarm. These invariances make the learned policy robust to changes in the swarm and also reduce the search space for the policy search method significantly. We show that the resulting system is able to solve assembly tasks with varying object shapes in multiple simulation scenarios and evaluate the robustness of our representation to changes in the swarm size. Furthermore, we demonstrate that the policies learned in simulation are robust enough to be transferred to real robots.

@inproceedings{lirolem31674,
month = {May},
booktitle = {IEEE International Conference on Robotics and Automation},
title = {Learning robust policies for object manipulation with robot swarms},
year = {2018},
author = {G. H. W. Gebhardt and K. Daun and M. Schnaubelt and G. Neumann},
abstract = {Swarm robotics investigates how a large population of robots with simple actuation and limited sensors can collectively solve complex tasks. One particular interesting application with robot swarms is autonomous object assembly.
Such tasks have been solved successfully with robot swarms that are controlled by a human operator using a light source.
In this paper, we present a method to solve such assembly tasks autonomously based on policy search methods. We split the assembly process in two subtasks: generating a high-level assembly plan and learning a low-level object movement policy.  The assembly policy plans the trajectories for each object and the object movement policy controls the trajectory execution.  Learning the object movement policy is challenging as it depends on the complex state of the swarm which consists of an individual state for each agent. To approach this problem, we introduce a representation of the swarm which is based on Hilbert space embeddings of distributions. This representation is invariant to the number of agents in the swarm as well as to the allocation of an agent to its position in the swarm.  These invariances make the learned policy robust to changes in the swarm and also reduce the search space for the policy search method significantly. We show that the resulting system is able to solve assembly tasks with varying object shapes in multiple simulation scenarios and evaluate the robustness of our representation to changes in the swarm size. Furthermore, we demonstrate that the policies learned in simulation are robust enough to be transferred to real robots.},
keywords = {ARRAY(0x56147f7c9288)},
url = {http://eprints.lincoln.ac.uk/31674/}
}
• G. H. W. Gebhardt, K. Daun, M. Schnaubelt, and G. Neumann, “Robust learning of object assembly tasks with an invariant representation of robot swarms,” in International Conference on Robotics and Automation (ICRA), 2018.

— Swarm robotics investigates how a large population of robots with simple actuation and limited sensors can collectively solve complex tasks. One particular interesting application with robot swarms is autonomous object assembly. Such tasks have been solved successfully with robot swarms that are controlled by a human operator using a light source. In this paper, we present a method to solve such assembly tasks autonomously based on policy search methods. We split the assembly process in two subtasks: generating a high-level assembly plan and learning a low-level object movement policy. The assembly policy plans the trajectories for each object and the object movement policy controls the trajectory execution. Learning the object movement policy is challenging as it depends on the complex state of the swarm which consists of an individual state for each agent. To approach this problem, we introduce a representation of the swarm which is based on Hilbert space embeddings of distributions. This representation is invariant to the number of agents in the swarm as well as to the allocation of an agent to its position in the swarm. These invariances make the learned policy robust to changes in the swarm and also reduce the search space for the policy search method significantly. We show that the resulting system is able to solve assembly tasks with varying object shapes in multiple simulation scenarios and evaluate the robustness of our representation to changes in the swarm size. Furthermore, we demonstrate that the policies learned in simulation are robust enough to be transferred to real robots.

@inproceedings{lirolem30920,
author = {G. H. W. Gebhardt and K. Daun and M. Schnaubelt and G. Neumann},
year = {2018},
title = {Robust learning of object assembly tasks with an invariant representation of robot swarms},
month = {May},
booktitle = {International Conference on Robotics and Automation (ICRA)},
keywords = {ARRAY(0x56147f37e378)},
url = {http://eprints.lincoln.ac.uk/30920/},
abstract = {{--} Swarm robotics investigates how a large population of robots with simple actuation and limited sensors can collectively solve complex tasks. One particular interesting application with robot swarms is autonomous object assembly. Such tasks have been solved successfully with robot swarms that are controlled by a human operator using a light source. In this paper, we present a method to solve such assembly tasks autonomously based on policy search methods. We split the assembly process in two subtasks: generating a high-level assembly plan and learning a low-level object movement policy. The assembly policy plans the trajectories for each object and the object movement policy controls the trajectory execution.
Learning the object movement policy is challenging as it depends on the complex state of the swarm which consists of an individual state for each agent. To approach this problem, we introduce a representation of the swarm which is based on Hilbert space embeddings of distributions. This representation is invariant to the number of agents in the swarm as well as to the allocation of an agent to its position in the swarm.  These invariances make the learned policy robust to changes in the swarm and also reduce the search space for the policy search method significantly. We show that the resulting system is able to solve assembly tasks with varying object shapes in multiple simulation scenarios and evaluate the robustness of our representation to changes in the swarm size. Furthermore, we demonstrate that the policies learned in simulation are robust enough to be transferred to real robots.}
}
• H. van Hoof, G. Neumann, and J. Peters, “Non-parametric policy search with limited information loss,” Journal of Machine Learning Research, 2018.

Learning complex control policies from non-linear and redundant sensory input is an important challenge for reinforcement learning algorithms. Non-parametric methods that approximate values functions or transition models can address this problem, by adapting to the complexity of the dataset. Yet, many current non-parametric approaches rely on unstable greedy maximization of approximate value functions, which might lead to poor convergence or oscillations in the policy update. A more robust policy update can be obtained by limiting the information loss between successive state-action distributions. In this paper, we develop a policy search algorithm with policy updates that are both robust and non-parametric. Our method can learn non-parametric control policies for infinite horizon continuous Markov decision processes with non-linear and redundant sensory representations. We investigate how we can use approximations of the kernel function to reduce the time requirements of the demanding non-parametric computations. In our experiments, we show the strong performance of the proposed method, and how it can be approximated effi- ciently. Finally, we show that our algorithm can learn a real-robot underpowered swing-up task directly from image data.

@article{lirolem28020,
month = {December},
publisher = {Journal of Machine Learning Research},
author = {Herke van Hoof and Gerhard Neumann and Jan Peters},
title = {Non-parametric policy search with limited information loss},
year = {2018},
journal = {Journal of Machine Learning Research},
abstract = {Learning complex control policies from non-linear and redundant sensory input is an important
challenge for reinforcement learning algorithms. Non-parametric methods that
approximate values functions or transition models can address this problem, by adapting
to the complexity of the dataset. Yet, many current non-parametric approaches rely on
unstable greedy maximization of approximate value functions, which might lead to poor
convergence or oscillations in the policy update. A more robust policy update can be obtained
by limiting the information loss between successive state-action distributions. In this
paper, we develop a policy search algorithm with policy updates that are both robust and
non-parametric. Our method can learn non-parametric control policies for infinite horizon
continuous Markov decision processes with non-linear and redundant sensory representations.
We investigate how we can use approximations of the kernel function to reduce the
time requirements of the demanding non-parametric computations. In our experiments, we
show the strong performance of the proposed method, and how it can be approximated effi-
ciently. Finally, we show that our algorithm can learn a real-robot underpowered swing-up
keywords = {ARRAY(0x56147f6fafb0)},
url = {http://eprints.lincoln.ac.uk/28020/}
}
• C. Hu, Q. Fu, and S. Yue, “Colias IV: the a?ordable micro robot platform with bio-inspired vision,” in 19th Towards Autonomous Robotic Systems (TAROS) Conference, 2018.

Vision is one of the most important sensing modalities for robots and has been realized on mostly large platforms. However for micro robots which are commonly utilized in swarm robotic studies, the visual ability is seldom applied or with only limited functions and resolution, due to the challenging requirements on the computation power and high data volume to deal with. This research has proposed the low-cost micro ground robot Colias IV, which is particularly designed to meet the requirements to allow embedded vision based tasks onboard, such as bio-inspired collision detection neural networks. Numerous of successful approaches have demonstrated that the proposed micro robot Colias IV to be a feasible platform for introducing visual based algorithms into swarm robotics.

@inproceedings{lirolem31672,
year = {2018},
title = {Colias IV: the a?ordable micro robot platform with bio-inspired vision},
author = {Cheng Hu and Qinbing Fu and Shigang Yue},
booktitle = {19th Towards Autonomous Robotic Systems (TAROS) Conference},
month = {July},
keywords = {ARRAY(0x56147f82e120)},
url = {http://eprints.lincoln.ac.uk/31672/},
abstract = {Vision is one of the most important sensing modalities for robots and has been realized on mostly large platforms. However for micro robots which are commonly utilized in swarm robotic studies, the visual ability is seldom applied or with only limited functions and resolution, due to the challenging requirements on the computation power and high data volume to deal with. This research has proposed the low-cost micro ground robot Colias IV, which is particularly designed to meet the requirements to allow embedded vision based tasks onboard, such as bio-inspired collision detection neural networks. Numerous of successful approaches have demonstrated that the proposed micro robot Colias IV to be a feasible platform for introducing visual based algorithms into swarm robotics.}
}
• D. Koert, G. Maeda, G. Neumann, and J. Peters, “Learning coupled forward-inverse models with combined prediction errors,” in International Conference on Robotics and Automation (ICRA), 2018.

Challenging tasks in unstructured environments require robots to learn complex models. Given a large amount of information, learning multiple simple models can offer an efficient alternative to a monolithic complex network. Training multiple models–that is, learning their parameters and their responsibilities–has been shown to be prohibitively hard as optimization is prone to local minima. To efficiently learn multiple models for different contexts, we thus develop a new algorithm based on expectation maximization (EM). In contrast to comparable concepts, this algorithm trains multiple modules of paired forward-inverse models by using the prediction errors of both forward and inverse models simultaneously. In particular, we show that our method yields a substantial improvement over only considering the errors of the forward models on tasks where the inverse space contains multiple solutions

@inproceedings{lirolem31686,
booktitle = {International Conference on Robotics and Automation (ICRA)},
month = {May},
title = {Learning coupled forward-inverse models with combined prediction errors},
year = {2018},
author = {D. Koert and G. Maeda and G. Neumann and J. Peters},
abstract = {Challenging tasks in unstructured environments require robots to learn complex models. Given a large amount of information, learning multiple simple models can offer an efficient alternative to a monolithic complex network. Training multiple models{--}that is, learning their parameters and their responsibilities{--}has been shown to be prohibitively hard as optimization is prone to local minima. To efficiently learn multiple models for different contexts, we thus develop a new algorithm based on expectation maximization (EM). In contrast to comparable concepts, this algorithm trains multiple modules of paired forward-inverse models by using the prediction errors of both forward and inverse models simultaneously.  In particular, we show that our method yields a substantial improvement over only considering the errors of the forward models on tasks where the inverse space contains multiple solutions},
url = {http://eprints.lincoln.ac.uk/31686/},
keywords = {ARRAY(0x56147f7bf0e0)}
}
• A. Kucukyilmaz and Y. Demiris, “Learning shared control by demonstration for personalized wheelchair assistance,” IEEE Transactions on Haptics, 2018.

An emerging research problem in assistive robotics is the design of methodologies that allow robots to provide personalized assistance to users. For this purpose, we present a method to learn shared control policies from demonstrations offered by a human assistant. We train a Gaussian process (GP) regression model to continuously regulate the level of assistance between the user and the robot, given the user’s previous and current actions and the state of the environment. The assistance policy is learned after only a single human demonstration, i.e. in one-shot. Our technique is evaluated in a one-of-a-kind experimental study, where the machine-learned shared control policy is compared to human assistance. Our analyses show that our technique is successful in emulating human shared control, by matching the location and amount of offered assistance on different trajectories. We observed that the effort requirement of the users were comparable between human-robot and human-human settings. Under the learned policy, the jerkiness of the user’s joystick movements dropped significantly, despite a significant increase in the jerkiness of the robot assistant’s commands. In terms of performance, even though the robotic assistance increased task completion time, the average distance to obstacles stayed in similar ranges to human assistance.

@article{lirolem31131,
author = {Ayse Kucukyilmaz and Yiannis Demiris},
publisher = {Institute of Electrical and Electronics Engineers (IEEE)},
title = {Learning shared control by demonstration for personalized wheelchair assistance},
year = {2018},
journal = {IEEE Transactions on Haptics},
month = {December},
abstract = {An emerging research problem in assistive robotics is the design of methodologies that allow robots to provide personalized assistance to users. For this purpose, we present a method to learn shared control policies from demonstrations offered by a human assistant. We train a Gaussian process (GP) regression model to continuously regulate the level of assistance between the user and the robot, given the user's previous and current actions and the state of the environment. The assistance policy is learned after only a single human demonstration, i.e. in one-shot. Our technique is evaluated in a one-of-a-kind experimental study, where the machine-learned shared control policy is compared to human assistance. Our analyses show that our technique is successful in emulating human shared control, by matching the location and amount of offered assistance on different trajectories. We observed that the effort requirement of the users were comparable between human-robot and human-human settings. Under the learned policy, the jerkiness of the user's joystick movements dropped significantly, despite a significant increase in the jerkiness of the robot assistant's commands. In terms of performance, even though the robotic assistance increased task completion time, the average distance to obstacles stayed in similar ranges to human assistance.},
url = {http://eprints.lincoln.ac.uk/31131/},
keywords = {ARRAY(0x56147f79f648)}
}
• P. Lightbody, P. Baxter, and M. Hanheide, “Studying table-top manipulation tasks: a robust framework for object tracking in collaboration,” in The 13th Annual ACM/IEEE International Conference on Human Robot Interaction, 2018.

Table-top object manipulation is a well-established test bed on which to study both basic foundations of general human-robot interaction and more specific collaborative tasks. A prerequisite, both for studies and for actual collaborative or assistive tasks, is the robust perception of any objects involved. This paper presents a real-time capable and ROS-integrated approach, bringing together state-of-the-art detection and tracking algorithms, integrating perceptual cues from multiple cameras and solving detection, sensor fusion and tracking in one framework. The highly scalable framework was tested in a HRI use-case scenario with 25 objects being reliably tracked under significant temporary occlusions. The use-case demonstrates the suitability of the approach when working with multiple objects in small table-top environments and highlights the versatility and range of analysis available with this framework.

@inproceedings{lirolem31204,
publisher = {ACM/IEEE},
author = {Peter Lightbody and Paul Baxter and Marc Hanheide},
year = {2018},
title = {Studying table-top manipulation tasks: a robust framework for object tracking in collaboration},
month = {March},
booktitle = {The 13th Annual ACM/IEEE International Conference on Human Robot Interaction},
abstract = {Table-top object manipulation is a well-established test bed on which to study both basic foundations of general human-robot interaction and more specific collaborative tasks. A prerequisite, both for studies and for actual collaborative or assistive tasks, is the robust perception of any objects involved. This paper presents a real-time capable and ROS-integrated approach, bringing together state-of-the-art detection and tracking algorithms, integrating perceptual cues from multiple cameras and solving detection, sensor fusion and tracking in one framework. The highly scalable framework was tested in a HRI use-case scenario with 25 objects being reliably tracked under significant temporary occlusions. The use-case demonstrates the suitability of the approach when working with multiple objects in small table-top environments and highlights the versatility and range of analysis available with this framework.},
keywords = {ARRAY(0x56147f5f92a8)},
url = {http://eprints.lincoln.ac.uk/31204/}
}
• P. Liu, K. Elgeneidy, S. Pearson, N. Huda, and G. Neumann, “Towards real-time robotic motion planning for grasping in cluttered and uncertain environments,” in 19th Towards Autonomous Robotic Systems (TAROS) Conference, 2018.

Adaptation to unorganized, congested and uncertain environment is a desirable capability but challenging task in development of robotic motion planning algorithms for object grasping. We have to make a tradeoff between coping with the environmental complexities using computational expensive approaches, and enforcing practical manipulation and grasping in real-time. In this paper, we present a brief overview and research objectives towards real-time motion planning for grasping in cluttered and uncertain environments. We present feasible ways in approaching this goal, in which key challenges and plausible solutions are discussed.

@inproceedings{lirolem31679,
publisher = {Springer},
author = {Pengcheng Liu and Khaled Elgeneidy and Simon Pearson and Nazmul Huda and Gerhard Neumann},
year = {2018},
title = {Towards real-time robotic motion planning for grasping in cluttered and uncertain environments},
booktitle = {19th Towards Autonomous Robotic Systems (TAROS) Conference},
month = {July},
abstract = {Adaptation to unorganized, congested and uncertain environment is a desirable capability but challenging task in development of robotic motion planning algorithms for object grasping. We have to make a tradeoff between coping with the environmental complexities using computational expensive approaches, and enforcing practical manipulation and grasping in real-time. In this paper, we present a brief overview and research objectives towards real-time motion planning for grasping in cluttered and uncertain environments. We present feasible ways in approaching this goal, in which key challenges and plausible solutions are discussed.},
keywords = {ARRAY(0x56147f6f45e8)},
url = {http://eprints.lincoln.ac.uk/31679/}
}
• D. Liu and S. Yue, “Event-driven continuous STDP learning with deep structure for visual pattern recognition,” IEEE Transactions on Cybernetics, 2018.

Human beings can achieve reliable and fast visual pattern recognition with limited time and learning samples. Underlying this capability, ventral stream plays an important role in object representation and form recognition. Modeling the ventral steam may shed light on further understanding the visual brain in humans and building artificial vision systems for pattern recognition. The current methods to model the mechanism of ventral stream are far from exhibiting fast, continuous and event-driven learning like the human brain. To create a visual system similar to ventral stream in human with fast learning capability, in this study, we propose a new spiking neural system with an event-driven continuous spike timing dependent plasticity (STDP) learning method using specific spiking timing sequences. Two novel continuous input mechanisms have been used to obtain the continuous input spiking pattern sequence. With the event-driven STDP learning rule, the proposed learning procedure will be activated if the neuron receive one pre- or post-synaptic spike event. The experimental results on MNIST database show that the proposed method outperforms all other methods in fast learning scenarios and most of the current models in exhaustive learning experiments.

@article{lirolem31010,
month = {May},
title = {Event-driven continuous STDP learning with deep structure for visual pattern recognition},
year = {2018},
journal = {IEEE Transactions on Cybernetics},
publisher = {Institute of Electrical and Electronics Engineers (IEEE)},
author = {Daqi Liu and Shigang Yue},
keywords = {ARRAY(0x56147f606340)},
url = {http://eprints.lincoln.ac.uk/31010/},
abstract = {Human beings can achieve reliable and fast visual pattern recognition with limited time and learning samples. Underlying this capability, ventral stream plays an important role in object representation and form recognition. Modeling the ventral steam may shed light on further understanding the visual brain in humans and building artificial vision systems for pattern recognition. The current methods to model the mechanism of ventral stream are far from exhibiting fast, continuous and event-driven learning like the human brain. To create a visual system similar to ventral stream in human with fast learning capability, in this study, we propose a new spiking neural system with an event-driven continuous spike timing dependent plasticity (STDP) learning method using specific spiking timing sequences. Two novel continuous input mechanisms have been used to obtain the continuous input spiking pattern sequence. With the event-driven STDP learning rule, the proposed learning procedure will be activated if the neuron receive one pre- or post-synaptic spike event. The experimental results on MNIST database show that the proposed method outperforms all other methods in fast learning scenarios and most of the current models in exhaustive learning experiments.}
}
• T. Osa, J. Pajarinen, G. Neumann, A. J. Bagnell, P. Abbeel, and J. Peters, “An algorithmic perspective on imitation learning,” Foundations and Trends in Robotics, vol. 7, iss. 1-2, pp. 1-179, 2018.

As robots and other intelligent agents move from simple environments and problems to more complex, unstructured settings, manually programming their behavior has become increasingly challenging and expensive. Often, it is easier for a teacher to demonstrate a desired behavior rather than attempt to manually engineer it. This process of learning from demonstrations, and the study of algorithms to do so, is called imitation learning. This work provides an introduction to imitation learning. It covers the underlying assumptions, approaches, and how they relate; the rich set of algorithms developed to tackle the problem; and advice on effective tools and implementation. We intend this paper to serve two audiences. First, we want to familiarize machine learning experts with the challenges of imitation learning, particularly those arising in robotics, and the interesting theoretical and practical distinctions between it and more familiar frameworks like statistical supervised learning theory and reinforcement learning. Second, we want to give roboticists and experts in applied artificial intelligence a broader appreciation for the frameworks and tools available for imitation learning. We pay particular attention to the intimate connection between imitation learning approaches and those of structured prediction Daumé III et al. [2009]. To structure this discussion, we categorize imitation learning techniques based on the following key criteria which drive algorithmic decisions: 1) The structure of the policy space. Is the learned policy a time-index trajectory (trajectory learning), a mapping from observations to actions (so called behavioral cloning [Bain and Sammut, 1996]), or the result of a complex optimization or planning problem at each execution as is common in inverse optimal control methods [Kalman, 1964, Moylan and Anderson, 1973]. 2) The information available during training and testing. In particular, is the learning algorithm privy to the full state that the teacher possess? Is the learner able to interact with the teacher and gather corrections or more data? Does the learner have a (typically a priori) model of the system with which it interacts? Does the learner have access to the reward (cost) function that the teacher is attempting to optimize? 3) The notion of success. Different algorithmic approaches provide varying guarantees on the resulting learned behavior. These guarantees range from weaker (e.g., measuring disagreement with the agent?s decision) to stronger (e.g., providing guarantees on the performance of the learner with respect to a true cost function, either known or unknown). We organize our work by paying particular attention to distinction (1): dividing imitation learning into directly replicating desired behavior (sometimes called behavioral cloning) and learning the hidden objectives of the desired behavior from demonstrations (called inverse optimal control or inverse reinforcement learning [Russell, 1998]). In the latter case, behavior arises as the result of an optimization problem solved for each new instance that the learner faces. In addition to method analysis, we discuss the design decisions a practitioner must make when selecting an imitation learning approach. Moreover, application examples–such as robots that play table tennis [Kober and Peters, 2009], programs that play the game of Go [Silver et al., 2016], and systems that understand natural language [Wen et al., 2015]– illustrate the properties and motivations behind different forms of imitation learning. We conclude by presenting a set of open questions and point towards possible future research directions for machine learning.

@article{lirolem31687,
author = {Takayuki Osa and Joni Pajarinen and Gerhard Neumann and J. Andrew Bagnell and Pieter Abbeel and Jan Peters},
publisher = {Now publishers},
pages = {1--179},
month = {March},
number = {1-2},
year = {2018},
journal = {Foundations and Trends in Robotics},
title = {An algorithmic perspective on imitation learning},
volume = {7},
keywords = {ARRAY(0x56147f76ffc0)},
url = {http://eprints.lincoln.ac.uk/31687/},
abstract = {As robots and other intelligent agents move from simple environments and problems to more complex, unstructured settings, manually programming their behavior has become increasingly challenging and expensive. Often, it is easier for a teacher to demonstrate a desired behavior rather than attempt to manually engineer it. This process of learning from demonstrations, and the study of algorithms to do so, is called imitation learning. This work provides an introduction to imitation learning. It covers the underlying assumptions, approaches, and how they relate; the rich set of algorithms developed to tackle the problem; and advice on effective tools and implementation. We intend this paper to serve two audiences. First, we want to familiarize machine learning experts with the challenges of imitation learning, particularly those arising in robotics, and the interesting theoretical and practical distinctions between it and more familiar frameworks like statistical supervised learning theory and reinforcement learning. Second, we want to give roboticists and experts in applied artificial intelligence a broader appreciation for the frameworks and tools available for imitation learning. We pay particular attention to the intimate connection between imitation learning approaches and those of structured prediction Daum{\'e} III et al. [2009]. To structure this discussion, we categorize imitation learning techniques based on the following key criteria which drive algorithmic decisions:

1) The structure of the policy space. Is the learned policy a time-index trajectory (trajectory learning), a mapping from observations to actions (so called behavioral cloning [Bain and Sammut, 1996]), or the result of a complex optimization or planning problem at each execution as is common in inverse optimal control methods [Kalman, 1964, Moylan and Anderson, 1973].

2) The information available during training and testing. In particular, is the learning algorithm privy to the full state that the teacher possess? Is the learner able to interact with the teacher and gather corrections or more data? Does the learner have a (typically a priori) model of the system with which it interacts? Does the learner have access to the reward (cost) function that the teacher is attempting to optimize?

3) The notion of success. Different algorithmic approaches provide varying guarantees on the resulting learned behavior. These guarantees range from weaker (e.g., measuring disagreement with the agent?s decision) to stronger (e.g., providing guarantees on the performance of the learner with respect to a true cost function, either known or unknown). We organize our work by paying particular attention to distinction (1): dividing imitation learning into directly replicating desired behavior (sometimes called behavioral cloning) and learning the hidden objectives of the desired behavior from demonstrations (called inverse optimal control or inverse reinforcement learning [Russell, 1998]). In the latter case, behavior arises as the result of an optimization problem solved for each new instance that the learner faces. In addition to method analysis, we discuss the design decisions a practitioner must make when selecting an imitation learning approach. Moreover, application examples{--}such as robots that play table tennis [Kober and Peters, 2009], programs that play the game of Go [Silver et al., 2016], and systems that understand natural language [Wen et al., 2015]{--} illustrate the properties and motivations behind different forms of imitation learning. We conclude by presenting a set of open questions and point towards possible future research directions for machine learning.}
}
• A. Paraschos, C. Daniel, J. Peters, and G. Neumann, “Using probabilistic movement primitives in robotics,” Autonomous Robots, vol. 42, iss. 3, pp. 529-551, 2018.

Movement Primitives are a well-established paradigm for modular movement representation and generation. They provide a data-driven representation of movements and support generalization to novel situations, temporal modulation, sequencing of primitives and controllers for executing the primitive on physical systems. However, while many MP frameworks exhibit some of these properties, there is a need for a uni- fied framework that implements all of them in a principled way. In this paper, we show that this goal can be achieved by using a probabilistic representation. Our approach models trajectory distributions learned from stochastic movements. Probabilistic operations, such as conditioning can be used to achieve generalization to novel situations or to combine and blend movements in a principled way. We derive a stochastic feedback controller that reproduces the encoded variability of the movement and the coupling of the degrees of freedom of the robot. We evaluate and compare our approach on several simulated and real robot scenarios.

@article{lirolem27883,
number = {3},
year = {2018},
volume = {42},
journal = {Autonomous Robots},
title = {Using probabilistic movement primitives in robotics},
pages = {529--551},
month = {March},
author = {Alexandros Paraschos and Christian Daniel and Jan Peters and Gerhard Neumann},
publisher = {Springer Verlag},
keywords = {ARRAY(0x56147f8f4198)},
url = {http://eprints.lincoln.ac.uk/27883/},
abstract = {Movement Primitives are a well-established
paradigm for modular movement representation and
generation. They provide a data-driven representation
of movements and support generalization to novel situations,
temporal modulation, sequencing of primitives
and controllers for executing the primitive on physical
systems. However, while many MP frameworks exhibit
some of these properties, there is a need for a uni-
fied framework that implements all of them in a principled
way. In this paper, we show that this goal can be
achieved by using a probabilistic representation. Our
approach models trajectory distributions learned from
stochastic movements. Probabilistic operations, such as
conditioning can be used to achieve generalization to
novel situations or to combine and blend movements in
a principled way. We derive a stochastic feedback controller
that reproduces the encoded variability of the
movement and the coupling of the degrees of freedom
of the robot. We evaluate and compare our approach
on several simulated and real robot scenarios.}
}
• R. Pinsler, R. Akrour, T. Osa, J. Peters, and G. Neumann, “Sample and feedback efficient hierarchical reinforcement learning from human preferences,” in IEEE International Conference on Robotics and Automation (ICRA), 2018.

While reinforcement learning has led to promising results in robotics, defining an informative reward function can sometimes prove to be challenging. Prior work considered including the human in the loop to jointly learn the reward function and the optimal policy. Generating samples from a physical robot and requesting human feedback are both taxing efforts for which efficiency is critical. In contrast to prior work, in this paper we propose to learn reward functions from both the robot and the human perspectives in order to improve on both efficiency metrics. On one side, learning a reward function from the human perspective increases feedback efficiency by assuming that humans rank trajectories according to an outcome space of reduced dimensionaltiy. On the other side, learning a reward function from the robot perspective circumvents the need for learning a dynamics model while retaining the sample efficiency of model-based approaches. We provide an algorithm that incorporates bi-perspective reward learning into a general hierarchical reinforcement learning framework and demonstrate the merits of our approach on a toy task and a simulated robot grasping task.

@inproceedings{lirolem31675,
booktitle = {IEEE International Conference on Robotics and Automation (ICRA)},
month = {May},
title = {Sample and feedback efficient hierarchical reinforcement learning from human preferences},
year = {2018},
author = {R. Pinsler and R. Akrour and T. Osa and J. Peters and G. Neumann},
abstract = {While reinforcement learning has led to promising results in robotics, defining an informative reward function can sometimes prove to be challenging. Prior work considered including the human in the loop to jointly learn the reward function and the optimal policy. Generating samples from a physical robot and requesting human feedback are both taxing efforts for which efficiency is critical. In contrast to prior work, in this paper we propose to learn reward functions from both the robot and the human perspectives in order to improve on both efficiency metrics. On one side, learning a reward function from the human perspective increases feedback efficiency by assuming that humans rank trajectories according to an outcome space of reduced dimensionaltiy. On the other side, learning a reward function from the robot perspective circumvents the need for learning a dynamics model while retaining the sample efficiency of model-based approaches. We provide an algorithm that incorporates bi-perspective reward learning into a general hierarchical reinforcement learning framework and demonstrate the merits of our approach on a toy task and a simulated robot grasping task.},
url = {http://eprints.lincoln.ac.uk/31675/},
keywords = {ARRAY(0x56147f583050)}
}

### 2017

• A. Abdolmaleki, B. Price, N. Lau, P. Reis, and G. Neumann, “Contextual CMA-ES,” in International Joint Conference on Artificial Intelligence (IJCAI), 2017.

Many stochastic search algorithms are designed to optimize a fixed objective function to learn a task, i.e., if the objective function changes slightly, for example, due to a change in the situation or context of the task, relearning is required to adapt to the new context. For instance, if we want to learn a kicking movement for a soccer robot, we have to relearn the movement for different ball locations. Such relearning is undesired as it is highly inefficient and many applications require a fast adaptation to a new context/situation. Therefore, we investigate contextual stochastic search algorithms that can learn multiple, similar tasks simultaneously. Current contextual stochastic search methods are based on policy search algorithms and suffer from premature convergence and the need for parameter tuning. In this paper, we extend the well known CMA-ES algorithm to the contextual setting and illustrate its performance on several contextual tasks. Our new algorithm, called contextual CMAES, leverages from contextual learning while it preserves all the features of standard CMA-ES such as stability, avoidance of premature convergence, step size control and a minimal amount of parameter tuning.

@inproceedings{lirolem28141,
year = {2017},
title = {Contextual CMA-ES},
author = {A. Abdolmaleki and B. Price and N. Lau and P. Reis and G. Neumann},
month = {August},
booktitle = {International Joint Conference on Artificial Intelligence (IJCAI)},
url = {http://eprints.lincoln.ac.uk/28141/},
keywords = {ARRAY(0x56147f9f37a8)},
abstract = {Many stochastic search algorithms are designed to optimize a fixed objective function to learn a task, i.e., if the objective function changes slightly, for example, due to a change in the situation or context of the task, relearning is required to adapt to the new context. For instance, if we want to learn a kicking movement for a soccer robot, we have to relearn the movement for different ball locations. Such relearning is undesired as it is highly inefficient and many applications require a fast adaptation to a new context/situation. Therefore, we investigate contextual stochastic search algorithms
that can learn multiple, similar tasks simultaneously. Current contextual stochastic search methods are based on policy search algorithms and suffer from premature convergence and the need for parameter tuning. In this paper, we extend the well known CMA-ES algorithm to the contextual setting and illustrate its performance on several contextual
tasks. Our new algorithm, called contextual CMAES, leverages from contextual learning while it preserves all the features of standard CMA-ES such as stability, avoidance of premature convergence, step size control and a minimal amount of parameter tuning.}
}
• A. Abdolmaleki, B. Price, N. Lau, L. P. Reis, and G. Neumann, “Deriving and improving CMA-ES with Information geometric trust regions,” in The Genetic and Evolutionary Computation Conference (GECCO 2017), 2017.

CMA-ES is one of the most popular stochastic search algorithms. It performs favourably in many tasks without the need of extensive parameter tuning. The algorithm has many beneficial properties, including automatic step-size adaptation, efficient covariance updates that incorporates the current samples as well as the evolution path and its invariance properties. Its update rules are composed of well established heuristics where the theoretical foundations of some of these rules are also well understood. In this paper we will fully derive all CMA-ES update rules within the framework of expectation-maximisation-based stochastic search algorithms using information-geometric trust regions. We show that the use of the trust region results in similar updates to CMA-ES for the mean and the covariance matrix while it allows for the derivation of an improved update rule for the step-size. Our new algorithm, Trust-Region Covariance Matrix Adaptation Evolution Strategy (TR-CMA-ES) is fully derived from first order optimization principles and performs favourably in compare to standard CMA-ES algorithm.

@inproceedings{lirolem27056,
title = {Deriving and improving CMA-ES with Information geometric trust regions},
year = {2017},
author = {Abbas Abdolmaleki and Bob Price and Nuno Lau and Luis Paulo Reis and Gerhard Neumann},
month = {July},
booktitle = {The Genetic and Evolutionary Computation Conference (GECCO 2017)},
url = {http://eprints.lincoln.ac.uk/27056/},
keywords = {ARRAY(0x56147f5c02b0)},
abstract = {CMA-ES is one of the most popular stochastic search algorithms.
It performs favourably in many tasks without the need of extensive
parameter tuning. The algorithm has many beneficial properties,
that incorporates the current samples as well as the evolution
path and its invariance properties. Its update rules are composed
of well established heuristics where the theoretical foundations of
some of these rules are also well understood. In this paper we
will fully derive all CMA-ES update rules within the framework of
expectation-maximisation-based stochastic search algorithms using
information-geometric trust regions. We show that the use of the trust
region results in similar updates to CMA-ES for the mean and the
covariance matrix while it allows for the derivation of an improved
update rule for the step-size. Our new algorithm, Trust-Region Covariance
Matrix Adaptation Evolution Strategy (TR-CMA-ES) is
fully derived from first order optimization principles and performs
favourably in compare to standard CMA-ES algorithm.}
}
• H. Abdulsamad, O. Arenz, J. Peters, and G. Neumann, “State-regularized policy search for linearized dynamical systems,” in Proceedings of the International Conference on Automated Planning and Scheduling (ICAPS), 2017.

Trajectory-Centric Reinforcement Learning and Trajectory Optimization methods optimize a sequence of feedbackcontrollers by taking advantage of local approximations of model dynamics and cost functions. Stability of the policy update is a major issue for these methods, rendering them hard to apply for highly nonlinear systems. Recent approaches combine classical Stochastic Optimal Control methods with information-theoretic bounds to control the step-size of the policy update and could even be used to train nonlinear deep control policies. These methods bound the relative entropy between the new and the old policy to ensure a stable policy update. However, despite the bound in policy space, the state distributions of two consecutive policies can still differ significantly, rendering the used local approximate models invalid. To alleviate this issue we propose enforcing a relative entropy constraint not only on the policy update, but also on the update of the state distribution, around which the dynamics and cost are being approximated. We present a derivation of the closed-form policy update and show that our approach outperforms related methods on two nonlinear and highly dynamic simulated systems.

@inproceedings{lirolem27055,
booktitle = {Proceedings of the International Conference on Automated Planning and Scheduling (ICAPS)},
month = {June},
author = {Hany Abdulsamad and Oleg Arenz and Jan Peters and Gerhard Neumann},
title = {State-regularized policy search for linearized dynamical systems},
year = {2017},
keywords = {ARRAY(0x56147f88e760)},
url = {http://eprints.lincoln.ac.uk/27055/},
abstract = {Trajectory-Centric Reinforcement Learning and Trajectory
Optimization methods optimize a sequence of feedbackcontrollers
by taking advantage of local approximations of
model dynamics and cost functions. Stability of the policy update
is a major issue for these methods, rendering them hard
to apply for highly nonlinear systems. Recent approaches
combine classical Stochastic Optimal Control methods with
information-theoretic bounds to control the step-size of the
policy update and could even be used to train nonlinear deep
control policies. These methods bound the relative entropy
between the new and the old policy to ensure a stable policy
update. However, despite the bound in policy space, the
state distributions of two consecutive policies can still differ
significantly, rendering the used local approximate models invalid.
To alleviate this issue we propose enforcing a relative
entropy constraint not only on the policy update, but also on
the update of the state distribution, around which the dynamics
and cost are being approximated. We present a derivation
of the closed-form policy update and show that our approach
outperforms related methods on two nonlinear and highly dynamic
simulated systems.}
}
• R. Akrour, D. Sorokin, J. Peters, and G. Neumann, “Local Bayesian optimization of motor skills,” in International Conference on Machine Learning (ICML), 2017.

Bayesian optimization is renowned for its sample efficiency but its application to higher dimensional tasks is impeded by its focus on global optimization. To scale to higher dimensional problems, we leverage the sample efficiency of Bayesian optimization in a local context. The optimization of the acquisition function is restricted to the vicinity of a Gaussian search distribution which is moved towards high value areas of the objective. The proposed informationtheoretic update of the search distribution results in a Bayesian interpretation of local stochastic search: the search distribution encodes prior knowledge on the optimum?s location and is weighted at each iteration by the likelihood of this location?s optimality. We demonstrate the effectiveness of our algorithm on several benchmark objective functions as well as a continuous robotic task in which an informative prior is obtained by imitation learning.

@inproceedings{lirolem27902,
title = {Local Bayesian optimization of motor skills},
year = {2017},
author = {R. Akrour and D. Sorokin and J. Peters and G. Neumann},
booktitle = {International Conference on Machine Learning (ICML)},
month = {August},
url = {http://eprints.lincoln.ac.uk/27902/},
keywords = {ARRAY(0x56147f5c0268)},
abstract = {Bayesian optimization is renowned for its sample
efficiency but its application to higher dimensional
tasks is impeded by its focus on global
optimization. To scale to higher dimensional
problems, we leverage the sample efficiency of
Bayesian optimization in a local context. The
optimization of the acquisition function is restricted
to the vicinity of a Gaussian search distribution
which is moved towards high value areas
of the objective. The proposed informationtheoretic
update of the search distribution results
in a Bayesian interpretation of local stochastic
search: the search distribution encodes prior
knowledge on the optimum?s location and is
weighted at each iteration by the likelihood of
this location?s optimality. We demonstrate the
effectiveness of our algorithm on several benchmark
objective functions as well as a continuous
robotic task in which an informative prior is obtained
by imitation learning.}
}
• P. Baxter, E. Ashurst, R. Read, J. Kennedy, and T. Belpaeme, “Robot education peers in a situated primary school study: personalisation promotes child learning,” PLoS One, 2017.

The benefit of social robots to support child learning in an educational context over an extended period of time is evaluated. Specifically, the effect of personalisation and adaptation of robot social behaviour is assessed. Two autonomous robots were embedded within two matched classrooms of a primary school for a continuous two week period without experimenter supervision to act as learning companions for the children for familiar and novel subjects. Results suggest that while children in both personalised and non-personalised conditions learned, there was increased child learning of a novel subject exhibited when interacting with a robot that personalised its behaviours, with indications that this benefit extended to other class-based performance. Additional evidence was obtained suggesting that there is increased acceptance of the personalised robot peer over a non-personalised version. These results provide the first evidence in support of peer-robot behavioural personalisation having a positive influence on learning when embedded in a learning environment for an extended period of time.

@article{lirolem27582,
month = {May},
author = {Paul Baxter and Emily Ashurst and Robin Read and James Kennedy and Tony Belpaeme},
publisher = {Public Library of Science},
journal = {PLoS One},
year = {2017},
title = {Robot education peers in a situated primary school study: personalisation promotes child learning},
keywords = {ARRAY(0x56147f883fc0)},
url = {http://eprints.lincoln.ac.uk/27582/},
abstract = {The benefit of social robots to support child learning in an educational context over an extended period of time is evaluated. Specifically, the effect of personalisation and adaptation of robot social behaviour is assessed. Two autonomous robots were embedded within two matched classrooms of a primary school for a continuous two week period without experimenter supervision to act as learning companions for the children for familiar and novel subjects. Results suggest that while children in both personalised and non-personalised conditions learned, there was increased child learning of a novel subject exhibited when interacting with a robot that personalised its behaviours, with indications that this benefit extended to other class-based performance. Additional evidence was obtained suggesting that there is increased acceptance of the personalised robot peer over a non-personalised version. These results provide the first evidence in support of peer-robot behavioural personalisation having a positive influence on learning when embedded in a learning environment for an extended period of time.}
}
• N. Bellotto, M. Fernandez-Carmona, and S. Cosar, “ENRICHME integration of ambient intelligence and robotics for AAL,” in Wellbeing AI: From Machine Learning to Subjectivity Oriented Computing (AAAI 2017 Spring Symposium), 2017.

Technological advances and affordability of recent smart sensors, as well as the consolidation of common software platforms for the integration of the latter and robotic sensors, are enabling the creation of complex active and assisted living environments for improving the quality of life of the elderly and the less able people. One such example is the integrated system developed by the European project ENRICHME, the aim of which is to monitor and prolong the independent living of old people affected by mild cognitive impairments with a combination of smart-home, robotics and web technologies. This paper presents in particular the design and technological solutions adopted to integrate, process and store the information provided by a set of fixed smart sensors and mobile robot sensors in a domestic scenario, including presence and contact detectors, environmental sensors, and RFID-tagged objects, for long-term user monitoring and

@inproceedings{lirolem25362,
month = {March},
booktitle = {Wellbeing AI: From Machine Learning to Subjectivity Oriented Computing (AAAI 2017 Spring Symposium)},
author = {Nicola Bellotto and Manuel Fernandez-Carmona and Serhan Cosar},
publisher = {AAAI},
title = {ENRICHME integration of ambient intelligence and robotics for AAL},
year = {2017},
abstract = {Technological advances and affordability of recent smart sensors, as well as the consolidation of common software platforms for the integration of the latter and robotic sensors, are enabling the creation of complex active and assisted living environments for improving the quality of life of the elderly and the less able people. One such example is the integrated system developed by the European project ENRICHME, the aim of which is to monitor and prolong the independent living of old people affected by mild cognitive impairments with a combination of smart-home, robotics and web technologies. This paper presents in particular the design and technological solutions adopted to integrate, process and store the information provided by a set of fixed smart sensors and mobile robot sensors in a domestic scenario, including presence and contact detectors, environmental sensors, and RFID-tagged objects, for long-term user monitoring and},
url = {http://eprints.lincoln.ac.uk/25362/},
keywords = {ARRAY(0x56147f611198)}
}
• C. Coppola, S. Cosar, D. Faria, and N. Bellotto, “Automatic detection of human interactions from RGB-D data for social activity classification,” in IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), 2017.

We present a system for the temporal detection of social interactions. Many of the works until now have succeeded in recognising activities from clipped videos in datasets, but for robotic applications, it is important to be able to move to more realistic data. For this reason, it is important to be able to detect temporally the intervals of time in which humans are performing an individual activity or a social one. Recognition of the human activities is a key feature for analysing the human behaviour. In particular, recognition of social activities could be useful to trigger human-robot interactions or to detect situations of potential danger. Based on that, this research has three goals: (1) define a new set of descriptors able to represent the phenomena; (2) develop a computational model able to discern the intervals in which a pair of people are interacting or performing individual activities; (3) provide a public dataset with RGB-D videos where social interactions and individual activities happen in a continuous stream. Results show that using the proposed approach allows to reach a good performance in the temporal segmentation of social activities.

@inproceedings{lirolem27647,
publisher = {IEEE},
author = {Claudio Coppola and Serhan Cosar and Diego Faria and Nicola Bellotto},
title = {Automatic detection of human interactions from RGB-D data for social activity classification},
year = {2017},
booktitle = {IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN)},
keywords = {ARRAY(0x56147f5f93c8)},
url = {http://eprints.lincoln.ac.uk/27647/},
abstract = {We present a system for the temporal detection of social interactions. Many of the works until now have succeeded in recognising activities from clipped videos in datasets, but for robotic applications, it is important to be able to move to more realistic data. For this reason, it is important to be able to detect temporally the intervals of time in which humans are performing an individual activity or a social one. Recognition of the human activities is a key feature for analysing the human behaviour. In particular, recognition of social activities could be useful to trigger human-robot interactions or to detect situations of potential danger. Based on that, this research has three goals: (1) define a new set of descriptors able to represent the phenomena; (2) develop a computational model able to discern the intervals in which a pair of people are interacting or performing individual activities; (3) provide a public dataset with RGB-D videos where social interactions and individual activities happen in a continuous stream. Results show that using the proposed approach allows to reach a good performance in the temporal segmentation of social activities.}
}
• S. Cosar, C. Coppola, and N. Bellotto, “Volume-based human re-identification with RGB-D cameras,” in VISAPP – International Conference on Computer Vision Theory and Applications, 2017.

This paper presents an RGB-D based human re-identification approach using novel biometrics features from the body’s volume. Existing work based on RGB images or skeleton features have some limitations for real-world robotic applications, most notably in dealing with occlusions and orientation of the user. Here, we propose novel features that allow performing re-identification when the person is facing side/backward or the person is partially occluded. The proposed approach has been tested for various scenarios including different views, occlusion and the public BIWI RGBD-ID dataset.

@inproceedings{lirolem25360,
booktitle = {VISAPP - International Conference on Computer Vision Theory and Applications},
month = {February},
author = {Serhan Cosar and Claudio Coppola and Nicola Bellotto},
year = {2017},
title = {Volume-based human re-identification with RGB-D cameras},
keywords = {ARRAY(0x56147f59d3e8)},
url = {http://eprints.lincoln.ac.uk/25360/},
abstract = {This paper presents an RGB-D based human re-identification approach using novel biometrics features from the body's volume. Existing work based on RGB images or skeleton features have some limitations for real-world robotic applications, most notably in dealing with occlusions and orientation of the user. Here, we propose novel features that allow performing re-identification when the person is facing side/backward or the person is partially occluded. The proposed approach has been tested for various scenarios including different views, occlusion and the public BIWI RGBD-ID dataset.}
}
• H. Cuayahuitl, “Deep reinforcement learning for conversational robots playing games,” in IEEE RAS International Conference on Humanoid Robots, 2017.

Deep reinforcement learning for interactive multimodal robots is attractive for endowing machines with trainable skill acquisition. But this form of learning still represents several challenges. The challenge that we focus in this paper is effective policy learning. To address that, in this paper we compare the Deep Q-Networks (DQN) method against a variant that aims for stronger decisions than the original method by avoiding decisions with the lowest negative rewards. We evaluated our baseline and proposed algorithms in agents playing the game of Noughts and Crosses with two grid sizes (3×3 and 5×5). Experimental results show evidence that our proposed method can lead to more effective policies than the baseline DQN method, which can be used for training interactive social robots.

@inproceedings{lirolem29060,
author = {Heriberto Cuayahuitl},
publisher = {IEEE},
title = {Deep reinforcement learning for conversational robots playing games},
year = {2017},
month = {November},
booktitle = {IEEE RAS International Conference on Humanoid Robots},
abstract = {Deep reinforcement learning for interactive multimodal robots is attractive for endowing machines with trainable skill acquisition. But this form of learning still represents several challenges. The challenge that we focus in this paper is effective policy learning. To address that, in this paper we compare the Deep Q-Networks (DQN) method against a variant that aims for stronger decisions than the original method by avoiding decisions with the lowest negative rewards. We evaluated our baseline and proposed algorithms in agents playing the game of Noughts and Crosses with two grid sizes (3x3 and 5x5). Experimental results show evidence that our proposed method can lead to more effective policies than the baseline DQN method, which can be used for training interactive social robots.},
keywords = {ARRAY(0x56147f7e3768)},
url = {http://eprints.lincoln.ac.uk/29060/}
}
• H. Cuayahuitl and S. Yu, “Deep reinforcement learning of dialogue policies with less weight updates,” in International Conference of the Speech Communication Association (INTERSPEECH), 2017.

Deep reinforcement learning dialogue systems are attractive because they can jointly learn their feature representations and policies without manual feature engineering. But its application is challenging due to slow learning. We propose a two-stage method for accelerating the induction of single or multi-domain dialogue policies. While the first stage reduces the amount of weight updates over time, the second stage uses very limited minibatches (of as much as two learning experiences) sampled from experience replay memories. The former frequently updates the weights of the neural nets at early stages of training, and decreases the amount of updates as training progresses by performing updates during exploration and by skipping updates during exploitation. The learning process is thus accelerated through less weight updates in both stages. An empirical evaluation in three domains (restaurants, hotels and tv guide) confirms that the proposed method trains policies 5 times faster than a baseline without the proposed method. Our findings are useful for training larger-scale neural-based spoken dialogue systems.

@inproceedings{lirolem27676,
month = {August},
booktitle = {International Conference of the Speech Communication Association (INTERSPEECH)},
author = {Heriberto Cuayahuitl and Seunghak Yu},
year = {2017},
title = {Deep reinforcement learning of dialogue policies with less weight updates},
keywords = {ARRAY(0x56147f8eb340)},
url = {http://eprints.lincoln.ac.uk/27676/},
abstract = {Deep reinforcement learning dialogue systems are attractive because they can jointly learn their feature representations and policies without manual feature engineering. But its application is challenging due to slow learning. We propose a two-stage method for accelerating the induction of single or multi-domain dialogue policies. While the first stage reduces the amount of weight updates over time, the second stage uses very limited minibatches (of as much as two learning experiences) sampled from experience replay memories. The former frequently updates the weights of the neural nets at early stages of training, and decreases the amount of updates as training progresses by performing updates during exploration and by skipping updates during exploitation. The learning process is thus accelerated
through less weight updates in both stages. An empirical evaluation in three domains (restaurants, hotels and tv guide) confirms that the proposed method trains policies 5 times faster than a baseline without the proposed method. Our findings are useful for training larger-scale neural-based spoken dialogue systems.}
}
• H. Cuayahuitl, S. Yu, A. Williamson, and J. Carse, “Scaling up deep reinforcement learning for multi-domain dialogue systems,” in International Joint Conference on Neural Networks (IJCNN), 2017.

Standard deep reinforcement learning methods such as Deep Q-Networks (DQN) for multiple tasks (domains) face scalability problems due to large search spaces. This paper proposes a three-stage method for multi-domain dialogue policy learning–termed NDQN, and applies it to an information-seeking spoken dialogue system in the domains of restaurants and hotels. In this method, the first stage does multi-policy learning via a network of DQN agents; the second makes use of compact state representations by compressing raw inputs; and the third stage applies a pre-training phase for bootstraping the behaviour of agents in the network. Experimental results comparing DQN (baseline) versus NDQN (proposed) using simulations report that the proposed method exhibits better scalability and is promising for optimising the behaviour of multi-domain dialogue systems. An additional evaluation reports that the NDQN agents outperformed a K-Nearest Neighbour baseline in task success and dialogue length, yielding more efficient and successful dialogues.

@inproceedings{lirolem26622,
publisher = {IEEE},
author = {Heriberto Cuayahuitl and Seunghak Yu and Ashley Williamson and Jacob Carse},
title = {Scaling up deep reinforcement learning for multi-domain dialogue systems},
year = {2017},
month = {May},
booktitle = {International Joint Conference on Neural Networks (IJCNN)},
url = {http://eprints.lincoln.ac.uk/26622/},
keywords = {ARRAY(0x56147f87b1d8)},
abstract = {Standard deep reinforcement learning methods such as Deep Q-Networks (DQN) for multiple tasks (domains) face scalability problems due to large search spaces. This paper proposes a three-stage method for multi-domain dialogue policy learning{--}termed NDQN, and applies it to an information-seeking spoken dialogue system in the domains of restaurants and hotels. In this method, the first stage does multi-policy learning via a network of DQN agents; the second makes use of compact state representations by compressing raw inputs; and the third stage applies a pre-training phase for bootstraping the behaviour of agents in the network. Experimental results comparing DQN
(baseline) versus NDQN (proposed) using simulations report that the proposed method exhibits better scalability and is
promising for optimising the behaviour of multi-domain dialogue systems. An additional evaluation reports that the NDQN agents outperformed a K-Nearest Neighbour baseline in task success and dialogue length, yielding more efficient and successful dialogues.}
}
• N. Dethlefs, M. Milders, H. Cuayáhuitl, T. Al-Salkini, and L. Douglas, “A natural language-based presentation of cognitive stimulation to people with dementia in assistive technology: a pilot study,” Informatics for Health and Social Care, pp. 1-12, 2017.

Currently, an estimated 36 million people worldwide are affected by Alzheimer?s disease or related dementias. In the absence of a cure, non-pharmacological interventions, such as cognitive stimulation, which slow down the rate of deterioration can benefit people with dementia and their caregivers. Such interventions have shown to improve well-being and slow down the rate of cognitive decline. It has further been shown that cognitive stimulation in interaction with a computer is as effective as with a human. However, the need to operate a computer often represents a difficulty for the elderly and stands in the way of widespread adoption. A possible solution to this obstacle is to provide a spoken natural language interface that allows people with dementia to interact with the cognitive stimulation software in the same way as they would interact with a human caregiver. This makes the assistive technology accessible to users regardless of their technical skills and provides a fully intuitive user experience. This article describes a pilot study that evaluated the feasibility of computer-based cognitive stimulation through a spoken natural language interface. Prototype software was evaluated with 23 users, including healthy elderly people and people with dementia. Feedback was overwhelmingly positive.

@article{lirolem28284,
publisher = {Taylor \& Francis: STM},
author = {Nina Dethlefs and Maarten Milders and Heriberto Cuay{\'a}huitl and Turkey Al-Salkini and Lorraine Douglas},
year = {2017},
journal = {Informatics for Health and Social Care},
title = {A natural language-based presentation of cognitive stimulation to people with dementia in assistive technology: a pilot study},
pages = {1--12},
month = {December},
abstract = {Currently, an estimated 36 million people worldwide are affected by Alzheimer?s disease or related dementias. In the absence of a cure, non-pharmacological interventions, such as cognitive stimulation, which slow down the rate of deterioration can benefit people with dementia and their caregivers. Such interventions have shown to improve well-being and slow down the rate of cognitive decline. It has further been shown that cognitive stimulation in interaction with a computer is as effective as with a human. However, the need to operate a computer often represents a difficulty for the elderly and stands in the way of widespread adoption. A possible solution to this obstacle is to provide a spoken natural language interface that allows people with dementia to interact with the cognitive stimulation software in the same way as they would interact with a human caregiver. This makes the assistive technology accessible to users regardless of their technical skills and provides a fully intuitive user experience. This article describes a pilot study that evaluated the feasibility of computer-based cognitive stimulation through a spoken natural language interface. Prototype software was evaluated with 23 users, including healthy elderly people and people with dementia. Feedback was overwhelmingly positive.},
url = {http://eprints.lincoln.ac.uk/28284/},
keywords = {ARRAY(0x56147f63ed08)}
}
• T. Duckett, A. Tapus, and N. Bellotto, “Editorial to special issue on the Seventh European Conference on Mobile Robots (ECMR?15),” Robotics and Autonomous Systems, vol. 91, p. 348, 2017.

This Special Issue is based on a selection of the best papers presented at the Seventh European Conference on Mobile Robots (ECMR?15), September 2nd?4th, 2015, in Lincoln, UK.

@article{lirolem28034,
title = {Editorial to special issue on the Seventh European Conference on Mobile Robots (ECMR?15)},
journal = {Robotics and Autonomous Systems},
volume = {91},
year = {2017},
month = {May},
pages = {348},
publisher = {Elsevier},
author = {Tom Duckett and Adriana Tapus and Nicola Bellotto},
url = {http://eprints.lincoln.ac.uk/28034/},
keywords = {ARRAY(0x56147f9840e8)},
abstract = {This Special Issue is based on a selection of the best papers presented at the Seventh European Conference on Mobile Robots (ECMR?15), September 2nd?4th, 2015, in Lincoln, UK.}
}
• F. End, R. Akrour, J. Peters, and G. Neumann, “Layered direct policy search for learning hierarchical skills,” in International Conference on Robotics and Automation (ICRA), 2017.

Solutions to real world robotic tasks often require complex behaviors in high dimensional continuous state and action spaces. Reinforcement Learning (RL) is aimed at learning such behaviors but often fails for lack of scalability. To address this issue, Hierarchical RL (HRL) algorithms leverage hierarchical policies to exploit the structure of a task. However, many HRL algorithms rely on task specific knowledge such as a set of predefined sub-policies or sub-goals. In this paper we propose a new HRL algorithm based on information theoretic principles to autonomously uncover a diverse set of sub-policies and their activation policies. Moreover, the learning process mirrors the policys structure and is thus also hierarchical, consisting of a set of independent optimization problems. The hierarchical structure of the learning process allows us to control the learning rate of the sub-policies and the gating individually and add specific information theoretic constraints to each layer to ensure the diversification of the subpolicies. We evaluate our algorithm on two high dimensional continuous tasks and experimentally demonstrate its ability to autonomously discover a rich set of sub-policies.

@inproceedings{lirolem26737,
author = {F. End and R. Akrour and J. Peters and G. Neumann},
title = {Layered direct policy search for learning hierarchical skills},
year = {2017},
booktitle = {International Conference on Robotics and Automation (ICRA)},
month = {May},
url = {http://eprints.lincoln.ac.uk/26737/},
keywords = {ARRAY(0x56147f7fee00)},
abstract = {Solutions to real world robotic tasks often require
complex behaviors in high dimensional continuous state and
action spaces. Reinforcement Learning (RL) is aimed at learning
such behaviors but often fails for lack of scalability. To
address this issue, Hierarchical RL (HRL) algorithms leverage
hierarchical policies to exploit the structure of a task. However,
many HRL algorithms rely on task specific knowledge such
as a set of predefined sub-policies or sub-goals. In this paper
we propose a new HRL algorithm based on information
theoretic principles to autonomously uncover a diverse set
of sub-policies and their activation policies. Moreover, the
learning process mirrors the policys structure and is thus also
hierarchical, consisting of a set of independent optimization
problems. The hierarchical structure of the learning process
allows us to control the learning rate of the sub-policies and
the gating individually and add specific information theoretic
constraints to each layer to ensure the diversification of the subpolicies.
We evaluate our algorithm on two high dimensional
continuous tasks and experimentally demonstrate its ability to
autonomously discover a rich set of sub-policies.}
}
• P. G. Esteban, P. Baxter, T. Belpaeme, E. Billing, H. Cai, H. Cao, M. Coeckelbergh, C. Costescu, D. David, A. D. Beir, Y. Fang, Z. Ju, J. Kennedy, H. Liu, A. Mazel, A. Pandey, K. Richardson, E. Senft, S. Thill, G. V. de Perre, B. Vanderborght, D. Vernon, H. Yu, and T. Ziemke, “How to build a supervised autonomous system for robot-enhanced therapy for children with autism spectrum disorder,” Paladyn, Journal of Behavioral Robotics, vol. 8, iss. 1, 2017.

Robot-Assisted Therapy (RAT) has successfully been used to improve social skills in children with autism spectrum disorders (ASD) through remote control of the robot in so-called Wizard of Oz (WoZ) paradigms.However, there is a need to increase the autonomy of the robot both to lighten the burden on human therapists (who have to remain in control and, importantly, supervise the robot) and to provide a consistent therapeutic experience. This paper seeks to provide insight into increasing the autonomy level of social robots in therapy to move beyond WoZ. With the final aim of improved human-human social interaction for the children, this multidisciplinary research seeks to facilitate the use of social robots as tools in clinical situations by addressing the challenge of increasing robot autonomy.We introduce the clinical framework in which the developments are tested, alongside initial data obtained from patients in a first phase of the project using a WoZ set-up mimicking the targeted supervised-autonomy behaviour. We further describe the implemented system architecture capable of providing the robot with supervised autonomy.

@article{lirolem27519,
number = {1},
journal = {Paladyn, Journal of Behavioral Robotics},
volume = {8},
year = {2017},
title = {How to build a supervised autonomous system for robot-enhanced therapy for children with autism spectrum disorder},
author = {Pablo G. Esteban and Paul Baxter and Tony Belpaeme and Erik Billing and Haibin Cai and Hoang-Long Cao and Mark Coeckelbergh and Cristina Costescu and Daniel David and Albert De Beir and Yinfeng Fang and Zhaojie Ju and James Kennedy and Honghai Liu and Alexandre Mazel and Amit Pandey and Kathleen Richardson and Emmanue Senft and Serge Thill and Greet Van de Perre and Bram Vanderborght and David Vernon and Hui Yu and Tom Ziemke},
publisher = {Springer/Versita with DeGruyter},
month = {May},
abstract = {Robot-Assisted Therapy (RAT) has successfully been used to improve social skills in children with autism spectrum disorders (ASD) through remote control of the robot in so-called Wizard of Oz (WoZ) paradigms.However, there is a need to increase the autonomy of the robot both to lighten the burden on human therapists (who have to remain in control and, importantly, supervise the robot) and to provide a consistent therapeutic experience. This paper seeks to provide insight into increasing the autonomy level of social robots in therapy to move beyond WoZ. With the final aim of improved human-human social interaction for the children, this multidisciplinary research seeks to facilitate the use of social robots as tools in clinical situations by addressing the challenge of increasing robot autonomy.We introduce the clinical framework in which the developments are tested, alongside initial data obtained from patients in a first phase of the project using a WoZ set-up mimicking the targeted supervised-autonomy behaviour. We further describe the implemented system architecture capable of providing the robot with supervised autonomy.},
url = {http://eprints.lincoln.ac.uk/27519/},
keywords = {ARRAY(0x56147f66aea8)}
}
• F. B. Farraj, T. Osa, N. Pedemonte, J. Peters, G. Neumann, and P. R. Giordano, “A learning-based shared control architecture for interactive task execution,” in IEEE International Conference on Robotics and Automation (ICRA), 2017.

Shared control is a key technology for various robotic applications in which a robotic system and a human operator are meant to collaborate efficiently. In order to achieve efficient task execution in shared control, it is essential to predict the desired behavior for a given situation or context to simplify the control task for the human operator. To do this prediction, we use Learning from Demonstration (LfD), which is a popular approach for transferring human skills to robots. We encode the demonstrated behavior as trajectory distributions and generalize the learned distributions to new situations. The goal of this paper is to present a shared control framework that uses learned expert distributions to gain more autonomy. Our approach controls the balance between the controller?s autonomy and the human preference based on the distributions of the demonstrated trajectories. Moreover, the learned distributions are autonomously refined from collaborative task executions, resulting in a master-slave system with increasing autonomy that requires less user input with an increasing number of task executions. We experimentally validated that our shared control approach enables efficient task executions. Moreover, the conducted experiments demonstrated that the developed system improves its performances through interactive task executions with our shared control.

@inproceedings{lirolem26738,
year = {2017},
title = {A learning-based shared control architecture for interactive task execution},
author = {F. B. Farraj and T. Osa and N. Pedemonte and J. Peters and G. Neumann and P. R. Giordano},
publisher = {IEEE},
booktitle = {IEEE International Conference on Robotics and Automation (ICRA)},
month = {May},
keywords = {ARRAY(0x56147f5e78d8)},
url = {http://eprints.lincoln.ac.uk/26738/},
abstract = {Shared control is a key technology for various
robotic applications in which a robotic system and a human
operator are meant to collaborate efficiently. In order to achieve
efficient task execution in shared control, it is essential to
predict the desired behavior for a given situation or context
to simplify the control task for the human operator. To do this
prediction, we use Learning from Demonstration (LfD), which is
a popular approach for transferring human skills to robots. We
encode the demonstrated behavior as trajectory distributions
and generalize the learned distributions to new situations. The
goal of this paper is to present a shared control framework
that uses learned expert distributions to gain more autonomy.
Our approach controls the balance between the controller?s
autonomy and the human preference based on the distributions
of the demonstrated trajectories. Moreover, the learned
distributions are autonomously refined from collaborative task
executions, resulting in a master-slave system with increasing
autonomy that requires less user input with an increasing
number of task executions. We experimentally validated that
our shared control approach enables efficient task executions.
Moreover, the conducted experiments demonstrated that the
developed system improves its performances through interactive
task executions with our shared control.}
}
• J. P. Fentanes, C. Dondrup, and M. Hanheide, “Navigation testing for continuous integration in robotics,” in UK-RAS Conference on Robotics and Autonomous Systems, 2017.

Robots working in real-world applications need to be robust and reliable. However, ensuring robust software in an academic development environment with dozens of developers poses a significant challenge. This work presents a testing framework, successfully employed in a large-scale integrated robotics project, based on continuous integration and the fork-and-pull model of software development, implementing automated system regression testing for robot navigation. It presents a framework suitable for both regression testing and also providing processes for parameter optimisation and benchmarking.

@inproceedings{lirolem31547,
booktitle = {UK-RAS Conference on Robotics and Autonomous Systems},
month = {December},
year = {2017},
title = {Navigation testing for continuous integration in robotics},
author = {Jaime Pulido Fentanes and Christian Dondrup and Marc Hanheide},
publisher = {UK-RAS Conference on Robotics and Autonomous Systems (RAS 2017)},
abstract = {Robots working in real-world applications need to be robust and reliable. However, ensuring robust software in an academic development environment with dozens of developers poses a significant challenge. This work presents a testing framework, successfully employed in a large-scale integrated robotics project, based on continuous integration and the fork-and-pull model of software development, implementing automated system regression testing for robot navigation. It presents a framework suitable for both regression testing and also providing processes for parameter optimisation and benchmarking.},
keywords = {ARRAY(0x56147f741648)},
url = {http://eprints.lincoln.ac.uk/31547/}
}
• M. Fernandez-Carmona, S. Cosar, C. Coppola, and N. Bellotto, “Entropy-based abnormal activity detection fusing RGB-D and domotic sensors,” in IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems (MFI), 2017.

The automatic detection of anomalies in Active and Assisted Living (AAL) environments is important for monitoring the wellbeing and safety of the elderly at home. The integration of smart domotic sensors (e.g. presence detectors) with those ones equipping modern mobile robots (e.g. RGBD camera) provides new opportunities for addressing this challenge. In this paper, we propose a novel solution to combine local activity levels detected by a single RGBD camera with the global activity perceived by a network of domotic sensors. Our approach relies on a new method for computing such a global activity using various presence detectors, based on the concept of entropy from information theory. This entropy effectively shows how active a particular room or environment?s area is. The solution includes also a new application of Hybrid Markov Logic Networks (HMLNs) to merge different information sources for local and global anomaly detection. The system has been tested with RGBD data and a comprehensive domotic dataset containing data entries from 37 different domotic sensors (presence, temperature, light, energy consumption, door contact), which is made publicly available. The experimental results show the effectiveness of our approach and the potential for complex anomaly detection in AAL settings.

@inproceedings{lirolem28779,
title = {Entropy-based abnormal activity detection fusing RGB-D and domotic sensors},
year = {2017},
publisher = {IEEE},
author = {Manuel Fernandez-Carmona and Serhan Cosar and Claudio Coppola and Nicola Bellotto},
booktitle = {IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems (MFI)},
month = {November},
keywords = {ARRAY(0x56147f7c14d0)},
url = {http://eprints.lincoln.ac.uk/28779/},
abstract = {The automatic detection of anomalies in Active and Assisted Living (AAL) environments is important for monitoring the wellbeing and safety of the elderly at home. The integration of smart domotic sensors (e.g. presence detectors) with those ones equipping modern mobile robots (e.g. RGBD camera) provides new opportunities for addressing this challenge. In this paper, we propose a novel solution to combine local activity levels detected by a single RGBD camera with the global activity perceived by a network of domotic sensors. Our approach relies on a new method for computing such a global activity using various presence detectors, based on the concept of entropy from information theory. This entropy effectively shows how active a particular room or environment?s area is. The solution includes also a new application of Hybrid Markov Logic Networks (HMLNs) to merge different information sources for local and global anomaly detection. The system has been tested with RGBD data and a comprehensive domotic dataset containing data entries from 37 different domotic sensors (presence, temperature, light, energy consumption, door contact), which is made publicly available. The experimental results show the effectiveness of our approach and the potential for complex anomaly detection in AAL settings.}
}
• Q. Fu, C. Hu, T. Liu, and S. Yue, “Collision selective LGMDs neuron models research benefits from a vision-based autonomous micro robot,” in 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2017.

The developments of robotics inform research across a broad range of disciplines. In this paper, we will study and compare two collision selective neuron models via a vision-based autonomous micro robot. In the locusts’ visual brain, two Lobula Giant Movement Detectors (LGMDs), i.e. LGMD1 and LGMD2, have been identified as looming sensitive neurons responding to rapidly expanding objects, yet with different collision selectivity. Both neurons have been built for perceiving potential collisions in an efficient and reliable manner; a few modeling works have also demonstrated their effectiveness for robotic implementations. In this research, for the first time, we set up binocular neuronal models, combining the functionalities of LGMD1 and LGMD2 neurons, in the visual modality of a ground mobile robot. The results of systematic on-line experiments demonstrated three contributions: (1) The arena tests involving multiple robots verified the robustness and efficiency of a reactive motion control strategy via integrating a bilateral pair of LGMD1 and LGMD2 models for collision detection in dynamic scenarios. (2) We pinpointed the different collision selectivity between LGMD1 and LGMD2 neuron models fulfilling corresponded biological research results. (3) The low-cost robot may also shed lights on similar bio-inspired embedded vision systems and swarm robotics applications.

@inproceedings{lirolem27834,
title = {Collision selective LGMDs neuron models research benefits from a vision-based autonomous micro robot},
year = {2017},
author = {Qinbing Fu and Cheng Hu and Tian Liu and Shigang Yue},
booktitle = {2017 IEEE/RSJ International Conference on Intelligent Robots and Systems},
month = {September},
keywords = {ARRAY(0x56147f5d7450)},
url = {http://eprints.lincoln.ac.uk/27834/},
abstract = {The developments of robotics inform research across a broad range of disciplines. In this paper, we will study and compare two collision selective neuron models via a vision-based autonomous micro robot. In the locusts' visual brain, two Lobula Giant Movement Detectors (LGMDs), i.e. LGMD1 and LGMD2, have been identified as looming sensitive neurons responding to rapidly expanding objects, yet with different collision selectivity. Both neurons have been built for perceiving potential collisions in an efficient and reliable manner; a few modeling works have also demonstrated their effectiveness for robotic implementations. In this research, for the first time, we set up binocular neuronal models, combining the functionalities of LGMD1 and LGMD2 neurons, in the visual modality of a ground mobile robot. The results of systematic on-line experiments demonstrated three contributions: (1) The arena tests involving multiple robots verified the robustness and efficiency of a reactive motion control strategy via integrating a bilateral pair of LGMD1 and LGMD2 models for collision detection in dynamic scenarios. (2) We pinpointed the different collision selectivity between LGMD1 and LGMD2 neuron models fulfilling corresponded biological research results. (3) The low-cost robot may also shed lights on similar bio-inspired embedded vision systems and swarm robotics applications.}
}
• Q. Fu and S. Yue, “Modeling direction selective visual neural network with ON and OFF pathways for extracting motion cues from cluttered background,” in The 2017 International Joint Conference on Neural Networks (IJCNN 2017), 2017.

The nature endows animals robustvision systems for extracting and recognizing differentmotion cues, detectingpredators, chasing preys/mates in dynamic and cluttered environments. Direction selective neurons (DSNs), with preference to certain orientation visual stimulus, have been found in both vertebrates and invertebrates for decades. In thispaper, with respectto recent biological research progress in motion-detecting circuitry, we propose a novel way to model DSNs for recognizing movements on four cardinal directions. It is based on an architecture of ON and OFF visual pathways underlies a theory of splitting motion signals into parallel channels, encoding brightness increments and decrements separately. To enhance the edge selectivity and speed response to moving objects, we put forth a bio-plausible spatial-temporal network structure with multiple connections of same polarity ON/OFF cells. Each pair-wised combination is ?ltered with dynamic delay depending on sampling distance. The proposed vision system was challenged against image streams from both synthetic and cluttered real physical scenarios. The results demonstrated three major contributions: ?rst, the neural network ful?lled the characteristics of a postulated physiological map of conveying visual information through different neuropile layers; second, the DSNs model can extract useful directional motion cues from cluttered background robustly and timely, which hits at potential of quick implementation in visionbased micro mobile robots; moreover, it also represents better speed response compared to a state-of-the-art elementary motion detector.

@inproceedings{lirolem26619,
author = {Qinbing Fu and Shigang Yue},
year = {2017},
title = {Modeling direction selective visual neural network with ON and OFF pathways for extracting motion cues from cluttered background},
booktitle = {The 2017 International Joint Conference on Neural Networks (IJCNN 2017)},
month = {May},
abstract = {The nature endows animals robustvision systems for extracting and recognizing differentmotion cues, detectingpredators, chasing preys/mates in dynamic and cluttered environments. Direction selective neurons (DSNs), with preference to certain orientation visual stimulus, have been found in both vertebrates and invertebrates for decades. In thispaper, with respectto recent biological research progress in motion-detecting circuitry, we propose a novel way to model DSNs for recognizing movements on four cardinal directions. It is based on an architecture of ON and OFF visual pathways underlies a theory of splitting motion signals into parallel channels, encoding brightness increments and decrements separately. To enhance the edge selectivity and speed response to moving objects, we put forth a bio-plausible spatial-temporal network structure with multiple connections of same polarity ON/OFF cells. Each pair-wised combination is ?ltered with dynamic delay depending on sampling distance. The proposed vision system was challenged against image streams from both synthetic and cluttered real physical scenarios. The results demonstrated three major contributions: ?rst, the neural network ful?lled the characteristics of a postulated physiological map of conveying visual information through different neuropile layers; second, the DSNs model can extract useful directional motion cues from cluttered background robustly and timely, which hits at potential of quick implementation in visionbased micro mobile robots; moreover, it also represents better speed response compared to a state-of-the-art elementary motion detector.},
keywords = {ARRAY(0x56147f603b18)},
url = {http://eprints.lincoln.ac.uk/26619/}
}
• Q. Fu and S. Yue, “Mimicking fly motion tracking and fixation behaviors with a hybrid visual neural network,” in IEEE Int. Conf. on Robotics and Biomimetics, 2017.

How do animals, e.g. insects, detect meaningful visual motion cues involving directional and locational information of moving objects in visual clutter accurately and efficiently? This open question has been very attractive for decades. In this paper, with respect to latest biological research progress made on motion detection circuitry, we conduct a novel hybrid visual neural network, combining the functionality of two bio-plausible, namely motion and position pathways explored in fly visual system, for mimicking the tracking and fixation behaviors. This modeling study extends a former direction selective neurons model to the higher level of behavior. The motivated algorithms can be used to guide a system that extracts location information on moving objects in a scene regardless of background clutter, using entirely low-level visual processing. We tested it against translational movements in synthetic and real-world scenes. The results demonstrated the following contributions: (1) Compared to conventional computer vision techniques, it turns out the computational simplicity of this model may benefit the utility in small robots for real time fixating. (2) The hybrid neural network structure fulfills the characteristics of a putative signal tuning map in physiology. (3) It also satisfies with a profound implication proposed by biologists: visual fixation behaviors could be simply tuned via only the position pathway; nevertheless, the motion-detecting pathway enhances the tracking precision.

@inproceedings{lirolem28879,
booktitle = {IEEE Int. Conf. on Robotics and Biomimetics},
month = {December},
author = {Qinbing Fu and Shigang Yue},
title = {Mimicking fly motion tracking and fixation behaviors with a hybrid visual neural network},
year = {2017},
abstract = {How do animals, e.g. insects, detect meaningful visual motion cues involving directional and locational information of moving objects in visual clutter accurately and efficiently? This open question has been very attractive for decades. In this paper, with respect to latest biological research progress made on motion detection circuitry, we conduct a novel hybrid visual neural network, combining the functionality of two bio-plausible, namely motion and position pathways explored in fly visual system, for mimicking the tracking and fixation behaviors. This modeling study extends a former direction selective neurons model to the higher level of behavior. The motivated algorithms can be used to guide a system that extracts location information on moving objects in a scene regardless of background clutter, using entirely low-level visual processing. We tested it against translational movements in synthetic and real-world scenes. The results demonstrated the following contributions: (1) Compared to conventional computer vision techniques, it turns out the computational simplicity of this model may benefit the utility in small robots for real time fixating. (2) The hybrid neural network structure fulfills the characteristics of a putative signal tuning map in physiology. (3) It also satisfies with a profound implication proposed by biologists: visual fixation behaviors could be simply tuned via only the position pathway; nevertheless, the motion-detecting pathway enhances the tracking precision.},
url = {http://eprints.lincoln.ac.uk/28879/},
keywords = {ARRAY(0x56147f843440)}
}
• A. Gabriel, R. Akrour, J. Peters, and G. Neumann, “Empowered skills,” in International Conference on Robotics and Automation (ICRA), 2017.

Robot Reinforcement Learning (RL) algorithms return a policy that maximizes a global cumulative reward signal but typically do not create diverse behaviors. Hence, the policy will typically only capture a single solution of a task. However, many motor tasks have a large variety of solutions and the knowledge about these solutions can have several advantages. For example, in an adversarial setting such as robot table tennis, the lack of diversity renders the behavior predictable and hence easy to counter for the opponent. In an interactive setting such as learning from human feedback, an emphasis on diversity gives the human more opportunity for guiding the robot and to avoid the latter to be stuck in local optima of the task. In order to increase diversity of the learned behaviors, we leverage prior work on intrinsic motivation and empowerment. We derive a new intrinsic motivation signal by enriching the description of a task with an outcome space, representing interesting aspects of a sensorimotor stream. For example, in table tennis, the outcome space could be given by the return position and return ball speed. The intrinsic motivation is now given by the diversity of future outcomes, a concept also known as empowerment. We derive a new policy search algorithm that maximizes a trade-off between the extrinsic reward and this intrinsic motivation criterion. Experiments on a planar reaching task and simulated robot table tennis demonstrate that our algorithm can learn a diverse set of behaviors within the area of interest of the tasks.

@inproceedings{lirolem26736,
author = {A. Gabriel and R. Akrour and J. Peters and G. Neumann},
title = {Empowered skills},
year = {2017},
month = {May},
booktitle = {International Conference on Robotics and Automation (ICRA)},
abstract = {Robot Reinforcement Learning (RL) algorithms
return a policy that maximizes a global cumulative reward
signal but typically do not create diverse behaviors. Hence, the
policy will typically only capture a single solution of a task.
However, many motor tasks have a large variety of solutions
and the knowledge about these solutions can have several
robot table tennis, the lack of diversity renders the behavior
predictable and hence easy to counter for the opponent. In an
interactive setting such as learning from human feedback, an
emphasis on diversity gives the human more opportunity for
guiding the robot and to avoid the latter to be stuck in local
optima of the task. In order to increase diversity of the learned
behaviors, we leverage prior work on intrinsic motivation and
empowerment. We derive a new intrinsic motivation signal by
enriching the description of a task with an outcome space,
representing interesting aspects of a sensorimotor stream. For
example, in table tennis, the outcome space could be given
by the return position and return ball speed. The intrinsic
motivation is now given by the diversity of future outcomes,
a concept also known as empowerment. We derive a new
policy search algorithm that maximizes a trade-off between
the extrinsic reward and this intrinsic motivation criterion.
Experiments on a planar reaching task and simulated robot
table tennis demonstrate that our algorithm can learn a diverse
set of behaviors within the area of interest of the tasks.},
url = {http://eprints.lincoln.ac.uk/26736/},
keywords = {ARRAY(0x56147f88fe00)}
}
• G. H. W. Gebhardt, K. Daun, M. Schnaubelt, A. Hendrich, D. Kauth, and G. Neumann, “Learning to assemble objects with a robot swarm,” in Proceedings of the 16th Conference on Autonomous Agents and MultiAgent Systems (AAMAS 17), 2017, pp. 1547-1549.

Large populations of simple robots can solve complex tasks, but controlling them is still a challenging problem, due to limited communication and computation power. In order to assemble objects, have shown that a human controller can solve such a task. Instead, we investigate how to learn the assembly of multiple objects with a single central controller. We propose splitting the assembly process in two sub-tasks — generating a top-level assembly policy and learning an object movement policy. The assembly policy plans the trajectories for each object and the object movement policy controls the trajectory execution.The resulting system is able to solve assembly tasks with varying object shapes being assembled as shown in multiple simulation scenarios.

@inproceedings{lirolem28089,
month = {May},
pages = {1547--1549},
booktitle = {Proceedings of the 16th Conference on Autonomous Agents and MultiAgent Systems (AAMAS 17)},
publisher = {international foundation for autonomous agents and multiagent systems},
author = {Gregor H. W. Gebhardt and Kevin Daun and Marius Schnaubelt and Alexander Hendrich and Daniel Kauth and Gerhard Neumann},
note = {Extended abstract},
year = {2017},
title = {Learning to assemble objects with a robot swarm},
url = {http://eprints.lincoln.ac.uk/28089/},
keywords = {ARRAY(0x56147f604220)},
abstract = {Large populations of simple robots can solve complex tasks, but controlling them is still a challenging problem, due to limited communication and computation power. In order to assemble objects, have shown that a human controller can solve such a task. Instead, we investigate how to learn the assembly of multiple objects with a single central controller. We propose splitting the assembly process in two sub-tasks -- generating a top-level assembly policy and learning an object movement policy. The assembly policy plans the trajectories for each object and the object movement policy controls the trajectory execution.The resulting system is able to solve assembly tasks with varying object shapes being assembled as shown in multiple simulation scenarios.}
}
• G. H. W. Gebhardt, A. Kupcsik, and G. Neumann, “The kernel Kalman rule: efficient nonparametric inference with recursive least squares,” in Thirty-First AAAI Conference on Artificial Intelligence, 2017.

Nonparametric inference techniques provide promising tools for probabilistic reasoning in high-dimensional nonlinear systems. Most of these techniques embed distributions into reproducing kernel Hilbert spaces (RKHS) and rely on the kernel Bayes? rule (KBR) to manipulate the embeddings. However, the computational demands of the KBR scale poorly with the number of samples and the KBR often suffers from numerical instabilities. In this paper, we present the kernel Kalman rule (KKR) as an alternative to the KBR. The derivation of the KKR is based on recursive least squares, inspired by the derivation of the Kalman innovation update. We apply the KKR to filtering tasks where we use RKHS embeddings to represent the belief state, resulting in the kernel Kalman filter (KKF). We show on a nonlinear state estimation task with high dimensional observations that our approach provides a significantly improved estimation accuracy while the computational demands are significantly decreased.

@inproceedings{lirolem26739,
author = {G. H. W. Gebhardt and A. Kupcsik and G. Neumann},
publisher = {AAAI},
year = {2017},
title = {The kernel Kalman rule: efficient nonparametric inference with recursive least squares},
booktitle = {Thirty-First AAAI Conference on Artificial Intelligence},
month = {February},
abstract = {Nonparametric inference techniques provide promising tools
for probabilistic reasoning in high-dimensional nonlinear systems.
Most of these techniques embed distributions into reproducing
kernel Hilbert spaces (RKHS) and rely on the kernel
Bayes? rule (KBR) to manipulate the embeddings. However,
the computational demands of the KBR scale poorly
with the number of samples and the KBR often suffers from
numerical instabilities. In this paper, we present the kernel
Kalman rule (KKR) as an alternative to the KBR. The derivation
of the KKR is based on recursive least squares, inspired
by the derivation of the Kalman innovation update. We apply
the KKR to filtering tasks where we use RKHS embeddings
to represent the belief state, resulting in the kernel Kalman filter
(KKF). We show on a nonlinear state estimation task with
high dimensional observations that our approach provides a
significantly improved estimation accuracy while the computational
demands are significantly decreased.},
url = {http://eprints.lincoln.ac.uk/26739/},
keywords = {ARRAY(0x56147f72beb8)}
}
• M. Hanheide, M. Göbelbecker, G. S. Horn, A. Pronobis, K. Sjöö, A. Aydemir, P. Jensfelt, C. Gretton, R. Dearden, M. Janicek, H. Zender, G. Kruijff, N. Hawes, and J. L. Wyatt, “Robot task planning and explanation in open and uncertain worlds,” Artificial Intelligence, vol. 247, pp. 119-150, 2017.

A long-standing goal of AI is to enable robots to plan in the face of uncertain and incomplete information, and to handle task failure intelligently. This paper shows how to achieve this. There are two central ideas. The first idea is to organize the robot’s knowledge into three layers: instance knowledge at the bottom, commonsense knowledge above that, and diagnostic knowledge on top. Knowledge in a layer above can be used to modify knowledge in the layer(s) below. The second idea is that the robot should represent not just how its actions change the world, but also what it knows or believes. There are two types of knowledge effects the robot’s actions can have: epistemic effects (I believe X because I saw it) and assumptions (I’ll assume X to be true). By combining the knowledge layers with the models of knowledge effects, we can simultaneously solve several problems in robotics: (i) task planning and execution under uncertainty; (ii) task planning and execution in open worlds; (iii) explaining task failure; (iv) verifying those explanations. The paper describes how the ideas are implemented in a three-layer architecture on a mobile robot platform. The robot implementation was evaluated in five different experiments on object search, mapping, and room categorization.

@article{lirolem18592,
publisher = {Elsevier},
author = {Marc Hanheide and Moritz G{\"o}belbecker and Graham S. Horn and Andrzej Pronobis and Kristoffer Sj{\"o}{\"o} and Alper Aydemir and Patric Jensfelt and Charles Gretton and Richard Dearden and Miroslav Janicek and Hendrik Zender and Geert-Jan Kruijff and Nick Hawes and Jeremy L. Wyatt},
month = {June},
pages = {119--150},
journal = {Artificial Intelligence},
title = {Robot task planning and explanation in open and uncertain worlds},
year = {2017},
volume = {247},
abstract = {A long-standing goal of AI is to enable robots to plan in the face of uncertain and incomplete information, and to handle task failure intelligently. This paper shows how to achieve this. There are two central ideas. The first idea is to organize the robot's knowledge into three layers: instance knowledge at the bottom, commonsense knowledge above that, and diagnostic knowledge on top. Knowledge in a layer above can be used to modify knowledge in the layer(s) below. The second idea is that the robot should represent not just how its actions change the world, but also what it knows or believes. There are two types of knowledge effects the robot's actions can have: epistemic effects (I believe X because I saw it) and assumptions (I'll assume X to be true). By combining the knowledge layers with the models of knowledge effects, we can simultaneously solve several problems in robotics: (i) task planning and execution under uncertainty; (ii) task planning and execution in open worlds; (iii) explaining task failure; (iv) verifying those explanations. The paper describes how the ideas are implemented in a three-layer architecture on a mobile robot platform. The robot implementation was evaluated in five different experiments on object search, mapping, and room categorization.},
keywords = {ARRAY(0x56147f662b18)},
url = {http://eprints.lincoln.ac.uk/18592/}
}
• M. Hanheide, D. Hebesberger, and T. Krajnik, “The when, where, and how: an adaptive robotic info-terminal for care home residents ? a long-term study,” in Int. Conf. on Human-Robot Interaction (HRI), Vienna, 2017.

Adapting to users’ intentions is a key requirement for autonomous robots in general, and in care settings in particular. In this paper, a comprehensive long-term study of a mobile robot providing information services to residents, visitors, and staff of a care home is presented with a focus on adapting to the when and where the robot should be offering its services to best accommodate the users’ needs. Rather than providing a fixed schedule, the presented system takes the opportunity of long-term deployment to explore the space of possibilities of interaction while concurrently exploiting the model learned to provide better services. But in order to provide effective services to users in a care home, not only then when and where are relevant, but also the way how the information is provided and accessed. Hence, also the usability of the deployed system is studied specifically, in order to provide a most comprehensive overall assessment of a robotic info-terminal implementation in a care setting. Our results back our hypotheses, (i) that learning a spatiotemporal model of users’ intentions improves efficiency and usefulness of the system, and (ii) that the specific information sought after is indeed dependent on the location the info-terminal is offered.

@inproceedings{lirolem25866,
booktitle = {Int. Conf. on Human-Robot Interaction (HRI)},
month = {March},
year = {2017},
title = {The when, where, and how: an adaptive robotic info-terminal for care home residents ? a long-term study},
author = {Marc Hanheide and Denise Hebesberger and Tomas Krajnik},
publisher = {ACM/IEEE},
abstract = {Adapting to users' intentions is a key requirement for autonomous robots in general, and in care settings in particular. In this paper, a comprehensive long-term study of a mobile robot providing information services to residents, visitors, and staff of a care home is presented with a focus on adapting to the when and where the robot should be offering its services to best accommodate the users' needs. Rather than providing a fixed schedule, the presented system takes the opportunity of long-term deployment to explore the space of possibilities of interaction while concurrently exploiting the model learned to provide better services. But in order to provide effective services to users in a care home, not only then when and where are relevant, but also the way how the information is provided and accessed. Hence, also the usability of the deployed system is studied specifically, in order to provide a most comprehensive overall assessment of a robotic info-terminal implementation in a care setting. Our results back our hypotheses, (i) that learning a spatiotemporal model of users' intentions improves efficiency and usefulness of the system, and (ii) that the specific information sought after is indeed dependent on the location the info-terminal is offered.},
url = {http://eprints.lincoln.ac.uk/25866/}
}
• D. Hebesberger, C. Dondrup, C. Gisinger, and M. Hanheide, “Patterns of use: how older adults with progressed dementia interact with a robot,” in Proc ACM/IEEE Int. Conf. on Human-Robot Interaction (HRI) Late Breaking Reports, Vienna, 2017.

Older adults represent a new user group of robots that are deployed in their private homes or in care facilities. In the presented study tangible aspects of older adults’ interaction with an autonomous robot were focused. The robot was deployed as a companion in physical therapy for older adults with progressed dementia. Interaction was possible via a mounted touch screen. The menu was structured in a single layer and icons were big and with strong contrast. Employing a detailed observation protocol, interaction frequencies and contexts were assessed. Thereby, it was found that most of the interaction was encouraged by the therapists and that two out of 12 older adults with progressed dementia showed self-inducted interactions.

@inproceedings{lirolem25867,
title = {Patterns of use: how older adults with progressed dementia interact with a robot},
year = {2017},
publisher = {ACM/IEEE},
author = {Denise Hebesberger and Christian Dondrup and Christoph Gisinger and Marc Hanheide},
booktitle = {Proc ACM/IEEE Int. Conf. on Human-Robot Interaction (HRI) Late Breaking Reports},
month = {March},
abstract = {Older adults represent a new user group of robots that are deployed in their private homes or in care facilities. In the presented study tangible aspects of older adults' interaction with an autonomous robot were focused. The robot was deployed as a companion in physical therapy for older adults with progressed dementia. Interaction was possible via a mounted touch screen. The menu was structured in a single layer and icons were big and with strong contrast. Employing a detailed observation protocol, interaction frequencies and contexts were assessed. Thereby, it was found that most of the interaction was encouraged by the therapists and that two out of 12 older adults with progressed dementia showed self-inducted interactions.},
keywords = {ARRAY(0x56147f87ae60)},
url = {http://eprints.lincoln.ac.uk/25867/}
}
• M. Heshmat, M. Fernandez-Carmona, Z. Yan, and N. Bellotto, “Active human detection with a mobile robot,” in UK-RAS Conference on Robotics and Autonomous Systems, 2017.

The problem of active human detection with a mobile robot equipped with an RGB-D camera is considered in this work. Traditional human detection algorithms for indoor mobile robots face several challenges, including occlusions due to cluttered dynamic environments, changing backgrounds, and large variety of human movements. Active human detection aims to improve classic detection systems by actively selecting new and potentially better observation points of the person. In this preliminary work, we present a system that actively guides a mobile robot towards high-confidence human detections, including initial simulation tests that highlight pros and cons of the proposed approach.

@inproceedings{lirolem29946,
author = {Mohamed Heshmat and Manuel Fernandez-Carmona and Zhi Yan and Nicola Bellotto},
title = {Active human detection with a mobile robot},
year = {2017},
booktitle = {UK-RAS Conference on Robotics and Autonomous Systems},
month = {December},
keywords = {ARRAY(0x56147f774080)},
url = {http://eprints.lincoln.ac.uk/29946/},
abstract = {The problem of active human detection with a mobile robot equipped with an RGB-D camera is considered in this work. Traditional human detection algorithms for indoor mobile robots face several challenges, including occlusions due to cluttered dynamic environments, changing backgrounds, and large variety of human movements. Active human detection aims to improve classic detection systems by actively selecting new and potentially better observation points of the person. In this preliminary work, we present a system that actively guides a mobile robot towards high-confidence human detections, including initial simulation tests that highlight pros and cons of the proposed approach.}
}
• B. Hu, S. Yue, and Z. Zhang, “A rotational motion perception neural network based on asymmetric spatiotemporal visual information processing,” IEEE Transactions on Neural Networks and Learning Systems, vol. 28, iss. 11, pp. 2803-2821, 2017.

All complex motion patterns can be decomposed into several elements, including translation, expansion/contraction, and rotational motion. In biological vision systems, scientists have found that specific types of visual neurons have specific preferences to each of the three motion elements. There are computational models on translation and expansion/contraction perceptions; however, little has been done in the past to create computational models for rotational motion perception. To fill this gap, we proposed a neural network that utilizes a specific spatiotemporal arrangement of asymmetric lateral inhibited direction selective neural networks (DSNNs) for rotational motion perception. The proposed neural network consists of two parts-presynaptic and postsynaptic parts. In the presynaptic part, there are a number of lateral inhibited DSNNs to extract directional visual cues. In the postsynaptic part, similar to the arrangement of the directional columns in the cerebral cortex, these direction selective neurons are arranged in a cyclic order to perceive rotational motion cues. In the postsynaptic network, the delayed excitation from each direction selective neuron is multiplied by the gathered excitation from this neuron and its unilateral counterparts depending on which rotation, clockwise (cw) or counter-cw (ccw), to perceive. Systematic experiments under various conditions and settings have been carried out and validated the robustness and reliability of the proposed neural network in detecting cw or ccw rotational motion. This research is a critical step further toward dynamic visual information processing.

@article{lirolem24936,
author = {Bin Hu and Shigang Yue and Zhuhong Zhang},
publisher = {IEEE},
month = {November},
pages = {2803--2821},
volume = {28},
year = {2017},
title = {A rotational motion perception neural network based on asymmetric spatiotemporal visual information processing},
journal = {IEEE Transactions on Neural Networks and Learning Systems},
number = {11},
keywords = {ARRAY(0x56147f6a8ca8)},
url = {http://eprints.lincoln.ac.uk/24936/},
abstract = {All complex motion patterns can be decomposed into several elements, including translation, expansion/contraction, and rotational motion. In biological vision systems, scientists have found that specific types of visual neurons have specific preferences to each of the three motion elements. There are computational models on translation and expansion/contraction perceptions; however, little has been done in the past to create computational models for rotational motion perception. To fill this gap, we proposed a neural network that utilizes a specific spatiotemporal arrangement of asymmetric lateral inhibited direction selective neural networks (DSNNs) for rotational motion perception. The proposed neural network consists of two parts-presynaptic and postsynaptic parts. In the presynaptic part, there are a number of lateral inhibited DSNNs to extract directional visual cues. In the postsynaptic part, similar to the arrangement of the directional columns in the cerebral cortex, these direction selective neurons are arranged in a cyclic order to perceive rotational motion cues. In the postsynaptic network, the delayed excitation from each direction selective neuron is multiplied by the gathered excitation from this neuron and its unilateral counterparts depending on which rotation, clockwise (cw) or counter-cw (ccw), to perceive. Systematic experiments under various conditions and settings have been carried out and validated the robustness and reliability of the proposed neural network in detecting cw or ccw rotational motion. This research is a critical step further toward dynamic visual information processing.}
}
• C. Keeble, P. A. Thwaites, P. D. Baxter, S. Barber, R. C. Parslow, and G. R. Law, “Learning through chain event graphs: the role of maternal factors in childhood type I diabetes,” American Journal of Epidemiology, vol. 186, iss. 10, pp. 1204-1208, 2017.

.

@article{lirolem26599,
pages = {1204--1208},
month = {November},
author = {C. Keeble and P. A. Thwaites and P. D. Baxter and S. Barber and R. C. Parslow and G. R. Law},
publisher = {Oxford University Press},
number = {10},
journal = {American Journal of Epidemiology},
year = {2017},
title = {Learning through chain event graphs: the role of maternal factors in childhood type I diabetes},
volume = {186},
url = {http://eprints.lincoln.ac.uk/26599/},
keywords = {ARRAY(0x56147f93b320)},
abstract = {.}
}
• C. Keeble, P. A. Thwaites, S. Barber, G. R. Law, and P. D. Baxter, “Adaptation of chain event graphs for use with case-Control studies in epidemiology,” The International Journal of Biostatistics, vol. 13, iss. 2, 2017.

Case-control studies are used in epidemiology to try to uncover the causes of diseases, but are a retrospective study design known to suffer from non-participation and recall bias, which may explain their decreased popularity in recent years. Traditional analyses report usually only the odds ratio for given exposures and the binary disease status. Chain event graphs are a graphical representation of a statistical model derived from event trees which have been developed in artificial intelligence and statistics, and only recently introduced to the epidemiology literature. They are a modern Bayesian technique which enable prior knowledge to be incorporated into the data analysis using the agglomerative hierarchical clustering algorithm, used to form a suitable chain event graph. Additionally, they can account for missing data and be used to explore missingness mechanisms. Here we adapt the chain event graph framework to suit scenarios often encountered in case-control studies, to strengthen this study design which is time and financially efficient. We demonstrate eight adaptations to the graphs, which consist of two suitable for full case-control study analysis, four which can be used in interim analyses to explore biases, and two which aim to improve the ease and accuracy of analyses. The adaptations are illustrated with complete, reproducible, fully-interpreted examples, including the event tree and chain event graph. Chain event graphs are used here for the first time to summarise non-participation, data collection techniques, data reliability, and disease severity in case-control studies. We demonstrate how these features of a case-control study can be incorporated into the analysis to provide further insight, which can help to identify potential biases and lead to more accurate study results.

@article{lirolem29511,
volume = {13},
journal = {The International Journal of Biostatistics},
title = {Adaptation of chain event graphs for use with case-Control studies in epidemiology},
year = {2017},
number = {2},
publisher = {De Gruyter},
author = {Claire Keeble and Peter Adam Thwaites and Stuart Barber and Graham Richard Law and Paul David Baxter},
month = {December},
keywords = {ARRAY(0x56147f515618)},
url = {http://eprints.lincoln.ac.uk/29511/},
abstract = {Case-control studies are used in epidemiology to try to uncover the causes of diseases, but are a retrospective study design known to suffer from non-participation and recall bias, which may explain their decreased popularity in recent years. Traditional analyses report usually only the odds ratio for given exposures and the binary disease status. Chain event graphs are a graphical representation of a statistical model derived from event trees which have been developed in artificial intelligence and statistics, and only recently introduced to the epidemiology literature. They are a modern Bayesian technique which enable prior knowledge to be incorporated into the data analysis using the agglomerative hierarchical clustering algorithm, used to form a suitable chain event graph. Additionally, they can account for missing data and be used to explore missingness mechanisms. Here we adapt the chain event graph framework to suit scenarios often encountered in case-control studies, to strengthen this study design which is time and financially efficient. We demonstrate eight adaptations to the graphs, which consist of two suitable for full case-control study analysis, four which can be used in interim analyses to explore biases, and two which aim to improve the ease and accuracy of analyses. The adaptations are illustrated with complete, reproducible, fully-interpreted examples, including the event tree and chain event graph. Chain event graphs are used here for the first time to summarise non-participation, data collection techniques, data reliability, and disease severity in case-control studies. We demonstrate how these features of a case-control study can be incorporated into the analysis to provide further insight, which can help to identify potential biases and lead to more accurate study results.}
}
• S. Keizer, M. Guhe, H. Cuayahuitl, I. Efstathiou, K. Engelbrecht, M. Dobre, A. Lascarides, and O. Lemon, “Evaluating persuasion strategies and deep reinforcement learning methods for negotiation dialogue agents,” in 15th Conference of the European chapter of the Association for Computational Linguistics, 2017.

In this paper we present a comparative evaluation of various negotiation strategies within an online version of the game ?Settlers of Catan?. The comparison is based on human subjects playing games against artificial game-playing agents (?bots?) which implement different negotiation dialogue strategies, using a chat dialogue interface to negotiate trades. Our results suggest that a negotiation strategy that uses persuasion, as well as a strategy that is trained from data using Deep Reinforcement Learning, both lead to an improved win rate against humans, compared to previous rule-based and supervised learning baseline dialogue negotiators.

@inproceedings{lirolem26621,
author = {Simon Keizer and Markus Guhe and Heriberto Cuayahuitl and Ioannis Efstathiou and Klaus-Peter Engelbrecht and Mihai Dobre and Alex Lascarides and Oliver Lemon},
publisher = {ACL},
title = {Evaluating persuasion strategies and deep reinforcement learning methods for negotiation dialogue agents},
year = {2017},
booktitle = {15th Conference of the European chapter of the Association for Computational Linguistics},
month = {April},
url = {http://eprints.lincoln.ac.uk/26621/},
keywords = {ARRAY(0x56147f575a18)},
abstract = {In this paper we present a comparative evaluation of various negotiation strategies within an online version of the game ?Settlers of Catan?. The comparison is based on human subjects playing games against artificial game-playing
agents (?bots?) which implement different negotiation dialogue strategies, using a chat dialogue interface to negotiate trades. Our results suggest that a negotiation strategy that uses persuasion, as well as a strategy that is trained from data using Deep Reinforcement Learning, both lead to an improved win rate against humans, compared to previous rule-based and supervised learning baseline dialogue negotiators.}
}
• J. Kennedy, P. Baxter, and T. Belpaeme, “Nonverbal immediacy as a characterisation of social behaviour for human-robot interaction,” International Journal of Social Robotics, vol. 9, iss. 1, pp. 109-128, 2017.

An increasing amount of research has started to explore the impact of robot social behaviour on the outcome of a goal for a human interaction partner, such as cognitive learning gains. However, it remains unclear from what principles the social behaviour for such robots should be derived. Human models are often used, but in this paper an alternative approach is proposed. First, the concept of nonverbal immediacy from the communication literature is introduced, with a focus on how it can provide a characterisation of social behaviour, and the subsequent outcomes of such behaviour. A literature review is conducted to explore the impact on learning of the social cues which form the nonverbal immediacy measure. This leads to the production of a series of guidelines for social robot behaviour. The resulting behaviour is evaluated in a more general context, where both children and adults judge the immediacy of humans and robots in a similar manner, and their recall of a short story is tested. Children recall more of the story when the robot is more immediate, which demonstrates an e?ffect predicted by the literature. This study provides validation for the application of nonverbal immediacy to child-robot interaction. It is proposed that nonverbal immediacy measures could be used as a means of characterising robot social behaviour for human-robot interaction.

@article{lirolem24215,
number = {1},
year = {2017},
volume = {9},
journal = {International Journal of Social Robotics},
title = {Nonverbal immediacy as a characterisation of social behaviour for human-robot interaction},
author = {James Kennedy and Paul Baxter and Tony Belpaeme},
publisher = {Springer},
pages = {109--128},
month = {January},
keywords = {ARRAY(0x56147f62e9e8)},
url = {http://eprints.lincoln.ac.uk/24215/},
abstract = {An increasing amount of research has started
to explore the impact of robot social behaviour on the
outcome of a goal for a human interaction partner, such
as cognitive learning gains. However, it remains unclear
from what principles the social behaviour for such robots
should be derived. Human models are often used, but
in this paper an alternative approach is proposed. First,
the concept of nonverbal immediacy from the communication
literature is introduced, with a focus on how it
can provide a characterisation of social behaviour, and
the subsequent outcomes of such behaviour. A literature
review is conducted to explore the impact on learning
of the social cues which form the nonverbal immediacy
measure. This leads to the production of a series
of guidelines for social robot behaviour. The resulting
behaviour is evaluated in a more general context, where
both children and adults judge the immediacy of humans
and robots in a similar manner, and their recall of
a short story is tested. Children recall more of the story
when the robot is more immediate, which demonstrates
an e?ffect predicted by the literature. This study provides
validation for the application of nonverbal immediacy
to child-robot interaction. It is proposed that nonverbal
immediacy measures could be used as a means of
characterising robot social behaviour for human-robot
interaction.}
}
• J. Kennedy, P. Baxter, and T. Belpaeme, “The impact of robot tutor nonverbal social behavior on child learning,” Frontiers in ICT, 2017.

Several studies have indicated that interacting with social robots in educational contexts may lead to a greater learning than interactions with computers or virtual agents. As such, an increasing amount of social human?robot interaction research is being conducted in the learning domain, particularly with children. However, it is unclear precisely what social behavior a robot should employ in such interactions. Inspiration can be taken from human?human studies; this often leads to an assumption that the more social behavior an agent utilizes, the better the learning outcome will be. We apply a nonverbal behavior metric to a series of studies in which children are taught how to identify prime numbers by a robot with various behavioral manipulations. We find a trend, which generally agrees with the pedagogy literature, but also that overt nonverbal behavior does not account for all learning differences. We discuss the impact of novelty, child expectations, and responses to social cues to further the understanding of the relationship between robot social behavior and learning. We suggest that the combination of nonverbal behavior and social cue congruency is necessary to facilitate learning.

@article{lirolem27043,
month = {April},
author = {James Kennedy and Paul Baxter and Tony Belpaeme},
publisher = {Frontiers Media},
journal = {Frontiers in ICT},
year = {2017},
title = {The impact of robot tutor nonverbal social behavior on child learning},
Affective and Social Signals for HRI},
url = {http://eprints.lincoln.ac.uk/27043/},
keywords = {ARRAY(0x56147f790828)},
abstract = {Several studies have indicated that interacting with social robots in educational contexts may lead to a greater learning than interactions with computers or virtual agents. As such, an increasing amount of social human?robot interaction research is being conducted in the learning domain, particularly with children. However, it is unclear precisely what social behavior a robot should employ in such interactions. Inspiration can be taken from human?human studies; this often leads to an assumption that the more social behavior an agent utilizes, the better the learning outcome will be. We apply a nonverbal behavior metric to a series of studies in which children are taught how to identify prime numbers by a robot with various behavioral manipulations. We find a trend, which generally agrees with the pedagogy literature, but also that overt nonverbal behavior does not account for all learning differences. We discuss the impact of novelty, child expectations, and responses to social cues to further the understanding of the relationship between robot social behavior and learning. We suggest that the combination of nonverbal behavior and social cue congruency is necessary to facilitate learning.}
}
• T. Krajnik, J. P. Fentanes, J. Santos, and T. Duckett, “FreMEn: Frequency map enhancement for long-term mobile robot autonomy in changing environments,” Robotics, IEEE Transactions on [see also Robotics and Automation, IEEE Transactions on], vol. 33, iss. 4, pp. 964-977, 2017.

We present a new approach to long-term mobile robot mapping in dynamic indoor environments. Unlike traditional world models that are tailored to represent static scenes, our approach explicitly models environmental dynamics. We assume that some of the hidden processes that influence the dynamic environment states are periodic and model the uncertainty of the estimated state variables by their frequency spectra. The spectral model can represent arbitrary timescales of environment dynamics with low memory requirements. Transformation of the spectral model to the time domain allows for the prediction of the future environment states, which improves the robot’s long-term performance in dynamic environments. Experiments performed over time periods of months to years demonstrate that the approach can efficiently represent large numbers of observations and reliably predict future environment states. The experiments indicate that the model’s predictive capabilities improve mobile robot localisation and navigation in changing environments.

@article{lirolem26196,
publisher = {IEEE},
author = {Tomas Krajnik and Jaime Pulido Fentanes and Joao Santos and Tom Duckett},
month = {August},
pages = {964--977},
year = {2017},
title = {FreMEn: Frequency map enhancement for long-term mobile robot autonomy in changing environments},
journal = {Robotics, IEEE Transactions on [see also Robotics and Automation, IEEE Transactions on]},
volume = {33},
number = {4},
abstract = {We present a new approach to long-term mobile robot mapping in dynamic indoor environments. Unlike traditional world models that are tailored to represent static scenes, our approach explicitly models environmental dynamics. We assume that some of the hidden processes that influence the dynamic environment states are periodic and model the uncertainty of the estimated state variables by their frequency spectra. The spectral model can represent arbitrary timescales of environment dynamics with low memory requirements. Transformation of the spectral model to the time domain allows for the prediction of the future environment states, which improves the robot's long-term performance in dynamic environments. Experiments performed over time periods of months to years demonstrate that the approach can efficiently represent large numbers of observations and reliably predict future environment states. The experiments indicate that the model's predictive capabilities improve mobile robot localisation and navigation in changing environments.},
url = {http://eprints.lincoln.ac.uk/26196/},
keywords = {ARRAY(0x56147f983fb0)}
}
• T. Krajnik, P. Cristoforis, K. Kusumam, P. Neubert, and T. Duckett, “Image features for visual teach-and-repeat navigation in changing environments,” Robotics and Autonomous Systems, vol. 88, pp. 127-141, 2017.

We present an evaluation of standard image features in the context of long-term visual teach-and-repeat navigation of mobile robots, where the environment exhibits significant changes in appearance caused by seasonal weather variations and daily illumination changes. We argue that for long-term autonomous navigation, the viewpoint-, scale- and rotation- invariance of the standard feature extractors is less important than their robustness to the mid- and long-term environment appearance changes. Therefore, we focus our evaluation on the robustness of image registration to variable lighting and naturally-occurring seasonal changes. We combine detection and description components of different image extractors and evaluate their performance on five datasets collected by mobile vehicles in three different outdoor environments over the course of one year. Moreover, we propose a trainable feature descriptor based on a combination of evolutionary algorithms and Binary Robust Independent Elementary Features, which we call GRIEF (Generated BRIEF). In terms of robustness to seasonal changes, the most promising results were achieved by the SpG/CNN and the STAR/GRIEF feature, which was slightly less robust, but faster to calculate.

@article{lirolem25239,
year = {2017},
journal = {Robotics and Autonomous Systems},
title = {Image features for visual teach-and-repeat navigation in changing environments},
volume = {88},
pages = {127--141},
month = {February},
author = {Tomas Krajnik and Pablo Cristoforis and Keerthy Kusumam and Peer Neubert and Tom Duckett},
publisher = {Elsevier},
keywords = {ARRAY(0x56147f60eee8)},
url = {http://eprints.lincoln.ac.uk/25239/},
abstract = {We present an evaluation of standard image features in the context of long-term visual teach-and-repeat navigation of mobile robots, where the environment exhibits significant changes in appearance caused by seasonal weather variations and daily illumination changes. We argue that for long-term autonomous navigation, the viewpoint-, scale- and rotation- invariance of the standard feature extractors is less important than their robustness to the mid- and long-term environment appearance changes. Therefore, we focus our evaluation on the robustness of image registration to variable lighting and naturally-occurring seasonal changes. We combine detection and description components of different image extractors and evaluate their performance on five datasets collected by mobile vehicles in three different outdoor environments over the course of one year. Moreover, we propose a trainable feature descriptor based on a combination of evolutionary algorithms and Binary Robust Independent Elementary Features, which we call GRIEF (Generated BRIEF). In terms of robustness to seasonal changes, the most promising results were achieved by the SpG/CNN and the STAR/GRIEF feature, which was slightly less robust, but faster to calculate.}
}
• A. Kupcsik, M. P. Deisenroth, J. Peters, A. P. Loh, P. Vadakkepat, and G. Neumann, “Model-based contextual policy search for data-efficient generalization of robot skills,” Artificial Intelligence, vol. 247, pp. 415-439, 2017.

In robotics, lower-level controllers are typically used to make the robot solve a specific task in a fixed context. For example, the lower-level controller can encode a hitting movement while the context defines the target coordinates to hit. However, in many learning problems the context may change between task executions. To adapt the policy to a new context, we utilize a hierarchical approach by learning an upper-level policy that generalizes the lower-level controllers to new contexts. A common approach to learn such upper-level policies is to use policy search. However, the majority of current contextual policy search approaches are model-free and require a high number of interactions with the robot and its environment. Model-based approaches are known to significantly reduce the amount of robot experiments, however, current model-based techniques cannot be applied straightforwardly to the problem of learning contextual upper-level policies. They rely on specific parametrizations of the policy and the reward function, which are often unrealistic in the contextual policy search formulation. In this paper, we propose a novel model-based contextual policy search algorithm that is able to generalize lower-level controllers, and is data-efficient. Our approach is based on learned probabilistic forward models and information theoretic policy search. Unlike current algorithms, our method does not require any assumption on the parametrization of the policy or the reward function. We show on complex simulated robotic tasks and in a real robot experiment that the proposed learning framework speeds up the learning process by up to two orders of magnitude in comparison to existing methods, while learning high quality policies.

@article{lirolem25774,
title = {Model-based contextual policy search for data-efficient generalization of robot skills},
journal = {Artificial Intelligence},
volume = {247},
year = {2017},
pages = {415--439},
month = {June},
publisher = {Elsevier},
author = {A. Kupcsik and M. P. Deisenroth and J. Peters and A. P. Loh and P. Vadakkepat and G. Neumann},
keywords = {ARRAY(0x56147f6037b8)},
url = {http://eprints.lincoln.ac.uk/25774/},
abstract = {In robotics, lower-level controllers are typically used to make the robot solve a specific task in a fixed context. For example, the lower-level controller can encode a hitting movement while the context defines the target coordinates to hit. However, in many learning problems the context may change between task executions. To adapt the policy to a new context, we utilize a hierarchical approach by learning an upper-level policy that generalizes the lower-level controllers to new contexts. A common approach to learn such upper-level policies is to use policy search. However, the majority of current contextual policy search approaches are model-free and require a high number of interactions with the robot and its environment. Model-based approaches are known to significantly reduce the amount of robot experiments, however, current model-based techniques cannot be applied straightforwardly to the problem of learning contextual upper-level policies. They rely on specific parametrizations of the policy and the reward function, which are often unrealistic in the contextual policy search formulation. In this paper, we propose a novel model-based contextual policy search algorithm that is able to generalize lower-level controllers, and is data-efficient. Our approach is based on learned probabilistic forward models and information theoretic policy search. Unlike current algorithms, our method does not require any assumption on the parametrization of the policy or the reward function. We show on complex simulated robotic tasks and in a real robot experiment that the proposed learning framework speeds up the learning process by up to two orders of magnitude in comparison to existing methods, while learning high quality policies.}
}
• K. Kusumam, T. Krajnik, S. Pearson, T. Duckett, and G. Cielniak, “3D-vision based detection, localization, and sizing of broccoli heads in the field,” Journal of Field Robotics, vol. 34, iss. 8, pp. 1505-1518, 2017.

@article{lirolem27782,
year = {2017},
title = {3D-vision based detection, localization, and sizing of broccoli heads in the field},
volume = {34},
journal = {Journal of Field Robotics},
number = {8},
publisher = {Wiley Periodicals, Inc.},
author = {Keerthy Kusumam and Tomas Krajnik and Simon Pearson and Tom Duckett and Grzegorz Cielniak},
month = {December},
pages = {1505--1518},
url = {http://eprints.lincoln.ac.uk/27782/},
keywords = {ARRAY(0x56147f79f6f0)}
}
• D. Liciotti, T. Duckett, N. Bellotto, E. Frontoni, and P. Zingaretti, “HMM-based activity recognition with a ceiling RGB-D camera,” in ICPRAM – 6th International Conference on Pattern Recognition Applications and Methods, 2017.

Automated recognition of Activities of Daily Living allows to identify possible health problems and apply corrective strategies in Ambient Assisted Living (AAL). Activities of Daily Living analysis can provide very useful information for elder care and long-term care services. This paper presents an automated RGB-D video analysis system that recognises human ADLs activities, related to classical daily actions. The main goal is to predict the probability of an analysed subject action. Thus, the abnormal behaviour can be detected. The activity detection and recognition is performed using an affordable RGB-D camera. Human activities, despite their unstructured nature, tend to have a natural hierarchical structure; for instance, generally making a coffee involves a three-step process of turning on the coffee machine, putting sugar in cup and opening the fridge for milk. Action sequence recognition is then handled using a discriminative Hidden Markov Model (HMM). RADiaL, a dataset with RGB-D images and 3D position of each person for training as well as evaluating the HMM, has been built and made publicly available.

@inproceedings{lirolem25361,
month = {February},
booktitle = {ICPRAM - 6th International Conference on Pattern Recognition Applications and Methods},
author = {Daniele Liciotti and Tom Duckett and Nicola Bellotto and Emanuele Frontoni and Primo Zingaretti},
title = {HMM-based activity recognition with a ceiling RGB-D camera},
year = {2017},
abstract = {Automated recognition of Activities of Daily Living allows to identify possible health problems and apply corrective strategies in Ambient Assisted Living (AAL). Activities of Daily Living analysis can provide very useful information for elder care and long-term care services. This paper presents an automated RGB-D video analysis system that recognises human ADLs activities, related to classical daily actions. The main goal is to predict the probability of an analysed subject action. Thus, the abnormal behaviour can be detected. The activity detection and recognition is performed using an affordable RGB-D camera. Human activities, despite their unstructured nature, tend to have a natural hierarchical structure; for instance, generally making a coffee involves a three-step process of turning on the coffee machine, putting sugar in cup and opening the fridge for milk. Action sequence recognition is then handled using a discriminative Hidden Markov Model (HMM). RADiaL, a dataset with RGB-D images and 3D position of each person for training as well as evaluating the HMM, has been built and made publicly available.},
url = {http://eprints.lincoln.ac.uk/25361/},
keywords = {ARRAY(0x56147f5aa2b8)}
}
• P. Lightbody, M. Hanheide, and T. Krajnik, “An efficient visual fiducial localisation system,” Applied Computing Review, vol. 17, iss. 3, pp. 28-37, 2017.

With use cases that range from external localisation of single robots or robotic swarms to self-localisation in marker-augmented environments and simplifying perception by tagging objects in a robot’s surrounding, fiducial markers have a wide field of application in the robotic world. We propose a new family of circular markers which allow for both computationally efficient detection, tracking and identification and full 6D position estimation. At the core of the proposed approach lies the separation of the detection and identification steps, with the former using computationally efficient circular marker detection and the latter utilising an open-ended necklace encoding’, allowing scalability to a large number of individual markers. While the proposed algorithm achieves similar accuracy to other state-of-the-art methods, its experimental evaluation in realistic conditions demonstrates that it can detect markers from larger distances while being up to two orders of magnitude faster than other state-of-the-art fiducial marker detection methods. In addition, the entire system is available as an open-source package at $\backslash$url\https://github.com/LCAS/whycon\.

@article{lirolem29678,
publisher = {ACM},
author = {Peter Lightbody and Marc Hanheide and Tomas Krajnik},
note = {Copyright is held by the authors. This work is based on an earlier work: SAC?17 Proceedings of the 2017 ACM Symposium on Applied Computing, Copyright 2017 ACM 978-1-4503-4486-9. http://dx.doi.org/10. 1145/3019612.3019709},
month = {September},
pages = {28--37},
year = {2017},
title = {An efficient visual fiducial localisation system},
volume = {17},
journal = {Applied Computing Review},
number = {3},
abstract = {With use cases that range from external localisation of single robots or robotic swarms to self-localisation in marker-augmented environments and simplifying perception by tagging objects in a robot's surrounding, fiducial markers have a wide field of application in the robotic world.
We propose a new family of circular markers which allow for both computationally efficient detection, tracking and identification and full 6D position estimation.
At the core of the proposed approach lies the separation of the detection and identification steps, with the former using computationally efficient circular marker detection and the latter utilising an open-ended necklace encoding', allowing scalability to a large number of individual markers.
While the proposed algorithm achieves similar accuracy to other state-of-the-art methods, its experimental evaluation in realistic conditions demonstrates that it can detect markers from larger distances while being up to two orders of magnitude faster than other state-of-the-art fiducial marker detection methods. In addition, the entire system is available as an open-source package at {$\backslash$}url\{https://github.com/LCAS/whycon\}.},
keywords = {ARRAY(0x56147f6e73c8)},
url = {http://eprints.lincoln.ac.uk/29678/}
}
• P. Lightbody, M. Hanheide, and T. Krajnik, “A versatile high-performance visual fiducial marker detection system with scalable identity encoding,” in 32nd ACM Symposium on Applied Computing, 2017, pp. 1-7.

Fiducial markers have a wide field of applications in robotics, ranging from external localisation of single robots or robotic swarms, over self-localisation in marker-augmented environments, to simplifying perception by tagging objects in a robot?s surrounding. We propose a new family of circular markers allowing for a computationally efficient detection, identification and full 3D position estimation. A key concept of our system is the separation of the detection and identification steps, where the first step is based on a computationally efficient circular marker detection, and the identification step is based on an open-ended ?Necklace code?, which allows for a theoretically infinite number of individually identifiable markers. The experimental evaluation of the system on a real robot indicates that while the proposed algorithm achieves similar accuracy to other state-of-the-art methods, it is faster by two orders of magnitude and it can detect markers from longer distances.

@inproceedings{lirolem25828,
year = {2017},
title = {A versatile high-performance visual fiducial marker detection system with scalable identity encoding},
author = {Peter Lightbody and Marc Hanheide and Tomas Krajnik},
publisher = {Association for Computing Machinery},
month = {April},
pages = {1--7},
booktitle = {32nd ACM Symposium on Applied Computing},
abstract = {Fiducial markers have a wide field of applications in robotics, ranging from external localisation of single robots or robotic swarms, over self-localisation in marker-augmented environments, to simplifying perception by tagging objects in a robot?s surrounding. We propose a new family of circular markers allowing for a computationally efficient detection, identification and full 3D position estimation. A key concept of our system is the separation of the detection and identification steps, where the first step is based on a computationally efficient circular marker detection, and the identification step is based on an open-ended ?Necklace code?, which allows for a theoretically infinite number of individually identifiable markers. The experimental evaluation of the system on a real robot indicates that while the proposed algorithm achieves similar accuracy to other state-of-the-art methods, it is faster by two orders of magnitude and it can detect markers from longer distances.},
keywords = {ARRAY(0x56147f7a43b0)},
url = {http://eprints.lincoln.ac.uk/25828/}
}
• R. Lioutikov, G. Neumann, G. Maeda, and J. Peters, “Learning movement primitive libraries through probabilistic segmentation,” International Journal of Robotics Research (IJRR), vol. 36, iss. 8, pp. 879-894, 2017.

Movement primitives are a well established approach for encoding and executing movements. While the primitives themselves have been extensively researched, the concept of movement primitive libraries has not received similar attention. Libraries of movement primitives represent the skill set of an agent. Primitives can be queried and sequenced in order to solve specific tasks. The goal of this work is to segment unlabeled demonstrations into a representative set of primitives. Our proposed method differs from current approaches by taking advantage of the often neglected, mutual dependencies between the segments contained in the demonstrations and the primitives to be encoded. By exploiting this mutual dependency, we show that we can improve both the segmentation and the movement primitive library. Based on probabilistic inference our novel approach segments the demonstrations while learning a probabilistic representation of movement primitives. We demonstrate our method on two real robot applications. First, the robot segments sequences of different letters into a library, explaining the observed trajectories. Second, the robot segments demonstrations of a chair assembly task into a movement primitive library. The library is subsequently used to assemble the chair in an order not present in the demonstrations.

@article{lirolem28021,
year = {2017},
title = {Learning movement primitive libraries through probabilistic segmentation},
journal = {International Journal of Robotics Research (IJRR)},
volume = {36},
number = {8},
author = {Rudolf Lioutikov and Gerhard Neumann and Guilherme Maeda and Jan Peters},
publisher = {SAGE},
month = {July},
pages = {879--894},
keywords = {ARRAY(0x56147f5fb0d0)},
url = {http://eprints.lincoln.ac.uk/28021/},
abstract = {Movement primitives are a well established approach for encoding and executing movements. While the primitives
themselves have been extensively researched, the concept of movement primitive libraries has not received similar
attention. Libraries of movement primitives represent the skill set of an agent. Primitives can be queried and sequenced
in order to solve specific tasks. The goal of this work is to segment unlabeled demonstrations into a representative
set of primitives. Our proposed method differs from current approaches by taking advantage of the often neglected,
mutual dependencies between the segments contained in the demonstrations and the primitives to be encoded. By
exploiting this mutual dependency, we show that we can improve both the segmentation and the movement primitive
library. Based on probabilistic inference our novel approach segments the demonstrations while learning a probabilistic
representation of movement primitives. We demonstrate our method on two real robot applications. First, the robot
segments sequences of different letters into a library, explaining the observed trajectories. Second, the robot segments
demonstrations of a chair assembly task into a movement primitive library. The library is subsequently used to assemble the chair in an order not present in the demonstrations.}
}
• D. Liu and S. Yue, “Fast unsupervised learning for visual pattern recognition using spike timing dependent plasticity,” Neurocomputing, vol. 249, pp. 212-224, 2017.

Real-time learning needs algorithms operating in a fast speed comparable to human or animal, however this is a huge challenge in processing visual inputs. Research shows a biological brain can process complicated real-life recognition scenarios at milliseconds scale. Inspired by biological system, in this paper, we proposed a novel real-time learning method by combing the spike timing-based feed-forward spiking neural network (SNN) and the fast unsupervised spike timing dependent plasticity learning method with dynamic post-synaptic thresholds. Fast cross-validated experiments using MNIST database showed the high e?ciency of the proposed method at an acceptable accuracy.

@article{lirolem26922,
journal = {Neurocomputing},
title = {Fast unsupervised learning for visual pattern recognition using spike timing dependent plasticity},
volume = {249},
year = {2017},
author = {Daqi Liu and Shigang Yue},
publisher = {Elsevier},
month = {August},
pages = {212--224},
abstract = {Real-time learning needs algorithms operating in a fast speed comparable to human or animal, however this is a huge challenge in processing visual inputs. Research shows a biological brain can process complicated real-life recognition scenarios at milliseconds scale. Inspired by biological system, in this paper, we proposed a novel real-time learning method by combing the spike timing-based feed-forward spiking neural network (SNN) and the fast unsupervised spike timing dependent plasticity learning method with dynamic post-synaptic thresholds. Fast cross-validated experiments using MNIST database showed the high e?ciency of the proposed method at an acceptable accuracy.},
keywords = {ARRAY(0x56147f8290e8)},
url = {http://eprints.lincoln.ac.uk/26922/}
}
• J. Lock, G. Cielniak, and N. Bellotto, “Portable navigations system with adaptive multimodal interface for the blind,” in AAAI 2017 Spring Symposium – Designing the User Experience of Machine Learning Systems, 2017.

Recent advances in mobile technology have the potential to radically change the quality of tools available for people with sensory impairments, in particular the blind. Nowadays almost every smart-phone and tablet is equipped with high resolutions cameras, which are typically used for photos and videos, communication purposes, games and virtual reality applications. Very little has been proposed to exploit these sensors for user localisation and navigation instead. To this end, the ?Active Vision with Human-in-the-Loop for the Visually Impaired? (ActiVis) project aims to develop a novel electronic travel aid to tackle the ?last 10 yards problem? and enable the autonomous navigation of blind users in unknown environments, ultimately enhancing or replacing existing solutions, such as guide dogs and white canes. This paper describes some of the key project?s challenges, in particular with respect to the design of the user interface that translate visual information from the camera to guiding instructions for the blind person, taking into account limitations due to the visual impairment and proposing a multimodal interface that embeds human-machine co-adaptation.

@inproceedings{lirolem25413,
booktitle = {AAAI 2017 Spring Symposium - Designing the User Experience of Machine Learning Systems},
month = {March},
publisher = {AAAI},
author = {Jacobus Lock and Grzegorz Cielniak and Nicola Bellotto},
title = {Portable navigations system with adaptive multimodal interface for the blind},
year = {2017},
url = {http://eprints.lincoln.ac.uk/25413/},
keywords = {ARRAY(0x56147f5c03b8)},
abstract = {Recent advances in mobile technology have the potential to radically change the quality of tools available for people with sensory impairments, in particular the blind. Nowadays almost every smart-phone and tablet is equipped with high resolutions cameras, which are typically used for photos and videos, communication purposes, games and virtual reality applications. Very little has been proposed to exploit these sensors for user localisation and navigation instead. To this end, the ?Active Vision with Human-in-the-Loop for the Visually Impaired? (ActiVis) project aims to develop a novel electronic travel aid to tackle the ?last 10 yards problem? and enable the autonomous navigation of blind users in unknown environments, ultimately enhancing or replacing existing solutions, such as guide dogs and white canes. This paper describes some of the key project?s challenges, in particular with respect to the design of the user interface that translate visual information from the camera to guiding instructions for the blind person, taking into account limitations due to the visual impairment and proposing a multimodal interface that embeds human-machine co-adaptation.}
}
• G. J. Maeda, G. Neumann, M. Ewerton, R. Lioutikov, O. Kroemer, and J. Peters, “Probabilistic movement primitives for coordination of multiple human?robot collaborative tasks,” Autonomous Robots, vol. 41, iss. 3, pp. 593-612, 2017.

This paper proposes an interaction learning method for collaborative and assistive robots based on movement primitives. The method allows for both action recognition and human?robot movement coordination. It uses imitation learning to construct a mixture model of human?robot interaction primitives. This probabilistic model allows the assistive trajectory of the robot to be inferred from human observations. The method is scalable in relation to the number of tasks and can learn nonlinear correlations between the trajectories that describe the human?robot interaction. We evaluated the method experimentally with a lightweight robot arm in a variety of assistive scenarios, including the coordinated handover of a bottle to a human, and the collaborative assembly of a toolbox. Potential applications of the method are personal caregiver robots, control of intelligent prosthetic devices, and robot coworkers in factories.

@article{lirolem25744,
number = {3},
volume = {41},
year = {2017},
journal = {Autonomous Robots},
title = {Probabilistic movement primitives for coordination of multiple human?robot collaborative tasks},
note = {Special Issue on Assistive and Rehabilitation Robotics},
author = {G. J. Maeda and G. Neumann and M. Ewerton and R. Lioutikov and O. Kroemer and J. Peters},
publisher = {Springer},
pages = {593--612},
month = {March},
keywords = {ARRAY(0x56147f917e98)},
url = {http://eprints.lincoln.ac.uk/25744/},
abstract = {This paper proposes an interaction learning method for collaborative and assistive robots based on movement primitives. The method allows for both action recognition and human?robot movement coordination. It uses imitation learning to construct a mixture model of human?robot interaction primitives. This probabilistic model allows the assistive trajectory of the robot to be inferred from human observations. The method is scalable in relation to the number of tasks and can learn nonlinear correlations between the trajectories that describe the human?robot interaction. We evaluated the method experimentally with a lightweight robot arm in a variety of assistive scenarios, including the coordinated handover of a bottle to a human, and the collaborative assembly of a toolbox. Potential applications of the method are personal caregiver robots, control of intelligent prosthetic devices, and robot coworkers in factories.}
}
• G. Maeda, M. Ewerton, G. Neumann, R. Lioutikov, and J. Peters, “Phase estimation for fast action recognition and trajectory generation in human?robot collaboration,” The International Journal of Robotics Research, vol. 36, iss. 13-14, pp. 1579-1594, 2017.

This paper proposes a method to achieve fast and fluid human?robot interaction by estimating the progress of the movement of the human. The method allows the progress, also referred to as the phase of the movement, to be estimated even when observations of the human are partial and occluded; a problem typically found when using motion capture systems in cluttered environments. By leveraging on the framework of Interaction Probabilistic Movement Primitives, phase estimation makes it possible to classify the human action, and to generate a corresponding robot trajectory before the human finishes his/her movement. The method is therefore suited for semi-autonomous robots acting as assistants and coworkers. Since observations may be sparse, our method is based on computing the probability of different phase candidates to find the phase that best aligns the Interaction Probabilistic Movement Primitives with the current observations. The method is fundamentally different from approaches based on Dynamic Time Warping that must rely on a consistent stream of measurements at runtime. The resulting framework can achieve phase estimation, action recognition and robot trajectory coordination using a single probabilistic representation. We evaluated the method using a seven-degree-of-freedom lightweight robot arm equipped with a five-finger hand in single and multi-task collaborative experiments. We compare the accuracy achieved by phase estimation with our previous method based on dynamic time warping.

@article{lirolem26734,
pages = {1579--1594},
month = {December},
author = {Guilherme Maeda and Marco Ewerton and Gerhard Neumann and Rudolf Lioutikov and Jan Peters},
publisher = {SAGE},
number = {13-14},
year = {2017},
journal = {The International Journal of Robotics Research},
volume = {36},
title = {Phase estimation for fast action recognition and trajectory generation in human?robot collaboration},
url = {http://eprints.lincoln.ac.uk/26734/},
keywords = {ARRAY(0x56147f8d6a78)},
abstract = {This paper proposes a method to achieve fast and fluid human?robot interaction by estimating the progress of the movement of the human. The method allows the progress, also referred to as the phase of the movement, to be estimated even when observations of the human are partial and occluded; a problem typically found when using motion capture systems in cluttered environments. By leveraging on the framework of Interaction Probabilistic Movement Primitives, phase estimation makes it possible to classify the human action, and to generate a corresponding robot trajectory before the human finishes his/her movement. The method is therefore suited for semi-autonomous robots acting as assistants and coworkers. Since observations may be sparse, our method is based on computing the probability of different phase candidates to find the phase that best aligns the Interaction Probabilistic Movement Primitives with the current observations. The method is fundamentally different from approaches based on Dynamic Time Warping that must rely on a consistent stream of measurements at runtime. The resulting framework can achieve phase estimation, action recognition and robot trajectory coordination using a single probabilistic representation. We evaluated the method using a seven-degree-of-freedom lightweight robot arm equipped with a five-finger hand in single and multi-task collaborative experiments. We compare the accuracy achieved by phase estimation with our previous method based on dynamic time warping.}
}
• S. M. Mellado, G. Cielniak, T. Krajnik, and T. Duckett, “Modelling and predicting rhythmic flow patterns in dynamic environments,” in UK-RAS Network Conference, 2017.

In this paper, we introduce a time-dependent probabilistic map able to model and predict future flow patterns of people in indoor environments. The proposed representation models the likelihood of motion direction by a set of harmonic functions, which efficiently capture long-term (hours to months) variations of crowd movements over time, so from a robotics perspective, this model could be useful to add the predicted human behaviour into the control loop to influence the actions of the robot. Our approach is evaluated with data collected from a real environment and initial qualitative results are presented.

@inproceedings{lirolem31053,
year = {2017},
title = {Modelling and predicting rhythmic flow patterns in dynamic environments},
author = {Sergi Molina Mellado and Grzegorz Cielniak and Tomas Krajnik and Tom Duckett},
booktitle = {UK-RAS Network Conference},
month = {December},
abstract = {In this paper, we introduce a time-dependent probabilistic map able to model and predict future flow patterns of people in indoor environments. The proposed representation models the likelihood of motion direction by a set of harmonic functions, which efficiently capture long-term (hours to months) variations of crowd movements over time, so from a robotics perspective, this model could be useful to add the predicted human behaviour into the control loop to influence the actions of the robot. Our approach is evaluated with data collected from a real environment and initial qualitative results are presented.},
url = {http://eprints.lincoln.ac.uk/31053/},
keywords = {ARRAY(0x56147f9b2630)}
}
• O. M. Mozos, V. Sandulescu, S. Andrews, D. Ellis, N. Bellotto, R. Dobrescu, and J. M. Ferrandez, “Stress detection using wearable physiological and sociometric sensors,” International Journal of Neural Systems, vol. 27, iss. 2, p. 1650041, 2017.

Stress remains a significant social problem for individuals in modern societies. This paper presents a machine learning approach for the automatic detection of stress of people in a social situation by combining two sensor systems that capture physiological and social responses. We compare the performance using different classifiers including support vector machine, AdaBoost, and k-nearest neighbour. Our experimental results show that by combining the measurements from both sensor systems, we could accurately discriminate between stressful and neutral situations during a controlled Trier social stress test (TSST). Moreover, this paper assesses the discriminative ability of each sensor modality individually and considers their suitability for real time stress detection. Finally, we present an study of the most discriminative features for stress detection.

@article{lirolem23128,
year = {2017},
title = {Stress detection using wearable physiological and sociometric sensors},
journal = {International Journal of Neural Systems},
volume = {27},
number = {2},
publisher = {World Scientific Publishing},
author = {Oscar Martinez Mozos and Virginia Sandulescu and Sally Andrews and David Ellis and Nicola Bellotto and Radu Dobrescu and Jose Manuel Ferrandez},
month = {March},
pages = {1650041},
url = {http://eprints.lincoln.ac.uk/23128/},
keywords = {ARRAY(0x56147f667a20)},
abstract = {Stress remains a significant social problem for individuals in modern societies. This paper presents a machine learning approach for the automatic detection of stress of people in a social situation by combining two sensor systems that capture physiological and social responses. We compare the performance using different classifiers including support vector machine, AdaBoost, and k-nearest neighbour. Our experimental results show that by combining the measurements from both sensor systems, we could accurately discriminate between stressful and neutral situations during a controlled Trier social stress test (TSST). Moreover, this paper assesses the discriminative ability of each sensor modality individually and considers their suitability for real time stress detection. Finally, we present an study of the most discriminative features for stress detection.}
}
• T. Osa, A. G. M. Esfahani, R. Stolkin, R. Lioutikov, J. Peters, and G. Neumann, “Guiding trajectory optimization by demonstrated distributions,” IEEE Robotics and Automation Letters (RA-L), vol. 2, iss. 2, pp. 819-826, 2017.

Trajectory optimization is an essential tool for motion planning under multiple constraints of robotic manipulators. Optimization-based methods can explicitly optimize a trajectory by leveraging prior knowledge of the system and have been used in various applications such as collision avoidance. However, these methods often require a hand-coded cost function in order to achieve the desired behavior. Specifying such cost function for a complex desired behavior, e.g., disentangling a rope, is a nontrivial task that is often even infeasible. Learning from demonstration (LfD) methods offer an alternative way to program robot motion. LfD methods are less dependent on analytical models and instead learn the behavior of experts implicitly from the demonstrated trajectories. However, the problem of adapting the demonstrations to new situations, e.g., avoiding newly introduced obstacles, has not been fully investigated in the literature. In this paper, we present a motion planning framework that combines the advantages of optimization-based and demonstration-based methods. We learn a distribution of trajectories demonstrated by human experts and use it to guide the trajectory optimization process. The resulting trajectory maintains the demonstrated behaviors, which are essential to performing the task successfully, while adapting the trajectory to avoid obstacles. In simulated experiments and with a real robotic system, we verify that our approach optimizes the trajectory to avoid obstacles and encodes the demonstrated behavior in the resulting trajectory

@article{lirolem26731,
year = {2017},
volume = {2},
journal = {IEEE Robotics and Automation Letters (RA-L)},
title = {Guiding trajectory optimization by demonstrated distributions},
number = {2},
publisher = {IEEE},
author = {Takayuki Osa and Amir M. Ghalamzan Esfahani and Rustam Stolkin and Rudolf Lioutikov and Jan Peters and Gerhard Neumann},
month = {January},
pages = {819--826},
url = {http://eprints.lincoln.ac.uk/26731/},
keywords = {ARRAY(0x56147f7cb1f8)},
abstract = {Trajectory optimization is an essential tool for motion
planning under multiple constraints of robotic manipulators.
Optimization-based methods can explicitly optimize a trajectory
by leveraging prior knowledge of the system and have been used
in various applications such as collision avoidance. However, these
methods often require a hand-coded cost function in order to
achieve the desired behavior. Specifying such cost function for
a complex desired behavior, e.g., disentangling a rope, is a nontrivial
task that is often even infeasible. Learning from demonstration
(LfD) methods offer an alternative way to program robot
motion. LfD methods are less dependent on analytical models
and instead learn the behavior of experts implicitly from the
demonstrated trajectories. However, the problem of adapting the
demonstrations to new situations, e.g., avoiding newly introduced
obstacles, has not been fully investigated in the literature. In this
paper, we present a motion planning framework that combines
the advantages of optimization-based and demonstration-based
methods. We learn a distribution of trajectories demonstrated by
human experts and use it to guide the trajectory optimization
process. The resulting trajectory maintains the demonstrated
behaviors, which are essential to performing the task successfully,
while adapting the trajectory to avoid obstacles. In simulated
experiments and with a real robotic system, we verify that our
approach optimizes the trajectory to avoid obstacles and encodes
the demonstrated behavior in the resulting trajectory}
}
• J. Pajarinen, V. Kyrki, M. Koval, S. Srinivasa, J. Peters, and G. Neumann, “Hybrid control trajectory optimization under uncertainty,” in IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2017.

Trajectory optimization is a fundamental problem in robotics. While optimization of continuous control trajectories is well developed, many applications require both discrete and continuous, i.e. hybrid controls. Finding an optimal sequence of hybrid controls is challenging due to the exponential explosion of discrete control combinations. Our method, based on Differential Dynamic Programming (DDP), circumvents this problem by incorporating discrete actions inside DDP: we first optimize continuous mixtures of discrete actions, and, subsequently force the mixtures into fully discrete actions. Moreover, we show how our approach can be extended to partially observable Markov decision processes (POMDPs) for trajectory planning under uncertainty. We validate the approach in a car driving problem where the robot has to switch discrete gears and in a box pushing application where the robot can switch the side of the box to push. The pose and the friction parameters of the pushed box are initially unknown and only indirectly observable.

@inproceedings{lirolem28257,
booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
month = {September},
year = {2017},
title = {Hybrid control trajectory optimization under uncertainty},
author = {J. Pajarinen and V. Kyrki and M. Koval and S Srinivasa and J. Peters and G. Neumann},
abstract = {Trajectory optimization is a fundamental problem in robotics. While optimization of continuous control trajectories is well developed, many applications require both discrete and continuous, i.e. hybrid controls. Finding an optimal sequence of hybrid controls is challenging due to the exponential explosion of discrete control combinations. Our method, based on Differential Dynamic Programming (DDP), circumvents this problem by incorporating discrete actions inside DDP: we first optimize continuous mixtures of discrete actions, and, subsequently force the mixtures into fully discrete actions. Moreover, we show how our approach can be extended to partially observable Markov decision processes (POMDPs) for trajectory planning under uncertainty. We validate the approach in a car driving problem where the robot has to switch discrete gears and in a box pushing application where the robot can switch the side of the box to push. The pose and the friction parameters of the pushed box are initially unknown and only indirectly observable.},
url = {http://eprints.lincoln.ac.uk/28257/},
keywords = {ARRAY(0x56147f6903b8)}
}
• A. Paraschos, R. Lioutikov, J. Peters, and G. Neumann, “Probabilistic prioritization of movement primitives,” IEEE Robotics and Automation Letters, vol. PP, iss. 99, 2017.

@article{lirolem27901,
booktitle = {, Proceedings of the International Conference on Intelligent Robot Systems, and IEEE Robotics and Automation Letters (RA-L)},
month = {July},
author = {Alexandros Paraschos and Rudolf Lioutikov and Jan Peters and Gerhard Neumann},
publisher = {IEEE},
number = {99},
title = {Probabilistic prioritization of movement primitives},
year = {2017},
journal = {IEEE Robotics and Automation Letters},
volume = {PP},
url = {http://eprints.lincoln.ac.uk/27901/},
keywords = {ARRAY(0x56147f731d40)},
abstract = {Movement prioritization is a common approach
to combine controllers of different tasks for redundant robots,
where each task is assigned a priority. The priorities of the
tasks are often hand-tuned or the result of an optimization,
but seldomly learned from data. This paper combines Bayesian
task prioritization with probabilistic movement primitives to
prioritize full motion sequences that are learned from demonstrations.
Probabilistic movement primitives (ProMPs) can
encode distributions of movements over full motion sequences
and provide control laws to exactly follow these distributions.
The probabilistic formulation allows for a natural application of
Bayesian task prioritization. We extend the ProMP controllers
with an additional feedback component that accounts inaccuracies
in following the distribution and allows for a more
robust prioritization of primitives. We demonstrate how the
task priorities can be obtained from imitation learning and
how different primitives can be combined to solve even unseen
task-combinations. Due to the prioritization, our approach can
efficiently learn a combination of tasks without requiring individual
models per task combination. Further, our approach can
controllers, for example, for implementing obstacle avoidance.
Hence, the need of retraining the whole library is avoided in
many cases. We evaluate our approach on reaching movements
under constraints with redundant simulated planar robots and
two physical robot platforms, the humanoid robot ?iCub? and
a KUKA LWR robot arm.}
}
• A. Rahman, A. Ahmed, and S. Yue, “Classification of tongue – glossitis abnormality,” in International Conference of Data Mining and Knowledge Engineering, 2017, pp. 1-4.

An approach to classify tongue abnormality related to Diabetes Mellitus (DM) following Western Medicine (WM) approach. Glossitis abnormality is one of the common tongue abnormalities that affects patients who suffer from Diabetes Mellitus (DM). The novelty of the proposed approach is attributed to utilising visual signs that appear on tongue due to Glossitis abnormality causes by high blood sugar level in the human body. The test for the blood sugar level is inconvenient for some patients in rural and poor areas where medical services are minimal or may not be available at all. To screen and monitor human organ effectively, the proposed computer aided model predicts and classifies abnormality appears on the tongue or tongue surface using visual signs caused by the abnormality. The visual signs were extracted following a logically formed medical approach, which complies with Western Medicine (WM) approach. Using Random Forest classifier on the extracted visual tongue signs, from 572 tongue samples for 166 patients, the experimental results have shown promising accuracy of 95.8\% for Glossitis abnormality.

@inproceedings{lirolem27400,
title = {Classification of tongue - glossitis abnormality},
year = {2017},
author = {Ashiqur Rahman and Amr Ahmed and Shigang Yue},
month = {July},
pages = {1--4},
booktitle = {International Conference of Data Mining and Knowledge Engineering},
url = {http://eprints.lincoln.ac.uk/27400/},
keywords = {ARRAY(0x56147f9cff78)},
abstract = {An approach to classify tongue abnormality related to Diabetes Mellitus (DM) following Western Medicine (WM) approach. Glossitis abnormality is one of the common tongue abnormalities that affects patients who suffer from Diabetes Mellitus (DM).

The novelty of the proposed approach is attributed to utilising visual signs that appear on tongue due to Glossitis abnormality causes by high blood sugar level in the human body. The test for the blood sugar level is inconvenient for some patients in rural and poor areas where medical services are minimal or may not be available at all. To screen and monitor human organ effectively, the proposed computer aided model predicts and classifies abnormality appears on the tongue or tongue surface using visual signs caused by the abnormality. The visual signs were extracted following a logically formed medical approach, which complies with Western Medicine (WM) approach. Using Random Forest classifier on the extracted visual tongue signs, from 572 tongue samples for 166 patients, the experimental results have shown promising accuracy of 95.8\% for Glossitis abnormality.}
}
• M. Salem, A. Weiss, and P. Baxter, “New frontiers in human-robot interaction [special section on interdisciplinary human-centred approaches],” Interaction Studies, vol. 17, iss. 3, pp. 405-407, 2017.

@article{lirolem27044,
year = {2017},
volume = {17},
title = {New frontiers in human-robot interaction [special section on interdisciplinary human-centred approaches]},
journal = {Interaction Studies},
number = {3},
author = {Maha Salem and Astrid Weiss and Paul Baxter},
publisher = {John Benjamins Publishers},
month = {March},
pages = {405--407},
url = {http://eprints.lincoln.ac.uk/27044/},
keywords = {ARRAY(0x56147f93f050)},
abstract = {-}
}
• J. M. Santos, T. s Krajník, and T. Duckett, “Spatio-temporal exploration strategies for long-term autonomy of mobile robots,” Robotics and Autonomous Systems, vol. 88, pp. 116-126, 2017.

We present a study of spatio-temporal environment representations and exploration strategies for long-term deployment of mobile robots in real-world, dynamic environments. We propose a new concept for life-long mobile robot spatio-temporal exploration that aims at building, updating and maintaining the environment model during the long-term deployment. The addition of the temporal dimension to the explored space makes the exploration task a never-ending data-gathering process, which we address by application of information-theoretic exploration techniques to world representations that model the uncertainty of environment states as probabilistic functions of time. We evaluate the performance of different exploration strategies and temporal models on real-world data gathered over the course of several months. The combination of dynamic environment representations with information-gain exploration principles allows to create and maintain up-to-date models of continuously changing environments, enabling efficient and self-improving long-term operation of mobile robots.

@article{lirolem25412,
title = {Spatio-temporal exploration strategies for long-term autonomy of mobile robots},
journal = {Robotics and Autonomous Systems},
volume = {88},
year = {2017},
pages = {116--126},
month = {February},
publisher = {Elsevier},
author = {Jo{\~a}o Machado Santos and Tom{\'a}{\v s} Krajn{\'i}k and Tom Duckett},
url = {http://eprints.lincoln.ac.uk/25412/},
keywords = {ARRAY(0x56147f9ef228)},
abstract = {We present a study of spatio-temporal environment representations and exploration strategies for long-term deployment of mobile robots in real-world, dynamic environments.
We propose a new concept for life-long mobile robot spatio-temporal exploration that aims at building, updating and maintaining the environment model during the long-term deployment.
The addition of the temporal dimension to the explored space makes the exploration task a never-ending data-gathering process, which we address by application of information-theoretic exploration techniques to world representations that model the uncertainty of environment states as probabilistic functions of time.
We evaluate the performance of different exploration strategies and temporal models on real-world data gathered over the course of several months.
The combination of dynamic environment representations with information-gain exploration principles allows to create and maintain up-to-date models of continuously changing environments, enabling efficient and self-improving long-term operation of mobile robots.}
}
• E. Senft, S. Lemaignan, P. E. Baxter, and T. Belpaeme, “Leveraging human inputs in interactive machine learning for human robot interaction,” in ACM/IEEE International Conference on Human-Robot Interaction – HRI ’17, Vienna, Austria, 2017, pp. 281-282.

A key challenge of HRI is allowing robots to be adaptable, especially as robots are expected to penetrate society at large and to interact in unexpected environments with non- technical users. One way of providing this adaptability is to use Interactive Machine Learning, i.e. having a human supervisor included in the learning process who can steer the action selection and the learning in the desired direction. We ran a study exploring how people use numeric rewards to evaluate a robot’s behaviour and guide its learning. From the results we derive a number of challenges when design- ing learning robots: what kind of input should the human provide? How should the robot communicate its state or its intention? And how can the teaching process by made easier for human supervisors?

@inproceedings{lirolem30192,
year = {2017},
title = {Leveraging human inputs in interactive machine learning for human robot interaction},
author = {Emmanuel Senft and Severin Lemaignan and Paul E. Baxter and Tony Belpaeme},
month = {March},
pages = {281--282},
booktitle = {ACM/IEEE International Conference on Human-Robot Interaction - HRI '17},
abstract = {A key challenge of HRI is allowing robots to be adaptable, especially as robots are expected to penetrate society at large and to interact in unexpected environments with non- technical users. One way of providing this adaptability is to use Interactive Machine Learning, i.e. having a human supervisor included in the learning process who can steer the action selection and the learning in the desired direction. We ran a study exploring how people use numeric rewards to evaluate a robot's behaviour and guide its learning. From the results we derive a number of challenges when design- ing learning robots: what kind of input should the human provide? How should the robot communicate its state or its intention? And how can the teaching process by made easier for human supervisors?},
url = {http://eprints.lincoln.ac.uk/30192/},
keywords = {ARRAY(0x56147f84c588)}
}
• E. Senft, S. Lemaignan, P. Baxter, and T. Belpaeme, “Toward supervised reinforcement learning with partial states for social HRI,” in 4th AAAI FSS on Artificial Intelligence for Social Human-Robot Interaction (AI-HRI), Arlington, Virginia, U.S.A., 2017, pp. 109-113.

Social interacting is a complex task for which machine learning holds particular promise. However, as no sufficiently accurate simulator of human interactions exists today, the learning of social interaction strategies has to happen online in the real world. Actions executed by the robot impact on humans, and as such have to be carefully selected, making it impossible to rely on random exploration. Additionally, no clear reward function exists for social interactions. This implies that traditional approaches used for Reinforcement Learning cannot be directly applied for learning how to interact with the social world. As such we argue that robots will profit from human expertise and guidance to learn social interactions. However, as the quantity of input a human can provide is limited, new methods have to be designed to use human input more efficiently. In this paper we describe a setup in which we combine a framework called Supervised Progressively Autonomous Robot Competencies (SPARC), which allows safer online learning with Reinforcement Learning, with the use of partial states rather than full states to accelerate generalisation and obtain a usable action policy more quickly.

@inproceedings{lirolem30193,
year = {2017},
title = {Toward supervised reinforcement learning with partial states for social HRI},
publisher = {AAAI Press},
author = {Emmanuel Senft and Severin Lemaignan and Paul Baxter and Tony Belpaeme},
pages = {109--113},
booktitle = {4th AAAI FSS on Artificial Intelligence for Social Human-Robot Interaction (AI-HRI)},
month = {November},
abstract = {Social interacting is a complex task for which machine learning holds particular promise. However, as no sufficiently accurate simulator of human interactions exists today, the learning of social interaction strategies has to happen online in the real world. Actions executed by the robot impact on humans, and as such have to be carefully selected, making it impossible to rely on random exploration. Additionally, no clear reward function exists for social interactions. This implies that traditional approaches used for Reinforcement Learning cannot be directly applied for learning how to interact with the social world. As such we argue that robots will profit from human expertise and guidance to learn social interactions. However, as the quantity of input a human can provide is limited, new methods have to be designed to use human input more efficiently. In this paper we describe a setup in which we combine a framework called Supervised Progressively Autonomous Robot Competencies (SPARC), which allows safer online learning with Reinforcement Learning, with the use of partial states rather than full states to accelerate generalisation and obtain a usable action policy more quickly.},
url = {http://eprints.lincoln.ac.uk/30193/},
keywords = {ARRAY(0x56147f6a04f0)}
}
• E. Senft, P. Baxter, J. Kennedy, S. Lemaignan, and T. Belpaeme, “Supervised autonomy for online learning in human-robot interaction,” Pattern Recognition Letters, vol. 96, pp. 77-86, 2017.

When a robot is learning it needs to explore its environment and how its environment responds on its actions. When the environment is large and there are a large number of possible actions the robot can take, this exploration phase can take prohibitively long. However, exploration can often be optimised by letting a human expert guide the robot during its learning. Interactive machine learning, in which a human user interactively guides the robot as it learns, has been shown to be an effective way to teach a robot. It requires an intuitive control mechanism to allow the human expert to provide feedback on the robot?s progress. This paper presents a novel method which combines Reinforcement Learning and Supervised Progressively Autonomous Robot Competencies (SPARC). By allowing the user to fully control the robot and by treating rewards as implicit, SPARC aims to learn an action policy while maintaining human supervisory oversight of the robot?s behaviour. This method is evaluated and compared to Interactive Reinforcement Learning in a robot teaching task. Qualitative and quantitative results indicate that SPARC allows for safer and faster learning by the robot, whilst not placing a high workload on the human teacher.

@article{lirolem26857,
month = {November},
pages = {77--86},
author = {Emmanuel Senft and Paul Baxter and James Kennedy and Severin Lemaignan and Tony Belpaeme},
publisher = {Elsevier / North Holland for International Association for Pattern Recognition},
title = {Supervised autonomy for online learning in human-robot interaction},
volume = {96},
journal = {Pattern Recognition Letters},
year = {2017},
url = {http://eprints.lincoln.ac.uk/26857/},
keywords = {ARRAY(0x56147f64a730)},
abstract = {When a robot is learning it needs to explore its environment and how its environment responds on its
actions. When the environment is large and there are a large number of possible actions the robot can
take, this exploration phase can take prohibitively long. However, exploration can often be optimised
by letting a human expert guide the robot during its learning. Interactive machine learning, in which a
human user interactively guides the robot as it learns, has been shown to be an effective way to teach a
robot. It requires an intuitive control mechanism to allow the human expert to provide feedback on
the robot?s progress. This paper presents a novel method which combines Reinforcement Learning
and Supervised Progressively Autonomous Robot Competencies (SPARC). By allowing the user to
fully control the robot and by treating rewards as implicit, SPARC aims to learn an action policy
while maintaining human supervisory oversight of the robot?s behaviour. This method is evaluated and
compared to Interactive Reinforcement Learning in a robot teaching task. Qualitative and quantitative
results indicate that SPARC allows for safer and faster learning by the robot, whilst not placing a high
}
• V. Tangkaratt, H. van Hoof, S. Parisi, G. Neumann, J. Peters, and M. Sugiyama, “Policy search with high-dimensional context variables,” in AAAI Conference on Artificial Intelligence (AAAI), 2017.

Direct contextual policy search methods learn to improve policy parameters and simultaneously generalize these parameters to different context or task variables. However, learning from high-dimensional context variables, such as camera images, is still a prominent problem in many real-world tasks. A naive application of unsupervised dimensionality reduction methods to the context variables, such as principal component analysis, is insufficient as task-relevant input may be ignored. In this paper, we propose a contextual policy search method in the model-based relative entropy stochastic search framework with integrated dimensionality reduction. We learn a model of the reward that is locally quadratic in both the policy parameters and the context variables. Furthermore, we perform supervised linear dimensionality reduction on the context variables by nuclear norm regularization. The experimental results show that the proposed method outperforms naive dimensionality reduction via principal component analysis and a state-of-the-art contextual policy search method.

@inproceedings{lirolem26740,
booktitle = {AAAI Conference on Artificial Intelligence (AAAI)},
month = {February},
year = {2017},
title = {Policy search with high-dimensional context variables},
publisher = {Association for the Advancement of Artificial Intelligence},
author = {V. Tangkaratt and H. van Hoof and S. Parisi and G. Neumann and J. Peters and M. Sugiyama},
keywords = {ARRAY(0x56147f58cfb0)},
url = {http://eprints.lincoln.ac.uk/26740/},
abstract = {Direct contextual policy search methods learn to improve policy
parameters and simultaneously generalize these parameters
to different context or task variables. However, learning
from high-dimensional context variables, such as camera images,
is still a prominent problem in many real-world tasks.
A naive application of unsupervised dimensionality reduction
methods to the context variables, such as principal component
analysis, is insufficient as task-relevant input may be ignored.
In this paper, we propose a contextual policy search method in
the model-based relative entropy stochastic search framework
with integrated dimensionality reduction. We learn a model of
the reward that is locally quadratic in both the policy parameters
and the context variables. Furthermore, we perform supervised
linear dimensionality reduction on the context variables
by nuclear norm regularization. The experimental results
show that the proposed method outperforms naive dimensionality
reduction via principal component analysis and
a state-of-the-art contextual policy search method.}
}
• T. Vintr, S. M. Mellado, G. Cielniak, T. Duckett, and T. Krajnik, “Spatiotemporal models for motion planning in human populated environments,” in Student Conference on Planning in Artificial Intelligence and Robotics (PAIR), 2017.

In this paper we present an effective spatio-temporal model for motion planning computed using a novel representation known as the temporary warp space-hypertime continuum. Such a model is suitable for robots that are expected to be helpful to humans in their natural environments. This method allows to capture natural periodicities of human behavior by adding additional time dimensions. The model created thus represents the temporal structure of the human habits within a given space and can be analyzed using regular analytical methods. We visualize the results on a real-world dataset using heatmaps.

@inproceedings{lirolem31052,
title = {Spatiotemporal models for motion planning in human populated environments},
year = {2017},
publisher = {Czech Technical University in Prague, Faculty of Electrical Engineering},
author = {Tomas Vintr and Sergi Molina Mellado and Grzegorz Cielniak and Tom Duckett and Tomas Krajnik},
month = {September},
booktitle = {Student Conference on Planning in Artificial Intelligence and Robotics (PAIR)},
abstract = {In this paper we present an effective spatio-temporal model for motion planning computed using a novel representation known as the temporary warp space-hypertime continuum. Such a model is suitable for robots that are expected to be helpful to humans in their natural environments. This method allows to capture natural periodicities of human behavior by adding additional time dimensions. The model created thus represents the temporal structure of the human habits within a given space and can be analyzed using regular analytical methods. We visualize the results on a real-world dataset using heatmaps.},
keywords = {ARRAY(0x56147f5757a8)},
url = {http://eprints.lincoln.ac.uk/31052/}
}
• D. Wang, X. Hou, J. Xu, S. Yue, and C. Liu, “Traffic sign detection using a cascade method with fast feature extraction and saliency test,” IEEE Transactions on Intelligent Transportation Systems, vol. 18, iss. 12, pp. 3290-3302, 2017.

Automatic traffic sign detection is challenging due to the complexity of scene images, and fast detection is required in real applications such as driver assistance systems. In this paper, we propose a fast traffic sign detection method based on a cascade method with saliency test and neighboring scale awareness. In the cascade method, feature maps of several channels are extracted efficiently using approximation techniques. Sliding windows are pruned hierarchically using coarse-to-fine classifiers and the correlation between neighboring scales. The cascade system has only one free parameter, while the multiple thresholds are selected by a data-driven approach. To further increase speed, we also use a novel saliency test based on mid-level features to pre-prune background windows. Experiments on two public traffic sign data sets show that the proposed method achieves competing performance and runs 27 times as fast as most of the state-of-the-art methods.

@article{lirolem27022,
number = {12},
year = {2017},
journal = {IEEE Transactions on Intelligent Transportation Systems},
volume = {18},
title = {Traffic sign detection using a cascade method with fast feature extraction and saliency test},
author = {Dongdong Wang and Xinwen Hou and Jiawei Xu and Shigang Yue and Cheng-Lin Liu},
publisher = {IEEE},
pages = {3290--3302},
month = {December},
url = {http://eprints.lincoln.ac.uk/27022/},
keywords = {ARRAY(0x56147f995090)},
abstract = {Automatic traffic sign detection is challenging due to the complexity of scene images, and fast detection is required in real applications such as driver assistance systems. In this paper, we propose a fast traffic sign detection method based on a cascade method with saliency test and neighboring scale awareness. In the cascade method, feature maps of several channels are extracted efficiently using approximation techniques. Sliding windows are pruned hierarchically using coarse-to-fine classifiers and the correlation between neighboring scales. The cascade system has only one free parameter, while the multiple thresholds are selected by a data-driven approach. To further increase speed, we also use a novel saliency test based on mid-level features to pre-prune background windows. Experiments on two public traffic sign data sets show that the proposed method achieves competing performance and runs 27 times as fast as most of the state-of-the-art methods.}
}
• C. Wirth, R. Akrour, G. Neumann, and J. Fürnkranz, “A survey of preference-based reinforcement learning methods,” Journal of Machine Learning Research, vol. 18, iss. 136, pp. 1-46, 2017.

Reinforcement learning (RL) techniques optimize the accumulated long-term reward of a suitably chosen reward function. However, designing such a reward function often requires a lot of task- specific prior knowledge. The designer needs to consider different objectives that do not only influence the learned behavior but also the learning progress. To alleviate these issues, preference-based reinforcement learning algorithms (PbRL) have been proposed that can directly learn from an expert’s preferences instead of a hand-designed numeric reward. PbRL has gained traction in recent years due to its ability to resolve the reward shaping problem, its ability to learn from non numeric rewards and the possibility to reduce the dependence on expert knowledge. We provide a unified framework for PbRL that describes the task formally and points out the different design principles that affect the evaluation task for the human as well as the computational complexity. The design principles include the type of feedback that is assumed, the representation that is learned to capture the preferences, the optimization problem that has to be solved as well as how the exploration/exploitation problem is tackled. Furthermore, we point out shortcomings of current algorithms, propose open research questions and briefly survey practical tasks that have been solved using PbRL.

@article{lirolem30636,
pages = {1--46},
month = {December},
publisher = {Journal of Machine Learning Research / Massachusetts Institute of Technology Press (MIT Press) / Microtome},
author = {Christian Wirth and Riad Akrour and Gerhard Neumann and Johannes F{\"u}rnkranz},
number = {136},
year = {2017},
journal = {Journal of Machine Learning Research},
volume = {18},
title = {A survey of preference-based reinforcement learning methods},
url = {http://eprints.lincoln.ac.uk/30636/},
keywords = {ARRAY(0x56147f6e74a0)},
abstract = {Reinforcement learning (RL) techniques optimize the accumulated long-term reward of a suitably chosen reward function. However, designing such a reward function often requires a lot of task- specific prior knowledge. The designer needs to consider different objectives that do not only influence the learned behavior but also the learning progress. To alleviate these issues, preference-based reinforcement learning algorithms (PbRL) have been proposed that can directly learn from an expert's preferences instead of a hand-designed numeric reward. PbRL has gained traction in recent years due to its ability to resolve the reward shaping problem, its ability to learn from non numeric rewards and the possibility to reduce the dependence on expert knowledge. We provide a unified framework for PbRL that describes the task formally and points out the different design principles that affect the evaluation task for the human as well as the computational complexity. The design principles include the type of feedback that is assumed, the representation that is learned to capture the preferences, the optimization problem that has to be solved as well as how the exploration/exploitation problem is tackled. Furthermore, we point out shortcomings of current algorithms, propose open research questions and briefly survey practical tasks that have been solved using PbRL.}
}
• J. Xu, S. Yue, F. Menchinelli, and K. Guo, “What has been missed for predicting human attention in viewing driving clips?,” PeerJ, vol. 5, p. e2946, 2017.

Recent research progress on the topic of human visual attention allocation in scene perception and its simulation is based mainly on studies with static images. However, natural vision requires us to extract visual information that constantly changes due to egocentric movements or dynamics of the world. It is unclear to what extent spatio-temporal regularity, an inherent regularity in dynamic vision, affects human gaze distribution and saliency computation in visual attention models. In this free-viewing eye-tracking study we manipulated the spatio-temporal regularity of traffic videos by presenting them in normal video sequence, reversed video sequence, normal frame sequence, and randomised frame sequence. The recorded human gaze allocation was then used as the ?ground truth? to examine the predictive ability of a number of state-of-the-art visual attention models. The analysis revealed high inter-observer agreement across individual human observers, but all the tested attention models performed significantly worse than humans. The inferior predictability of the models was evident from indistinguishable gaze prediction irrespective of stimuli presentation sequence, and weak central fixation bias. Our findings suggest that a realistic visual attention model for the processing of dynamic scenes should incorporate human visual sensitivity with spatio-temporal regularity and central fixation bias.

@article{lirolem25963,
author = {Jiawei Xu and Shigang Yue and Federica Menchinelli and Kun Guo},
publisher = {PeerJ},
month = {February},
pages = {e2946},
title = {What has been missed for predicting human attention in viewing driving clips?},
volume = {5},
year = {2017},
journal = {PeerJ},
abstract = {Recent research progress on the topic of human visual attention allocation in scene perception and its simulation is based mainly on studies with static images. However, natural vision requires us to extract visual information that constantly changes due to egocentric movements or dynamics of the world. It is unclear to what extent spatio-temporal regularity, an inherent regularity in dynamic vision, affects human gaze distribution and saliency computation in visual attention models. In this free-viewing eye-tracking study we manipulated the spatio-temporal regularity of traffic videos by presenting them in normal video sequence, reversed video sequence, normal frame sequence, and randomised frame sequence. The recorded human gaze allocation was then used as the ?ground truth? to examine the predictive ability of a number of state-of-the-art visual attention models. The analysis revealed high inter-observer agreement across individual human observers, but all the tested attention models performed significantly worse than humans. The inferior predictability of the models was evident from indistinguishable gaze prediction irrespective of stimuli presentation sequence, and weak central fixation bias. Our findings suggest that a realistic visual attention model for the processing of dynamic scenes should incorporate human visual sensitivity with spatio-temporal regularity and central fixation bias.},
keywords = {ARRAY(0x56147f93db98)},
url = {http://eprints.lincoln.ac.uk/25963/}
}
• Z. Yan, T. Duckett, and N. Bellotto, “Online learning for human classification in 3D LiDAR-based tracking,” in IEEE/RSJ International Conference on Itelligent Robots and Systems (IROS), 2017.

Human detection and tracking is one of the most important aspects to be considered in service robotics, as the robot often shares its workspace and interacts closely with humans. This paper presents an online learning framework for human classification in 3D LiDAR scans, taking advantage of robust multi-target tracking to avoid the need for data annotation by a human expert. The system learns iteratively by retraining a classifier online with the samples collected by the robot over time. A novel aspect of our approach is that errors in training data can be corrected using the information provided by the 3D LiDAR-based tracking. In order to do this, an efficient 3D cluster detector of potential human targets has been implemented. We evaluate the framework using a new 3D LiDAR dataset of people moving in a large indoor public space, which is made available to the research community. The experiments analyse the real-time performance of the cluster detector and show that our online-trained human classifier matches and in some cases outperforms its offline version.

@inproceedings{lirolem27675,
publisher = {IEEE},
author = {Zhi Yan and Tom Duckett and Nicola Bellotto},
title = {Online learning for human classification in 3D LiDAR-based tracking},
year = {2017},
month = {September},
booktitle = {IEEE/RSJ International Conference on Itelligent Robots and Systems (IROS)},
abstract = {Human detection and tracking is one of the most important aspects to be considered in service robotics, as the robot often shares its workspace and interacts closely with humans. This paper presents an online learning framework for human classification in 3D LiDAR scans, taking advantage of robust multi-target tracking to avoid the need for data annotation by a human expert. The system learns iteratively by retraining a classifier online with the samples collected by the robot over time. A novel aspect of our approach is that errors in training data can be corrected using the information provided by the 3D LiDAR-based tracking. In order to do this, an efficient 3D cluster detector of potential human targets has been implemented. We evaluate the framework using a new 3D LiDAR dataset of people moving in a large indoor public space, which is made available to the research community. The experiments analyse the real-time performance of the cluster detector and show that our online-trained human classifier matches and in some cases outperforms its offline version.},
keywords = {ARRAY(0x56147f682740)},
url = {http://eprints.lincoln.ac.uk/27675/}
}
• A. Zaganidis, M. Magnusson, T. Duckett, and G. Cielniak, “Semantic-assisted 3D Normal Distributions Transform for scan registration in environments with limited structure,” in International Conference on Intelligent Robots and Systems (IROS), 2017.

Point cloud registration is a core problem of many robotic applications, including simultaneous localization and mapping. The Normal Distributions Transform (NDT) is a method that fits a number of Gaussian distributions to the data points, and then uses this transform as an approximation of the real data, registering a relatively small number of distributions as opposed to the full point cloud. This approach contributes to NDT?s registration robustness and speed but leaves room for improvement in environments of limited structure. To address this limitation we propose a method for the introduction of semantic information extracted from the point clouds into the registration process. The paper presents a large scale experimental evaluation of the algorithm against NDT on two publicly available benchmark data sets. For the purpose of this test a measure of smoothness is used for the semantic partitioning of the point clouds. The results indicate that the proposed method improves the accuracy, robustness and speed of NDT registration, especially in unstructured environments, making NDT suitable for a wider range of applications.

@inproceedings{lirolem28481,
publisher = {IEEE/RSJ},
author = {Anestis Zaganidis and Martin Magnusson and Tom Duckett and Grzegorz Cielniak},
title = {Semantic-assisted 3D Normal Distributions Transform for scan registration in environments with limited structure},
year = {2017},
month = {September},
booktitle = {International Conference on Intelligent Robots and Systems (IROS)},
url = {http://eprints.lincoln.ac.uk/28481/},
keywords = {ARRAY(0x56147f616f30)},
abstract = {Point cloud registration is a core problem of many robotic applications, including simultaneous localization and mapping. The Normal Distributions Transform (NDT) is a method that fits a number of Gaussian distributions to the data points, and then uses this transform as an approximation of the real data, registering a relatively small number of distributions as opposed to the full point cloud. This approach contributes to NDT?s registration robustness and speed but leaves room for improvement in environments of limited structure.
To address this limitation we propose a method for the introduction of semantic information extracted from the point clouds into the registration process. The paper presents a large scale experimental evaluation of the algorithm against NDT on two publicly available benchmark data sets. For the purpose of this test a measure of smoothness is used for the semantic partitioning of the point clouds. The results indicate that the proposed method improves the accuracy, robustness and speed of NDT registration, especially in unstructured environments, making NDT suitable for a wider range of applications.}
}
• X. Zheng, F. Lv, F. Zhao, S. Yue, C. Zhang, Z. Wang, F. Li, H. Jiang, and Z. Wang, “A 10 GHz 56 fsrms-integrated-jitter and -247 dB FOM ring-VCO based injection-locked clock multiplier with a continuous frequency-tracking loop in 65 nm CMOS,” in 38th Annual Custom Integrated Circuits Conference, CICC 2017, 2017.

This paper presents a low jitter ring-VCO based injection-locked clock multiplier (RILCM) with a phase-shift detection based hybrid frequency tracking loop (FTL). A full-swing pseudo-differential delay cell (FS-PDDC) is proposed to lower the device noise to phase noise conversion. To obtain high operation speed, high detection accuracy, and low output disturbance, a compact timing-adjusted phase detector (TPD) tightly combining with a well-matched charge pump (CP) is designed. Additionally, a lock-loss detection and lock recovery (LLD-LR) scheme is devised to equip the RILCM with a similar lock-acquisition ability to conventional PLL, thus excluding the initial frequency setup aid and preventing potential lock loss. Implemented in 65 nm CMOS, the RILCM occupies an active area of 0.07 mm2 and consumes 59.4 mW at 10 GHz. The measured results show that it achieves 56.1 fs rms-jitter and -57.13 dBc spur level. The calculated figure-of-merit (FOM) is -247.3 dB, which is better than previous RILCMs and even comparable to those large-area LC-ILCMs. Â\copyright 2017 IEEE.

@inproceedings{lirolem29190,
volume = {2017-A},
title = {A 10 GHz 56 fsrms-integrated-jitter and -247 dB FOM ring-VCO based injection-locked clock multiplier with a continuous frequency-tracking loop in 65 nm CMOS},
year = {2017},
booktitle = {38th Annual Custom Integrated Circuits Conference, CICC 2017},
month = {April},
note = {Conference Code:129634},
author = {X Zheng and F. Lv and F. Zhao and S. Yue and C. Zhang and Z. Wang and F. Li and H. Jiang and Z. Wang},
publisher = {Institute of Electrical and Electronics Engineers Inc.},
keywords = {ARRAY(0x56147f747498)},
url = {http://eprints.lincoln.ac.uk/29190/},
abstract = {This paper presents a low jitter ring-VCO based injection-locked clock multiplier (RILCM) with a phase-shift detection based hybrid frequency tracking loop (FTL). A full-swing pseudo-differential delay cell (FS-PDDC) is proposed to lower the device noise to phase noise conversion. To obtain high operation speed, high detection accuracy, and low output disturbance, a compact timing-adjusted phase detector (TPD) tightly combining with a well-matched charge pump (CP) is designed. Additionally, a lock-loss detection and lock recovery (LLD-LR) scheme is devised to equip the RILCM with a similar lock-acquisition ability to conventional PLL, thus excluding the initial frequency setup aid and preventing potential lock loss. Implemented in 65 nm CMOS, the RILCM occupies an active area of 0.07 mm2 and consumes 59.4 mW at 10 GHz. The measured results show that it achieves 56.1 fs rms-jitter and -57.13 dBc spur level. The calculated figure-of-merit (FOM) is -247.3 dB, which is better than previous RILCMs and even comparable to those large-area LC-ILCMs. {\^A}{\copyright} 2017 IEEE.}
}
• X. Zheng, C. Zhang, F. Lv, F. Zhao, S. Yue, Z. Wang, F. Li, H. Jiang, and Z. Wang, “A 4-40 Gb/s PAM4 transmitter with output linearity optimization in 65 nm CMOS,” in 38th Annual Custom Integrated Circuits Conference, CICC 2017, 2017.

This paper presents a 4-40 Gb/s current mode PAM4 transmitter with an optimized eye linearity. By embedding an additional mixed combiner and an extra current source into the output driver and developing a coherent scaled-replica based bias generator, the channel-length modulation caused tail-current variations for both DC and AC coupling modes can be effectively compensated. Implemented in 65 nm CMOS, the transmitter occupies an area of 1.02 mm2 and consumes 102 mW at 40 Gb/s. After applying the proposed linearity optimization, the measured eye linearity can be optimized from 1.28 to 1.01 with a single-end swing of 480 mV in AC coupling mode. Â\copyright 2017 IEEE.

@inproceedings{lirolem29189,
booktitle = {38th Annual Custom Integrated Circuits Conference, CICC 2017},
note = {Conference Code:129634},
publisher = {Institute of Electrical and Electronics Engineers Inc.},
author = {X. Zheng and C. Zhang and F. Lv and F. Zhao and S. Yue and Z. Wang and F. Li and H. Jiang and Z. Wang},
year = {2017},
title = {A 4-40 Gb/s PAM4 transmitter with output linearity optimization in 65 nm CMOS},
volume = {2017-A},
abstract = {This paper presents a 4-40 Gb/s current mode PAM4 transmitter with an optimized eye linearity. By embedding an additional mixed combiner and an extra current source into the output driver and developing a coherent scaled-replica based bias generator, the channel-length modulation caused tail-current variations for both DC and AC coupling modes can be effectively compensated. Implemented in 65 nm CMOS, the transmitter occupies an area of 1.02 mm2 and consumes 102 mW at 40 Gb/s. After applying the proposed linearity optimization, the measured eye linearity can be optimized from 1.28 to 1.01 with a single-end swing of 480 mV in AC coupling mode. {\^A}{\copyright} 2017 IEEE.},
keywords = {ARRAY(0x56147f60b3b0)},
url = {http://eprints.lincoln.ac.uk/29189/}
}

### 2016

• A. Abdolmaleki, R. Lioutikov, N. Lua, P. L. Reis, J. Peters, and G. Neumann, “Model-based relative entropy stochastic search,” in Advances in Neural Information Processing Systems (NIPS), 2016, pp. 153-154.

Stochastic search algorithms are general black-box optimizers. Due to their ease of use and their generality, they have recently also gained a lot of attention in operations research, machine learning and policy search. Yet, these algorithms require a lot of evaluations of the objective, scale poorly with the problem dimension, are affected by highly noisy objective functions and may converge prematurely. To alleviate these problems, we introduce a new surrogate-based stochastic search approach. We learn simple, quadratic surrogate models of the objective function. As the quality of such a quadratic approximation is limited, we do not greedily exploit the learned models. The algorithm can be misled by an inaccurate optimum introduced by the surrogate. Instead, we use information theoretic constraints to bound the ?distance? between the new and old data distribution while maximizing the objective function. Additionally the new method is able to sustain the exploration of the search distribution to avoid premature convergence. We compare our method with state of art black-box optimization methods on standard uni-modal and multi-modal optimization functions, on simulated planar robot tasks and a complex robot ball throwing task. The proposed method considerably outperforms the existing approaches.

@inproceedings{lirolem25741,
booktitle = {Advances in Neural Information Processing Systems (NIPS)},
pages = {153--154},
author = {A. Abdolmaleki and R. Lioutikov and N. Lua and L. Paulo Reis and J. Peters and G. Neumann},
year = {2016},
title = {Model-based relative entropy stochastic search},
journal = {GECCO 2016 Companion - Proceedings of the 2016 Genetic and Evolutionary Computation Conference},
abstract = {Stochastic search algorithms are general black-box optimizers. Due to their ease
of use and their generality, they have recently also gained a lot of attention in operations
research, machine learning and policy search. Yet, these algorithms require
a lot of evaluations of the objective, scale poorly with the problem dimension, are
affected by highly noisy objective functions and may converge prematurely. To
alleviate these problems, we introduce a new surrogate-based stochastic search
approach. We learn simple, quadratic surrogate models of the objective function.
As the quality of such a quadratic approximation is limited, we do not greedily exploit
the learned models. The algorithm can be misled by an inaccurate optimum
introduced by the surrogate. Instead, we use information theoretic constraints to
bound the ?distance? between the new and old data distribution while maximizing
the objective function. Additionally the new method is able to sustain the exploration
of the search distribution to avoid premature convergence. We compare our
method with state of art black-box optimization methods on standard uni-modal
and multi-modal optimization functions, on simulated planar robot tasks and a
complex robot ball throwing task. The proposed method considerably outperforms
the existing approaches.},
url = {http://eprints.lincoln.ac.uk/25741/},
keywords = {ARRAY(0x56147f690718)}
}
• A. Abdolmaleki, N. Lau, L. P. Reis, and G. Neumann, “Non-parametric contextual stochastic search,” in Intelligent Robots and Systems (IROS), 2016 IEEE/RSJ International Conference on, 2016, pp. 2643-2648.

Stochastic search algorithms are black-box optimizer of an objective function. They have recently gained a lot of attention in operations research, machine learning and policy search of robot motor skills due to their ease of use and their generality. Yet, many stochastic search algorithms require relearning if the task or objective function changes slightly to adapt the solution to the new situation or the new context. In this paper, we consider the contextual stochastic search setup. Here, we want to find multiple good parameter vectors for multiple related tasks, where each task is described by a continuous context vector. Hence, the objective function might change slightly for each parameter vector evaluation of a task or context. Contextual algorithms have been investigated in the field of policy search, however, the search distribution typically uses a parametric model that is linear in the some hand-defined context features. Finding good context features is a challenging task, and hence, non-parametric methods are often preferred over their parametric counter-parts. In this paper, we propose a non-parametric contextual stochastic search algorithm that can learn a non-parametric search distribution for multiple tasks simultaneously. In difference to existing methods, our method can also learn a context dependent covariance matrix that guides the exploration of the search process. We illustrate its performance on several non-linear contextual tasks.

@inproceedings{lirolem25738,
booktitle = {Intelligent Robots and Systems (IROS), 2016 IEEE/RSJ International Conference on},
pages = {2643--2648},
month = {October},
author = {A. Abdolmaleki and N. Lau and L.P. Reis and G. Neumann},
title = {Non-parametric contextual stochastic search},
volume = {2016-N},
year = {2016},
journal = {IEEE International Conference on Intelligent Robots and Systems},
keywords = {ARRAY(0x56147f883930)},
url = {http://eprints.lincoln.ac.uk/25738/},
abstract = {Stochastic search algorithms are black-box optimizer of an objective function. They have recently gained a lot of attention in operations research, machine learning and policy search of robot motor skills due to their ease of use and their generality. Yet, many stochastic search algorithms require relearning if the task or objective function changes slightly to adapt the solution to the new situation or the new context. In this paper, we consider the contextual stochastic search setup. Here, we want to find multiple good parameter vectors for multiple related tasks, where each task is described by a continuous context vector. Hence, the objective function might change slightly for each parameter vector evaluation of a task or context. Contextual algorithms have been investigated in the field of policy search, however, the search distribution typically uses a parametric model that is linear in the some hand-defined context features. Finding good context features is a challenging task, and hence, non-parametric methods are often preferred over their parametric counter-parts. In this paper, we propose a non-parametric contextual stochastic search algorithm that can learn a non-parametric search distribution for multiple tasks simultaneously. In difference to existing methods, our method can also learn a context dependent covariance matrix that guides the exploration of the search process. We illustrate its performance on several non-linear contextual tasks.}
}
• A. Abdolmaleki, N. Lau, P. L. Reis, and G. Neumann, “Contextual stochastic search,” in Genetic and Evolutionary Computation Conference GECCO 2016, 2016, pp. 29-30.

Stochastic search algorithms have recently also gained a lot of attention in operations research, machine learning and policy search of robot motor skills due to their ease of use and their generality. Yet, many stochastic search algorithms require relearning if the task changes slightly to adapt the solution to the new situation or the new context. Therefore we consider the contextual stochastic search setup. Here, we want to find good parameter vectors for multiple related tasks, where each task is described by a continuous context vector. Hence, the objective might change slightly for each parameter vector evaluation. In this research, we investigate the contextual stochastic search algorithms that can learn from multiple tasks simultaneously.

@inproceedings{lirolem25679,
title = {Contextual stochastic search},
year = {2016},
author = {A. Abdolmaleki and N. Lau and L. Paulo Reis and G. Neumann},
publisher = {ACM},
pages = {29--30},
booktitle = {Genetic and Evolutionary Computation Conference GECCO 2016},
month = {July},
url = {http://eprints.lincoln.ac.uk/25679/},
keywords = {ARRAY(0x56147f7c60e0)},
abstract = {Stochastic search algorithms have recently also gained a lot of attention in operations research, machine learning and policy search of robot motor skills due to their ease of use and their generality. Yet, many stochastic search algorithms require relearning if the task changes slightly to adapt the solution to the new situation or the new context. Therefore we consider the contextual stochastic search setup. Here, we want to find good parameter vectors for multiple related tasks, where each task is described by a continuous context vector. Hence, the objective might change slightly for each parameter vector evaluation. In this research, we investigate the contextual stochastic search algorithms that can learn from multiple tasks simultaneously.}
}
• A. Abdolmaleki, N. Lau, L. P. Reis, J. Peters, and G. Neumann, “Contextual policy search for linear and nonlinear generalization of a humanoid walking controller,” Journal of Intelligent and Robotic Systems: Theory and Applications, vol. 83, iss. 3, pp. 393-408, 2016.

We investigate learning of flexible robot locomotion controllers, i.e., the controllers should be applicable for multiple contexts, for example different walking speeds, various slopes of the terrain or other physical properties of the robot. In our experiments, contexts are desired walking linear speed of the gait. Current approaches for learning control parameters of biped locomotion controllers are typically only applicable for a single context. They can be used for a particular context, for example to learn a gait with highest speed, lowest energy consumption or a combination of both. The question of our research is, how can we obtain a flexible walking controller that controls the robot (near) optimally for many different contexts? We achieve the desired flexibility of the controller by applying the recently developed contextual relative entropy policy search(REPS) method which generalizes the robot walking controller for different contexts, where a context is described by a real valued vector. In this paper we also extend the contextual REPS algorithm to learn a non-linear policy instead of a linear policy over the contexts which call it RBF-REPS as it uses Radial Basis Functions. In order to validate our method, we perform three simulation experiments including a walking experiment using a simulated NAO humanoid robot. The robot learns a policy to choose the controller parameters for a continuous set of forward walking speeds.

@article{lirolem25745,
year = {2016},
volume = {83},
journal = {Journal of Intelligent and Robotic Systems: Theory and Applications},
title = {Contextual policy search for linear and nonlinear generalization of a humanoid walking controller},
number = {3},
publisher = {Springer},
author = {Abbas Abdolmaleki and Nuno Lau and Luis Paulo Reis and Jan Peters and Gerhard Neumann},
month = {September},
pages = {393--408},
abstract = {We investigate learning of flexible robot locomotion controllers, i.e., the controllers should be applicable for multiple contexts, for example different walking speeds, various slopes of the terrain or other physical properties of the robot. In our experiments, contexts are desired walking linear speed of the gait. Current approaches for learning control parameters of biped locomotion controllers are typically only applicable for a single context. They can be used for a particular context, for example to learn a gait with highest speed, lowest energy consumption or a combination of both. The question of our research is, how can we obtain a flexible walking controller that controls the robot (near) optimally for many different contexts? We achieve the desired flexibility of the controller by applying the recently developed contextual relative entropy policy search(REPS) method which generalizes the robot walking controller for different contexts, where a context is described by a real valued vector. In this paper we also extend the contextual REPS algorithm to learn a non-linear policy instead of a linear policy over the contexts which call it RBF-REPS as it uses Radial Basis Functions. In order to validate our method, we perform three simulation experiments including a walking experiment using a simulated NAO humanoid robot. The robot learns a policy to choose the controller parameters for a continuous set of forward walking speeds.},
keywords = {ARRAY(0x56147f8c7a78)},
url = {http://eprints.lincoln.ac.uk/25745/}
}
• R. Akrour, A. Abdolmaleki, H. Abdulsamad, and G. Neumann, “Model-free trajectory optimization for reinforcement learning,” in Proceedings of the International Conference on Machine Learning (ICML), 2016, pp. 4342-4352.

Many of the recent Trajectory Optimization algorithms alternate between local approximation of the dynamics and conservative policy update. However, linearly approximating the dynamics in order to derive the new policy can bias the update and prevent convergence to the optimal policy. In this article, we propose a new model-free algorithm that backpropagates a local quadratic time-dependent Q-Function, allowing the derivation of the policy update in closed form. Our policy update ensures exact KL-constraint satisfaction without simplifying assumptions on the system dynamics demonstrating improved performance in comparison to related Trajectory Optimization algorithms linearizing the dynamics.

@inproceedings{lirolem25747,
volume = {6},
title = {Model-free trajectory optimization for reinforcement learning},
year = {2016},
journal = {33rd International Conference on Machine Learning, ICML 2016},
month = {June},
pages = {4342--4352},
booktitle = {Proceedings of the International Conference on Machine Learning (ICML)},
author = {R. Akrour and A. Abdolmaleki and H. Abdulsamad and G. Neumann},
keywords = {ARRAY(0x56147f6f97d0)},
url = {http://eprints.lincoln.ac.uk/25747/},
abstract = {Many of the recent Trajectory Optimization algorithms
alternate between local approximation
of the dynamics and conservative policy update.
However, linearly approximating the dynamics
in order to derive the new policy can bias the update
and prevent convergence to the optimal policy.
algorithm that backpropagates a local quadratic
time-dependent Q-Function, allowing the derivation
of the policy update in closed form. Our policy
update ensures exact KL-constraint satisfaction
without simplifying assumptions on the system
dynamics demonstrating improved performance
in comparison to related Trajectory Optimization
algorithms linearizing the dynamics.}
}
• O. Arenz, H. Abdulsamad, and G. Neumann, “Optimal control and inverse optimal control by distribution matching,” in Intelligent Robots and Systems (IROS), 2016 IEEE/RSJ International Conference on, 2016, pp. 4046-4053.

Optimal control is a powerful approach to achieve optimal behavior. However, it typically requires a manual specification of a cost function which often contains several objectives, such as reaching goal positions at different time steps or energy efficiency. Manually trading-off these objectives is often difficult and requires a high engineering effort. In this paper, we present a new approach to specify optimal behavior. We directly specify the desired behavior by a distribution over future states or features of the states. For example, the experimenter could choose to reach certain mean positions with given accuracy/variance at specified time steps. Our approach also unifies optimal control and inverse optimal control in one framework. Given a desired state distribution, we estimate a cost function such that the optimal controller matches the desired distribution. If the desired distribution is estimated from expert demonstrations, our approach performs inverse optimal control. We evaluate our approach on several optimal and inverse optimal control tasks on non-linear systems using incremental linearizations similar to differential dynamic programming approaches.

@inproceedings{lirolem25737,
journal = {IEEE International Conference on Intelligent Robots and Systems},
title = {Optimal control and inverse optimal control by distribution matching},
year = {2016},
volume = {2016-N},
author = {O. Arenz and H. Abdulsamad and G. Neumann},
month = {October},
booktitle = {Intelligent Robots and Systems (IROS), 2016 IEEE/RSJ International Conference on},
pages = {4046--4053},
url = {http://eprints.lincoln.ac.uk/25737/},
keywords = {ARRAY(0x56147f8bd9e8)},
abstract = {Optimal control is a powerful approach to achieve optimal behavior. However, it typically requires a manual specification of a cost function which often contains several objectives, such as reaching goal positions at different time steps or energy efficiency. Manually trading-off these objectives is often difficult and requires a high engineering effort. In this paper, we present a new approach to specify optimal behavior. We directly specify the desired behavior by a distribution over future states or features of the states. For example, the experimenter could choose to reach certain mean positions with given accuracy/variance at specified time steps. Our approach also unifies optimal control and inverse optimal control in one framework. Given a desired state distribution, we estimate a cost function such that the optimal controller matches the desired distribution. If the desired distribution is estimated from expert demonstrations, our approach performs inverse optimal control. We evaluate our approach on several optimal and inverse optimal control tasks on non-linear systems using incremental linearizations similar to differential dynamic programming approaches.}
}
• F. Arvin, A. E. Turgut, T. Krajnik, and S. Yue, “Investigation of cue-based aggregation in static and dynamic environments with a mobile robot swarm,” Adaptive Behavior, vol. 24, iss. 2, pp. 102-118, 2016.

Aggregation is one of the most fundamental behaviors that has been studied in swarm robotic researches for more than two decades. The studies in biology revealed that environment is a preeminent factor in especially cue-based aggregation that can be defined as aggregation at a particular location which is a heat or a light source acting as a cue indicating an optimal zone. In swarm robotics, studies on cue-based aggregation mainly focused on different methods of aggregation and different parameters such as population size. Although of utmost importance, environmental effects on aggregation performance have not been studied systematically. In this paper, we study the effects of different environmental factors; size, texture and number of cues in a static setting and moving cues in a dynamic setting using real robots. We used aggregation time and size of the aggregate as the two metrics to measure aggregation performance. We performed real robot experiments with different population sizes and evaluated the performance of aggregation using the defined metrics. We also proposed a probabilistic aggregation model and predicted the aggregation performance accurately in most of the settings. The results of the experiments show that environmental conditions affect the aggregation performance considerably and have to be studied in depth.

@article{lirolem22466,
number = {2},
year = {2016},
title = {Investigation of cue-based aggregation in static and dynamic environments with a mobile robot swarm},
volume = {24},
pages = {102--118},
month = {April},
author = {Farshad Arvin and Ali Emre Turgut and Tomas Krajnik and Shigang Yue},
publisher = {SAGE},
url = {http://eprints.lincoln.ac.uk/22466/},
keywords = {ARRAY(0x56147f9f1930)},
abstract = {Aggregation is one of the most fundamental behaviors that has been studied in swarm robotic researches for more than two decades. The studies in biology revealed that environment is a preeminent factor in especially cue-based aggregation that can be defined as aggregation at a particular location which is a heat or a light source acting as a cue indicating an optimal zone. In swarm robotics, studies on cue-based aggregation mainly focused on different methods of aggregation and different parameters such as population size. Although of utmost importance, environmental effects on aggregation performance have not been studied systematically. In this paper, we study the effects of different environmental factors; size, texture and number of cues in a static setting and moving cues in a dynamic setting using real robots. We used aggregation time and size of the aggregate as the two metrics to measure aggregation performance. We performed real robot experiments with different population sizes and evaluated the performance of aggregation using the defined metrics. We also proposed a probabilistic aggregation model and predicted the aggregation performance accurately in most of the settings. The results of the experiments show that environmental conditions affect the aggregation performance considerably and have to be studied in depth.}
}
• P. Baxter, J. Kennedy, E. Senft, S. Lemaignan, and T. Belpaeme, “From characterising three years of HRI to methodology and reporting recommendations,” in altHRI 2016, Christchurch, New Zealand, 2016, pp. 391-398.

Human-Robot Interaction (HRI) research requires the integration and cooperation of multiple disciplines, technical and social, in order to make progress. In many cases using different motivations, each of these disciplines bring with them different assumptions and methodologies.We assess recent trends in the field of HRI by examining publications in the HRI conference over the past three years (over 100 full papers), and characterise them according to 14 categories.We focus primarily on aspects of methodology. From this, a series of practical rec- ommendations based on rigorous guidelines from other research fields that have not yet become common practice in HRI are proposed. Furthermore, we explore the primary implications of the observed recent trends for the field more generally, in terms of both methodology and research directions.We propose that the interdisciplinary nature of HRI must be maintained, but that a common methodological approach provides a much needed frame of reference to facilitate rigorous future progress.

@inproceedings{lirolem30203,
title = {From characterising three years of HRI to methodology and reporting recommendations},
year = {2016},
month = {March},
booktitle = {altHRI 2016},
pages = {391--398},
publisher = {ACM Press},
author = {Paul Baxter and James Kennedy and Emmanuel Senft and Severin Lemaignan and Tony Belpaeme},
keywords = {ARRAY(0x56147f9e4fe8)},
url = {http://eprints.lincoln.ac.uk/30203/},
abstract = {Human-Robot Interaction (HRI) research requires
the integration and cooperation of multiple disciplines, technical and social, in order to make progress. In many cases using different motivations, each of these disciplines bring with them different assumptions and methodologies.We assess recent trends in the field of HRI by examining publications in the HRI conference over the past three years (over 100 full papers), and characterise them according to 14 categories.We focus primarily on aspects of methodology. From this, a series of practical rec- ommendations based on rigorous guidelines from other research fields that have not yet become common practice in HRI are proposed. Furthermore, we explore the primary implications of the observed recent trends for the field more generally, in terms of both methodology and research directions.We propose that the interdisciplinary nature of HRI must be maintained, but that a common methodological approach provides a much needed frame of reference to facilitate rigorous future progress.}
}
• P. Baxter, “Solve memory to solve cognition,” in Proceedings of the EUCognition Meeting (European Association for Cognitive Systems) “Cognitive Robot Architectures”, Vienna, Austria, 2016, pp. 58-59.

The foundations of cognition and cognitive behaviour are consistently proposed to be built upon the capability to predict (at various levels of abstraction). For autonomous cognitive agents, this implicitly assumes a foundational role for memory, as a mechanism by which prior experience can be brought to bear in the service of present and future behaviour. In this contribution, this idea is extended to propose that an active process of memory provides the substrate for cognitive processing, particularly when considering it as fundamentally associative and from a developmental perspective. It is in this context that the claim is made that in order to solve the question of cognition, the role and function of memory must be fully resolved.

@inproceedings{lirolem30191,
publisher = {CEUR Workshop Proceedings},
author = {Paul Baxter},
month = {December},
pages = {58--59},
booktitle = {Proceedings of the EUCognition Meeting (European Association for Cognitive Systems) "Cognitive Robot Architectures"},
year = {2016},
title = {Solve memory to solve cognition},
keywords = {ARRAY(0x56147f9e90a0)},
url = {http://eprints.lincoln.ac.uk/30191/},
abstract = {The foundations of cognition and cognitive behaviour are consistently proposed to be built upon the capability to predict (at various levels of abstraction). For autonomous cognitive agents, this implicitly assumes a foundational role for memory, as a mechanism by which prior experience can be brought to bear in the service of present and future behaviour. In this contribution, this idea is extended to propose that an active process of memory provides the substrate for cognitive processing, particularly when considering it as fundamentally associative and from a developmental perspective. It is in this context that the claim is made that in order to solve the question of cognition, the role and function of memory must be fully resolved.}
}
• G. Broughton, T. Krajnik, M. Fernandez-carmona, G. Cielniak, and N. Bellotto, “RFID-based Object Localisation with a Mobile Robot to Assist the Elderly with Mild Cognitive Impairments,” in International Workshop on Intelligent Environments Supporting Healthcare and Well-being (WISHWell), 2016.

Mild Cognitive Impairments (MCI) disrupt the quality of life and reduce the independence of many elderly at home. People with MCI can increasingly become forgetful, hence solutions to help them ?finding lost objects are useful. This paper presents a framework for mobile robots to localise objects in a domestic environment using Radio Frequency Identification (RFID) technology. In particular, it describes the development of a new library for interacting with RFID readers, readily available for the Robot Operating System (ROS), and introduces some methods for its application to RFID-based object localisation with a single antenna. The framework adopts occupancy grids to create a probabilistic representations of tags location in the environment. A robot traversing the environment can then make use of this framework to keep an internal record of where objects were last spotted, and where they are most likely to be at any given point in time. Some preliminary results are presented, together with directions for future research.

@inproceedings{lirolem23298,
month = {September},
booktitle = {International Workshop on Intelligent Environments Supporting Healthcare and Well-being (WISHWell)},
title = {RFID-based Object Localisation with a Mobile Robot to Assist the Elderly with Mild Cognitive Impairments},
year = {2016},
author = {George Broughton and Tomas Krajnik and Manuel Fernandez-carmona and Grzegorz Cielniak and Nicola Bellotto},
abstract = {Mild Cognitive Impairments (MCI) disrupt the quality of life and reduce the independence of many elderly at home. People with MCI can increasingly become forgetful, hence solutions to help them ?finding lost objects are useful. This paper presents a framework for mobile robots to localise objects in a domestic environment using Radio Frequency Identification (RFID) technology. In particular, it describes the development of a new library for interacting with RFID readers, readily available for the Robot Operating System (ROS), and introduces some methods for its application to RFID-based object localisation with a single antenna. The framework adopts occupancy grids to create a probabilistic representations of tags location in the environment. A robot traversing the environment can then make use of this framework to keep an internal record of where objects were last spotted, and where they are most likely to be at any given point in time. Some preliminary results are presented, together with directions for future research.},
url = {http://eprints.lincoln.ac.uk/23298/},
keywords = {ARRAY(0x56147f82e2b8)}
}
• W. Chen, C. Xiong, and S. Yue, “On configuration trajectory formation in spatiotemporal profile for reproducing human hand reaching movement,” IEEE Transactions on Cybernetics, vol. 46, iss. 3, 2016.

Most functional reaching activities in daily living generally require a hand to reach the functional position in appropriate orientation with invariant spatiotemporal profile. Effectively reproducing such spatiotemporal feature of hand configuration trajectory in real time is essential to understand the human motor control and plan human-like motion on anthropomorphic robotic arm. However, there are no novel computational models in literature toward reproducing hand configuration-to-configuration movement in spatiotemporal profile. In response to the problem, this paper presents a computational framework for hand configuration trajectory formation based on hierarchical principle of human motor control. The composite potential field is constructed on special Euclidean Group to induce time-varying configuration toward target. The dynamic behavior of hand is described by a second-order kinematic model to produce the external representation of high-level motor control. The multivariate regression relation between intrinsic and extrinsic coordinates of arm, is statistically analyzed for determining the arm orientation in real time, which produces the external representation of low-level motor control. The proposed method is demonstrated in an anthropomorphic arm by performing several highly curved self-reaching movements. The generated configuration trajectories are compared with actual human movement in spatiotemporal profile to validate the proposed method.

@article{lirolem17880,
volume = {46},
journal = {IEEE Transactions on Cybernetics},
title = {On configuration trajectory formation in spatiotemporal profile for reproducing human hand reaching movement},
year = {2016},
number = {3},
month = {March},
author = {Wenbin Chen and Caihua Xiong and Shigang Yue},
publisher = {IEEE},
abstract = {Most functional reaching activities in daily living generally require a hand to reach the functional position in appropriate orientation with invariant spatiotemporal profile. Effectively reproducing such spatiotemporal feature of hand configuration trajectory in real time is essential to understand the human motor control and plan human-like motion on anthropomorphic robotic arm. However, there are no novel computational models in literature toward reproducing hand configuration-to-configuration movement in spatiotemporal profile. In response to the problem, this paper presents a computational framework for hand configuration trajectory formation based on hierarchical principle of human motor control. The composite potential field is constructed on special Euclidean Group to induce time-varying configuration toward target. The dynamic behavior of hand is described by a second-order kinematic model to produce the external representation of high-level motor control. The multivariate regression relation between intrinsic and extrinsic coordinates of arm, is statistically analyzed for determining the arm orientation in real time, which produces the external representation of low-level motor control. The proposed method is demonstrated in an anthropomorphic arm by performing several highly curved self-reaching movements. The generated configuration trajectories are compared with actual human movement in spatiotemporal profile to validate the proposed method.},
keywords = {ARRAY(0x56147f5246e0)},
url = {http://eprints.lincoln.ac.uk/17880/}
}
• A. Coninx, P. Baxter, E. Oleari, S. Bellini, B. Bierman, O. B. Henkemans, L. Canamero, P. Cosi, V. Enescu, R. R. Espinoza, A. Hiolle, R. Humbert, B. Kiefer, I. Kruijff-korbayova, R. Looije, M. Mosconi, M. Neerincx, G. Paci, G. Patsis, C. Pozzi, F. Sacchitelli, H. Sahli, A. Sanna, G. Sommavilla, F. Tesser, Y. Demiris, and T. Belpaeme, “Towards long-term social child-robot interaction: using multi-activity switching to engage young users,” Journal of Human-Robot Interaction, vol. 5, iss. 1, pp. 32-67, 2016.

Social robots have the potential to provide support in a number of practical domains, such as learning and behaviour change. This potential is particularly relevant for children, who have proven receptive to interactions with social robots. To reach learning and therapeutic goals, a number of issues need to be investigated, notably the design of an effective child-robot interaction (cHRI) to ensure the child remains engaged in the relationship and that educational goals are met. Typically, current cHRI research experiments focus on a single type of interaction activity (e.g. a game). However, these can suffer from a lack of adaptation to the child, or from an increasingly repetitive nature of the activity and interaction. In this paper, we motivate and propose a practicable solution to this issue: an adaptive robot able to switch between multiple activities within single interactions. We describe a system that embodies this idea, and present a case study in which diabetic children collaboratively learn with the robot about various aspects of managing their condition. We demonstrate the ability of our system to induce a varied interaction and show the potential of this approach both as an educational tool and as a research method for long-term cHRI.

@article{lirolem23074,
pages = {32--67},
year = {2016},
title = {Towards long-term social child-robot interaction: using multi-activity switching to engage young users},
volume = {5},
journal = {Journal of Human-Robot Interaction},
author = {Alexandre Coninx and Paul Baxter and Elettra Oleari and Sara Bellini and Bert Bierman and Olivier Blanson Henkemans and Lola Canamero and Piero Cosi and Valentin Enescu and Raquel Ros Espinoza and Antoine Hiolle and Remi Humbert and Bernd Kiefer and Ivana Kruijff-korbayova and Rosemarijn Looije and Marco Mosconi and Mark Neerincx and Giulio Paci and Georgios Patsis and Clara Pozzi and Francesca Sacchitelli and Hichem Sahli and Alberto Sanna and Giacomo Sommavilla and Fabio Tesser and Yiannis Demiris and Tony Belpaeme},
number = {1},
abstract = {Social robots have the potential to provide support in a number of practical domains, such as learning and behaviour change. This potential is particularly relevant for children, who have proven receptive to interactions with social robots. To reach learning and therapeutic goals, a number of issues need to be investigated, notably the design of an effective child-robot interaction (cHRI) to ensure the child remains engaged in the relationship and that educational goals are met. Typically, current cHRI research experiments focus on a single type of interaction activity (e.g. a game). However, these can suffer from a lack of adaptation to the child, or from an increasingly repetitive nature of the activity and interaction. In this paper, we motivate and propose a practicable solution to this issue: an adaptive robot able to switch between multiple activities within single interactions. We describe a system that embodies this idea, and present a case study in which diabetic children collaboratively learn with the robot about various aspects of managing their condition. We demonstrate the ability of our system to induce a varied interaction and show the potential of this approach both as an educational tool and as a research method for long-term cHRI.},
keywords = {ARRAY(0x56147f625710)},
url = {http://eprints.lincoln.ac.uk/23074/}
}
• C. Coppola, D. Faria, U. Nunes, and N. Bellotto, “Social activity recognition based on probabilistic merging of skeleton features with proximity priors from RGB-D data,” in IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2016.

Social activity based on body motion is a key feature for non-verbal and physical behavior defined as function for communicative signal and social interaction between individuals. Social activity recognition is important to study human-human communication and also human-robot interaction. Based on that, this research has threefold goals: (1) recognition of social behavior (e.g. human-human interaction) using a probabilistic approach that merges spatio-temporal features from individual bodies and social features from the relationship between two individuals; (2) learn priors based on physical proximity between individuals during an interaction using proxemics theory to feed a probabilistic ensemble of classifiers; and (3) provide a public dataset with RGB-D data of social daily activities including risk situations useful to test approaches for assisted living, since this type of dataset is still missing. Results show that using a modified dynamic Bayesian mixture model designed to merge features with different semantics and also with proximity priors, the proposed framework can correctly recognize social activities in different situations, e.g. using data from one or two individuals.

@inproceedings{lirolem23425,
booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
month = {October},
title = {Social activity recognition based on probabilistic merging of skeleton features with proximity priors from RGB-D data},
year = {2016},
author = {Claudio Coppola and Diego Faria and Urbano Nunes and Nicola Bellotto},
publisher = {IEEE},
abstract = {Social activity based on body motion is a key feature for non-verbal and physical behavior defined as function for communicative signal and social interaction between individuals. Social activity recognition is important to study human-human communication and also human-robot interaction. Based on that, this research has threefold goals: (1) recognition of social behavior (e.g. human-human interaction) using a probabilistic approach that merges spatio-temporal features from individual bodies and social features from the relationship between two individuals; (2) learn priors based on physical proximity between individuals during an interaction using proxemics theory to feed a probabilistic ensemble of classifiers; and (3) provide a public dataset with RGB-D data
of social daily activities including risk situations useful to test approaches for assisted living, since this type of dataset is still missing. Results show that using a modified dynamic Bayesian mixture model designed to merge features with different semantics and also with proximity priors, the proposed framework can correctly recognize social activities in different situations, e.g. using data from one or two individuals.},
keywords = {ARRAY(0x56147f7437e0)},
url = {http://eprints.lincoln.ac.uk/23425/}
}
• C. Coppola, T. Krajnik, T. Duckett, and N. Bellotto, “Learning temporal context for activity recognition,” in European Conference on Artificial Intelligence (ECAI), 2016.

We investigate how incremental learning of long-term human activity patterns improves the accuracy of activity classification over time. Rather than trying to improve the classification methods themselves, we assume that they can take into account prior probabilities of activities occurring at a particular time. We use the classification results to build temporal models that can provide these priors to the classifiers. As our system gradually learns about typical patterns of human activities, the accuracy of activity classification improves, which results in even more accurate priors. Two datasets collected over several months containing hand-annotated activity in residential and office environments were chosen to evaluate the approach. Several types of temporal models were evaluated for each of these datasets. The results indicate that incremental learning of daily routines leads to a significant improvement in activity classification.

@inproceedings{lirolem23297,
month = {August},
booktitle = {European Conference  on Artificial Intelligence (ECAI)},
author = {Claudio Coppola and Tomas Krajnik and Tom Duckett and Nicola Bellotto},
title = {Learning temporal context for activity recognition},
year = {2016},
abstract = {We investigate how incremental learning of long-term human activity patterns improves the accuracy of activity classification over time. Rather than trying to improve the classification methods themselves, we assume that they can take into account prior probabilities of activities occurring at a particular time. We use the classification results to build temporal models that can provide these priors to the classifiers. As our system gradually learns about typical patterns of human activities, the accuracy of activity classification improves, which results in even more accurate priors. Two datasets collected over several months containing hand-annotated activity in residential and office environments were chosen to evaluate the approach. Several types of temporal models were evaluated for each of these datasets. The results indicate that incremental learning of daily routines leads to a significant improvement in activity classification.},
keywords = {ARRAY(0x56147f655c18)},
url = {http://eprints.lincoln.ac.uk/23297/}
}
• H. Cuayahuitl, G. Couly, and C. Olalainty, “Training an interactive humanoid robot using multimodal deep reinforcement learning,” in NIPS Workshop on Deep Reinforcement Learning, 2016.

Training robots to perceive, act and communicate using multiple modalities still represents a challenging problem, particularly if robots are expected to learn efficiently from small sets of example interactions. We describe a learning approach as a step in this direction, where we teach a humanoid robot how to play the game of noughts and crosses. Given that multiple multimodal skills can be trained to play this game, we focus our attention to training the robot to perceive the game, and to interact in this game. Our multimodal deep reinforcement learning agent perceives multimodal features and exhibits verbal and non-verbal actions while playing. Experimental results using simulations show that the robot can learn to win or draw up to 98\% of the games. A pilot test of the proposed multimodal system for the targeted game—integrating speech, vision and gestures—reports that reasonable and fluent interactions can be achieved using the proposed approach.

@inproceedings{lirolem25937,
booktitle = {NIPS Workshop on Deep Reinforcement Learning},
month = {December},
author = {Heriberto Cuayahuitl and Guillaume Couly and Clement Olalainty},
publisher = {arXiv},
volume = {abs/16},
journal = {CoRR},
year = {2016},
title = {Training an interactive humanoid robot using multimodal deep reinforcement learning},
abstract = {Training robots to perceive, act and communicate using multiple modalities still represents a challenging problem, particularly if robots are expected to learn efficiently from small sets of example interactions. We describe a learning approach as a step in this direction, where we teach a humanoid robot how to play the game of noughts and crosses. Given that multiple multimodal skills can be trained to play this game, we focus our attention to training the robot to perceive the game, and to interact in this game. Our multimodal deep reinforcement learning agent perceives multimodal features and exhibits verbal and non-verbal actions while playing. Experimental results using simulations show that the robot can learn to win or draw up to 98\% of the games. A pilot test of the proposed multimodal system for the targeted game---integrating speech, vision and gestures---reports that reasonable and fluent interactions can be achieved using the proposed approach.},
keywords = {ARRAY(0x56147f9e1748)},
url = {http://eprints.lincoln.ac.uk/25937/}
}
• H. Cuayahuitl, S. Yu, A. Williamson, and J. Carse, “Deep reinforcement learning for multi-domain dialogue systems,” in NIPS Workshop on Deep Reinforcement Learning, 2016.

Standard deep reinforcement learning methods such as Deep Q-Networks (DQN) for multiple tasks (domains) face scalability problems. We propose a method for multi-domain dialogue policy learning—termed NDQN, and apply it to an information-seeking spoken dialogue system in the domains of restaurants and hotels. Experimental results comparing DQN (baseline) versus NDQN (proposed) using simulations report that our proposed method exhibits better scalability and is promising for optimising the behaviour of multi-domain dialogue systems.

@inproceedings{lirolem25935,
title = {Deep reinforcement learning for multi-domain dialogue systems},
journal = {CoRR},
volume = {abs/16},
year = {2016},
author = {Heriberto Cuayahuitl and Seunghak Yu and Ashley Williamson and Jacob Carse},
publisher = {arXiv},
month = {December},
booktitle = {NIPS Workshop on Deep Reinforcement Learning},
url = {http://eprints.lincoln.ac.uk/25935/},
keywords = {ARRAY(0x56147f71a740)},
abstract = {Standard deep reinforcement learning methods such as Deep Q-Networks (DQN) for multiple tasks (domains) face scalability problems. We propose a method for multi-domain dialogue policy learning---termed NDQN, and apply it to an information-seeking spoken dialogue system in the domains of restaurants and hotels. Experimental results comparing DQN (baseline) versus NDQN (proposed) using simulations report that our proposed method exhibits better scalability and is promising for optimising the behaviour of multi-domain dialogue systems.}
}
• C. Daniel, H. van Hoof, J. Peters, and G. Neumann, “Probabilistic inference for determining options in reinforcement learning,” Machine Learning, vol. 104, iss. 2-3, pp. 337-357, 2016.

Tasks that require many sequential decisions or complex solutions are hard to solve using conventional reinforcement learning algorithms. Based on the semi Markov decision process setting (SMDP) and the option framework, we propose a model which aims to alleviate these concerns. Instead of learning a single monolithic policy, the agent learns a set of simpler sub-policies as well as the initiation and termination probabilities for each of those sub-policies. While existing option learning algorithms frequently require manual specification of components such as the sub-policies, we present an algorithm which infers all relevant components of the option framework from data. Furthermore, the proposed approach is based on parametric option representations and works well in combination with current policy search methods, which are particularly well suited for continuous real-world tasks. We present results on SMDPs with discrete as well as continuous state-action spaces. The results show that the presented algorithm can combine simple sub-policies to solve complex tasks and can improve learning performance on simpler tasks.

@article{lirolem25739,
pages = {337--357},
month = {September},
author = {C. Daniel and H. van Hoof and J. Peters and G. Neumann},
publisher = {Springer},
number = {2-3},
journal = {Machine Learning},
year = {2016},
volume = {104},
title = {Probabilistic inference for determining options in reinforcement learning},
abstract = {Tasks that require many sequential decisions or complex solutions are hard to solve using conventional reinforcement learning algorithms. Based on the semi Markov decision process setting (SMDP) and the option framework, we propose a model which aims to alleviate these concerns. Instead of learning a single monolithic policy, the agent learns a set of simpler sub-policies as well as the initiation and termination probabilities for each of those sub-policies. While existing option learning algorithms frequently require manual specification of components such as the sub-policies, we present an algorithm which infers all relevant components of the option framework from data. Furthermore, the proposed approach is based on parametric option representations and works well in combination with current policy search methods, which are particularly well suited for continuous real-world tasks. We present results on SMDPs with discrete as well as continuous state-action spaces. The results show that the presented algorithm can combine simple sub-policies to solve complex tasks and can improve learning performance on simpler tasks.},
url = {http://eprints.lincoln.ac.uk/25739/},
keywords = {ARRAY(0x56147f9946b8)}
}
• C. Daniel, G. Neumann, O. Kroemer, and J. Peters, “Hierarchical relative entropy policy search,” Journal of Machine Learning Research, vol. 17, pp. 1-50, 2016.

Many reinforcement learning (RL) tasks, especially in robotics, consist of multiple sub-tasks that are strongly structured. Such task structures can be exploited by incorporating hierarchical policies that consist of gating networks and sub-policies. However, this concept has only been partially explored for real world settings and complete methods, derived from first principles, are needed. Real world settings are challenging due to large and continuous state-action spaces that are prohibitive for exhaustive sampling methods. We define the problem of learning sub-policies in continuous state action spaces as finding a hierarchical policy that is composed of a high-level gating policy to select the low-level sub-policies for execution by the agent. In order to efficiently share experience with all sub-policies, also called inter-policy learning, we treat these sub-policies as latent variables which allows for distribution of the update information between the sub-policies. We present three different variants of our algorithm, designed to be suitable for a wide variety of real world robot learning tasks and evaluate our algorithms in two real robot learning scenarios as well as several simulations and comparisons.

@article{lirolem25743,
title = {Hierarchical relative entropy policy search},
journal = {Journal of Machine Learning Research},
volume = {17},
year = {2016},
pages = {1--50},
month = {June},
author = {C. Daniel and G. Neumann and O. Kroemer and J. Peters},
publisher = {Massachusetts Institute of Technology Press (MIT Press) / Microtome Publishing},
abstract = {Many reinforcement learning (RL) tasks, especially in robotics, consist of multiple sub-tasks that
are strongly structured. Such task structures can be exploited by incorporating hierarchical policies
that consist of gating networks and sub-policies. However, this concept has only been partially explored
for real world settings and complete methods, derived from first principles, are needed. Real
world settings are challenging due to large and continuous state-action spaces that are prohibitive
for exhaustive sampling methods. We define the problem of learning sub-policies in continuous
state action spaces as finding a hierarchical policy that is composed of a high-level gating policy to
select the low-level sub-policies for execution by the agent. In order to efficiently share experience
with all sub-policies, also called inter-policy learning, we treat these sub-policies as latent variables
which allows for distribution of the update information between the sub-policies. We present three
different variants of our algorithm, designed to be suitable for a wide variety of real world robot
learning tasks and evaluate our algorithms in two real robot learning scenarios as well as several
simulations and comparisons.},
url = {http://eprints.lincoln.ac.uk/25743/},
keywords = {ARRAY(0x56147f99fff8)}
}
• N. Dethlefs, H. Hastie, H. Cuayahuitl, Y. Yu, V. Rieser, and O. Lemon, “Information density and overlap in spoken dialogue,” Computer Speech & Language, vol. 37, pp. 82-97, 2016.

Incremental dialogue systems are often perceived as more responsive and natural because they are able to address phenomena of turn-taking and overlapping speech, such as backchannels or barge-ins. Previous work in this area has often identified distinctive prosodic features, or features relating to syntactic or semantic completeness, as marking appropriate places of turn-taking. In a separate strand of work, psycholinguistic studies have established a connection between information density and prominence in language–the less expected a linguistic unit is in a particular context, the more likely it is to be linguistically marked. This has been observed across linguistic levels, including the prosodic, which plays an important role in predicting overlapping speech. In this article, we explore the hypothesis that information density (ID) also plays a role in turn-taking. Specifically, we aim to show that humans are sensitive to the peaks and troughs of information density in speech, and that overlapping speech at ID troughs is perceived as more acceptable than overlaps at ID peaks. To test our hypothesis, we collect human ratings for three models of generating overlapping speech based on features of: (1) prosody and semantic or syntactic completeness, (2) information density, and (3) both types of information. Results show that over 50\% of users preferred the version using both types of features, followed by a preference for information density features alone. This indicates a clear human sensitivity to the effects of information density in spoken language and provides a strong motivation to adopt this metric for the design, development and evaluation of turn-taking modules in spoken and incremental dialogue systems.

@article{lirolem22216,
title = {Information density and overlap in spoken dialogue},
volume = {37},
journal = {Computer Speech \& Language},
year = {2016},
author = {Nina Dethlefs and Helen Hastie and Heriberto Cuayahuitl and Yanchao Yu and Verena Rieser and Oliver Lemon},
publisher = {Elsevier for International Speech Communication Association (ISCA)},
month = {May},
pages = {82--97},
keywords = {ARRAY(0x56147f728498)},
url = {http://eprints.lincoln.ac.uk/22216/},
abstract = {Incremental dialogue systems are often perceived as more responsive and natural because they are able to address phenomena of turn-taking and overlapping speech, such as backchannels or barge-ins. Previous work in this area has often identified distinctive prosodic features, or features relating to syntactic or semantic completeness, as marking appropriate places of turn-taking. In a separate strand of work, psycholinguistic studies have established a connection between information density and prominence in language{--}the less expected a linguistic unit is in a particular context, the more likely it is to be linguistically marked. This has been observed across linguistic levels, including the prosodic, which plays an important role in predicting overlapping speech.

In this article, we explore the hypothesis that information density (ID) also plays a role in turn-taking. Specifically, we aim to show that humans are sensitive to the peaks and troughs of information density in speech, and that overlapping speech at ID troughs is perceived as more acceptable than overlaps at ID peaks. To test our hypothesis, we collect human ratings for three models of generating overlapping speech based on features of: (1) prosody and semantic or syntactic completeness, (2) information density, and (3) both types of information. Results show that over 50\% of users preferred the version using both types of features, followed by a preference for information density features alone. This indicates a clear human sensitivity to the effects of information density in spoken language and provides a strong motivation to adopt this metric for the design, development and evaluation of turn-taking modules in spoken and incremental dialogue systems.}
}
• P. Dickinson, O. Szymanezyk, G. Cielniak, and M. Mannion, “Indoor positioning of shoppers using a network of bluetooth low energy beacons,” in 2016 International Conference on Indoor Positioning and Indoor Navigation (IPIN), 4-7 October 2016, Alcalá de Henares, Spain, 2016.

In this paper we present our work on the indoor positioning of users (shoppers), using a network of Bluetooth Low Energy (BLE) beacons deployed in a large wholesale shopping store. Our objective is to accurately determine which product sections a user is adjacent to while traversing the store, using RSSI readings from multiple beacons, measured asynchronously on a standard commercial mobile device. We further wish to leverage the store layout (which imposes natural constraints on the movement of users) and the physical configuration of the beacon network, to produce a robust and efficient solution. We start by describing our application context and hardware configuration, and proceed to introduce our node-graph model of user location. We then describe our experimental work which begins with an investigation of signal characteristics along and across aisles. We propose three methods of localization, using a ?nearest-beacon? approach as a base-line; exponentially averaged weighted range estimates; and a particle-filter method based on the RSSI attenuation model and Gaussian-noise. Our results demonstrate that the particle filter method significantly out-performs the others. Scalability also makes this method ideal for applications run on mobile devices with more limited computational capabilities

@inproceedings{lirolem24589,
author = {Patrick Dickinson and Olivier Szymanezyk and Grzegorz Cielniak and Mike Mannion},
publisher = {IEEE Xplore},
year = {2016},
title = {Indoor positioning of shoppers using a network of bluetooth low energy beacons},
booktitle = {2016 International Conference on Indoor Positioning and Indoor Navigation (IPIN), 4-7 October 2016, Alcal{\'a} de Henares, Spain},
month = {October},
keywords = {ARRAY(0x56147f74f6a8)},
url = {http://eprints.lincoln.ac.uk/24589/},
abstract = {In this paper we present our work on the indoor positioning of users (shoppers), using a network of Bluetooth Low Energy (BLE) beacons deployed in a large wholesale shopping store. Our objective is to accurately determine which product sections a user is adjacent to while traversing the store, using RSSI readings from multiple beacons, measured asynchronously on a standard commercial mobile device. We further wish to leverage the store layout (which imposes natural constraints on the movement of users) and the physical configuration of the beacon network, to produce a robust and efficient solution. We start by describing our application context and hardware configuration, and proceed to introduce our node-graph model of user location. We then describe our experimental work which begins with an investigation of signal characteristics along and across aisles. We propose three methods of localization, using a ?nearest-beacon? approach as a base-line; exponentially averaged weighted range estimates; and a particle-filter method based on the RSSI attenuation model and Gaussian-noise. Our results demonstrate that the particle filter method significantly out-performs the others. Scalability also makes this method ideal for applications run on mobile devices with more limited computational capabilities}
}
• C. Dondrup and M. Hanheide, “Qualitative constraints for human-aware robot navigation using Velocity Costmaps,” in 2016 25th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), 2016, pp. 586-592.

In this work, we propose the combination of a state-of-the-art sampling-based local planner with so-called Velocity Costmaps to achieve human-aware robot navigation. Instead of introducing humans as ?special obstacles? into the representation of the environment, we restrict the sample space of a ?Dynamic Window Approach? local planner to only allow trajectories based on a qualitative description of the future unfolding of the encounter. To achieve this, we use a Bayesian temporal model based on a Qualitative Trajectory Calculus to represent the mutual navigation intent of human and robot, and translate these descriptors into sample space constraints for trajectory generation. We show how to learn these models from demonstration and evaluate our approach against standard Gaussian cost models in simulation and in real-world using a non-holonomic mobile robot. Our experiments show that our approach exceeds the performance and safety of the Gaussian models in pass-by and path crossing situations.

@inproceedings{lirolem27957,
month = {August},
pages = {586--592},
booktitle = {2016 25th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN)},
year = {2016},
title = {Qualitative constraints for human-aware robot navigation using Velocity Costmaps},
publisher = {IEEE},
author = {Christian Dondrup and Marc Hanheide},
url = {http://eprints.lincoln.ac.uk/27957/},
keywords = {ARRAY(0x56147f5795d0)},
abstract = {In this work, we propose the combination of a state-of-the-art sampling-based local planner with so-called Velocity Costmaps to achieve human-aware robot navigation. Instead of introducing humans as ?special obstacles? into the representation of the environment, we restrict the sample space of a ?Dynamic Window Approach? local planner to only allow trajectories based on a qualitative description of the future unfolding of the encounter. To achieve this, we use a Bayesian temporal model based on a Qualitative Trajectory Calculus to represent the mutual navigation intent of human and robot, and translate these descriptors into sample space constraints for trajectory generation. We show how to learn these models from demonstration and evaluate our approach against standard Gaussian cost models in simulation and in real-world using a non-holonomic mobile robot. Our experiments show that our approach exceeds the performance and safety of the Gaussian models in pass-by and path crossing situations.}
}
• M. Ewerton, G. Maeda, G. Neumann, V. Kisner, G. Kollegger, J. Wiemeyer, and J. Peters, “Movement primitives with multiple phase parameters,” in Robotics and Automation (ICRA), 2016 IEEE International Conference on, 2016, pp. 201-206.

Movement primitives are concise movement representations that can be learned from human demonstrations, support generalization to novel situations and modulate the speed of execution of movements. The speed modulation mechanisms proposed so far are limited though, allowing only for uniform speed modulation or coupling changes in speed to local measurements of forces, torques or other quantities. Those approaches are not enough when dealing with general velocity constraints. We present a movement primitive formulation that can be used to non-uniformly adapt the speed of execution of a movement in order to satisfy a given constraint, while maintaining similarity in shape to the original trajectory. We present results using a 4-DoF robot arm in a minigolf setup.

@inproceedings{lirolem25742,
journal = {Proceedings - IEEE International Conference on Robotics and Automation},
volume = {2016-J},
year = {2016},
title = {Movement primitives with multiple phase parameters},
author = {M. Ewerton and G. Maeda and G. Neumann and V. Kisner and G. Kollegger and J. Wiemeyer and J. Peters},
month = {June},
pages = {201--206},
booktitle = {Robotics and Automation (ICRA), 2016 IEEE International Conference on},
abstract = {Movement primitives are concise movement representations that can be learned from human demonstrations, support generalization to novel situations and modulate the speed of execution of movements. The speed modulation mechanisms proposed so far are limited though, allowing only for uniform speed modulation or coupling changes in speed to local measurements of forces, torques or other quantities. Those approaches are not enough when dealing with general velocity constraints. We present a movement primitive formulation that can be used to non-uniformly adapt the speed of execution of a movement in order to satisfy a given constraint, while maintaining similarity in shape to the original trajectory. We present results using a 4-DoF robot arm in a minigolf setup.},
keywords = {ARRAY(0x56147f815018)},
url = {http://eprints.lincoln.ac.uk/25742/}
}
• J. P. Fentanes, T. Krajnik, M. Hanheide, and T. Duckett, “Persistent localization and life-long mapping in changing environments using the frequency map enhancement,” in IEEE/RSJ International Conference on Intelligent Robots ans Systems (IROS), 2016.

We present a lifelong mapping and localisation system for long-term autonomous operation of mobile robots in changing environments. The core of the system is a spatio-temporal occupancy grid that explicitly represents the persistence and periodicity of the individual cells and can predict the probability of their occupancy in the future. During navigation, our robot builds temporally local maps and integrates then into the global spatio-temporal grid. Through re-observation of the same locations, the spatio-temporal grid learns the long-term environment dynamics and gains the ability to predict the future environment states. This predictive ability allows to generate time-specific 2d maps used by the robot’s localisation and planning modules. By analysing data from a long-term deployment of the robot in a human-populated environment, we show that the proposed representation improves localisation accuracy and the efficiency of path planning. We also show how to integrate the method into the ROS navigation stack for use by other roboticists.

@inproceedings{lirolem24088,
month = {October},
booktitle = {IEEE/RSJ International Conference on Intelligent Robots ans Systems (IROS)},
publisher = {IEEE},
author = {Jaime Pulido Fentanes and Tomas Krajnik and Marc Hanheide and Tom Duckett},
title = {Persistent localization and life-long mapping in changing environments using the frequency map enhancement},
year = {2016},
url = {http://eprints.lincoln.ac.uk/24088/},
keywords = {ARRAY(0x56147f5f0120)},
abstract = {We present a lifelong mapping and localisation system for long-term autonomous operation of mobile robots in changing environments.
The core of the system is a spatio-temporal occupancy grid that explicitly represents the persistence and periodicity of the individual cells and can predict the probability of their occupancy in the future.
During navigation, our robot builds temporally local maps and integrates then into the global spatio-temporal grid. Through re-observation of the same locations, the spatio-temporal grid learns the long-term environment dynamics and gains the ability to predict the future environment states. This predictive ability allows to generate time-specific 2d maps  used by the robot's localisation and planning modules. By analysing data from a long-term deployment of the robot in a human-populated environment, we show that the proposed  representation improves localisation accuracy and the efficiency of path planning. We also show how to integrate the method into the ROS navigation stack for use by other roboticists.}
}
• M. Fernandez-Carmona and N. Bellotto, “On-line inference comparison with Markov Logic Network engines for activity recognition in AAL environments,” in IEEE International Conference on Intelligent Environments, 2016.

We address possible solutions for a practical application of Markov Logic Networks to online activity recognition, based on domotic sensors, to be used for monitoring elderly with mild cognitive impairments. Our system has to provide responsive information about user activities throughout the day, so different inference engines are tested. We use an abstraction layer to gather information from commercial domotic sensors. Sensor events are stored using a non-relational database. Using this database, evidences are built to query a logic network about current activities. Markov Logic Networks are able to deal with uncertainty while keeping a structured knowledge. This makes them a suitable tool for ambient sensors based inference. However, in their previous application, inferences are usually made offline. Time is a relevant constrain in our system and hence logic networks are designed here accordingly. We compare in this work different engines to model a Markov Logic Network suitable for such circumstances. Results show some insights about how to design a low latency logic network and which kind of solutions should be avoided.

@inproceedings{lirolem23189,
title = {On-line inference comparison with Markov Logic Network engines for activity recognition in AAL environments},
year = {2016},
publisher = {IEEE},
author = {Manuel Fernandez-Carmona and Nicola Bellotto},
booktitle = {IEEE International Conference on Intelligent Environments},
month = {September},
abstract = {We address possible solutions for a practical application of Markov Logic Networks to online activity recognition, based on domotic sensors, to be used for monitoring elderly with mild cognitive impairments. Our system has to provide responsive information about user activities throughout the day, so different inference engines are tested. We use an abstraction layer to gather information from commercial domotic sensors.  Sensor events are stored using a non-relational database. Using this database, evidences are built to query a logic network about current activities. Markov Logic Networks are able to deal with uncertainty while keeping a structured knowledge. This makes them a suitable tool for ambient sensors based inference. However, in their previous application, inferences are usually made offline. Time is a relevant constrain in our system and hence logic networks are designed here accordingly. We compare in this work different engines to model a Markov Logic Network suitable for such circumstances. Results show some insights about how to design a low latency logic network and which kind of solutions should be avoided.},
keywords = {ARRAY(0x56147f7ff6d0)},
url = {http://eprints.lincoln.ac.uk/23189/}
}
• Q. Fu, S. Yue, and C. Hu, “Bio-inspired collision detector with enhanced selectivity for ground robotic vision system,” in 27th British Machine Vision Conference, 2016.

There are many ways of building collision-detecting systems. In this paper, we propose a novel collision selective visual neural network inspired by LGMD2 neurons in the juvenile locusts. Such collision-sensitive neuron matures early in the ?rst-aged or even hatching locusts, and is only selective to detect looming dark objects against bright background in depth, represents swooping predators, a situation which is similar to ground robots or vehicles. However, little has been done on modeling LGMD2, let alone its potential applications in robotics and other vision-based areas. Compared to other collision detectors, our major contributions are ?rst, enhancing the collision selectivity in a bio-inspired way, via constructing a computing ef?cient visual sensor, and realizing the revealed speci?c characteristic sofLGMD2. Second, we applied the neural network to help rearrange path navigation of an autonomous ground miniature robot in an arena. We also examined its neural properties through systematic experiments challenged against image streams from a visual sensor of the micro-robot.

@inproceedings{lirolem24941,
booktitle = {27th British Machine Vision Conference},
month = {September},
title = {Bio-inspired collision detector with enhanced selectivity for ground robotic vision system},
year = {2016},
author = {Qinbing Fu and Shigang Yue and Cheng Hu},
abstract = {There are many ways of building collision-detecting systems. In this paper, we propose a novel collision selective visual neural network inspired by LGMD2 neurons in the juvenile locusts. Such collision-sensitive neuron matures early in the ?rst-aged or even hatching locusts, and is only selective to detect looming dark objects against bright background in depth, represents swooping predators, a situation which is similar to ground robots or vehicles. However, little has been done on modeling LGMD2, let alone its potential applications in robotics and other vision-based areas. Compared to other collision detectors, our major contributions are ?rst, enhancing the collision selectivity in a bio-inspired way, via constructing a computing ef?cient visual sensor, and realizing the revealed speci?c characteristic sofLGMD2. Second, we applied the neural network to help rearrange path navigation of an autonomous ground miniature robot in an arena. We also examined its neural properties through systematic experiments challenged against image streams from a visual sensor of the micro-robot.},
url = {http://eprints.lincoln.ac.uk/24941/},
keywords = {ARRAY(0x56147fa12630)}
}
• Y. Gatsoulis, M. Alomari, C. Burbridge, C. Dondrup, P. Duckworth, P. Lightbody, M. Hanheide, N. Hawes, D. C. Hogg, and A. G. Cohn, “QSRlib: a software library for online acquisition of qualitative spatial relations from video,” in 29th International Workshop on Qualitative Reasoning (QR16), at IJCAI-16, 2016.

There is increasing interest in using Qualitative Spatial Relations as a formalism to abstract from noisy and large amounts of video data in order to form high level conceptualisations, e.g. of activities present in video. We present a library to support such work. It is compatible with the Robot Operating System (ROS) but can also be used stand alone. A number of QSRs are built in; others can be easily added.

@inproceedings{lirolem24853,
author = {Y. Gatsoulis and M. Alomari and C. Burbridge and C. Dondrup and P. Duckworth and P. Lightbody and M. Hanheide and N. Hawes and D. C. Hogg and A. G. Cohn},
year = {2016},
title = {QSRlib: a software library for online acquisition of qualitative spatial relations from video},
month = {July},
booktitle = {29th International Workshop on Qualitative Reasoning (QR16), at IJCAI-16},
abstract = {There is increasing interest in using Qualitative Spatial
Relations as a formalism to abstract from noisy and
large amounts of video data in order to form high level
conceptualisations, e.g. of activities present in video.
We present a library to support such work. It is compatible
with the Robot Operating System (ROS) but can
also be used stand alone. A number of QSRs are built
in; others can be easily added.},
url = {http://eprints.lincoln.ac.uk/24853/},
keywords = {ARRAY(0x56147f9fac68)}
}
• K. Gerling, D. Hebesberger, C. Dondrup, T. K$backslash$”ortner, and M. Hanheide, “Robot deployment in long-term care: a case study of a mobile robot in physical therapy,” Zeitschrift für Geriatrie und Gerontologie, vol. 49, iss. 4, pp. 288-297, 2016.

Background. Healthcare systems in industrialised countries are challenged to provide care for a growing number of older adults. Information technology holds the promise of facilitating this process by providing support for care staff, and improving wellbeing of older adults through a variety of support systems. Goal. Little is known about the challenges that arise from the deployment of technology in care settings; yet, the integration of technology into care is one of the core determinants of successful support. In this paper, we discuss challenges and opportunities associated with technology integration in care using the example of a mobile robot to support physical therapy among older adults with cognitive impairment in the European project STRANDS. Results and discussion. We report on technical challenges along with perspectives of physical therapists, and provide an overview of lessons learned which we hope will help inform the work of researchers and practitioners wishing to integrate robotic aids in the caregiving process.

@article{lirolem22902,
month = {June},
pages = {288--297},
publisher = {Springer for Bundesverband Geriatrie / Deutsche Gesellschaft f{\"u}r Gerontologie und Geriatrie},
author = {Kathrin Gerling and Denise Hebesberger and Christian Dondrup and Tobias K{$\backslash$}"ortner and Marc Hanheide},
title = {Robot deployment in long-term care: a case study of a mobile robot in physical therapy},
year = {2016},
journal = {Zeitschrift f{\"u}r Geriatrie und Gerontologie},
volume = {49},
number = {4},
abstract = {Background. Healthcare systems in industrialised countries are challenged to provide
care for a growing number of older adults. Information technology holds the promise of
facilitating this process by providing support for care staff, and improving wellbeing of
older adults through a variety of support systems. Goal. Little is known about the
challenges that arise from the deployment of technology in care settings; yet, the
integration of technology into care is one of the core determinants of successful
support. In this paper, we discuss challenges and opportunities associated with
technology integration in care using the example of a mobile robot to support physical
therapy among older adults with cognitive impairment in the European project
STRANDS. Results and discussion. We report on technical challenges along with
perspectives of physical therapists, and provide an overview of lessons learned which
we hope will help inform the work of researchers and practitioners wishing to integrate
robotic aids in the caregiving process.},
keywords = {ARRAY(0x56147f6e7428)},
url = {http://eprints.lincoln.ac.uk/22902/}
}
• E. Gyebi, M. Hanheide, and G. Cielniak, “The effectiveness of integrating educational robotic activities into higher education Computer Science curricula: a case study in a developing country,” in Edurobotics 2016, 2016, pp. 73-87.

In this paper, we present a case study to investigate the effects of educational robotics on a formal undergraduate Computer Science education in a developing country. The key contributions of this paper include a longitudinal study design, spanning the whole duration of one taught course, and its focus on continually assessing the effectiveness and the impact of robotic-based exercises. The study assessed the students’ motivation, engagement and level of understanding in learning general computer programming. The survey results indicate that there are benefits which can be gained from such activities and educational robotics is a promising tool in developing engaging study curricula. We hope that our experience from this study together with the free materials and data available for download will be beneficial to other practitioners working with educational robotics in different parts of the world.

@inproceedings{lirolem25579,
month = {November},
booktitle = {Edurobotics 2016},
pages = {73--87},
author = {Ernest Gyebi and Marc Hanheide and Grzegorz Cielniak},
publisher = {Springer},
year = {2016},
title = {The effectiveness of integrating educational robotic activities into higher education Computer Science curricula: a case study in a developing country},
url = {http://eprints.lincoln.ac.uk/25579/},
keywords = {ARRAY(0x56147f6fca38)},
abstract = {In this paper, we present a case study to investigate the effects of educational robotics on a formal undergraduate Computer Science education in a developing country. The key contributions of this paper include a longitudinal study design, spanning the whole duration of one taught course, and its focus on continually assessing the effectiveness and the impact of robotic-based exercises. The study assessed the  students' motivation, engagement and level of understanding in learning general computer programming. The survey results indicate that there are benefits which can be gained from such activities and educational robotics is a promising tool in developing engaging study curricula. We hope that our experience from this study together with the free materials and data available for download will be beneficial to other practitioners working with educational robotics in different parts of the world.}
}
• C. Hu, F. Arvin, C. Xiong, and S. Yue, “A bio-inspired embedded vision system for autonomous micro-robots: the LGMD case,” IEEE Transactions on Cognitive and Developmental Systems, vol. PP, iss. 99, pp. 1-14, 2016.

In this paper, we present a new bio-inspired vision system embedded for micro-robots. The vision system takes inspiration from locusts in detecting fast approaching objects. Neurophysiological research suggested that locusts use a wide-field visual neuron called lobula giant movement detector (LGMD) to respond to imminent collisions. In this work, we present the implementation of the selected neuron model by a low-cost ARM processor as part of a composite vision module. As the first embedded LGMD vision module fits to a micro-robot, the developed system performs all image acquisition and processing independently. The vision module is placed on top of a microrobot to initiate obstacle avoidance behaviour autonomously. Both simulation and real-world experiments were carried out to test the reliability and robustness of the vision system. The results of the experiments with different scenarios demonstrated the potential of the bio-inspired vision system as a low-cost embedded module for autonomous robots.

@article{lirolem25279,
volume = {PP},
year = {2016},
title = {A bio-inspired embedded vision system for autonomous micro-robots: the LGMD case},
journal = {IEEE Transactions on Cognitive and Developmental Systems},
number = {99},
month = {May},
pages = {1--14},
publisher = {IEEE},
author = {Cheng Hu and Farshad Arvin and Caihua Xiong and Shigang Yue},
keywords = {ARRAY(0x56147f889348)},
url = {http://eprints.lincoln.ac.uk/25279/},
abstract = {In this paper, we present a new bio-inspired vision
system embedded for micro-robots. The vision system takes inspiration
from locusts in detecting fast approaching objects. Neurophysiological
research suggested that locusts use a wide-field
visual neuron called lobula giant movement detector (LGMD)
to respond to imminent collisions. In this work, we present
the implementation of the selected neuron model by a low-cost
ARM processor as part of a composite vision module. As the
first embedded LGMD vision module fits to a micro-robot, the
developed system performs all image acquisition and processing
independently. The vision module is placed on top of a microrobot
to initiate obstacle avoidance behaviour autonomously. Both
simulation and real-world experiments were carried out to test
the reliability and robustness of the vision system. The results
of the experiments with different scenarios demonstrated the
potential of the bio-inspired vision system as a low-cost embedded
module for autonomous robots.}
}
• C. Keeble, P. D. Baxter, S. Barber, and G. R. Law, “Participation rates In epidemiology studies and surveys: a review 2005 – 2007,” The Internet Journal of Epidemiology, vol. 14, iss. 1, pp. 1-14, 2016.

Understanding the factors associated with participation is key in addressing the problem of declining participation rates in epidemiological studies. This review aims to summarise factors affecting participation rates in articles published during the last nine years, to compare with previous findings to determine whether the research focus for non-participation has changed and whether the findings have been consistent over time. Web of Science was used to search titles of English articles from 2007?2015 for a range of synonymous words concerning participation rates. A predefined inclusion criteria was used to determine whether the resulting articles referred to participation in the context of study enrolment. Factors associated with participation were extracted from included articles. The search returned 626 articles, of which 162 satisfied the inclusion criteria. Compared with pre-2007, participant characteristics generally remained unchanged, but were topic-dependent. An increased focus on study design and a greater use of technology for enrolment and data collection was found, suggesting a transition towards technology-based methods. In addition to increased participation rates, studies should consider any bias arising from non-participation. When reporting results, authors are encouraged to include a standardised participation rate, a calculation of potential bias, and to apply an appropriate statistical method where appropriate. Requirements from journals to include these would allow for easier comparison of results between studies.

@article{lirolem26606,
year = {2016},
volume = {14},
title = {Participation rates In epidemiology studies and surveys: a review 2005 - 2007},
journal = {The Internet Journal of Epidemiology},
number = {1},
publisher = {Internet Scientific Publications, LLC},
author = {C. Keeble and P. D. Baxter and S. Barber and G. R. Law},
month = {January},
pages = {1--14},
abstract = {Understanding the factors associated with participation is key in addressing the problem of declining participation rates in epidemiological studies. This review aims to summarise factors affecting participation rates in articles published during the last nine years, to compare with previous findings to determine whether the research focus for non-participation has changed and whether the findings have been consistent over time. Web of Science was used to search titles of English articles from 2007?2015 for a range of synonymous words concerning participation rates. A predefined inclusion criteria was used to determine whether the resulting articles referred to participation in the context of study enrolment. Factors associated with participation were extracted from included articles. The search returned 626 articles, of which 162 satisfied the inclusion criteria. Compared with pre-2007, participant characteristics generally remained unchanged, but were topic-dependent. An increased focus on study design and a greater use of technology for enrolment and data collection was found, suggesting a transition towards technology-based methods. In addition to increased participation rates, studies should consider any bias arising from non-participation. When reporting results, authors are encouraged to include a standardised participation rate, a calculation of potential bias, and to apply an appropriate statistical method where appropriate. Requirements from journals to include these would allow for easier comparison of results between studies.},
keywords = {ARRAY(0x56147f928198)},
url = {http://eprints.lincoln.ac.uk/26606/}
}
• J. Kennedy, P. Baxter, E. Senft, and T. Belpaeme, “Social robot tutoring for child second language learning,” in Proceedings of the Tenth Annual ACM/IEEE International Conference on Human-Robot Interaction HRI 2016, Christchurch, New Zealand, 2016, pp. 231-238.

An increasing amount of research is being conducted to determine how a robot tutor should behave socially in educa- tional interactions with children. Both human-human and human- robot interaction literature predicts an increase in learning with increased social availability of a tutor, where social availability has verbal and nonverbal components. Prior work has shown that greater availability in the nonverbal behaviour of a robot tutor has a positive impact on child learning. This paper presents a study with 67 children to explore how social aspects of a tutor robot?s speech influences their perception of the robot and their language learning in an interaction. Children perceive the difference in social behaviour between ?low? and ?high? verbal availability conditions, and improve significantly between a pre- and a post-test in both conditions. A longer-term retention test taken the following week showed that the children had retained almost all of the information they had learnt. However, learning was not affected by which of the robot behaviours they had been exposed to. It is suggested that in this short-term interaction context, additional effort in developing social aspects of a robot?s verbal behaviour may not return the desired positive impact on learning gains.

@inproceedings{lirolem24855,
publisher = {ACM Press},
author = {James Kennedy and Paul Baxter and Emmanuel Senft and Tony Belpaeme},
month = {March},
pages = {231--238},
booktitle = {Proceedings of the Tenth Annual ACM/IEEE International Conference on Human-Robot Interaction HRI 2016},
title = {Social robot tutoring for child second language learning},
year = {2016},
abstract = {An increasing amount of research is being conducted
to determine how a robot tutor should behave socially in educa- tional interactions with children. Both human-human and human- robot interaction literature predicts an increase in learning with increased social availability of a tutor, where social availability has verbal and nonverbal components. Prior work has shown that greater availability in the nonverbal behaviour of a robot tutor has a positive impact on child learning. This paper presents a study with 67 children to explore how social aspects of a tutor robot?s speech influences their perception of the robot and their language learning in an interaction. Children perceive the difference in social behaviour between ?low? and ?high? verbal availability conditions, and improve significantly between a pre- and a post-test in both conditions. A longer-term retention test taken the following week showed that the children had retained almost all of the information they had learnt. However, learning was not affected by which of the robot behaviours they had been exposed to. It is suggested that in this short-term interaction context, additional effort in developing social aspects of a robot?s verbal behaviour may not return the desired positive impact on learning gains.},
keywords = {ARRAY(0x56147f99ea58)},
url = {http://eprints.lincoln.ac.uk/24855/}
}
• T. Krajnik, J. P. Fentanes, J. Santos, and T. Duckett, “Frequency map enhancement: introducing dynamics into static environment models,” in ICRA Workshop AI for Long-Term Autonomy, 2016.

We present applications of the Frequency Map Enhancement (FreMEn), which improves the performance of mobile robots in long-term scenarios by introducing the notion of dynamics into their (originally static) environment models. Rather than using a fixed probability value, the method models the uncertainty of the elementary environment states by their frequency spectra. This allows to integrate sparse and irregular observations obtained during long-term deployments of mobile robots into memory-efficient spatio-temporal models that reflect mid- and long-term pseudo-periodic environment variations. The frequency-enhanced spatio-temporal models allow to predict the future environment states, which improves the efficiency of mobile robot operation in changing environments. In a series of experiments performed over periods of weeks to years, we demonstrate that the proposed approach improves mobile robot localization, path and task planning, activity recognition and allows for life-long spatio-temporal exploration.

@inproceedings{lirolem23261,
author = {Tomas Krajnik and Jaime Pulido Fentanes and Joao Santos and Tom Duckett},
year = {2016},
title = {Frequency map enhancement: introducing dynamics into static environment models},
month = {May},
booktitle = {ICRA Workshop AI for Long-Term Autonomy},
abstract = {We present applications of the Frequency Map Enhancement (FreMEn), which improves the performance of mobile robots in long-term scenarios by introducing the notion of dynamics into their (originally static) environment models. Rather than using a fixed probability value, the method models the uncertainty of the elementary environment states by their frequency spectra. This allows to integrate sparse and irregular observations obtained during long-term deployments of mobile robots into memory-efficient spatio-temporal models that reflect mid- and long-term pseudo-periodic environment variations. The frequency-enhanced spatio-temporal models allow to predict the future environment states, which improves the efficiency of mobile robot operation in changing environments.   In a series of experiments performed over periods of weeks to years, we demonstrate that the proposed approach improves mobile robot localization, path and task planning, activity recognition and allows for life-long spatio-temporal exploration.},
url = {http://eprints.lincoln.ac.uk/23261/},
keywords = {ARRAY(0x56147f565390)}
}
• M. Kulich, T. Krajnik, L. Preucil, and T. Duckett, “To explore or to exploit? Learning humans’ behaviour to maximize interactions with them,” in International Workshop on Modelling and Simulation for Autonomous Systems, 2016, pp. 48-63.

Assume a robot operating in a public space (e.g., a library, a museum) and serving visitors as a companion, a guide or an information stand. To do that, the robot has to interact with humans, which presumes that it actively searches for humans in order to interact with them. This paper addresses the problem how to plan robot’s actions in order to maximize the number of such interactions in the case human behavior is not known in advance. We formulate this problem as the exploration/exploitation problem and design several strategies for the robot. The main contribution of the paper than lies in evaluation and comparison of the designed strategies on two datasets. The evaluation shows interesting properties of the strategies, which are discussed.

@inproceedings{lirolem26195,
publisher = {Springer},
author = {Miroslav Kulich and Tomas Krajnik and Libor Preucil and Tom Duckett},
year = {2016},
title = {To explore or to exploit? Learning humans' behaviour to maximize interactions with them},
booktitle = {International Workshop on Modelling and Simulation for Autonomous Systems},
pages = {48--63},
month = {June},
abstract = {Assume a robot operating in a public space (e.g., a library, a museum) and serving visitors as a companion, a guide or an information stand. To do  that, the robot has to interact with humans, which presumes that it actively searches for humans in order to interact with them. This paper addresses the problem how to plan robot's actions in order to maximize the number of such interactions in the case human behavior is not known in advance. We formulate this problem as the exploration/exploitation problem and design several strategies for the robot. The main contribution of the paper than lies in evaluation and comparison of the designed strategies on two datasets. The evaluation shows interesting properties of the strategies, which are discussed.},
url = {http://eprints.lincoln.ac.uk/26195/},
keywords = {ARRAY(0x56147fa12318)}
}
• K. Kusumam, T. Krajnik, S. Pearson, G. Cielniak, and T. Duckett, “Can you pick a broccoli? 3D-vision based detection and localisation of broccoli heads in the field,” in IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2016.

This paper presents a 3D vision system for robotic harvesting of broccoli using low-cost RGB-D sensors. The presented method addresses the tasks of detecting mature broccoli heads in the field and providing their 3D locations relative to the vehicle. The paper evaluates different 3D features, machine learning and temporal filtering methods for detection of broccoli heads. Our experiments show that a combination of Viewpoint Feature Histograms, Support Vector Machine classifier and a temporal filter to track the detected heads results in a system that detects broccoli heads with 95.2\% precision. We also show that the temporal filtering can be used to generate a 3D map of the broccoli head positions in the field.

@inproceedings{lirolem24087,
title = {Can you pick a broccoli? 3D-vision based detection and localisation of broccoli heads in the field},
year = {2016},
author = {Keerthy Kusumam and Tomas Krajnik and Simon Pearson and Grzegorz Cielniak and Tom Duckett},
publisher = {IEEE},
month = {October},
booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
abstract = {This paper presents a 3D vision system for robotic harvesting of broccoli using low-cost RGB-D sensors. The presented method addresses the tasks of detecting mature broccoli heads in the field and providing their 3D locations relative to the vehicle. The paper evaluates different 3D features, machine learning and temporal filtering methods for detection of broccoli heads. Our experiments show that a combination of Viewpoint Feature Histograms, Support Vector Machine classifier and a temporal filter to track the detected heads results in a system that detects broccoli heads with 95.2\% precision. We also show that the temporal filtering can be used to generate a 3D map of the broccoli head positions in the field.},
url = {http://eprints.lincoln.ac.uk/24087/},
keywords = {ARRAY(0x56147f57fa08)}
}
• X. Li, W. Qi, and S. Yue, “An effective pansharpening method based on guided filtering,” in 2016 IEEE 11th Conference on Industrial Electronics and Applications (ICIEA), 2016, pp. 534-538.

Pansharpening is an important tool in remote sensing applications. It transforms a set of low-spatial-resolution multispectral images to high-spatial-resolution images by fusing with a co-registered high-spatial-resolution panchromatic image. To deal with the increasing high resolution satellite images, wide varieties of pansharpening techniques have been developed. In this paper, we present an effective pansharpening method based on guided filtering. The method takes advantage of the guided filter to refine the blocking edges in the upscaled multispectral images and extract sufficient high frequency details from the panchromatic image. Moreover, it can be implemented to sharpen multispectral imagery in a convenient band-by-band manner. The experimental evaluations are carried out on QuickBird satellite images. Subjective and objective evaluations show that our proposed method can achieve high spectral and spatial quality and outperforms some existing methods.

@inproceedings{lirolem27952,
title = {An effective pansharpening method based on guided filtering},
year = {2016},
author = {Xu Li and Weifeng Qi and Shigang Yue},
pages = {534--538},
booktitle = {2016 IEEE 11th Conference on Industrial Electronics and Applications (ICIEA)},
month = {June},
url = {http://eprints.lincoln.ac.uk/27952/},
keywords = {ARRAY(0x56147f7702c0)},
abstract = {Pansharpening is an important tool in remote sensing applications. It transforms a set of low-spatial-resolution multispectral images to high-spatial-resolution images by fusing with a co-registered high-spatial-resolution panchromatic image. To deal with the increasing high resolution satellite images, wide varieties of pansharpening techniques have been developed. In this paper, we present an effective pansharpening method based on guided filtering. The method takes advantage of the guided filter to refine the blocking edges in the upscaled multispectral images and extract sufficient high frequency details from the panchromatic image. Moreover, it can be implemented to sharpen multispectral imagery in a convenient band-by-band manner. The experimental evaluations are carried out on QuickBird satellite images. Subjective and objective evaluations show that our proposed method can achieve high spectral and spatial quality and outperforms some existing methods.}
}
• F. Lier, M. Hanheide, L. Natale, S. Schulz, J. Weisz, S. Wachsmuth, and S. Wrede, “Towards automated system and experiment reproduction in robotics,” in 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2016.

Even though research on autonomous robots and human-robot interaction accomplished great progress in recent years, and reusable soft- and hardware components are available, many of the reported findings are only hardly reproducible by fellow scientists. Usually, reproducibility is impeded because required information, such as the specification of software versions and their configuration, required data sets, and experiment protocols are not mentioned or referenced in most publications. In order to address these issues, we recently introduced an integrated tool chain and its underlying development process to facilitate reproducibility in robotics. In this contribution we instantiate the complete tool chain in a unique user study in order to assess its applicability and usability. To this end, we chose three different robotic systems from independent institutions and modeled them in our tool chain, including three exemplary experiments. Subsequently, we asked twelve researchers to reproduce one of the formerly unknown systems and the associated experiment. We show that all twelve scientists were able to replicate a formerly unknown robotics experiment using our tool chain.

@inproceedings{lirolem24852,
booktitle = {2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
month = {October},
title = {Towards automated system and experiment reproduction in robotics},
year = {2016},
author = {Florian Lier and Marc Hanheide and Lorenzo Natale and Simon Schulz and Jonathan Weisz and Sven Wachsmuth and Sebastian Wrede},
abstract = {Even though research on autonomous robots and
human-robot interaction accomplished great progress in recent
years, and reusable soft- and hardware components are
available, many of the reported findings are only hardly
reproducible by fellow scientists. Usually, reproducibility is
impeded because required information, such as the specification
of software versions and their configuration, required data sets,
and experiment protocols are not mentioned or referenced
in most publications. In order to address these issues, we
recently introduced an integrated tool chain and its underlying
development process to facilitate reproducibility in robotics.
In this contribution we instantiate the complete tool chain in
a unique user study in order to assess its applicability and
usability. To this end, we chose three different robotic systems
from independent institutions and modeled them in our tool
chain, including three exemplary experiments. Subsequently,
we asked twelve researchers to reproduce one of the formerly
unknown systems and the associated experiment. We show that
all twelve scientists were able to replicate a formerly unknown
robotics experiment using our tool chain.},
keywords = {ARRAY(0x56147f7fcd58)},
url = {http://eprints.lincoln.ac.uk/24852/}
}
• D. Liu and S. Yue, “Visual pattern recognition using unsupervised spike timing dependent plasticity learning,” in 2016 International Joint Conference on Neural Networks (IJCNN), 2016, pp. 285-292.

Neuroscience study shows mammalian brain only use millisecond scale time window to process complicated real-life recognition scenarios. However, such speed cannot be achieved by traditional rate-based spiking neural network (SNN). Compared with spiking rate, the specific spiking timing (also called spiking pattern) may convey much more information. In this paper, by using modified rank order coding scheme, the generated absolute analog features have been encoded into the first spike wave with specific spatiotemporal structural information. An intuitive yet powerful feed-forward spiking neural network framework has been proposed, along with its own unsupervised spike-timing-dependent plasticity (STDP) learning rule with dynamic post-synaptic potential threshold. Compared with other state-of-art spiking algorithms, the proposed method uses biologically plausible STDP learning method to learn the selectivity while the dynamic post-synaptic potential threshold guarantees no training sample will be ignored during the learning procedure. Furthermore, unlike the complicated frameworks used in those state-of-art spiking algorithms, the proposed intuitive spiking neural network is not time-consuming and quite capable of on-line learning. A satisfactory experimental result has been achieved on classic MNIST handwritten character database.

@inproceedings{lirolem27954,
title = {Visual pattern recognition using unsupervised spike timing dependent plasticity learning},
year = {2016},
author = {Daqi Liu and Shigang Yue},
pages = {285--292},
booktitle = {2016 International Joint Conference on Neural Networks (IJCNN)},
month = {July},
abstract = {Neuroscience study shows mammalian brain only use millisecond scale time window to process complicated real-life recognition scenarios. However, such speed cannot be achieved by traditional rate-based spiking neural network (SNN). Compared with spiking rate, the specific spiking timing (also called spiking pattern) may convey much more information. In this paper, by using modified rank order coding scheme, the generated absolute analog features have been encoded into the first spike wave with specific spatiotemporal structural information. An intuitive yet powerful feed-forward spiking neural network framework has been proposed, along with its own unsupervised spike-timing-dependent plasticity (STDP) learning rule with dynamic post-synaptic potential threshold. Compared with other state-of-art spiking algorithms, the proposed method uses biologically plausible STDP learning method to learn the selectivity while the dynamic post-synaptic potential threshold guarantees no training sample will be ignored during the learning procedure. Furthermore, unlike the complicated frameworks used in those state-of-art spiking algorithms, the proposed intuitive spiking neural network is not time-consuming and quite capable of on-line learning. A satisfactory experimental result has been achieved on classic MNIST handwritten character database.},
url = {http://eprints.lincoln.ac.uk/27954/},
keywords = {ARRAY(0x56147f9e4a18)}
}
• V. Modugno, G. Neumann, E. Rueckert, G. Oriolo, J. Peters, and S. Ivaldi, “Learning soft task priorities for control of redundant robots,” in IEEE International Conference on Robotics and Automation (ICRA) 2016, 2016.

Movement primitives (MPs) provide a powerful framework for data driven movement generation that has been successfully applied for learning from demonstrations and robot reinforcement learning. In robotics we often want to solve a multitude of different, but related tasks. As the parameters of the primitives are typically high dimensional, a common practice for the generalization of movement primitives to new tasks is to adapt only a small set of control variables, also called meta parameters, of the primitive. Yet, for most MP representations, the encoding of these control variables is precoded in the representation and can not be adapted to the considered tasks. In this paper, we want to learn the encoding of task-specific control variables also from data instead of relying on fixed meta-parameter representations. We use hierarchical Bayesian models (HBMs) to estimate a low dimensional latent variable model for probabilistic movement primitives (ProMPs), which is a recent movement primitive representation. We show on two real robot datasets that ProMPs based on HBMs outperform standard ProMPs in terms of generalization and learning from a small amount of data and also allows for an intuitive analysis of the movement. We also extend our HBM by a mixture model, such that we can model different movement types in the same dataset.

@inproceedings{lirolem25639,
title = {Learning soft task priorities for control of redundant robots},
year = {2016},
author = {V. Modugno and Gerhard Neumann and E. Rueckert and G. Oriolo and J. Peters and S. Ivaldi},
booktitle = {IEEE International Conference on Robotics and Automation (ICRA) 2016},
month = {May},
keywords = {ARRAY(0x56147f9fd9b8)},
url = {http://eprints.lincoln.ac.uk/25639/},
abstract = {Movement primitives (MPs) provide a powerful
framework for data driven movement generation that has been
successfully applied for learning from demonstrations and robot
reinforcement learning. In robotics we often want to solve a
multitude of different, but related tasks. As the parameters
of the primitives are typically high dimensional, a common
practice for the generalization of movement primitives to new
tasks is to adapt only a small set of control variables, also
called meta parameters, of the primitive. Yet, for most MP
representations, the encoding of these control variables is precoded
in the representation and can not be adapted to the
considered tasks. In this paper, we want to learn the encoding of
on fixed meta-parameter representations. We use hierarchical
Bayesian models (HBMs) to estimate a low dimensional latent
variable model for probabilistic movement primitives (ProMPs),
which is a recent movement primitive representation. We show
on two real robot datasets that ProMPs based on HBMs
outperform standard ProMPs in terms of generalization and
learning from a small amount of data and also allows for an
intuitive analysis of the movement. We also extend our HBM by
a mixture model, such that we can model different movement
types in the same dataset.}
}
• T. Osa, J. Peters, and G. Neumann, “Experiments with hierarchical reinforcement learning of multiple grasping policies,” in Proceedings of the International Symposium on Experimental Robotics (ISER), 2016.

Robotic grasping has attracted considerable interest, but it still remains a challenging task. The data-driven approach is a promising solution to the robotic grasping problem; this approach leverages a grasp dataset and generalizes grasps for various objects. However, these methods often depend on the quality of the given datasets, which are not trivial to obtain with sufficient quality. Although reinforcement learning approaches have been recently used to achieve autonomous collection of grasp datasets, the existing algorithms are often limited to specific grasp types. In this paper, we present a framework for hierarchical reinforcement learning of grasping policies. In our framework, the lowerlevel hierarchy learns multiple grasp types, and the upper-level hierarchy learns a policy to select from the learned grasp types according to a point cloud of a new object. Through experiments, we validate that our approach learns grasping by constructing the grasp dataset autonomously. The experimental results show that our approach learns multiple grasping policies and generalizes the learned grasps by using local point cloud information.

@inproceedings{lirolem26735,
author = {T. Osa and J. Peters and G. Neumann},
year = {2016},
title = {Experiments with hierarchical reinforcement learning of multiple grasping policies},
month = {April},
booktitle = {Proceedings of the International Symposium on Experimental Robotics (ISER)},
abstract = {Robotic grasping has attracted considerable interest, but it
still remains a challenging task. The data-driven approach is a promising
solution to the robotic grasping problem; this approach leverages a
grasp dataset and generalizes grasps for various objects. However, these
methods often depend on the quality of the given datasets, which are not
trivial to obtain with sufficient quality. Although reinforcement learning
approaches have been recently used to achieve autonomous collection
of grasp datasets, the existing algorithms are often limited to specific
grasp types. In this paper, we present a framework for hierarchical reinforcement
learning of grasping policies. In our framework, the lowerlevel
hierarchy learns multiple grasp types, and the upper-level hierarchy
learns a policy to select from the learned grasp types according to a point
cloud of a new object. Through experiments, we validate that our approach
learns grasping by constructing the grasp dataset autonomously.
The experimental results show that our approach learns multiple grasping
policies and generalizes the learned grasps by using local point cloud
information.},
url = {http://eprints.lincoln.ac.uk/26735/},
keywords = {ARRAY(0x56147f765de0)}
}
• W. Qi, X. Li, and S. Yue, “A guided filtering and HCT integrated pansharpening method for WorldView-2 satellite images,” in 2016 IEEE International Geoscience and Remote Sensing Symposium (IGARSS), 2016, pp. 7272-7275.

Pansharpening has been an important tool in remote sensing field, which is a process of providing multispectral images with higher spatial resolution. When dealing with WorldView-2 satellite imagery having more bands and higher resolution, most existing methods are not effective. In this paper, we propose a novel and effective pansharpening methods combing guided filtering and hyperspherical color transformation (HCT) for WorldView-2 images. We use panchromatic image as the guidance to further refine the intensity of multispectral data and also to extract the sufficient details from the panchromatic image itself. Moreover, the guided filtering and HCT integrated scheme can inject the extracted details into the multispectral data and the multispectral images can be sharpened all at once with an arbitrary order. The experimental results show that our proposed method can obtain high-quality pansharpened results and outperforms some existing methods.

@inproceedings{lirolem27953,
title = {A guided filtering and HCT integrated pansharpening method for WorldView-2 satellite images},
year = {2016},
author = {Weifeng Qi and Xu Li and Shigang Yue},
month = {July},
pages = {7272--7275},
booktitle = {2016 IEEE International Geoscience and Remote Sensing Symposium (IGARSS)},
url = {http://eprints.lincoln.ac.uk/27953/},
keywords = {ARRAY(0x56147f9c1850)},
abstract = {Pansharpening has been an important tool in remote sensing field, which is a process of providing multispectral images with higher spatial resolution. When dealing with WorldView-2 satellite imagery having more bands and higher resolution, most existing methods are not effective. In this paper, we propose a novel and effective pansharpening methods combing guided filtering and hyperspherical color transformation (HCT) for WorldView-2 images. We use panchromatic image as the guidance to further refine the intensity of multispectral data and also to extract the sufficient details from the panchromatic image itself. Moreover, the guided filtering and HCT integrated scheme can inject the extracted details into the multispectral data and the multispectral images can be sharpened all at once with an arbitrary order. The experimental results show that our proposed method can obtain high-quality pansharpened results and outperforms some existing methods.}
}
• F. Riccio, R. Capobianco, M. Hanheide, and D. Nardi, “Stam: a framework for spatio-temporal affordance maps,” in International Workshop on Modelling and Simulation for Autonomous Systems, 2016, pp. 271-280.

A?ordances have been introduced in literature as action op- portunities that objects o?er, and used in robotics to semantically rep- resent their interconnection. However, when considering an environment instead of an object, the problem becomes more complex due to the dynamism of its state. To tackle this issue, we introduce the concept of Spatio-Temporal A?ordances (STA) and Spatio-Temporal A?ordance Map (STAM). Using this formalism, we encode action semantics re- lated to the environment to improve task execution capabilities of an autonomous robot. We experimentally validate our approach to support the execution of robot tasks by showing that a?ordances encode accurate semantics of the environment.

@inproceedings{lirolem24851,
author = {Francesco Riccio and Roberto Capobianco and Marc Hanheide and Daniele Nardi},
publisher = {Springer},
year = {2016},
title = {Stam: a framework for spatio-temporal affordance maps},
booktitle = {International Workshop on Modelling and Simulation for Autonomous Systems},
pages = {271--280},
month = {June},
abstract = {A?ordances have been introduced in literature as action op-
portunities that objects o?er, and used in robotics to semantically rep-
resent their interconnection. However, when considering an environment
instead of an object, the problem becomes more complex due to the
dynamism of its state. To tackle this issue, we introduce the concept
of Spatio-Temporal A?ordances (STA) and Spatio-Temporal A?ordance
Map (STAM). Using this formalism, we encode action semantics re-
lated to the environment to improve task execution capabilities of an
autonomous robot. We experimentally validate our approach to support
the execution of robot tasks by showing that a?ordances encode accurate
semantics of the environment.},
keywords = {ARRAY(0x56147f79eeb0)},
url = {http://eprints.lincoln.ac.uk/24851/}
}
• C. Salatino, V. Gower, M. Ghrissi, A. Tapus, K. Wieczorowska-Tobis, A. Suwalska, P. Barattini, R. Rosso, G. Munaro, N. Bellotto, and H. van den Heuvel, “EnrichMe: a robotic solution for independence and active aging of elderly people with MCI,” in 15th International Conference on Computers Helping People with Special Needs (ICCHP 2016), 2016.

Mild cognitive impairment (MCI) is a state related to ageing, and sometimes evolves to dementia. As there is no pharmacological treatment for MCI, a non-pharmacological approach is very important. The use of Information and Communication Technologies (ICT) in care and assistance services for elderly people increases their chances of prolonging independence thanks to better cognitive efficiency. Robots are seen to have the potential to support the care and independence of elderly people. The project ENRICHME (funded by the EU H2020 Programme) focuses on developing and testing technologies for supporting elderly people with MCI in their living environment for a long time. This paper describes the results of the activities conducted during the first year of the ENRICHME project, in particular the definition of user needs and requirements and the resulting system architecture.

@inproceedings{lirolem22704,
month = {July},
booktitle = {15th International Conference on Computers Helping People with Special Needs (ICCHP 2016)},
year = {2016},
title = {EnrichMe: a robotic solution for independence and active aging of elderly people with MCI},
author = {Claudia Salatino and Valerio Gower and Meftah Ghrissi and Adriana Tapus and K Wieczorowska-Tobis and A Suwalska and Paolo Barattini and Roberto Rosso and Giulia Munaro and Nicola Bellotto and Herjan van den Heuvel},
abstract = {Mild cognitive impairment (MCI) is a state related to ageing, and sometimes evolves to dementia. As there is no pharmacological treatment for MCI, a non-pharmacological approach is very important. The use of Information and Communication Technologies (ICT) in care and assistance services for elderly people increases their chances of prolonging independence thanks to better cognitive efficiency. Robots are seen to have the potential to support the care and independence of elderly people. The project ENRICHME (funded by the EU H2020 Programme) focuses on developing and testing technologies for supporting elderly people with MCI in their living environment for a long time. This paper describes the results of the activities conducted during the first year of the ENRICHME project, in particular the definition of user needs and requirements and the resulting system architecture.},
keywords = {ARRAY(0x56147f9e4e20)},
url = {http://eprints.lincoln.ac.uk/22704/}
}
• J. Santos, T. Krajnik, J. P. Fentanes, and T. Duckett, “A 3D simulation environment with real dynamics: a tool for benchmarking mobile robot performance in long-term deployments,” in ICRA 2016 Workshop: AI for Long-term Autonomy, 2016.

This paper describes a method to compare and evaluate mobile robot algorithms for long-term deployment in changing environments. Typically, the long-term performance of state estimation algorithms for mobile robots is evaluated using pre-recorded sensory datasets. However such datasets are not suitable for evaluating decision-making and control algorithms where the behaviour of the robot will be different in every trial. Simulation allows to overcome this issue and while it ensures repeatability of experiments, the development of 3D simulations for an extended period of time is a costly exercise. In our approach long-term datasets comprising high-level tracks of dynamic entities such as people and furniture are recorded by ambient sensors placed in a real environment. The high-level tracks are then used to parameterise a 3D simulation containing its own geometric models of the dynamic entities and the background scene. This simulation, which is based on actual human activities, can then be used to benchmark and validate algorithms for long-term operation of mobile robots.

@inproceedings{lirolem23220,
month = {May},
booktitle = {ICRA 2016 Workshop: AI for Long-term Autonomy},
title = {A 3D simulation environment with real dynamics: a tool for benchmarking mobile robot performance in long-term deployments},
year = {2016},
author = {Joao Santos and Tomas Krajnik and Jaime Pulido Fentanes and Tom Duckett},
url = {http://eprints.lincoln.ac.uk/23220/},
keywords = {ARRAY(0x56147f5a7710)},
abstract = {This paper describes a method to compare and evaluate mobile robot algorithms for long-term deployment in changing   environments. Typically, the long-term performance of state estimation algorithms for mobile robots is evaluated using pre-recorded sensory datasets. However such datasets are not suitable for evaluating decision-making and control  algorithms where the behaviour of the robot will be different in every trial. Simulation allows to overcome this issue and while it ensures repeatability of experiments, the development of 3D simulations for an extended period of time is a costly exercise.
In our approach long-term datasets comprising high-level tracks of dynamic entities such as people and furniture are recorded by ambient sensors placed in a real environment. The high-level tracks are then used to parameterise a 3D  simulation containing its own geometric models of the dynamic entities and the background scene. This simulation,  which is based on actual human activities, can then be used to benchmark and validate algorithms for long-term  operation of mobile robots.}
}
• J. M. Santos, T. Krajnik, J. P. Fentanes, and T. Duckett, “Lifelong information-driven exploration to complete and refine 4-D spatio-temporal maps,” IEEE Robotics and Automation Letters, vol. 1, iss. 2, pp. 684-691, 2016.

This paper presents an exploration method that allows mobile robots to build and maintain spatio-temporal models of changing environments. The assumption of a perpetuallychanging world adds a temporal dimension to the exploration problem, making spatio-temporal exploration a never-ending, life-long learning process. We address the problem by application of information-theoretic exploration methods to spatio-temporal models that represent the uncertainty of environment states as probabilistic functions of time. This allows to predict the potential information gain to be obtained by observing a particular area at a given time, and consequently, to decide which locations to visit and the best times to go there. To validate the approach, a mobile robot was deployed continuously over 5 consecutive business days in a busy office environment. The results indicate that the robot?s ability to spot environmental changes im

@article{lirolem22698,
author = {Joao Machado Santos and Tomas Krajnik and Jaime Pulido Fentanes and Tom Duckett},
publisher = {IEEE},
month = {July},
pages = {684--691},
year = {2016},
title = {Lifelong information-driven exploration to complete and refine 4-D spatio-temporal maps},
volume = {1},
journal = {IEEE Robotics and Automation Letters},
number = {2},
abstract = {This paper presents an exploration method that allows
mobile robots to build and maintain spatio-temporal models
of changing environments. The assumption of a perpetuallychanging
world adds a temporal dimension to the exploration
problem, making spatio-temporal exploration a never-ending,
life-long learning process. We address the problem by application
of information-theoretic exploration methods to spatio-temporal
models that represent the uncertainty of environment states as
probabilistic functions of time. This allows to predict the potential
information gain to be obtained by observing a particular area
at a given time, and consequently, to decide which locations to
visit and the best times to go there.
To validate the approach, a mobile robot was deployed
continuously over 5 consecutive business days in a busy office
environment. The results indicate that the robot?s ability to spot
environmental changes im},
keywords = {ARRAY(0x56147fa23a68)},
url = {http://eprints.lincoln.ac.uk/22698/}
}
• E. Senft, S. Lemaignan, P. E. Baxter, T. Belpaeme, and and, “SPARC: an efficient way to combine reinforcement learning and supervised autonomy,” in Future of Interactive Learning Machines Workshop at NIPS’16, Los Angeles, USA, 2016.

Shortcomings of reinforcement learning for robot control include the sparsity of the environmental reward function, the high number of trials required before reaching an efficient action policy and the reliance on exploration to gather information about the environment, potentially resulting in undesired actions. These limits can be overcome by adding a human in the loop to provide additional information during the learning phase. In this paper, we propose a novel way to combine human inputs and reinforcement by following the Supervised Progressively Autonomous Robot Competencies (SPARC) approach. We compare this method to the principles of Interactive Reinforcement Learning as proposed by Thomaz and Breazeal. Results from a study involving 40 participants show that using SPARC increases the performance of the learning, reduces the time and number of inputs required for teaching and faces fewer errors during the learning process. These results support the use of SPARC as an efficient method to teach a robot to interact with humans.

@inproceedings{lirolem30194,
booktitle = {Future of Interactive Learning Machines Workshop at NIPS'16},
month = {December},
title = {SPARC: an efficient way to combine reinforcement learning and supervised autonomy},
year = {2016},
author = {Emmanuel Senft and Severin Lemaignan and Paul E. Baxter and Tony Belpaeme and   and  },
publisher = {?},
abstract = {Shortcomings of reinforcement learning for robot control include the sparsity of the environmental reward function, the high number of trials required before reaching an efficient action policy and the reliance on exploration to gather information about the environment, potentially resulting in undesired actions. These limits can be overcome by adding a human in the loop to provide additional information during the learning phase. In this paper, we propose a novel way to combine human inputs and reinforcement by following the Supervised Progressively Autonomous Robot Competencies (SPARC) approach. We compare this method to the principles of Interactive Reinforcement Learning as proposed by Thomaz and Breazeal. Results from a study involving 40 participants show that using SPARC increases the performance of the learning, reduces the time and number of inputs required for teaching and faces fewer errors during the learning process. These results support the use of SPARC as an efficient method to teach a robot to interact with humans.},
keywords = {ARRAY(0x56147f9fcf50)},
url = {http://eprints.lincoln.ac.uk/30194/}
}
• H. Wang, J. Peng, and S. Yue, “Bio-inspired small target motion detector with a new lateral inhibition mechanism,” in 2016 International Joint Conference on Neural Networks (IJCNN), 2016, pp. 4751-4758.

In nature, it is an important task for animals to detect small targets which move within cluttered background. In recent years, biologists have found that a class of neurons in the lobula complex, called STMDs (small target motion detectors) which have extreme selectivity for small targets moving within visual clutter. At the same time, some researchers assert that lateral inhibition plays an important role in discriminating the motion of the target from the motion of the background, even account for many features of the tuning of higher order visual neurons. Inspired by the finding that complete lateral inhibition can only be seen when the motion of the central region is identical to the motion of the peripheral region, we propose a new lateral inhibition mechanism combined with motion velocity and direction to improve the performance of ESTMD model (elementary small target motion detector). In this paper, we will elaborate on the biological plausibility and functionality of this new lateral inhibition mechanism in small target motion detection.

@inproceedings{lirolem27956,
pages = {4751--4758},
booktitle = {2016 International Joint Conference on Neural Networks (IJCNN)},
month = {July},
year = {2016},
title = {Bio-inspired small target motion detector with a new lateral inhibition mechanism},
author = {Hongxin Wang and Jigen Peng and Shigang Yue},
abstract = {In nature, it is an important task for animals to detect small targets which move within cluttered background. In recent years, biologists have found that a class of neurons in the lobula complex, called STMDs (small target motion detectors) which have extreme selectivity for small targets moving within visual clutter. At the same time, some researchers assert that lateral inhibition plays an important role in discriminating the motion of the target from the motion of the background, even account for many features of the tuning of higher order visual neurons. Inspired by the finding that complete lateral inhibition can only be seen when the motion of the central region is identical to the motion of the peripheral region, we propose a new lateral inhibition mechanism combined with motion velocity and direction to improve the performance of ESTMD model (elementary small target motion detector). In this paper, we will elaborate on the biological plausibility and functionality of this new lateral inhibition mechanism in small target motion detection.},
keywords = {ARRAY(0x56147f6f4258)},
url = {http://eprints.lincoln.ac.uk/27956/}
}
• C. Wirth, J. Furnkranz, and G. Neumann, “Model-free preference-based reinforcement learning,” in Thirtieth AAAI Conference on Artificial Intelligence, 2016, pp. 2222-2228.

Specifying a numeric reward function for reinforcement learning typically requires a lot of hand-tuning from a human expert. In contrast, preference-based reinforcement learning (PBRL) utilizes only pairwise comparisons between trajectories as a feedback signal, which are often more intuitive to specify. Currently available approaches to PBRL for control problems with continuous state/action spaces require a known or estimated model, which is often not available and hard to learn. In this paper, we integrate preference-based estimation of the reward function into a model-free reinforcement learning (RL) algorithm, resulting in a model-free PBRL algorithm. Our new algorithm is based on Relative Entropy Policy Search (REPS), enabling us to utilize stochastic policies and to directly control the greediness of the policy update. REPS decreases exploration of the policy slowly by limiting the relative entropy of the policy update, which ensures that the algorithm is provided with a versatile set of trajectories, and consequently with informative preferences. The preference-based estimation is computed using a sample-based Bayesian method, which can also estimate the uncertainty of the utility. Additionally, we also compare to a linear solvable approximation, based on inverse RL. We show that both approaches perform favourably to the current state-of-the-art. The overall result is an algorithm that can learn non-parametric continuous action policies from a small number of preferences.

@inproceedings{lirolem25746,
month = {February},
booktitle = {Thirtieth AAAI Conference on Artificial Intelligence},
pages = {2222--2228},
author = {C. Wirth and J. Furnkranz and G. Neumann},
year = {2016},
journal = {30th AAAI Conference on Artificial Intelligence, AAAI 2016},
title = {Model-free preference-based reinforcement learning},
abstract = {Specifying a numeric reward function for reinforcement learning typically requires a lot of hand-tuning from a human expert. In contrast, preference-based reinforcement learning (PBRL) utilizes only pairwise comparisons between trajectories as a feedback signal, which are often more intuitive to specify. Currently available approaches to PBRL for control problems with continuous state/action spaces require a known or estimated model, which is often not available and hard to learn. In this paper, we integrate preference-based estimation of the reward function into a model-free reinforcement learning (RL) algorithm, resulting in a model-free PBRL algorithm. Our new algorithm is based on Relative Entropy Policy Search (REPS), enabling us to utilize stochastic policies and to directly control the greediness of the policy update. REPS decreases exploration of the policy slowly by limiting the relative entropy of the policy update, which ensures that the algorithm is provided with a versatile set of trajectories, and consequently with informative preferences. The preference-based estimation is computed using a sample-based Bayesian method, which can also estimate the uncertainty of the utility. Additionally, we also compare to a linear solvable approximation, based on inverse RL. We show that both approaches perform favourably to the current state-of-the-art. The overall result is an algorithm that can learn non-parametric continuous action policies from a small number of preferences.},
keywords = {ARRAY(0x56147f8cc7b0)},
url = {http://eprints.lincoln.ac.uk/25746/}
}
• C. Xiong, W. Chen, B. Sun, M. Liu, S. Yue, and W. Chen, “Design and implementation of an anthropomorphic hand for replicating human grasping functions,” IEEE Transactions on Robotics, vol. 32, iss. 3, pp. 652-671, 2016.

How to design an anthropomorphic hand with a few actuators to replicate the grasping functions of the human hand is still a challenging problem. This paper aims to develop a general theory for designing the anthropomorphic hand and endowing the designed hand with natural grasping functions. A grasping experimental paradigm was set up for analyzing the grasping mechanism of the human hand in daily living. The movement relationship among joints in a digit, among digits in the human hand, and the postural synergic characteristic of the fingers were studied during the grasping. The design principle of the anthropomorphic mechanical digit that can reproduce the digit grasping movement of the human hand was developed. The design theory of the kinematic transmission mechanism that can be embedded into the palm of the anthropomorphic hand to reproduce the postural synergic characteristic of the fingers by using a limited number of actuators is proposed. The design method of the anthropomorphic hand for replicating human grasping functions was formulated. Grasping experiments are given to verify the effectiveness of the proposed design method of the anthropomorphic hand. Â\copyright 2016 IEEE.

@article{lirolem23735,
publisher = {Institute of Electrical and Electronics Engineers Inc.},
author = {Cai-Hua Xiong and Wen-Rui Chen and Bai-Yang Sun and Ming-Jin Liu and Shigang Yue and Wen-Bin Chen},
month = {June},
pages = {652--671},
year = {2016},
journal = {IEEE Transactions on Robotics},
title = {Design and implementation of an anthropomorphic hand for replicating human grasping functions},
volume = {32},
number = {3},
abstract = {How to design an anthropomorphic hand with a few actuators to replicate the grasping functions of the human hand is still a challenging problem. This paper aims to develop a general theory for designing the anthropomorphic hand and endowing the designed hand with natural grasping functions. A grasping experimental paradigm was set up for analyzing the grasping mechanism of the human hand in daily living. The movement relationship among joints in a digit, among digits in the human hand, and the postural synergic characteristic of the fingers were studied during the grasping. The design principle of the anthropomorphic mechanical digit that can reproduce the digit grasping movement of the human hand was developed. The design theory of the kinematic transmission mechanism that can be embedded into the palm of the anthropomorphic hand to reproduce the postural synergic characteristic of the fingers by using a limited number of actuators is proposed. The design method of the anthropomorphic hand for replicating human grasping functions was formulated. Grasping experiments are given to verify the effectiveness of the proposed design method of the anthropomorphic hand. {\^A}{\copyright} 2016 IEEE.},
keywords = {ARRAY(0x56147f9b8c88)},
url = {http://eprints.lincoln.ac.uk/23735/}
}
• Z. Xuqiang, L. Fule, W. Zhijun, L. Weitao, J. Wen, W. Zhihua, and S. Yue, “An S/H circuit with parasitics optimized for IF-sampling,” Journal of Semiconductors, vol. 37, iss. 6, p. 65005, 2016.

An IF-sampling S/H is presented, which adopts a flip-around structure, bottom-plate sampling technique and improved input bootstrapped switches. To achieve high sampling linearity over a wide input frequency range, the floating well technique is utilized to optimize the input switches. Besides, techniques of transistor load linearization and layout improvement are proposed to further reduce and linearize the parasitic capacitance. The S/H circuit has been fabricated in 0.18-\ensuremath\mum CMOS process as the front-end of a 14 bit, 250 MS/s pipeline ADC. For 30 MHz input, the measured SFDR/SNDR of the ADC is 94.7 dB/68. 5dB, which can remain over 84.3 dB/65.4 dB for input frequency up to 400 MHz. The ADC presents excellent dynamic performance at high input frequency, which is mainly attributed to the parasitics optimized S/H circuit.

@article{lirolem27940,
year = {2016},
volume = {37},
journal = {Journal of Semiconductors},
title = {An S/H circuit with parasitics optimized for IF-sampling},
number = {6},
author = {Zheng Xuqiang and Li Fule and Wang Zhijun and Li Weitao and Jia Wen and Wang Zhihua and Shigang Yue},
publisher = {IOP Publishing / Chinese Institute of Electronics},
month = {June},
pages = {065005},
keywords = {ARRAY(0x56147f6c5b08)},
url = {http://eprints.lincoln.ac.uk/27940/},
abstract = {An IF-sampling S/H is presented, which adopts a flip-around structure, bottom-plate sampling technique and improved input bootstrapped switches. To achieve high sampling linearity over a wide input frequency range, the floating well technique is utilized to optimize the input switches. Besides, techniques of transistor load linearization and layout improvement are proposed to further reduce and linearize the parasitic capacitance. The S/H circuit has been fabricated in 0.18-{\ensuremath{\mu}}m CMOS process as the front-end of a 14 bit, 250 MS/s pipeline ADC. For 30 MHz input, the measured SFDR/SNDR of the ADC is 94.7 dB/68. 5dB, which can remain over 84.3 dB/65.4 dB for input frequency up to 400 MHz. The ADC presents excellent dynamic performance at high input frequency, which is mainly attributed to the parasitics optimized S/H circuit.}
}
• Y. Yang, A. Ahmed, S. Yue, X. Xie, H. Chen, and Z. Wang, “An algorithm for accurate needle orientation,” in 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), 2016, pp. 5095-5098.

For the early diagnosis and treatment, a needle insertion for biopsy and treatment is a common and important means. To solve the low accuracy and high probability of repeat surgery in traditional surgical procedures, a computer-assisted system is an effective solution. In such a system, how to acquire the accurate orientation of the surgical needle is one of the most important factors. This paper proposes a ?Center Point Method? for needle axis extraction with high accuracy. The method makes full use of edge points from two sides of the needle in image and creates center points through which an accurate axis is extracted. Experiments show that the proposed method improves needle orientation accuracy by approximately 70\% compared to related work in binocular stereovision system.

@inproceedings{lirolem27949,
year = {2016},
title = {An algorithm for accurate needle orientation},
author = {Yifan Yang and Amr Ahmed and Shigang Yue and Xiang Xie and Hong Chen and Zhihua Wang},
month = {August},
pages = {5095--5098},
booktitle = {2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC)},
url = {http://eprints.lincoln.ac.uk/27949/},
keywords = {ARRAY(0x56147f8af868)},
abstract = {For the early diagnosis and treatment, a needle insertion for biopsy and treatment is a common and important means. To solve the low accuracy and high probability of repeat surgery in traditional surgical procedures, a computer-assisted system is an effective solution. In such a system, how to acquire the accurate orientation of the surgical needle is one of the most important factors. This paper proposes a ?Center Point Method? for needle axis extraction with high accuracy. The method makes full use of edge points from two sides of the needle in image and creates center points through which an accurate axis is extracted. Experiments show that the proposed method improves needle orientation accuracy by approximately 70\% compared to related work in binocular stereovision system.}
}
• G. Zhang, C. Zhang, and S. Yue, “LGMD and DSNs neural networks integration for collision predication,” in 2016 International Joint Conference on Neural Networks (IJCNN), 2016, pp. 1174-1179.

An ability to predict collisions is essential for current vehicles and autonomous robots. In this paper, an integrated collision predication system is proposed based on neural subsystems inspired from Lobula giant movement detector (LGMD) and directional selective neurons (DSNs) which focus on different part of the visual field separately. The two type of neurons found in the visual pathways of insects respond most strongly to moving objects with preferred motion patterns, i.e., the LGMD prefers looming stimuli and DSNs prefer specific lateral movements. We fuse the extracted information by each type of neurons to make final decision. By dividing the whole field of view into four regions for each subsystem to process, the proposed approaches can detect hazardous situations that had been difficult for single subsystem only. Our experiments show that the integrated system works in most of the hazardous scenarios.

@inproceedings{lirolem27955,
author = {Guopeng Zhang and Chun Zhang and Shigang Yue},
publisher = {IEEE},
year = {2016},
title = {LGMD and DSNs neural networks integration for collision predication},
month = {July},
booktitle = {2016 International Joint Conference on Neural Networks (IJCNN)},
pages = {1174--1179},
url = {http://eprints.lincoln.ac.uk/27955/},
keywords = {ARRAY(0x56147f936e78)},
abstract = {An ability to predict collisions is essential for current vehicles and autonomous robots. In this paper, an integrated collision predication system is proposed based on neural subsystems inspired from Lobula giant movement detector (LGMD) and directional selective neurons (DSNs) which focus on different part of the visual field separately. The two type of neurons found in the visual pathways of insects respond most strongly to moving objects with preferred motion patterns, i.e., the LGMD prefers looming stimuli and DSNs prefer specific lateral movements. We fuse the extracted information by each type of neurons to make final decision. By dividing the whole field of view into four regions for each subsystem to process, the proposed approaches can detect hazardous situations that had been difficult for single subsystem only. Our experiments show that the integrated system works in most of the hazardous scenarios.}
}
• X. Zheng, Z. Wang, F. Li, F. Zhao, S. Yue, C. Zhang, and Z. Wang, “A 14-bit 250 MS/s IF sampling pipelined ADC in 180 nm CMOS process,” IEEE Transactions on Circuits and Systems I: Regular Papers, vol. 63, iss. 9, pp. 1381-1392, 2016.

This paper presents a 14-bit 250 MS/s ADC fabricated in a 180 nm CMOS process, which aims at optimizing its linearity, operating speed, and power efficiency. The implemented ADC employs an improved SHA with parasitic optimized bootstrapped switches to achieve high sampling linearity over a wide input frequency range. It also explores a dedicated foreground calibration to correct the capacitor mismatches and the gain error of residue amplifier, where a novel configuration scheme with little cost for analog front-end is developed. Moreover, a partial non-overlapping clock scheme associated with a highspeed reference buffer and fast comparators is proposed to maximize the residue settling time. The implemented ADC is measured under different input frequencies with a sampling rate of 250 MS/s and it consumes 300 mW from a 1.8 V supply. For 30 MHz input, the measured SFDR and SNDR of the ADC is 94.7 dB and 68.5 dB, which can remain over 84.3 dB and 65.4 dB for up to 400 MHz. The measured DNL and INL after calibration are optimized to 0.15 LSB and 1.00 LSB, respectively, while the Walden FOM at Nyquist frequency is 0.57 pJ/step.

@article{lirolem25371,
publisher = {IEEE},
author = {Xuqiang Zheng and Zhijun Wang and Fule Li and Feng Zhao and Shigang Yue and Chun Zhang and Zhihua Wang},
month = {September},
pages = {1381--1392},
year = {2016},
title = {A 14-bit 250 MS/s IF sampling pipelined ADC in 180 nm CMOS process},
journal = {IEEE Transactions on Circuits and Systems I: Regular Papers},
volume = {63},
number = {9},
keywords = {ARRAY(0x56147ce9e1e8)},
url = {http://eprints.lincoln.ac.uk/25371/},
abstract = {This paper presents a 14-bit 250 MS/s ADC fabricated
in a 180 nm CMOS process, which aims at optimizing its
linearity, operating speed, and power efficiency. The implemented
ADC employs an improved SHA with parasitic optimized bootstrapped
switches to achieve high sampling linearity over a wide
input frequency range. It also explores a dedicated foreground
calibration to correct the capacitor mismatches and the gain
error of residue amplifier, where a novel configuration scheme
with little cost for analog front-end is developed. Moreover, a
partial non-overlapping clock scheme associated with a highspeed
reference buffer and fast comparators is proposed to
maximize the residue settling time. The implemented ADC is
measured under different input frequencies with a sampling rate
of 250 MS/s and it consumes 300 mW from a 1.8 V supply. For 30
MHz input, the measured SFDR and SNDR of the ADC is 94.7
dB and 68.5 dB, which can remain over 84.3 dB and 65.4 dB for
up to 400 MHz. The measured DNL and INL after calibration
are optimized to 0.15 LSB and 1.00 LSB, respectively, while the
Walden FOM at Nyquist frequency is 0.57 pJ/step.}
}
• X. Zheng, C. Zhang, F. Lv, F. Zhao, S. Yue, Z. Wang, F. Li, and Z. Wang, “A 5-50 Gb/s quarter rate transmitter with a 4-tap multiple-MUX based FFE in 65 nm CMOS,” in ESSCIRC Conference 2016: 42nd European Solid-State Circuits Conference, 2016, pp. 305-308.

This paper presents a 5-50 Gb/s quarter-rate transmitter with a 4-tap feed-forward equalization (FFE) based on multiple-multiplexer (MUX). A bandwidth enhanced 4:1 MUX with the capability of eliminating charge-sharing effect is proposed to increase the maximum operating speed. To produce the quarter-rate parallel data streams with appropriate delays, a compact latch array associated with an interleaved-retiming technique is designed. Implemented in 65 nm CMOS technology, the transmitter occupying an area of 0.6 mm2 achieves a maximum data rate of 50 Gb/s with an energy efficiency of 3.1 pJ/bit.

@inproceedings{lirolem27950,
title = {A 5-50 Gb/s quarter rate transmitter with a 4-tap multiple-MUX based FFE in 65 nm CMOS},
year = {2016},
author = {Xuqiang Zheng and Chun Zhang and Fangxu Lv and Feng Zhao and Shigang Yue and Ziqiang Wang and Fule Li and Zhihua Wang},
month = {September},
pages = {305--308},
booktitle = {ESSCIRC Conference 2016: 42nd European Solid-State Circuits Conference},
keywords = {ARRAY(0x56147f9e8b48)},
url = {http://eprints.lincoln.ac.uk/27950/},
abstract = {This paper presents a 5-50 Gb/s quarter-rate transmitter with a 4-tap feed-forward equalization (FFE) based on multiple-multiplexer (MUX). A bandwidth enhanced 4:1 MUX with the capability of eliminating charge-sharing effect is proposed to increase the maximum operating speed. To produce the quarter-rate parallel data streams with appropriate delays, a compact latch array associated with an interleaved-retiming technique is designed. Implemented in 65 nm CMOS technology, the transmitter occupying an area of 0.6 mm2 achieves a maximum data rate of 50 Gb/s with an energy efficiency of 3.1 pJ/bit.}
}
• D. S. caj, A. V. cko, M. M. c, M. J. cek, G. M. Kruijff, M. Hanheide, N. Hawes, J. L. Wyatt, T. Keller, K. Zhou, M. Zillich, and M. Kristan, “An integrated system for interactive continuous learning of categorical knowledge,” Journal of Experimental & Theoretical Artificial Intelligence, vol. 28, iss. 5, pp. 823-848, 2016.

This article presents an integrated robot system capable of interactive learning in dialogue with a human. Such a system needs to have several competencies and must be able to process different types of representations. In this article, we describe a collection of mechanisms that enable integration of heterogeneous competencies in a principled way. Central to our design is the creation of beliefs from visual and linguistic information, and the use of these beliefs for planning system behaviour to satisfy internal drives. The system is able to detect gaps in its knowledge and to plan and execute actions that provide information needed to fill these gaps. We propose a hierarchy of mechanisms which are capable of engaging in different kinds of learning interactions, e.g. those initiated by a tutor or by the system itself. We present the theory these mechanisms are build upon and an instantiation of this theory in the form of an integrated robot system. We demonstrate the operation of the system in the case of learning conceptual models of objects and their visual properties.

@article{lirolem22203,
number = {5},
year = {2016},
journal = {Journal of Experimental \& Theoretical Artificial Intelligence},
title = {An integrated system for interactive continuous learning of categorical knowledge},
volume = {28},
author = {Danijel Sko{\v c}aj and Alen Vre{\v c}ko and Marko Mahni{\v c} and Miroslav Jan{\'i}{\v c}ek and Geert-Jan M Kruijff and Marc Hanheide and Nick Hawes and Jeremy L Wyatt and Thomas Keller and Kai Zhou and Michael Zillich and Matej Kristan},
publisher = {Taylor \& Francis: STM, Behavioural Science and Public Health Titles},
pages = {823--848},
month = {August},
url = {http://eprints.lincoln.ac.uk/22203/},
keywords = {ARRAY(0x56147f7c0f18)},
abstract = {This article presents an integrated robot system capable of interactive learning in dialogue with a human. Such a system needs to have several competencies and must be able to process different types of representations. In this article, we describe a collection of mechanisms that enable integration of heterogeneous competencies in a principled way. Central to our design is the creation of beliefs from visual and linguistic information, and the use of these beliefs for planning system behaviour to satisfy internal drives. The system is able to detect gaps in its knowledge and to plan and execute actions that provide information needed to fill these gaps. We propose a hierarchy of mechanisms which are capable of engaging in different kinds of learning interactions, e.g. those initiated by a tutor or by the system itself. We present the theory these mechanisms are build upon and an instantiation of this theory in the form of an integrated robot system. We demonstrate the operation of the system in the case of learning conceptual models of objects and their visual properties.}
}

### 2015

• A. Abdolmaleki, N. Lau, L. P. Reis, J. Peters, and G. Neumann, “Contextual policy search for generalizing a parameterized biped walking controller,” in IEEE International Conference on Autonomous Robot Systems and Competitions (ICARSC), 2015, pp. 17-22.

We investigate learning of flexible Robot locomotion controller, i.e., the controllers should be applicable for multiple contexts, for example different walking speeds, various slopes of the terrain or other physical properties of the robot. In our experiments, contexts are desired walking linear speed and the direction of the gait. Current approaches for learning control parameters of biped locomotion controllers are typically only applicable for a single context. They can be used for a particular context, for example to learn a gait with highest speed, lowest energy consumption or a combination of both. The question of our research is, how can we obtain a flexible walking controller that controls the robot (near) optimally for many different contexts? We achieve the desired flexibility of the controller by applying the recently developed contextual relative entropy policy search(REPS) method. With such a contextual policy search algorithm, we can generalize the robot walking controller for different contexts, where a context is described by a real valued vector. In this paper we also extend the contextual REPS algorithm to learn a non-linear policy instead of a linear one over the contexts. In order to validate our method, we perform a simulation experiment using a simulated NAO humanoid robot. The robot now learns a policy to choose the controller parameters for a continuous set of walking speeds and directions.

@inproceedings{lirolem25698,
month = {April},
pages = {17--22},
booktitle = {IEEE International Conference on  Autonomous Robot Systems and Competitions (ICARSC)},
title = {Contextual policy search for generalizing a parameterized biped walking controller},
year = {2015},
author = {A. Abdolmaleki and N. Lau and L. P. Reis and J. Peters and G. Neumann},
publisher = {IEEE},
url = {http://eprints.lincoln.ac.uk/25698/},
keywords = {ARRAY(0x56147f71f7e0)},
abstract = {We investigate learning of flexible Robot locomotion controller, i.e., the controllers should be applicable for multiple contexts, for example different walking speeds, various slopes of the terrain or other physical properties of the robot. In our experiments, contexts are desired walking linear speed and the direction of the gait. Current approaches for learning control parameters of biped locomotion controllers are typically only applicable for a single context. They can be used for a particular context, for example to learn a gait with highest speed, lowest energy consumption or a combination of both. The question of our research is, how can we obtain a flexible walking controller that controls the robot (near) optimally for many different contexts? We achieve the desired flexibility of the controller by applying the recently developed contextual relative entropy policy search(REPS) method. With such a contextual policy search algorithm, we can generalize the robot walking controller for different contexts, where a context is described by a real valued vector. In this paper we also extend the contextual REPS algorithm to learn a non-linear policy instead of a linear one over the contexts. In order to validate our method, we perform a simulation experiment using a simulated NAO humanoid robot. The robot now learns a policy to choose the controller parameters for a continuous set of walking speeds and directions.}
}
• A. Abdolmaleki, N. Lau, L. P. Reis, and G. Neumann, “Regularized covariance estimation for weighted maximum likelihood policy search methods,” in Humanoid Robots (Humanoids), 2015 IEEE-RAS 15th International Conference on, 2015, pp. 154-159.

Many episode-based (or direct) policy search algorithms, maintain a multivariate Gaussian distribution as search distribution over the parameter space of some objective function. One class of algorithms, such as episodic REPS, PoWER or PI2 uses, a weighted maximum likelihood estimate (WMLE) to update the mean and covariance matrix of this distribution in each iteration. However, due to high dimensionality of covariance matrices and limited number of samples, the WMLE is an unreliable estimator. The use of WMLE leads to over-fitted covariance estimates, and, hence the variance/entropy of the search distribution decreases too quickly, which may cause premature convergence. In order to alleviate this problem, the estimated covariance matrix can be regularized in different ways, for example by using a convex combination of the diagonal covariance estimate and the sample covariance estimate. In this paper, we propose a new covariance matrix regularization technique for policy search methods that uses the convex combination of the sample covariance matrix and the old covariance matrix used in last iteration. The combination weighting is determined by specifying the desired entropy of the new search distribution. With this mechanism, the entropy of the search distribution can be gradually decreased without damage from the maximum likelihood estimate.

@inproceedings{lirolem25748,
title = {Regularized covariance estimation for weighted maximum likelihood policy search methods},
journal = {IEEE-RAS International Conference on Humanoid Robots},
volume = {2015-D},
year = {2015},
author = {A. Abdolmaleki and N. Lau and L. P. Reis and G. Neumann},
month = {November},
booktitle = {Humanoid Robots (Humanoids), 2015 IEEE-RAS 15th International Conference on},
pages = {154--159},
abstract = {Many episode-based (or direct) policy search algorithms, maintain a multivariate Gaussian distribution as search distribution over the parameter space of some objective function. One class of algorithms, such as episodic REPS, PoWER or PI2 uses, a weighted maximum likelihood estimate (WMLE) to update the mean and covariance matrix of this distribution in each iteration. However, due to high dimensionality of covariance matrices and limited number of samples, the WMLE is an unreliable estimator. The use of WMLE leads to over-fitted covariance estimates, and, hence the variance/entropy of the search distribution decreases too quickly, which may cause premature convergence. In order to alleviate this problem, the estimated covariance matrix can be regularized in different ways, for example by using a convex combination of the diagonal covariance estimate and the sample covariance estimate. In this paper, we propose a new covariance matrix regularization technique for policy search methods that uses the convex combination of the sample covariance matrix and the old covariance matrix used in last iteration. The combination weighting is determined by specifying the desired entropy of the new search distribution. With this mechanism, the entropy of the search distribution can be gradually decreased without damage from the maximum likelihood estimate.},
url = {http://eprints.lincoln.ac.uk/25748/},
keywords = {ARRAY(0x56147f9b1f28)}
}
• S. Albrecht, A. M. S. da Barreto, D. Braziunas, D. Buckeridge, and H. Cuayahuitl, “Reports of the AAAI 2014 Conference Workshops,” AI Magazine, vol. 36, iss. 1, pp. 87-98, 2015.

The AAAI-14 Workshop program was held Sunday and Monday, July 27?28, 2012, at the Québec City Convention Centre in Québec, Canada. Canada. The AAAI-14 workshop program included fifteen workshops covering a wide range of topics in artificial intelligence. The titles of the workshops were AI and Robotics; Artificial Intelligence Applied to Assistive Technologies and Smart Environments; Cognitive Computing for Augmented Human Intelligence; Computer Poker and Imperfect Information; Discovery Informatics; Incentives and Trust in Electronic Communities; Intelligent Cinematography and Editing; Machine Learning for Interactive Systems: Bridging the Gap between Perception, Action and Communication; Modern Artificial Intelligence for Health Analytics; Multiagent Interaction without Prior Coordination; Multidisciplinary Workshop on Advances in Preference Handling; Semantic Cities — Beyond Open Data to Models, Standards and Reasoning; Sequential Decision Making with Big Data; Statistical Relational AI; and The World Wide Web and Public Health Intelligence. This article presents short summaries of those events.

@article{lirolem22215,
publisher = {Association for the Advancemant of Artificial Intelligence},
author = {Stefano Albrecht and Andr{\'e} da Motta Salles Barreto and Darius Braziunas and David Buckeridge and Heriberto Cuayahuitl},
pages = {87--98},
month = {January},
number = {1},
volume = {36},
year = {2015},
title = {Reports of the AAAI 2014 Conference Workshops},
journal = {AI Magazine},
url = {http://eprints.lincoln.ac.uk/22215/},
keywords = {ARRAY(0x56147f93f650)},
abstract = {The AAAI-14 Workshop program was held Sunday and Monday, July 27?28, 2012, at the Qu{\'e}bec City Convention Centre in Qu{\'e}bec, Canada. Canada. The AAAI-14 workshop program included fifteen workshops covering a wide range of topics in artificial intelligence. The titles of the workshops were AI and Robotics; Artificial Intelligence Applied to Assistive Technologies and Smart Environments; Cognitive Computing for Augmented Human Intelligence; Computer Poker and Imperfect Information; Discovery Informatics; Incentives and Trust in Electronic Communities; Intelligent Cinematography and Editing; Machine Learning for Interactive Systems: Bridging the Gap between Perception, Action and Communication; Modern Artificial Intelligence for Health Analytics; Multiagent Interaction without Prior Coordination; Multidisciplinary Workshop on Advances in Preference Handling; Semantic Cities {--} Beyond Open Data to Models, Standards and Reasoning; Sequential Decision Making with Big Data; Statistical Relational AI; and The World Wide Web and Public Health Intelligence. This article presents short summaries of those events.}
}
• F. Arvin, C. Xiong, and S. Yue, “Colias-\ensuremath\Phi: an autonomous micro robot for artificial pheromone communication,” International Journal of Mechanical Engineering and Robotics Research, vol. 4, iss. 4, pp. 349-353, 2015.

Ants pheromone communication is an efficient mechanism which took inspiration from nature. It has been used in various artificial intelligence and multi robotics researches. This paper presents the development of an autonomous micro robot to be used in swarm robotic researches especially in pheromone based communication systems. The robot is an extended version of Colias micro robot with capability of decoding and following artificial pheromone trails. We utilize a low-cost experimental setup to implement pheromone-based scenarios using a flat LCD screen and a USB camera. The results of the performed experiments with group of robots demonstrated the feasibility of Colias-\ensuremath\Phi to be used in pheromone based experiments.

@article{lirolem19405,
pages = {349--353},
month = {October},
author = {Farshad Arvin and Caihua Xiong and Shigang Yue},
number = {4},
year = {2015},
journal = {International Journal of Mechanical Engineering and Robotics Research},
title = {Colias-{\ensuremath{\Phi}}: an autonomous micro robot for artificial pheromone communication},
volume = {4},
abstract = {Ants pheromone communication is an efficient mechanism which took inspiration from nature. It has been used in various artificial intelligence and multi robotics researches. This paper presents the development of an autonomous micro robot to be used in swarm robotic researches especially in pheromone based communication systems. The robot is an extended version of Colias micro robot with capability of decoding and following artificial pheromone trails. We utilize a low-cost experimental setup to implement pheromone-based scenarios using a flat LCD screen and a USB camera. The results of the performed experiments with group of robots demonstrated the feasibility of Colias-{\ensuremath{\Phi}} to be used in pheromone based experiments.},
url = {http://eprints.lincoln.ac.uk/19405/},
keywords = {ARRAY(0x56147f905d30)}
}
• F. Arvin, T. Krajnik, A. E. Turgut, and S. Yue, “COS-\ensuremath\Phi: artificial pheromone system for robotic swarms research,” IEEE/RSJ International Conference on Intelligent Robots and Systems 2015, 2015.

Pheromone-based communication is one of the most effective ways of communication widely observed in nature. It is particularly used by social insects such as bees, ants and termites; both for inter-agent and agent-swarm communications. Due to its effectiveness; artificial pheromones have been adopted in multi-robot and swarm robotic systems for more than a decade. Although, pheromone-based communication was implemented by different means like chemical (use of particular chemical compounds) or physical (RFID tags, light, sound) ways, none of them were able to replicate all the aspects of pheromones as seen in nature. In this paper, we propose a novel artificial pheromone system that is reliable, accurate and it uses off-the-shelf components only — LCD screen and low-cost USB camera. The system allows to simulate several pheromones and their interactions and to change parameters of the pheromones (diffusion, evaporation, etc.) on the fly allowing for controllable experiments. We tested the performance of the system using the Colias platform in single-robot and swarm scenarios. To allow the swarm robotics community to use the system for their research, we provide it as a freely available open-source package.

@article{lirolem17957,
month = {September},
year = {2015},
title = {COS-{\ensuremath{\Phi}}: artificial pheromone system for robotic swarms research},
journal = {IEEE/RSJ International Conference on Intelligent Robots and Systems 2015},
author = {Farshad Arvin and Tomas Krajnik and Ali Emre Turgut and Shigang Yue},
publisher = {IEEE},
note = {Conference:
2015 IEEE/RSJ International Conference on  Intelligent Robots and Systems (IROS 2015), 28 September - 2 October 2015, Hamburg, Germany},
keywords = {ARRAY(0x56147f85fdd8)},
url = {http://eprints.lincoln.ac.uk/17957/},
abstract = {Pheromone-based communication is one of the most effective ways of communication widely observed in nature. It is particularly used by social insects such as bees, ants and termites; both for inter-agent and agent-swarm communications. Due to its effectiveness; artificial pheromones have been adopted in multi-robot and swarm robotic systems for more than a decade. Although, pheromone-based communication was implemented by different means like chemical (use of particular chemical compounds) or physical (RFID tags, light, sound) ways, none of them were able to replicate all the aspects of pheromones as seen in nature. In this paper, we propose a novel artificial pheromone system that is reliable, accurate and it uses off-the-shelf components only -- LCD screen and low-cost USB camera. The system allows to simulate several pheromones and their interactions and to change parameters of the pheromones (diffusion, evaporation, etc.) on the fly allowing for controllable experiments. We tested the performance of the system using the Colias platform in single-robot and swarm scenarios. To allow the swarm robotics community to use the system for their research, we provide it as a freely available open-source package.}
}
• F. Arvin, R. Attar, A. E. Turgut, and S. Yue, “Power-law distribution of long-term experimental data in swarm robotics,” in International Conference on Swarm Intelligence, 2015, pp. 551-559.

Bio-inspired aggregation is one of the most fundamental behaviours that has been studied in swarm robotic for more than two decades. Biology revealed that the environmental characteristics are very important factors in aggregation of social insects and other animals. In this paper, we study the effects of different environmental factors such as size and texture of aggregation cues using real robots. In addition, we propose a mathematical model to predict the behaviour of the aggregation during an experiment.

@inproceedings{lirolem17627,
pages = {551--559},
booktitle = {International Conference on Swarm Intelligence},
month = {June},
title = {Power-law distribution of long-term experimental data in swarm robotics},
year = {2015},
author = {Farshad Arvin and Rahman Attar and Ali Emre Turgut and Shigang Yue},
publisher = {Springer},
abstract = {Bio-inspired aggregation is one of the most fundamental behaviours that has been
studied in swarm robotic for more than two decades. Biology revealed that the
environmental characteristics are very important factors in aggregation of social insects and
other animals. In this paper, we study the effects of different environmental factors such as
size and texture of aggregation cues using real robots. In addition, we propose a
mathematical model to predict the behaviour of the aggregation during an experiment.},
keywords = {ARRAY(0x56147f9744c0)},
url = {http://eprints.lincoln.ac.uk/17627/}
}
• W. Chen, C. Xiong, and S. Yue, “Mechanical implementation of kinematic synergy for continual grasping generation of anthropomorphic hand,” IEEE/ASME Transactions on Mechatronics, vol. 20, iss. 3, pp. 1249-1263, 2015.

The synergy-based motion generation of current anthropomorphic hands generally employ the static posture synergy, which is extracted from quantities of joint trajectory, to design the mechanism or control strategy. Under this framework, the temporal weight sequences of each synergy from pregrasp phase to grasp phase are required for reproducing any grasping task. Moreover, the zero-offset posture has to be preset before starting any grasp. Thus, the whole grasp phase appears to be unlike natural human grasp. Up until now, no work in the literature addresses these issues toward simplifying the continual grasp by only inputting the grasp pattern. In this paper, the kinematic synergies observed in angular velocity profile are employed to design the motion generation mechanism. The kinematic synergy extracted from quantities of grasp tasks is implemented by the proposed eigen cam group in tendon space. The completely continual grasp from the fully extending posture only require averagely rotating the two eigen cam groups one cycle. The change of grasp pattern only depends on respecifying transmission ratio pair for the two eigen cam groups. An illustrated hand prototype is developed based on the proposed design principle and the grasping experiments demonstrate the feasibility of the design method. The potential applications include the prosthetic hand that is controlled by the classified pattern from the bio-signal.

@article{lirolem17879,
publisher = {IEEE},
author = {Wenbin Chen and Caihua Xiong and Shigang Yue},
month = {June},
pages = {1249--1263},
title = {Mechanical implementation of kinematic synergy for continual grasping generation of anthropomorphic hand},
year = {2015},
journal = {IEEE/ASME Transactions on Mechatronics},
volume = {20},
number = {3},
keywords = {ARRAY(0x56147f5b81b0)},
url = {http://eprints.lincoln.ac.uk/17879/},
abstract = {The synergy-based motion generation of current anthropomorphic hands generally employ the static posture synergy, which is extracted from quantities of joint trajectory, to design the mechanism or control strategy. Under this framework, the temporal weight sequences of each synergy from pregrasp phase to grasp phase are required for reproducing any grasping task. Moreover, the zero-offset posture has to be preset before starting any grasp. Thus, the whole grasp phase appears to be unlike natural human grasp. Up until now, no work in the literature addresses these issues toward simplifying the continual grasp by only inputting the grasp pattern. In this paper, the kinematic synergies observed in angular velocity profile are employed to design the motion generation mechanism. The kinematic synergy extracted from quantities of grasp tasks is implemented by the proposed eigen cam group in tendon space. The completely continual grasp from the fully extending posture only require averagely rotating the two eigen cam groups one cycle. The change of grasp pattern only depends on respecifying transmission ratio pair for the two eigen cam groups. An illustrated hand prototype is developed based on the proposed design principle and the grasping experiments demonstrate the feasibility of the design method. The potential applications include the prosthetic hand that is controlled by the classified pattern from the bio-signal.}
}
• C. Coppola, O. M. Mozos, and N. Bellotto, “Applying a 3D qualitative trajectory calculus to human action recognition using depth cameras,” in IEEE/RSJ IROS Workshop on Assistance and Service Robotics in a Human Environment, 2015.

The life span of ordinary people is increasing steadily and many developed countries are facing the big challenge of dealing with an ageing population at greater risk of impairments and cognitive disorders, which hinder their quality of life. Monitoring human activities of daily living (ADLs) is important in order to identify potential health problems and apply corrective strategies as soon as possible. Towards this long term goal, the research here presented is a first step to monitor ADLs using 3D sensors in an Ambient Assisted Living (AAL) environment. In particular, the work here presented adopts a new 3D Qualitative Trajectory Calculus (QTC3D) to represent human actions that belong to such activities, designing and implementing a set of computational tools (i.e. Hidden Markov Models) to learn and classify them from standard datasets. Preliminary results show the good performance of our system and its potential application to a large number of scenarios, including mobile robots for AAL.

@inproceedings{lirolem18477,
month = {October},
booktitle = {IEEE/RSJ IROS Workshop on Assistance and Service Robotics in a Human Environment},
author = {Claudio Coppola and Oscar Martinez Mozos and Nicola Bellotto},
publisher = {IEEE},
year = {2015},
title = {Applying a 3D qualitative trajectory calculus to human action recognition using depth cameras},
note = {2015 IEEE/RSJ International Conference on Intelligent Robots and Systems},
keywords = {ARRAY(0x56147f9333d8)},
url = {http://eprints.lincoln.ac.uk/18477/},
abstract = {The life span of ordinary people is increasing steadily and many developed countries are facing the big challenge of dealing with an ageing population at greater risk of impairments and cognitive disorders, which hinder their quality of life. Monitoring human activities of daily living (ADLs) is important in order to identify potential health problems and apply corrective strategies as soon as possible. Towards this long term goal, the research here presented is a first step to monitor ADLs using 3D sensors in an Ambient Assisted Living (AAL) environment. In particular, the work here presented adopts a new 3D Qualitative Trajectory Calculus (QTC3D) to represent human actions that belong to such activities, designing and implementing a set of computational tools (i.e. Hidden Markov Models) to learn and classify them from standard datasets. Preliminary results show the good performance of our system and its potential application to a large number of scenarios, including mobile robots for AAL.}
}
• H. Cuayahuitl, S. Keizer, and O. Lemon, “Strategic dialogue management via deep reinforcement learning,” in NIPS Workshop on Deep Reinforcement Learning, 2015.

Artificially intelligent agents equipped with strategic skills that can negotiate during their interactions with other natural or artificial agents are still underdeveloped. This paper describes a successful application of Deep Reinforcement Learning (DRL) for training intelligent agents with strategic conversational skills, in a situated dialogue setting. Previous studies have modelled the behaviour of strategic agents using supervised learning and traditional reinforcement learning techniques, the latter using tabular representations or learning with linear function approximation. In this study, we apply DRL with a high-dimensional state space to the strategic board game of Settlers of Catan—where players can offer resources in exchange for others and they can also reply to offers made by other players. Our experimental results report that the DRL-based learnt policies significantly outperformed several baselines including random, rule-based, and supervised-based behaviours. The DRL-based policy has a 53\% win rate versus 3 automated players (bots’), whereas a supervised player trained on a dialogue corpus in this setting achieved only 27\%, versus the same 3 bots. This result supports the claim that DRL is a promising framework for training dialogue systems, and strategic agents with negotiation abilities.

@inproceedings{lirolem25994,
booktitle = {NIPS Workshop on Deep Reinforcement Learning},
year = {2015},
journal = {CoRR},
volume = {abs/16},
title = {Strategic dialogue management via deep reinforcement learning},
publisher = {arXiv},
author = {Heriberto Cuayahuitl and Simon Keizer and Oliver Lemon},
abstract = {Artificially intelligent agents equipped with strategic skills that can negotiate during their interactions with other natural or artificial agents are still underdeveloped. This paper describes a successful application of Deep Reinforcement Learning (DRL) for training intelligent agents with strategic conversational skills, in a situated dialogue setting. Previous studies have modelled the behaviour of strategic agents using supervised learning and traditional reinforcement learning techniques, the latter using tabular representations or learning with linear function approximation. In this study, we apply DRL with a high-dimensional state space to the strategic board game of Settlers of Catan---where players can offer resources in exchange for others and they can also reply to offers made by other players. Our experimental results report that the DRL-based learnt policies significantly outperformed several baselines including random, rule-based, and supervised-based behaviours. The DRL-based policy has a 53\% win rate versus 3 automated players (bots'), whereas a supervised player trained on a dialogue corpus in this setting achieved only 27\%, versus the same 3 bots. This result supports the claim that DRL is a promising framework for training dialogue systems, and strategic agents with negotiation abilities.},
keywords = {ARRAY(0x56147f64aec8)},
url = {http://eprints.lincoln.ac.uk/25994/}
}
• H. Cuayahuitl, K. Komatani, and G. Skantze, “Introduction for speech and language for interactive robots,” Computer Speech & Language, vol. 34, iss. 1, pp. 83-86, 2015.

This special issue includes research articles which apply spoken language processing to robots that interact with human users through speech, possibly combined with other modalities. Robots that can listen to human speech, understand it, interact according to the conveyed meaning, and respond represent major research and technological challenges. Their common aim is to equip robots with natural interaction abilities. However, robotics and spoken language processing are areas that are typically studied within their respective communities with limited communication across disciplinary boundaries. The articles in this special issue represent examples that address the need for an increased multidisciplinary exchange of ideas.

@article{lirolem22214,
author = {Heriberto Cuayahuitl and Kazunori Komatani and Gabriel Skantze},
publisher = {Elsevier for International Speech Communication Association (ISCA)},
pages = {83--86},
month = {November},
number = {1},
journal = {Computer Speech \& Language},
year = {2015},
title = {Introduction for speech and language for interactive robots},
volume = {34},
abstract = {This special issue includes research articles which apply spoken language processing to robots that interact with human users through speech, possibly combined with other modalities. Robots that can listen to human speech, understand it, interact according to the conveyed meaning, and respond represent major research and technological challenges. Their common aim is to equip robots with natural interaction abilities. However, robotics and spoken language processing are areas that are typically studied within their respective communities with limited communication across disciplinary boundaries. The articles in this special issue represent examples that address the need for an increased multidisciplinary exchange of ideas.},
keywords = {ARRAY(0x56147f97a888)},
url = {http://eprints.lincoln.ac.uk/22214/}
}
• C. Dondrup, N. Bellotto, M. Hanheide, K. Eder, and U. Leonards, “A computational model of human-robot spatial interactions based on a qualitative trajectory calculus,” Robotics, vol. 4, iss. 1, pp. 63-102, 2015.

In this paper we propose a probabilistic sequential model of Human-Robot Spatial Interaction (HRSI) using a well-established Qualitative Trajectory Calculus (QTC) to encode HRSI between a human and a mobile robot in a meaningful, tractable, and systematic manner. Our key contribution is to utilise QTC as a state descriptor and model HRSI as a probabilistic sequence of such states. Apart from the sole direction of movements of human and robot modelled by QTC, attributes of HRSI like proxemics and velocity profiles play vital roles for the modelling and generation of HRSI behaviour. In this paper, we particularly present how the concept of proxemics can be embedded in QTC to facilitate richer models. To facilitate reasoning on HRSI with qualitative representations, we show how we can combine the representational power of QTC with the concept of proxemics in a concise framework, enriching our probabilistic representation by implicitly modelling distances. We show the appropriateness of our sequential model of QTC by encoding different HRSI behaviours observed in two spatial interaction experiments. We classify these encounters, creating a comparative measurement, showing the representational capabilities of the model.

@article{lirolem16987,
number = {1},
volume = {4},
year = {2015},
journal = {Robotics},
title = {A computational model of human-robot spatial interactions based on a qualitative trajectory calculus},
note = {This article belongs to the Special Issue Representations and Reasoning for Robotics},
publisher = {MDPI},
author = {Christian Dondrup and Nicola Bellotto and Marc Hanheide and Kerstin Eder and Ute Leonards},
pages = {63--102},
month = {March},
abstract = {In this paper we propose a probabilistic sequential model of Human-Robot Spatial Interaction (HRSI) using a well-established Qualitative Trajectory Calculus (QTC) to encode HRSI between a human and a mobile robot in a meaningful, tractable, and systematic manner. Our key contribution is to utilise QTC as a state descriptor and model HRSI as a probabilistic sequence of such states. Apart from the sole direction of movements of human and robot modelled by QTC, attributes of HRSI like proxemics and velocity profiles play vital roles for the modelling and generation of HRSI behaviour. In this paper, we particularly present how the concept of proxemics can be embedded in QTC to facilitate richer models. To facilitate reasoning on HRSI with qualitative representations, we show how we can combine the representational power of QTC with the concept of proxemics in a concise framework, enriching our probabilistic representation by implicitly modelling distances. We show the appropriateness of our sequential model of QTC by encoding different HRSI behaviours observed in two spatial interaction experiments. We classify these encounters, creating a comparative measurement, showing the representational capabilities of the model.},
url = {http://eprints.lincoln.ac.uk/16987/},
keywords = {ARRAY(0x56147f64de10)}
}
• C. Dondrup, N. Bellotto, F. Jovan, and M. Hanheide, “Real-time multisensor people tracking for human-robot spatial interaction,” in Workshop on Machine Learning for Social Robotics at ICRA 2015, 2015.

All currently used mobile robot platforms are able to navigate safely through their environment, avoiding static and dynamic obstacles. However, in human populated environments mere obstacle avoidance is not sufficient to make humans feel comfortable and safe around robots. To this end, a large community is currently producing human-aware navigation approaches to create a more socially acceptable robot behaviour. Amajorbuilding block for all Human-Robot Spatial Interaction is the ability of detecting and tracking humans in the vicinity of the robot. We present a fully integrated people perception framework, designed to run in real-time on a mobile robot. This framework employs detectors based on laser and RGB-D data and a tracking approach able to fuse multiple detectors using different versions of data association and Kalman filtering. The resulting trajectories are transformed into Qualitative Spatial Relations based on a Qualitative Trajectory Calculus, to learn and classify different encounters using a Hidden Markov Model based representation. We present this perception pipeline, which is fully implemented into the Robot Operating System (ROS), in a small proof of concept experiment. All components are readily available for download, and free to use under the MIT license, to researchers in all fields, especially focussing on social interaction learning by providing different kinds of output, i.e. Qualitative Relations and trajectories.

@inproceedings{lirolem17545,
author = {Christian Dondrup and Nicola Bellotto and Ferdian Jovan and Marc Hanheide},
publisher = {ICRA / IEEE},
year = {2015},
title = {Real-time multisensor people tracking for human-robot spatial interaction},
month = {May},
booktitle = {Workshop on Machine Learning for Social Robotics at ICRA 2015},
abstract = {All currently used mobile robot platforms are able to navigate safely through their environment, avoiding static and dynamic obstacles. However, in human populated environments mere obstacle avoidance is not sufficient to make humans feel comfortable and safe around robots. To this end, a large community is currently producing human-aware navigation approaches to create a more socially acceptable robot behaviour. Amajorbuilding block for all Human-Robot Spatial Interaction is the ability of detecting and tracking humans in the vicinity of the robot. We present a fully integrated people perception framework, designed to run in real-time on a mobile robot. This framework employs detectors based on laser and RGB-D data and a tracking approach able to fuse multiple detectors using different versions of data association and Kalman filtering. The resulting trajectories are transformed into Qualitative Spatial Relations based on a Qualitative Trajectory Calculus, to learn and classify different encounters using a Hidden Markov Model based representation. We present this perception pipeline, which is fully implemented into the Robot Operating System (ROS), in a small proof of concept experiment. All components are readily available for download, and free to use under the MIT license, to researchers in all fields, especially focussing on social interaction learning by providing different kinds of output, i.e. Qualitative Relations and trajectories.},
url = {http://eprints.lincoln.ac.uk/17545/},
keywords = {ARRAY(0x56147f5f06a8)}
}
• M. Ewerton, G. Neumann, R. Lioutikov, H. B. Amor, J. Peters, and G. Maeda, “Learning multiple collaborative tasks with a mixture of interaction primitives,” in International Conference on Robotics and Automation (ICRA), 2015, pp. 1535-1542.

Robots that interact with humans must learn to not only adapt to different human partners but also to new interactions. Such a form of learning can be achieved by demonstrations and imitation. A recently introduced method to learn interactions from demonstrations is the framework of Interaction Primitives. While this framework is limited to represent and generalize a single interaction pattern, in practice, interactions between a human and a robot can consist of many different patterns. To overcome this limitation this paper proposes a Mixture of Interaction Primitives to learn multiple interaction patterns from unlabeled demonstrations. Specifically the proposed method uses Gaussian Mixture Models of Interaction Primitives to model nonlinear correlations between the movements of the different agents. We validate our algorithm with two experiments involving interactive tasks between a human and a lightweight robotic arm. In the first, we compare our proposed method with conventional Interaction Primitives in a toy problem scenario where the robot and the human are not linearly correlated. In the second, we present a proof-of-concept experiment where the robot assists a human in assembling a box.

@inproceedings{lirolem25762,
publisher = {IEEE},
author = {Marco Ewerton and Gerhard Neumann and Rudolf Lioutikov and Heni Ben Amor and Jan Peters and Guilherme Maeda},
note = {cited By 2},
month = {May},
pages = {1535--1542},
booktitle = {International Conference on Robotics and Automation (ICRA)},
year = {2015},
volume = {2015-J},
title = {Learning multiple collaborative tasks with a mixture of interaction primitives},
journal = {Proceedings - IEEE International Conference on Robotics and Automation},
number = {June},
keywords = {ARRAY(0x56147f7f82c0)},
url = {http://eprints.lincoln.ac.uk/25762/},
abstract = {Robots that interact with humans must learn to
not only adapt to different human partners but also to new
interactions. Such a form of learning can be achieved by
demonstrations and imitation. A recently introduced method
to learn interactions from demonstrations is the framework
of Interaction Primitives. While this framework is limited
to represent and generalize a single interaction pattern, in
practice, interactions between a human and a robot can consist
of many different patterns. To overcome this limitation this
paper proposes a Mixture of Interaction Primitives to learn
multiple interaction patterns from unlabeled demonstrations.
Specifically the proposed method uses Gaussian Mixture Models
of Interaction Primitives to model nonlinear correlations
between the movements of the different agents. We validate
our algorithm with two experiments involving interactive tasks
between a human and a lightweight robotic arm. In the first,
we compare our proposed method with conventional Interaction
Primitives in a toy problem scenario where the robot and the
human are not linearly correlated. In the second, we present a
proof-of-concept experiment where the robot assists a human
in assembling a box.}
}
• M. Ewerton, G. Maeda, J. Peters, and G. Neumann, “Learning motor skills from partially observed movements executed at different speeds,” in IEEE/RSJ Conference on Intelligent Robots and Systems (IROS), 2015, pp. 456-463.

Learning motor skills from multiple demonstrations presents a number of challenges. One of those challenges is the occurrence of occlusions and lack of sensor coverage, which may corrupt part of the recorded data. Another issue is the variability in speed of execution of the demonstrations, which may require a way of finding the correspondence between the time steps of the different demonstrations. In this paper, an approach to learn motor skills is proposed that accounts both for spatial and temporal variability of movements. This approach, based on an Expectation-Maximization algorithm to learn Probabilistic Movement Primitives, also allows for learning motor skills from partially observed demonstrations, which may result from occlusion or lack of sensor coverage. An application of the algorithm proposed in this work lies in the field of Human-Robot Interaction when the robot has to react to human movements executed at different speeds. Experiments in which a robotic arm receives a cup handed over by a human illustrate this application. The capabilities of the algorithm in learning and predicting movements are also evaluated in experiments using a data set of letters and a data set of golf putting movements.

@inproceedings{lirolem25753,
volume = {2015-D},
journal = {IEEE International Conference on Intelligent Robots and Systems},
title = {Learning motor skills from partially observed movements executed at different speeds},
year = {2015},
booktitle = {IEEE/RSJ Conference on Intelligent Robots and Systems (IROS)},
pages = {456--463},
month = {October},
author = {M. Ewerton and G. Maeda and J. Peters and G. Neumann},
abstract = {Learning motor skills from multiple demonstrations
presents a number of challenges. One of those challenges
is the occurrence of occlusions and lack of sensor coverage,
which may corrupt part of the recorded data. Another issue
is the variability in speed of execution of the demonstrations,
which may require a way of finding the correspondence between
the time steps of the different demonstrations. In this paper,
an approach to learn motor skills is proposed that accounts
both for spatial and temporal variability of movements. This
approach, based on an Expectation-Maximization algorithm to
learn Probabilistic Movement Primitives, also allows for learning
motor skills from partially observed demonstrations, which may
result from occlusion or lack of sensor coverage. An application
of the algorithm proposed in this work lies in the field of
Human-Robot Interaction when the robot has to react to human
movements executed at different speeds. Experiments in which
a robotic arm receives a cup handed over by a human illustrate
this application. The capabilities of the algorithm in learning
and predicting movements are also evaluated in experiments
using a data set of letters and a data set of golf putting
movements.},
keywords = {ARRAY(0x56147f883cf0)},
url = {http://eprints.lincoln.ac.uk/25753/}
}
• J. P. Fentanes, B. Lacerda, T. Krajnik, N. Hawes, and M. Hanheide, “Now or later? Predicting and maximising success of navigation actions from long-term experience,” in 2015 IEEE International Conference on Robotics and Automation (ICRA 2015), 2015.

In planning for deliberation or navigation in real-world robotic systems, one of the big challenges is to cope with change. It lies in the nature of planning that it has to make assumptions about the future state of the world, and the robot?s chances of successively accomplishing actions in this future. Hence, a robot?s plan can only be as good as its predictions about the world. In this paper, we present a novel approach to specifically represent changes that stem from periodic events in the environment (e.g. a door being opened or closed), which impact on the success probability of planned actions. We show that our approach to model the probability of action success as a set of superimposed periodic processes allows the robot to predict action outcomes in a long-term data obtained in two real-life offices better than a static model. We furthermore discuss and showcase how this knowledge gathered can be successfully employed in a probabilistic planning framework to devise better navigation plans. The key contributions of this paper are (i) the formation of the spectral model of action outcomes from non-uniform sampling, the (ii) analysis of its predictive power using two long-term datasets, and (iii) the application of the predicted outcomes in an MDP-based planning framework.

@inproceedings{lirolem17745,
publisher = {IEEE/RAS},
author = {Jaime Pulido Fentanes and Bruno Lacerda and Tomas Krajnik and Nick Hawes and Marc Hanheide},
year = {2015},
title = {Now or later? Predicting and maximising success of navigation actions from long-term experience},
booktitle = {2015 IEEE International Conference on Robotics and Automation (ICRA 2015)},
month = {May},
abstract = {In planning for deliberation or navigation in real-world robotic systems, one of the big challenges is to cope with change. It lies in the nature of planning that it has to make assumptions about the future state of the world, and the robot?s chances of successively accomplishing actions in this future.
Hence, a robot?s plan can only be as good as its predictions about the world. In this paper, we present a novel approach to specifically represent changes that stem from periodic events in the environment (e.g. a door being opened or closed), which impact on the success probability of planned actions. We show that our approach to model the probability of action success as a set of superimposed periodic processes allows the robot to predict action outcomes in a long-term data obtained in two real-life offices better than a static model. We furthermore discuss and showcase how this knowledge gathered can be successfully employed in a probabilistic planning framework to devise better navigation plans. The key contributions of this paper are (i) the formation of the spectral model of action outcomes from non-uniform sampling, the (ii) analysis of its predictive power using two long-term datasets, and (iii) the application of the predicted outcomes in an MDP-based planning framework.},
url = {http://eprints.lincoln.ac.uk/17745/},
keywords = {ARRAY(0x56147f7a3930)}
}
• Q. Fu and S. Yue, “Modelling LGMD2 visual neuron system,” in 2015 IEEE International Workshop on Machine Learning for Signal Processing, 2015.

Two Lobula Giant Movement Detectors (LGMDs) have been identified in the lobula region of the locust visual system: LGMD1 and LGMD2. LGMD1 had been successfully used in robot navigation to avoid impending collision. LGMD2 also responds to looming stimuli in depth, and shares most the same properties with LGMD1; however, LGMD2 has its specific collision selective responds when dealing with different visual stimulus. Therefore, in this paper, we propose a novel way to model LGMD2, in order to emulate its predicted bio-functions, moreover, to solve some defects of previous LGMD1 computational models. The mechanism of ON and OFF cells, as well as bioinspired nonlinear functions, are introduced in our model, to achieve LGMD2?s collision selectivity. Our model has been tested by a miniature mobile robot in real time. The results suggested this model has an ideal performance in both software and hardware for collision recognition.

@inproceedings{lirolem24940,
author = {Qinbing Fu and Shigang Yue},
title = {Modelling LGMD2 visual neuron system},
year = {2015},
booktitle = {2015 IEEE International Workshop on Machine Learning for Signal Processing},
month = {September},
url = {http://eprints.lincoln.ac.uk/24940/},
keywords = {ARRAY(0x56147f791350)},
abstract = {Two Lobula Giant Movement Detectors (LGMDs) have been identified in the lobula region of the locust visual system: LGMD1 and LGMD2. LGMD1 had been successfully used in robot navigation to avoid impending collision. LGMD2 also responds to looming stimuli in depth, and shares most the same properties with LGMD1; however, LGMD2 has its specific collision selective responds when dealing with different visual stimulus. Therefore, in this paper, we propose a novel way to model LGMD2, in order to emulate its predicted bio-functions, moreover, to solve some defects of previous LGMD1 computational models. The mechanism of ON and OFF cells, as well as bioinspired nonlinear functions, are introduced in our model, to achieve LGMD2?s collision selectivity. Our model has been tested by a miniature mobile robot in real time. The results suggested this model has an ideal performance in both software and hardware for collision recognition.}
}
• P. Gallina, N. Bellotto, and M. D. Luca, “Progressive co-adaptation in human-machine interaction,” in 12th International Conference on Informatics in Control, Automation and Robotics (ICINCO 2015), 2015.

In this paper we discuss the concept of co-adaptation between a human operator and a machine interface and we summarize its application with emphasis on two different domains, teleoperation and assistive technology. The analysis of the literature reveals that only in few cases the possibility of a temporal evolution of the co-adaptation parameters has been considered. In particular, it has been overlooked the role of time-related indexes that capture changes in motor and cognitive abilities of the human operator. We argue that for a more effective long-term co-adaptation process, the interface should be able to predict and adjust its parameters according to the evolution of human skills and performance. We thus propose a novel approach termed progressive co-adaptation, whereby human performance is continuously monitored and the system makes inferences about changes in the users’ cognitive and motor skills. We illustrate the features of progressive co-adaptation in two possible applications, robotic telemanipulation and active vision for the visually impaired.

@inproceedings{lirolem17501,
booktitle = {12th International Conference on Informatics in Control, Automation and Robotics (ICINCO 2015)},
month = {July},
year = {2015},
title = {Progressive co-adaptation in human-machine interaction},
author = {Paolo Gallina and Nicola Bellotto and Massimiliano Di Luca},
keywords = {ARRAY(0x56147f690028)},
url = {http://eprints.lincoln.ac.uk/17501/},
abstract = {In this paper we discuss the concept of co-adaptation between a human operator and a machine interface and we summarize its application with emphasis on two different domains, teleoperation and assistive technology. The analysis of the literature reveals that only in few cases the possibility of a temporal evolution of the co-adaptation parameters has been considered. In particular, it has been overlooked the role of time-related indexes that capture changes in motor and cognitive abilities of the human operator. We argue that for a more effective long-term co-adaptation process, the interface should be able to predict and adjust its parameters according to the evolution of human skills and performance. We thus propose a novel approach termed progressive co-adaptation, whereby human performance is continuously monitored and the system makes inferences about changes in the users' cognitive and motor skills. We illustrate the features of progressive co-adaptation in two possible applications, robotic telemanipulation and active vision for the visually impaired.}
}
• Y. Gao, W. Wang, and S. Yue, “On the rate of convergence by generalized Baskakov operators,” Advances in Mathematical Physics, vol. 2015, p. 564854, 2015.

We firstly construct generalized Baskakov operators V n, \ensuremath\alpha, q (f; x) and their truncated sum B n, \ensuremath\alpha, q (f; \ensuremath\gamma n, x). Secondly, we study the pointwise convergence and the uniform convergence of the operators V n, \ensuremath\alpha, q (f; x), respectively, and estimate that the rate of convergence by the operators V n, \ensuremath\alpha, q (f; x) is 1 / n q / 2. Finally, we study the convergence by the truncated operators B n, \ensuremath\alpha, q (f; \ensuremath\gamma n, x) and state that the finite truncated sum B n, \ensuremath\alpha, q (f; \ensuremath\gamma n, x) can replace the operators V n, \ensuremath\alpha, q (f; x) in the computational point of view provided that l i m n $\rightarrow$ ? n \ensuremath\gamma n = ?. \copyright 2015 Yi Gao et al.

@article{lirolem17367,
journal = {Advances in Mathematical Physics},
year = {2015},
title = {On the rate of convergence by generalized Baskakov operators},
volume = {2015},
permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.},
author = {Yi Gao and W. Wang and Shigang Yue},
publisher = {Hindawi Publishing Corporation},
pages = {564854},
month = {May},
abstract = {We firstly construct generalized Baskakov operators V n, {\ensuremath{\alpha}}, q (f; x) and their truncated sum B n, {\ensuremath{\alpha}}, q (f; {\ensuremath{\gamma}} n, x). Secondly, we study the pointwise convergence and the uniform convergence of the operators V n, {\ensuremath{\alpha}}, q (f; x), respectively, and estimate that the rate of convergence by the operators V n, {\ensuremath{\alpha}}, q (f; x) is 1 / n q / 2. Finally, we study the convergence by the truncated operators B n, {\ensuremath{\alpha}}, q (f; {\ensuremath{\gamma}} n, x) and state that the finite truncated sum B n, {\ensuremath{\alpha}}, q (f; {\ensuremath{\gamma}} n, x) can replace the operators V n, {\ensuremath{\alpha}}, q (f; x) in the computational point of view provided that l i m n {$\rightarrow$} ? n {\ensuremath{\gamma}} n = ?. {\copyright} 2015 Yi Gao et al.},
url = {http://eprints.lincoln.ac.uk/17367/},
keywords = {ARRAY(0x56147f723200)}
}
• Y. Gao, J. Peng, S. Yue, and Y. Zhao, “On the null space property of lq -minimization for 0\ensuremathJournal of Function Spaces, vol. 2015, p. 579853, 2015.

The paper discusses the relationship between the null space property (NSP) and the lq-minimization in compressed sensing. Several versions of the null space property, that is, the lq stable NSP, the lq robust NSP, and the lq,p robust NSP for 0\ensuremath

@article{lirolem17374,
journal = {Journal of Function Spaces},
title = {On the null space property of lq -minimization for 0{\ensuremath{
• E. Gyebi, M. Hanheide, and G. Cielniak, "Affordable mobile robotic platforms for teaching computer science at African universities," in 6th International Conference on Robotics in Education, 2015.

Educational robotics can play a key role in addressing some of the challenges faced by higher education in Africa. One of the major obstacles preventing a wider adoption of initiatives involving educational robotics in this part of the world is lack of robots that would be affordable by African institutions. In this paper, we present a survey and analysis of currently available affordable mobile robots and their suitability for teaching computer science at African universities. To this end, we propose a set of assessment criteria and review a number of platforms costing an order of magnitude less than the existing popular educational robots. Our analysis identifies suitable candidates offering contrasting features and benefits. We also discuss potential issues and promising directions which can be considered by both educators in Africa but also designers and manufacturers of future robot platforms.

@inproceedings{lirolem17557,
author = {Ernest Gyebi and Marc Hanheide and Grzegorz Cielniak},
year = {2015},
title = {Affordable mobile robotic platforms for teaching computer science at African universities},
month = {May},
booktitle = {6th International Conference on Robotics in Education},
url = {http://eprints.lincoln.ac.uk/17557/},
keywords = {ARRAY(0x56147f7a9ca8)},
abstract = {Educational robotics can play a key role in addressing some of the challenges faced by higher education in Africa. One of the major obstacles preventing a wider adoption of initiatives involving educational robotics in this part of the world is lack of robots that would be affordable by African institutions. In this paper, we present a survey and analysis of currently available affordable mobile robots and their suitability for teaching computer science at African universities. To this end, we propose a set of assessment criteria and review a number of platforms costing an order of magnitude less than the existing popular educational robots. Our analysis identifies suitable candidates offering contrasting features and benefits. We also discuss potential issues and promising directions which can be considered by both educators in Africa but also designers and manufacturers of future robot platforms.}
}
• E. Gyebi, F. Arvin, M. Hanheide, S. Yue, and G. Cielniak, "Colias: towards an affordable mobile robot for education in developing countries," in Developing Countries Forum at ICRA 2015, 2015.

Educational robotics can play a key role in addressing some of the important challenges faced by higher education in developing countries. One of the major obstacles preventing a wider adoption of initiatives involving educational robotics in these parts of the world is a lack of robot platforms which would be affordable for the local educational institutions. In this paper, we present our inexpensive mobile robot platform Colias and assess its potential for education in developing countries. To this end, we describe hardware and software components of the robot, assess its suitability for education and discuss the missing features which will need to be developed to turn Colias into a fully featured educational platform. The presented robot is one of the key components of our current efforts in popularising educational robotics at African universities.

@inproceedings{lirolem17558,
title = {Colias: towards an affordable mobile robot for education in developing countries},
year = {2015},
author = {Ernest Gyebi and Farshad Arvin and Marc Hanheide and Shigang Yue and Grzegorz Cielniak},
month = {May},
booktitle = {Developing Countries Forum at ICRA 2015},
abstract = {Educational robotics can play a key role in addressing some of the important challenges faced by higher education
in developing countries. One of the major obstacles preventing a wider adoption of initiatives involving educational robotics in these parts of the world is a lack of robot platforms which would be affordable for the local educational institutions. In this paper, we present our inexpensive mobile robot platform Colias and assess its potential for education in developing countries. To this end, we describe hardware and software components of the robot, assess its suitability for education and discuss the missing features which will need to be developed to turn Colias into a fully featured educational platform. The presented robot is one of the key components of our current efforts in popularising
educational robotics at African universities.},
keywords = {ARRAY(0x56147f64e470)},
url = {http://eprints.lincoln.ac.uk/17558/}
}
• E. Gyebi, M. Hanheide, and G. Cielniak, "Educational robotics for teaching computer science in Africa - pilot study," in WONDER 2015, First International Workshop on Educational Robotics, 2015.

Educational robotics can play a key role in addressing some of the challenges faced by higher education institutions in Africa. A remaining and open question is related to effectiveness of activities involving educational robots for teaching but also for improving learner's experience. This paper addresses that question by evaluating a short pilot study which introduced students at the Department of Computer Science, University of Ghana to robot programming. The initial positive results from the study indicate a potential for such activities to enhance teaching experience and practice at African institutions. The proposed integrated set-up including robotic hardware, software and educational tasks was effective and will form a solid base for a future, full scale integration of robotic activities into the undergraduate curricula at this particular institution. This evaluation should be valuable to other educators integrating educational robots into undergraduate curricula in developing countries and elsewhere.

@inproceedings{lirolem19407,
title = {Educational robotics for teaching computer science in Africa - pilot study},
year = {2015},
author = {Ernest Gyebi and Marc Hanheide and Grzegorz Cielniak},
booktitle = {WONDER 2015, First International Workshop on Educational Robotics},
month = {October},
url = {http://eprints.lincoln.ac.uk/19407/},
keywords = {ARRAY(0x56147f89dfd0)},
abstract = {Educational robotics can play a key role in addressing some of the challenges faced by higher education institutions in Africa. A remaining and open question is related to effectiveness of activities involving educational robots for teaching but also for improving learner's experience. This paper addresses that question by evaluating a short pilot study which introduced students at the Department of Computer Science, University of Ghana to robot programming. The initial positive results from the study indicate a potential for such activities to enhance teaching experience and practice at African institutions. The proposed integrated set-up including robotic hardware, software and educational tasks was effective and will form a solid base for a future, full scale integration of robotic activities into the undergraduate curricula at this particular institution. This evaluation should be valuable to other educators integrating educational robots into undergraduate curricula in developing countries and elsewhere.}
}
• D. Hebesberger, T. Körtner, J. Pripfl, C. Gisinger, and M. Hanheide, "What do staff in eldercare want a robot for? An assessment of potential tasks and user requirements for a long-term deployment," in IROS Workshop on "Bridging user needs to deployed applications of service robots", Hamburg, 2015.

Robotic aids could help to overcome the gap between rising numbers of older adults and at the same time declining numbers of care staff. Assessments of end-user requirements, especially focusing on staff in eldercare facilities are still sparse. Contributing to this field of research this study presents end-user requirements and task analysis gained from a methodological combination of interviews and focus group discussions. The findings suggest different tasks robots in eldercare could engage in such as ?fetch and carry? tasks, specific entertainment and information tasks, support in physical and occupational therapy, and in security. Furthermore this paper presents an iterative approach that closes the loop between requirements-assessments and subsequent implementations that follow the found requirements.

@inproceedings{lirolem18860,
booktitle = {IROS Workshop on "Bridging user needs to deployed applications of service robots"},
month = {September},
note = {The Robot-Era Project has received funding from the European Community's Seventh Framework Programme (FP7/2007-2013) under grant agreement num. 288899
FP7 - ICT - Challenge 5: ICT for Health, Ageing Well, Inclusion and Governance},
year = {2015},
title = {What do staff in eldercare want a robot for? An assessment of potential tasks and user requirements for a long-term deployment},
author = {Denise Hebesberger and Tobias K{\"o}rtner and J{\"u}rgen Pripfl and Christoph Gisinger and Marc Hanheide},
url = {http://eprints.lincoln.ac.uk/18860/},
keywords = {ARRAY(0x56147f8e5e60)},
abstract = {Robotic aids could help to overcome the gap between rising numbers of older adults and at the same time declining numbers of care staff. Assessments of end-user requirements, especially focusing on staff in eldercare facilities are still sparse. Contributing to this field of research this study presents end-user requirements and task analysis gained from a methodological combination of interviews and focus group discussions. The findings suggest different tasks robots in eldercare could engage in such as ?fetch and carry? tasks, specific entertainment and information tasks, support in physical and occupational therapy, and in security. Furthermore this paper presents an iterative approach that closes the loop between requirements-assessments and subsequent implementations that follow the found requirements.}
}
• H. V. Hoof, J. Peters, and G. Neumann, "Learning of non-parametric control policies with high-dimensional state features," Journal of Machine Learning Research: Workshop and Conference Proceedings, vol. 38, pp. 995-1003, 2015.

Learning complex control policies from highdimensional sensory input is a challenge for reinforcement learning algorithms. Kernel methods that approximate values functions or transition models can address this problem. Yet, many current approaches rely on instable greedy maximization. In this paper, we develop a policy search algorithm that integrates robust policy updates and kernel embeddings. Our method can learn nonparametric control policies for infinite horizon continuous MDPs with high-dimensional sensory representations. We show that our method outperforms related approaches, and that our algorithm can learn an underpowered swing-up task task directly from highdimensional image data.

@article{lirolem25757,
note = {Proceedings of the 18th International Conference
on Artificial Intelligence and Statistics (AISTATS), 9-12 May
2015, San Diego, CA,},
publisher = {MIT Press},
author = {Herke Van Hoof and Jan Peters and Gerhard Neumann},
pages = {995--1003},
booktitle = {18th International Conference on Artificial Intelligence and Statistics (AISTATS)},
month = {May},
volume = {38},
year = {2015},
title = {Learning of non-parametric control policies with high-dimensional state features},
journal = {Journal of Machine Learning Research: Workshop and Conference Proceedings},
abstract = {Learning complex control policies from highdimensional sensory input is a challenge for
reinforcement learning algorithms. Kernel methods that approximate values functions
or transition models can address this problem. Yet, many current approaches rely on
instable greedy maximization. In this paper, we develop a policy search algorithm that
integrates robust policy updates and kernel embeddings. Our method can learn nonparametric
control policies for infinite horizon continuous MDPs with high-dimensional
sensory representations. We show that our method outperforms related approaches, and
that our algorithm can learn an underpowered swing-up task task directly from highdimensional
image data.},
url = {http://eprints.lincoln.ac.uk/25757/},
keywords = {ARRAY(0x56147f7def80)}
}
• V. H. Hoof, T. Hermans, G. Neumann, and J. Peters, "Learning robot in-hand manipulation with tactile features," in International Conference on Humanoid Robots (HUMANOIDS), 2015, pp. 121-127.

Dexterous manipulation enables repositioning of objects and tools within a robot?s hand. When applying dexterous manipulation to unknown objects, exact object models are not available. Instead of relying on models, compliance and tactile feedback can be exploited to adapt to unknown objects. However, compliant hands and tactile sensors add complexity and are themselves difficult to model. Hence, we propose acquiring in-hand manipulation skills through reinforcement learning, which does not require analytic dynamics or kinematics models. In this paper, we show that this approach successfully acquires a tactile manipulation skill using a passively compliant hand. Additionally, we show that the learned tactile skill generalizes to novel objects.

@inproceedings{lirolem25750,
author = {H. Van Hoof and T. Hermans and G. Neumann and J. Peters},
pages = {121--127},
booktitle = {International Conference on Humanoid Robots (HUMANOIDS)},
month = {November},
title = {Learning robot in-hand manipulation with tactile features},
journal = {IEEE-RAS International Conference on Humanoid Robots},
year = {2015},
volume = {2015-D},
keywords = {ARRAY(0x56147f79c2d0)},
url = {http://eprints.lincoln.ac.uk/25750/},
abstract = {Dexterous manipulation enables repositioning of
objects and tools within a robot?s hand. When applying dexterous
manipulation to unknown objects, exact object models
are not available. Instead of relying on models, compliance and
tactile feedback can be exploited to adapt to unknown objects.
However, compliant hands and tactile sensors add complexity
and are themselves difficult to model. Hence, we propose acquiring
in-hand manipulation skills through reinforcement learning,
which does not require analytic dynamics or kinematics models.
In this paper, we show that this approach successfully acquires
a tactile manipulation skill using a passively compliant hand.
Additionally, we show that the learned tactile skill generalizes
to novel objects.}
}
• C. Keeble, G. R. Law, S. Barber, and P. D. Baxter, "Choosing a method to reduce selection bias: a tool for researchers," Open Journal of Epidemiology, vol. 5, iss. 3, pp. 155-162, 2015.

Selection bias is well known to affect surveys and epidemiological studies. There have been numerous methods proposed to reduce its effects, so many that researchers may be unclear which method is most suitable for their study; the wide choice may even deter some researchers, for fear of choosing a sub-optimal approach. We propose a straightforward tool to inform researchers of the most promising methods available to reduce selection bias and to assist the search for an appropriate method given their study design and details. We demonstrate the tool using three exam- ples where selection bias may occur; the tool quickly eliminates inappropriate methods and guides the researcher towards those to consider implementing. If more studies con- sider selection bias and adopt methods to reduce it, valuable time and resources will be saved, and should lead to more focused research towards disease prevention or cure.

@article{lirolem26608,
number = {3},
volume = {5},
year = {2015},
journal = {Open Journal of Epidemiology},
title = {Choosing a method to reduce selection bias: a tool for researchers},
pages = {155 -- 162},
month = {August},
author = {C. Keeble and G. R. Law and S. Barber and P. D. Baxter},
publisher = {Scientific Research Publishing},
keywords = {ARRAY(0x56147f65da18)},
url = {http://eprints.lincoln.ac.uk/26608/},
abstract = {Selection bias is well known to affect surveys and epidemiological studies. There have been numerous methods proposed to reduce its effects, so many that researchers may be unclear which method is most suitable for their study; the wide choice may even deter some researchers, for fear of choosing a sub-optimal approach. We propose a straightforward tool to inform researchers of the most promising methods available to reduce selection bias and to assist the search for an appropriate method given their study design and details. We demonstrate the tool using three exam- ples where selection bias may occur; the tool quickly eliminates inappropriate methods and guides the researcher towards those to consider implementing. If more studies con- sider selection bias and adopt methods to reduce it, valuable time and resources will be saved, and should lead to more focused research towards disease prevention or cure.}
}
• J. Kennedy, P. Baxter, and T. Belpaeme, "Comparing robot embodiments in a guided discovery learning interaction with children," International Journal of Social Robotics, vol. 7, iss. 2, pp. 293-308, 2015.

The application of social robots to the domain of education is becoming more prevalent. However, there re- main a wide range of open issues, such as the effectiveness of robots as tutors on student learning outcomes, the role of social behaviour in teaching interactions, and how the em- bodiment of a robot influences the interaction. In this paper, we seek to explore children?s behaviour towards a robot tutor for children in a novel guided discovery learning interac- tion. Since the necessity of real robots (as opposed to virtual agents) in education has not been definitively established in the literature, the effect of robot embodiment is assessed. The results demonstrate that children overcome strong incorrect biases in the material to be learned, but with no significant dif- ferences between embodiment conditions. However, the data do suggest that the use of real robots carries an advantage in terms of social presence that could provide educational benefits

@article{lirolem23075,
year = {2015},
volume = {7},
title = {Comparing robot embodiments in a guided discovery learning interaction with children},
journal = {International Journal of Social Robotics},
number = {2},
publisher = {Springer verlag},
author = {James Kennedy and Paul Baxter and Tony Belpaeme},
month = {April},
pages = {293--308},
keywords = {ARRAY(0x56147f737a30)},
url = {http://eprints.lincoln.ac.uk/23075/},
abstract = {The application of social robots to the domain of education is becoming more prevalent. However, there re- main a wide range of open issues, such as the effectiveness of robots as tutors on student learning outcomes, the role of social behaviour in teaching interactions, and how the em- bodiment of a robot influences the interaction. In this paper, we seek to explore children?s behaviour towards a robot tutor for children in a novel guided discovery learning interac- tion. Since the necessity of real robots (as opposed to virtual agents) in education has not been definitively established in the literature, the effect of robot embodiment is assessed. The results demonstrate that children overcome strong incorrect biases in the material to be learned, but with no significant dif- ferences between embodiment conditions. However, the data do suggest that the use of real robots carries an advantage in terms of social presence that could provide educational benefits}
}
• J. Kennedy, P. Baxter, and T. Belpaeme, "The robot who tried too hard: social behaviour of a robot tutor can negatively affect child learning," in Proceedings of the Tenth Annual ACM/IEEE International Conference on Human-Robot Interaction - HRI '15, 2015, pp. 67-74.

Social robots are finding increasing application in the domain of education, particularly for children, to support and augment learning opportunities. With an implicit assumption that social and adaptive behaviour is desirable, it is therefore of interest to determine precisely how these aspects of behaviour may be exploited in robots to support children in their learning. In this paper, we explore this issue by evaluating the effect of a social robot tutoring strategy with children learning about prime numbers. It is shown that the tutoring strategy itself leads to improvement, but that the presence of a robot employing this strategy amplifies this effect, resulting in significant learning. However, it was also found that children interacting with a robot using social and adaptive behaviours in addition to the teaching strategy did not learn a significant amount. These results indicate that while the presence of a physical robot leads to improved learning, caution is required when applying social behaviour to a robot in a tutoring context.

@inproceedings{lirolem24856,
booktitle = {Proceedings of the Tenth Annual ACM/IEEE International Conference on Human-Robot Interaction - HRI '15},
pages = {67--74},
month = {March},
author = {James Kennedy and Paul Baxter and Tony Belpaeme},
publisher = {ACM},
title = {The robot who tried too hard: social behaviour of a robot tutor can negatively affect child learning},
year = {2015},
abstract = {Social robots are finding increasing application in the domain of education, particularly for children, to support and augment learning opportunities. With an implicit assumption that social and adaptive behaviour is desirable, it is therefore of interest to determine precisely how these aspects of behaviour may be exploited in robots to support children in their learning. In this paper, we explore this issue by evaluating the effect of a social robot tutoring strategy with children learning about prime numbers. It is shown that the tutoring strategy itself leads to improvement, but that the presence of a robot employing this strategy amplifies this effect, resulting in significant learning. However, it was also found that children interacting with a robot using social and adaptive behaviours in addition to the teaching strategy did not learn a significant amount. These results indicate that while the presence of a physical robot leads to improved learning, caution is required when applying social behaviour to a robot in a tutoring context.},
url = {http://eprints.lincoln.ac.uk/24856/},
keywords = {ARRAY(0x56147f838c80)}
}
• T. Krajnik, J. Santos, and T. Duckett, "Life-long spatio-temporal exploration of dynamic environments," in European Conference on Mobile Robots 2015 (ECMR 15), 2015.

We propose a new idea for life-long mobile robot spatio-temporal exploration of dynamic environments. Our method assumes that the world is subject to perpetual change, which adds an extra, temporal dimension to the explored space and makes the exploration task a never-ending data-gathering process. To create and maintain a spatio-temporal model of a dynamic environment, the robot has to determine not only where, but also when to perform observations. We address the problem by application of information-theoretic exploration to world representations that model the uncertainty of environment states as probabilistic functions of time. We compare the performance of different exploration strategies and temporal models on real-world data gathered over the course of several months and show that combination of dynamic environment representations with information-gain exploration principles allows to create and maintain up-to-date models of constantly changing environments.

@inproceedings{lirolem17955,
month = {September},
booktitle = {European Conference on Mobile Robots 2015 (ECMR 15)},
title = {Life-long spatio-temporal exploration of dynamic environments},
year = {2015},
publisher = {IEEE},
author = {Tomas Krajnik and Joao Santos and Tom Duckett},
url = {http://eprints.lincoln.ac.uk/17955/},
keywords = {ARRAY(0x56147f6e9e60)},
abstract = {We propose a new idea for life-long mobile robot spatio-temporal exploration of dynamic environments.  Our method assumes that the world is subject to perpetual change, which adds an extra, temporal dimension to the explored space and makes the exploration task a never-ending data-gathering process. To create and maintain a spatio-temporal model of a dynamic environment, the robot has to determine not only where, but also when to perform observations.  We address the problem by application of information-theoretic exploration to world representations that model the uncertainty of environment states as probabilistic functions of time.

We compare the performance of different exploration strategies and temporal models on real-world data gathered over the course of several months and show that combination of dynamic environment representations with information-gain exploration principles allows to create and maintain up-to-date models of constantly changing environments.}
}
• T. Krajnik, J. P. Fentanes, O. M. Mozos, J. Ekekrantz, M. Hanheide, and T. Duckett, Long-term mobile robot localization in dynamic environments using spectral mapsAssociation for the Advancement of Artificial Intelligence (AAAI), 2015.

The video presents a novel approach for vision-based topological localisation in dynamic indoor environments. In contrast to other approaches that rely on static image features, our method explicitly models the temporal dynamics of the visibility of image features. The proposed spatio-temporal world model is able to predict the visibility of image features at different times of day, which allows to construct time-specific environment appearance models. This approach improves the robot localisation capabilities during long-term operations.

@misc{lirolem17950,
author = {Tomas Krajnik and Jaime Pulido Fentanes and Oscar Martinez Mozos and Johan Ekekrantz and Marc Hanheide and Tom Duckett},
publisher = {Association for the Advancement of Artificial Intelligence (AAAI)},
year = {2015},
title = {Long-term mobile robot localization in dynamic environments using spectral maps},
month = {January},
url = {http://eprints.lincoln.ac.uk/17950/},
keywords = {ARRAY(0x56147f6f9200)},
abstract = {The video presents a novel approach for vision-based topological localisation in dynamic indoor environments. In contrast to other approaches that rely on static image features, our method explicitly models the temporal dynamics of the visibility of image features. The proposed spatio-temporal world model is able to predict the visibility of image features at different times of day, which allows to construct time-specific environment appearance models. This approach improves the robot localisation capabilities during long-term operations.}
}
• T. Krajnik, P. deCristoforis, M. Nitsche, K. Kusumam, and T. Duckett, "Image features and seasons revisited," in European Conference on Mobile Robots 2015 (ECMR 15), 2015.

We present an evaluation of standard image features in the context of long-term visual teach-and-repeat mobile robot navigation, where the environment exhibits significant changes in appearance caused by seasonal weather variations and daily illumination changes. We argue that in the given long-term scenario, the viewpoint, scale and rotation invariance of the standard feature extractors is less important than their robustness to the mid- and long-term environment appearance changes. Therefore, we focus our evaluation on the robustness of image registration to variable lighting and naturally-occurring seasonal changes. We evaluate the image feature extractors on three datasets collected by mobile robots in two different outdoor environments over the course of one year. Based on this analysis, we propose a novel feature descriptor based on a combination of evolutionary algorithms and Binary Robust Independent Elementary Features, which we call GRIEF (Generated BRIEF). In terms of robustness to seasonal changes, the GRIEF feature descriptor outperforms the other ones while being computationally more efficient.

@inproceedings{lirolem17954,
year = {2015},
title = {Image features and seasons revisited},
author = {Tomas Krajnik and Pablo deCristoforis and Matias Nitsche and Keerthy Kusumam and Tom Duckett},
publisher = {IEEE},
month = {September},
booktitle = {European Conference on Mobile Robots 2015 (ECMR 15)},
url = {http://eprints.lincoln.ac.uk/17954/},
keywords = {ARRAY(0x56147f90d138)},
abstract = {We present an evaluation of standard image features in the context of long-term visual teach-and-repeat mobile robot navigation, where the environment exhibits significant changes in appearance caused by seasonal weather variations and daily illumination changes. We argue that in the given long-term scenario, the viewpoint, scale and rotation invariance of the standard feature extractors is less important than their robustness to the mid- and long-term environment appearance changes. Therefore, we focus our evaluation on the robustness of image registration to variable lighting and naturally-occurring seasonal changes.  We evaluate the image feature extractors on three datasets collected by mobile robots in two different outdoor environments over the course of one year. Based on this analysis, we propose a novel feature descriptor based on a combination of evolutionary algorithms and Binary Robust Independent Elementary Features, which we call GRIEF (Generated BRIEF). In terms of robustness to seasonal changes, the GRIEF feature descriptor outperforms the other ones while being computationally more efficient.}
}
• T. Krajnik, F. Arvin, A. E. Turgut, S. Yue, and T. Duckett, "COS\ensuremath\Phi: Vision-based artificial pheromone system for robotic swarms," in IEEE International Conference on Robotics and Automation (ICRA 2015), 2015.

We propose a novel spatio-temporal mobile-robot exploration method for dynamic, human-populated environments. In contrast to other exploration methods that model the environment as being static, our spatio-temporal exploration method creates and maintains a world model that not only represents the environment's structure, but also its dynamics over time. Consideration of the world dynamics adds an extra, temporal dimension to the explored space and makes the exploration task a never-ending data-gathering process to keep the robot's environment model up-to-date. Thus, the crucial question is not only where, but also when to observe the explored environment. We address the problem by application of information-theoretic exploration to world representations that model the environment states' uncertainties as probabilistic functions of time. The predictive ability of the spatio-temporal model allows the exploration method to decide not only where, but also when to make environment observations. To verify the proposed approach, an evaluation of several exploration strategies and spatio-temporal models was carried out using real-world data gathered over several months. The evaluation indicates that through understanding of the environment dynamics, the proposed spatio-temporal exploration method could predict which locations were going to change at a specific time and use this knowledge to guide the robot. Such an ability is crucial for long-term deployment of mobile robots in human-populated spaces that change over time.

@inproceedings{lirolem17952,
month = {May},
booktitle = {IEEE International Conference on Robotics and Automation (ICRA 2015)},
author = {Tomas Krajnik and Farshad Arvin and Ali Emre Turgut and Shigang Yue and Tom Duckett},
publisher = {IEEE},
title = {COS{\ensuremath{\Phi}}: Vision-based artificial pheromone system for robotic swarms},
year = {2015},
abstract = {We propose a novel spatio-temporal mobile-robot exploration method for dynamic, human-populated environments. In contrast to other exploration methods that model the environment as being static, our spatio-temporal exploration method creates and maintains a world model that not only represents the environment's structure, but also its dynamics over time.  Consideration of the world dynamics adds an extra, temporal dimension to the explored space and makes the exploration task a never-ending data-gathering process to keep the robot's environment model up-to-date.
Thus, the crucial question is not only where, but also when to observe the explored environment.
We address the problem by application of information-theoretic exploration to world representations that model the environment states' uncertainties as probabilistic functions of time. The predictive ability of the spatio-temporal model allows the exploration method to decide not only where, but also when to make environment observations.

To verify the proposed approach, an evaluation of several exploration strategies and spatio-temporal models was carried out using real-world data gathered over several months. The evaluation indicates that through understanding of the environment dynamics, the proposed spatio-temporal exploration method could predict which locations were going to change at a specific time and use this knowledge to guide the robot.  Such an ability is crucial for long-term deployment of mobile robots in human-populated spaces that change over time.},
url = {http://eprints.lincoln.ac.uk/17952/},
keywords = {ARRAY(0x56147f76df70)}
}
• T. Krajnik, M. Kulich, L. Mudrova, R. Ambrus, and T. Duckett, "Where's Waldo at time t? Using spatio-temporal models for mobile robot search," in IEEE International Conference on Robotics and Automation (ICRA), 2015, pp. 2140-2146.

We present a novel approach to mobile robot search for non-stationary objects in partially known environments. We formulate the search as a path planning problem in an environment where the probability of object occurrences at particular locations is a function of time. We propose to explicitly model the dynamics of the object occurrences by their frequency spectra. Using this spectral model, our path planning algorithm can construct plans that reflect the likelihoods of object locations at the time the search is performed. Three datasets collected over several months containing person and object occurrences in residential and office environments were chosen to evaluate the approach. Several types of spatio-temporal models were created for each of these datasets and the efficiency of the search method was assessed by measuring the time it took to locate a particular object. The results indicate that modeling the dynamics of object occurrences reduces the search time by 25\% to 65\% compared to maps that neglect these dynamics.

@inproceedings{lirolem17949,
year = {2015},
title = {Where's Waldo at time t? Using spatio-temporal models for mobile robot search},
publisher = {Institute of Electrical and Electronics Engineers},
author = {Tomas Krajnik and Miroslav Kulich and Lenka Mudrova and Rares Ambrus and Tom Duckett},
month = {May},
pages = {2140--2146},
booktitle = {IEEE International Conference on Robotics and Automation (ICRA)},
url = {http://eprints.lincoln.ac.uk/17949/},
keywords = {ARRAY(0x56147f745080)},
abstract = {We present a novel approach to mobile robot search for non-stationary objects in partially known environments. We formulate the search as a path planning problem in an environment where the probability of object occurrences at particular locations is a function of time. We propose to explicitly model the dynamics of the object occurrences by their frequency spectra. Using this spectral model, our path planning algorithm can construct plans that reflect the likelihoods of object locations at the time the search is performed. Three datasets collected over several months containing person and object occurrences in residential and office environments were chosen to evaluate the approach. Several types of spatio-temporal models were created for each of these datasets and the efficiency of the search method was assessed by measuring the time it took to locate a particular object. The results indicate that modeling the dynamics of object occurrences reduces the search time by 25\% to 65\% compared to maps that neglect these dynamics.}
}
• T. Krajnik, J. P. Fentanes, J. Santos, K. Kusumam, and T. Duckett, "FreMEn: frequency map enhancement for long-term mobile robot autonomy in changing environments," in ICRA 2015 Workshop on Visual Place Recognition in Changing Environments, 2015.

We present a method for introducing representation of dynamics into environment models that were originally tailored to represent static scenes. Rather than using a fixed probability value, the method models the uncertainty of the elementary environment states by probabilistic functions of time. These are composed of combinations of harmonic functions, which are obtained by means of frequency analysis. The use of frequency analysis allows to integrate long-term observations into memory-efficient spatio-temporal models that reflect the mid- to long-term environment dynamics. These frequency-enhanced spatio-temporal models allow to predict the future environment states, which improves the efficiency of mobile robot operation in changing environments. In a series of experiments performed over periods of days to years, we demonstrate that the proposed approach improves localization, path planning and exploration.

@inproceedings{lirolem17953,
booktitle = {ICRA 2015 Workshop on Visual Place Recognition in Changing Environments},
month = {May},
title = {FreMEn: frequency map enhancement for long-term mobile robot autonomy in changing environments},
year = {2015},
author = {Tomas Krajnik and Jaime Pulido Fentanes and Joao Santos and Keerthy Kusumam and Tom Duckett},
publisher = {IEEE},
url = {http://eprints.lincoln.ac.uk/17953/},
keywords = {ARRAY(0x56147f6a3538)},
abstract = {We present a method for introducing representation of dynamics into environment models that were originally tailored to represent static scenes. Rather than using a fixed probability value, the method models the uncertainty of the elementary environment states by probabilistic functions of time. These are composed of combinations of harmonic functions, which are obtained by means of frequency analysis. The use of frequency analysis allows to integrate long-term observations into memory-efficient spatio-temporal models that reflect the mid- to long-term environment dynamics. These frequency-enhanced spatio-temporal models allow to predict the future environment states, which improves the efficiency of mobile robot operation in changing environments.   In a series of experiments performed over periods of days to years, we demonstrate that the proposed approach improves localization, path planning and exploration.}
}
• O. Kroemer, C. Daniel, G. Neumann, H. V. Hoof, and J. Peters, "Towards learning hierarchical skills for multi-phase manipulation tasks," in International Conference on Robotics and Automation (ICRA), 2015, pp. 1503-1510.

Most manipulation tasks can be decomposed into a sequence of phases, where the robot?s actions have different effects in each phase. The robot can perform actions to transition between phases and, thus, alter the effects of its actions, e.g. grasp an object in order to then lift it. The robot can thus reach a phase that affords the desired manipulation. In this paper, we present an approach for exploiting the phase structure of tasks in order to learn manipulation skills more efficiently. Starting with human demonstrations, the robot learns a probabilistic model of the phases and the phase transitions. The robot then employs model-based reinforcement learning to create a library of motor primitives for transitioning between phases. The learned motor primitives generalize to new situations and tasks. Given this library, the robot uses a value function approach to learn a high-level policy for sequencing the motor primitives. The proposed method was successfully evaluated on a real robot performing a bimanual grasping task.

@inproceedings{lirolem25759,
year = {2015},
title = {Towards learning hierarchical skills for multi-phase manipulation tasks},
volume = {2015-J},
journal = {Proceedings - IEEE International Conference on Robotics and Automation},
number = {June},
month = {June},
booktitle = {International Conference on Robotics and Automation (ICRA)},
pages = {1503--1510},
publisher = {IEEE},
author = {Oliver Kroemer and Christian Daniel and Gerhard Neumann and Herke Van Hoof and Jan Peters},
abstract = {Most manipulation tasks can be decomposed into
a sequence of phases, where the robot?s actions have different
effects in each phase. The robot can perform actions to
transition between phases and, thus, alter the effects of its
actions, e.g. grasp an object in order to then lift it. The robot
can thus reach a phase that affords the desired manipulation.
In this paper, we present an approach for exploiting the
phase structure of tasks in order to learn manipulation skills
more efficiently. Starting with human demonstrations, the robot
learns a probabilistic model of the phases and the phase
transitions. The robot then employs model-based reinforcement
learning to create a library of motor primitives for transitioning
between phases. The learned motor primitives generalize to new
situations and tasks. Given this library, the robot uses a value
function approach to learn a high-level policy for sequencing
the motor primitives. The proposed method was successfully
evaluated on a real robot performing a bimanual grasping task.},
keywords = {ARRAY(0x56147f885a38)},
url = {http://eprints.lincoln.ac.uk/25759/}
}
• O. Kroemer, C. Daniel, G. Neumann, V. H. Hoof, and J. Peters, "Towards learning hierarchical skills for multi-phase manipulation tasks," in IEEE International Conference on Robotics and Automation (ICRA), 2015, 2015, pp. 1503-1510.

Most manipulation tasks can be decomposed into a sequence of phases, where the robot's actions have different effects in each phase. The robot can perform actions to transition between phases and, thus, alter the effects of its actions, e.g. grasp an object in order to then lift it. The robot can thus reach a phase that affords the desired manipulation. In this paper, we present an approach for exploiting the phase structure of tasks in order to learn manipulation skills more efficiently. Starting with human demonstrations, the robot learns a probabilistic model of the phases and the phase transitions. The robot then employs model-based reinforcement learning to create a library of motor primitives for transitioning between phases. The learned motor primitives generalize to new situations and tasks. Given this library, the robot uses a value function approach to learn a high-level policy for sequencing the motor primitives. The proposed method was successfully evaluated on a real robot performing a bimanual grasping task.

@inproceedings{lirolem25696,
month = {May},
pages = {1503--1510},
booktitle = {IEEE International Conference on Robotics and Automation (ICRA), 2015},
author = {O. Kroemer and C. Daniel and G. Neumann and H. Van Hoof and J. Peters},
publisher = {IEEE},
title = {Towards learning hierarchical skills for multi-phase manipulation tasks},
year = {2015},
volume = {2015-J},
number = {June},
abstract = {Most manipulation tasks can be decomposed into a sequence of phases, where the robot's actions have different effects in each phase. The robot can perform actions to transition between phases and, thus, alter the effects of its actions, e.g. grasp an object in order to then lift it. The robot can thus reach a phase that affords the desired manipulation. In this paper, we present an approach for exploiting the phase structure of tasks in order to learn manipulation skills more efficiently. Starting with human demonstrations, the robot learns a probabilistic model of the phases and the phase transitions. The robot then employs model-based reinforcement learning to create a library of motor primitives for transitioning between phases. The learned motor primitives generalize to new situations and tasks. Given this library, the robot uses a value function approach to learn a high-level policy for sequencing the motor primitives. The proposed method was successfully evaluated on a real robot performing a bimanual grasping task.},
keywords = {ARRAY(0x56147f85b460)},
url = {http://eprints.lincoln.ac.uk/25696/}
}
• A. Kucukyilmaz and Y. Demiris, "One-shot assistance estimation from expert demonstrations for a shared control wheelchair system," in Proceedings of the 24th IEEE International Symposium on Robot and Human Interactive Communication 2015 (RO-MAN'15), 2015.

An emerging research problem in the field of assistive robotics is the design of methodologies that allow robots to provide human-like assistance to the users. Especially within the rehabilitation domain, a grand challenge is to program a robot to mimic the operation of an occupational therapist, intervening with the user when necessary so as to improve the therapeutic power of the assistive robotic system. We propose a method to estimate assistance policies from expert demonstrations to present human-like intervention during navigation in a powered wheelchair setup. For this purpose, we constructed a setting, where a human offers assistance to the user over a haptic shared control system. The robot learns from human assistance demonstrations while the user is actively driving the wheelchair in an unconstrained environment. We train a Gaussian process regression model to learn assistance commands given past and current actions of the user and the state of the environment. The results indicate that the model can estimate human assistance after only a single demonstration, i.e. in one-shot, so that the robot can help the user by selecting the appropriate assistance in a human-like fashion.

@inproceedings{lirolem29367,
title = {One-shot assistance estimation from expert demonstrations for a shared control wheelchair system},
year = {2015},
journal = {Proceedings of the 24th IEEE International Symposium on Robot and Human Interactive Communication 2015 (RO-MAN'15)},
publisher = {IEEE},
author = {Ayse Kucukyilmaz and Yiannis Demiris},
booktitle = {Proceedings of the 24th IEEE International Symposium on Robot and Human Interactive Communication 2015 (RO-MAN'15)},
month = {August},
abstract = {An emerging research problem in the field of assistive robotics is the design of methodologies that allow robots to provide human-like assistance to the users. Especially within the rehabilitation domain, a grand challenge is to program a robot to mimic the operation of an occupational therapist, intervening with the user when necessary so as to improve the therapeutic power of the assistive robotic system. We propose a method to estimate assistance policies from expert demonstrations to present human-like intervention during navigation in a powered wheelchair setup. For this purpose, we constructed a setting, where a human offers assistance to the user over a haptic shared control system. The robot learns from human assistance demonstrations while the user is actively driving the wheelchair in an unconstrained environment. We train a Gaussian process regression model to learn assistance commands given past and current actions of the user and the state of the environment. The results indicate that the model can estimate human assistance after only a single demonstration, i.e. in one-shot, so that the robot can help the user by selecting the appropriate assistance in a human-like fashion.},
keywords = {ARRAY(0x56147f995ef8)},
url = {http://eprints.lincoln.ac.uk/29367/}
}
• H. Li, J. Peng, and S. Yue, "The sparsity of underdetermined linear system via lp minimization for 0 \ensuremath< p \ensuremath< 1," Mathematical Problems in Engineering, vol. 2015, 2015.

The sparsity problems have attracted a great deal of attention in recent years, which aim to find the sparsest solution of a representation or an equation. In the paper, we mainly study the sparsity of underdetermined linear system via lp minimization for 0\ensuremath

0 such that the following conclusions hold when p \ensuremath< \ensuremath\gamma(A, b): (1) the problem (pp) generates sparser solution as the value of p decreases; (2) the sparsest optimal solution to the problem (pp) is unique under the sense of absolute value permutation; (3) let X1 and X2 be the sparsest optimal solution to the problems (pp1) and (pp2) , respectively, and let X1 not be the absolute value permutation of X2. Then there exist t1,t2 \ensuremath\epsilon [p1,p2] such that X1 is the sparsest optimal solution to the problem (pt) (?t \ensuremath\epsilon [p1, t1]) and X2 is the sparsest optimal solution to the problem (pt) (?t \ensuremath\epsilon (t2, p2]).

@article{lirolem17577,
title = {The sparsity of underdetermined linear system via lp minimization for 0 {\ensuremath{}} 0 such that the following conclusions hold when p {\ensuremath{
• P. Lightbody, C. Dondrup, and M. Hanheide, "Make me a sandwich! Intrinsic human identification from their course of action," in Towards a Framework for Joint Action, 2015.

In order to allow humans and robots to work closely together and as a team, we need to equip robots not only with a general understanding of joint action, but also with an understanding of the idiosyncratic differences in the ways humans perform certain tasks. This will allow robots to be better colleagues, by anticipating an individual's actions, and acting accordingly. In this paper, we present a way of encoding a human's course of action as a probabilistic sequence of qualitative states, and show that such a model can be employed to identify individual humans from their respective course of action, even when accomplishing the very same goal state. We conclude from our findings that there are significant variations in the ways humans accomplish the very same task, and that our representation could in future work inform robot (task) planning in collaborative settings.

@inproceedings{lirolem19696,
author = {Peter Lightbody and Christian Dondrup and Marc Hanheide},
title = {Make me a sandwich! Intrinsic human identification from their course of action},
year = {2015},
booktitle = {Towards a Framework for Joint Action},
month = {October},
url = {http://eprints.lincoln.ac.uk/19696/},
keywords = {ARRAY(0x56147f5f01f8)},
abstract = {In order to allow humans and robots to work closely together and as a team, we need to equip robots not only with a general understanding of joint action, but also with an understanding of the idiosyncratic differences in the ways humans perform certain tasks. This will allow robots to be better colleagues, by anticipating an individual's actions, and acting accordingly. In this paper, we present a way of encoding a human's course of action as a probabilistic sequence of qualitative states, and show that such a model can be employed to identify individual humans from their respective course of action, even when accomplishing the very same goal state. We conclude from our findings that there are significant variations in the ways humans accomplish the very same task, and that our representation could in future work inform robot (task) planning in collaborative settings.}
}
• R. Lioutikov, G. Neumann, G. Maeda, and J. Peters, "Probabilistic segmentation applied to an assembly task," in 15th IEEE-RAS International Conference on Humanoid Robots, 2015, pp. 533-540.

Movement primitives are a well established approach for encoding and executing robot movements. While the primitives themselves have been extensively researched, the concept of movement primitive libraries has not received as much attention. Libraries of movement primitives represent the skill set of an agent and can be queried and sequenced in order to solve specific tasks. The goal of this work is to segment unlabeled demonstrations into an optimal set of skills. Our novel approach segments the demonstrations while learning a probabilistic representation of movement primitives. The method differs from current approaches by taking advantage of the often neglected, mutual dependencies between the segments contained in the demonstrations and the primitives to be encoded. Therefore, improving the combined quality of both segmentation and skill learning. Furthermore, our method allows incorporating domain specific insights using heuristics, which are subsequently evaluated and assessed through probabilistic inference methods. We demonstrate our method on a real robot application, where the robot segments demonstrations of a chair assembly task into a skill library. The library is subsequently used to assemble the chair in an order not present in the demonstrations.

@inproceedings{lirolem25751,
title = {Probabilistic segmentation applied to an assembly task},
volume = {2015-D},
journal = {IEEE-RAS International Conference on Humanoid Robots},
year = {2015},
month = {November},
pages = {533--540},
booktitle = {15th IEEE-RAS International Conference on Humanoid Robots},
author = {R. Lioutikov and G. Neumann and G. Maeda and J. Peters},
url = {http://eprints.lincoln.ac.uk/25751/},
keywords = {ARRAY(0x56147f6f1be0)},
abstract = {Movement primitives are a well established approach
for encoding and executing robot movements. While
the primitives themselves have been extensively researched, the
concept of movement primitive libraries has not received as
much attention. Libraries of movement primitives represent
the skill set of an agent and can be queried and sequenced in
order to solve specific tasks. The goal of this work is to segment
unlabeled demonstrations into an optimal set of skills. Our
novel approach segments the demonstrations while learning
a probabilistic representation of movement primitives. The
method differs from current approaches by taking advantage of
the often neglected, mutual dependencies between the segments
contained in the demonstrations and the primitives to be encoded.
Therefore, improving the combined quality of both segmentation
and skill learning. Furthermore, our method allows
incorporating domain specific insights using heuristics, which
are subsequently evaluated and assessed through probabilistic
inference methods. We demonstrate our method on a real robot
application, where the robot segments demonstrations of a chair
assembly task into a skill library. The library is subsequently
used to assemble the chair in an order not present in the
demonstrations.}
}
• C. E. Madan, A. Kucukyilmaz, T. M. Sezgin, and C. Basdogan, "Recognition of haptic interaction patterns in dyadic joint object manipulation," Haptics, IEEE Transactions on, vol. 8, iss. 1, pp. 54-66, 2015.

The development of robots that can physically cooperate with humans has attained interest in the last decades. Obviously, this effort requires a deep understanding of the intrinsic properties of interaction. Up to now, many researchers have focused on inferring human intents in terms of intermediate or terminal goals in physical tasks. On the other hand, working side by side with people, an autonomous robot additionally needs to come up with in-depth information about underlying haptic interaction patterns that are typically encountered during human-human cooperation. However, to our knowledge, no study has yet focused on characterizing such detailed information. In this sense, this work is pioneering as an effort to gain deeper understanding of interaction patterns involving two or more humans in a physical task. We present a labeled human-human-interaction dataset, which captures the interaction of two humans, who collaboratively transport an object in an haptics-enabled virtual environment. In the light of information gained by studying this dataset, we propose that the actions of cooperating partners can be examined under three interaction types: In any cooperative task, the interacting humans either 1) work in harmony, 2) cope with conflicts, or 3) remain passive during interaction. In line with this conception, we present a taxonomy of human interaction patterns; then propose five different feature sets, comprising force-, velocity-and power-related information, for the classification of these patterns. Our evaluation shows that using a multi-class support vector machine (SVM) classifier, we can accomplish a correct classification rate of 86 percent for the identification of interaction patterns, an accuracy obtained by fusing a selected set of most informative features by Minimum Redundancy Maximum Relevance (mRMR) feature selection method.

@article{lirolem29368,
number = {1},
year = {2015},
journal = {Haptics, IEEE Transactions on},
volume = {8},
title = {Recognition of haptic interaction patterns in dyadic joint object manipulation},
author = {Cigil Ece Madan and Ayse Kucukyilmaz and T.M. Sezgin and C. Basdogan},
publisher = {Institute of Electrical and Electronics Engineers (IEEE)},
pages = {54--66},
month = {January},
abstract = {The development of robots that can physically cooperate with humans has attained interest in the last decades. Obviously, this effort requires a deep understanding of the intrinsic properties of interaction. Up to now, many researchers have focused on inferring human intents in terms of intermediate or terminal goals in physical tasks. On the other hand, working side by side with people, an autonomous robot additionally needs to come up with in-depth information about underlying haptic interaction patterns that are typically encountered during human-human cooperation. However, to our knowledge, no study has yet focused on characterizing such detailed information. In this sense, this work is pioneering as an effort to gain deeper understanding of interaction patterns involving two or more humans in a physical task. We present a labeled human-human-interaction dataset, which captures the interaction of two humans, who collaboratively transport an object in an haptics-enabled virtual environment. In the light of information gained by studying this dataset, we propose that the actions of cooperating partners can be examined under three interaction types: In any cooperative task, the interacting humans either 1) work in harmony, 2) cope with conflicts, or 3) remain passive during interaction. In line with this conception, we present a taxonomy of human interaction patterns; then propose five different feature sets, comprising force-, velocity-and power-related information, for the classification of these patterns. Our evaluation shows that using a multi-class support vector machine (SVM) classifier, we can accomplish a correct classification rate of 86 percent for the identification of interaction patterns, an accuracy obtained by fusing a selected set of most informative features by Minimum Redundancy Maximum Relevance (mRMR) feature selection method.},
url = {http://eprints.lincoln.ac.uk/29368/},
keywords = {ARRAY(0x56147f779728)}
}
• N. Mavridis, N. Bellotto, K. Iliopoulos, and N. V. de Weghe, "QTC3D: extending the qualitative trajectory calculus to three dimensions," Information Sciences, vol. 322, pp. 20-30, 2015.

Spatial interactions between agents (humans, animals, or machines) carry information of high value to human or electronic observers. However, not all the information contained in a pair of continuous trajectories is important and thus the need for qualitative descriptions of interaction trajectories arises. The Qualitative Trajectory Calculus (QTC) (Van de Weghe, 2004) is a promising development towards this goal. Numerous variants of QTC have been proposed in the past and QTC has been applied towards analyzing various interaction domains. However, an inherent limitation of those QTC variations that deal with lateral movements is that they are limited to two-dimensional motion; therefore, complex three-dimensional interactions, such as those occurring between flying planes or birds, cannot be captured. Towards that purpose, in this paper QTC3D is presented: a novel qualitative trajectory calculus that can deal with full three-dimensional interactions. QTC3D is based on transformations of the Frenet-Serret frames accompanying the trajectories of the moving objects. Apart from the theoretical exposition, including definition and properties, as well as computational aspects, we also present an application of QTC3D towards modeling bird flight. Thus, the power of QTC is now extended to the full dimensionality of physical space, enabling succinct yet rich representations of spatial interactions between agents.

@article{lirolem17596,
volume = {322},
title = {QTC3D: extending the qualitative trajectory calculus to three dimensions},
journal = {Information Sciences},
year = {2015},
month = {November},
pages = {20--30},
author = {Nikolaos Mavridis and Nicola Bellotto and Konstantinos Iliopoulos and Nico Van de Weghe},
publisher = {Elsevier},
abstract = {Spatial interactions between agents (humans, animals, or machines) carry information of high value to human or electronic observers. However, not all the information contained in a pair of continuous trajectories is important and thus the need for qualitative descriptions of interaction trajectories arises. The Qualitative Trajectory Calculus (QTC) (Van de Weghe, 2004) is a promising development towards this goal. Numerous variants of QTC have been proposed in the past and QTC has been applied towards analyzing various interaction domains. However, an inherent limitation of those QTC variations that deal with lateral movements is that they are limited to two-dimensional motion; therefore, complex three-dimensional interactions, such as those occurring between flying planes or birds, cannot be captured. Towards that purpose, in this paper QTC3D is presented: a novel qualitative trajectory calculus that can deal with full three-dimensional interactions. QTC3D is based on transformations of the Frenet-Serret frames accompanying the trajectories of the moving objects. Apart from the theoretical exposition, including definition and properties, as well as computational aspects, we also present an application of QTC3D towards modeling bird flight. Thus, the power of QTC is now extended to the full dimensionality of physical space, enabling succinct yet rich representations of spatial interactions between agents.},
url = {http://eprints.lincoln.ac.uk/17596/},
keywords = {ARRAY(0x56147f9d8be8)}
}
• M. Nitsche, T. Krajnik, P. Cizek, M. Mejail, and T. Duckett, "WhyCon: an efficient, marker-based localization system," in IROS Workshop on Aerial Open-source Robotics, 2015.

We present an open-source marker-based localization system intended as a low-cost easy-to-deploy solution for aerial and swarm robotics. The main advantage of the presented method is its high computational efficiency, which allows its deployment on small robots with limited computational resources. Even on low-end computers, the core component of the system can detect and estimate 3D positions of hundreds of black and white markers at the maximum frame-rate of standard cameras. The method is robust to changing lighting conditions and achieves accuracy in the order of millimeters to centimeters. Due to its reliability, simplicity of use and availability as an open-source ROS module (http://purl.org/robotics/whycon), the system is now used in a number of aerial robotics projects where fast and precise relative localization is required.

@inproceedings{lirolem18877,
author = {Matias Nitsche and Tomas Krajnik and Petr Cizek and Marta Mejail and Tom Duckett},
title = {WhyCon: an efficient, marker-based localization system},
year = {2015},
booktitle = {IROS Workshop on Aerial Open-source Robotics},
month = {September},
url = {http://eprints.lincoln.ac.uk/18877/},
keywords = {ARRAY(0x56147f56af18)},
abstract = {We present an open-source marker-based localization system intended as a low-cost easy-to-deploy solution for aerial and swarm robotics. The main advantage of the presented method is its high computational efficiency, which allows its deployment on small robots with limited computational resources. Even on low-end computers, the core component of the system can detect and estimate 3D positions of hundreds of black and white markers at the maximum frame-rate of standard cameras. The method is robust to changing lighting conditions and achieves accuracy in the order of millimeters to centimeters. Due to its reliability, simplicity of use and availability as an open-source ROS module (http://purl.org/robotics/whycon), the system is now used in a number of aerial robotics projects where fast and precise relative localization is required.}
}
• A. Paraschos, G. Neumann, and J. Peters, "A probabilistic approach to robot trajectory generation," in International Conference on Humanoid Robots (HUMANOIDS), 2015, pp. 477-483.

Motor Primitives (MPs) are a promising approach for the data-driven acquisition as well as for the modular and re-usable generation of movements. However, a modular control architecture with MPs is only effective if the MPs support co-activation as well as continuously blending the activation from one MP to the next. In addition, we need efficient mechanisms to adapt a MP to the current situation. Common approaches to movement primitives lack such capabilities or their implementation is based on heuristics. We present a probabilistic movement primitive approach that overcomes the limitations of existing approaches. We encode a primitive as a probability distribution over trajectories. The representation as distribution has several beneficial properties. It allows encoding a time-varying variance profile. Most importantly, it allows performing new operations -- a product of distributions for the co-activation of MPs conditioning for generalizing the MP to different desired targets. We derive a feedback controller that reproduces a given trajectory distribution in closed form. We compare our approach to the existing state-of-the art and present real robot results for learning from demonstration.

@inproceedings{lirolem25755,
number = {Februa},
year = {2015},
volume = {2015-F},
title = {A probabilistic approach to robot trajectory generation},
journal = {IEEE-RAS International Conference on Humanoid Robots},
author = {A. Paraschos and Gerhard Neumann and J. Peters},
publisher = {IEEE},
booktitle = {International Conference on Humanoid Robots (HUMANOIDS)},
pages = {477--483},
month = {February},
abstract = {Motor Primitives (MPs) are a promising approach
for the data-driven acquisition as well as for the modular and
re-usable generation of movements. However, a modular control
architecture with MPs is only effective if the MPs support
co-activation as well as continuously blending the activation
from one MP to the next. In addition, we need efficient
mechanisms to adapt a MP to the current situation. Common
approaches to movement primitives lack such capabilities or
their implementation is based on heuristics. We present a
probabilistic movement primitive approach that overcomes the
limitations of existing approaches. We encode a primitive as a
probability distribution over trajectories. The representation as
distribution has several beneficial properties. It allows encoding
a time-varying variance profile. Most importantly, it allows
performing new operations {--} a product of distributions for
the co-activation of MPs conditioning for generalizing the MP
to different desired targets. We derive a feedback controller
that reproduces a given trajectory distribution in closed form.
We compare our approach to the existing state-of-the art and
present real robot results for learning from demonstration.},
url = {http://eprints.lincoln.ac.uk/25755/},
keywords = {ARRAY(0x56147f731638)}
}
• A. Paraschos, E. Rueckert, J. Peters, and G. Neumann, "Model-free Probabilistic Movement Primitives for physical interaction," in IEEE/RSJ Conference on Intelligent Robots and Systems (IROS), 2015, pp. 2860-2866.

Physical interaction in robotics is a complex problem that requires not only accurate reproduction of the kinematic trajectories but also of the forces and torques exhibited during the movement. We base our approach on Movement Primitives (MP), as MPs provide a framework for modelling complex movements and introduce useful operations on the movements, such as generalization to novel situations, time scaling, and others. Usually, MPs are trained with imitation learning, where an expert demonstrates the trajectories. However, MPs used in physical interaction either require additional learning approaches, e.g., reinforcement learning, or are based on handcrafted solutions. Our goal is to learn and generate movements for physical interaction that are learned with imitation learning, from a small set of demonstrated trajectories. The Probabilistic Movement Primitives (ProMPs) framework is a recent MP approach that introduces beneficial properties, such as combination and blending of MPs, and represents the correlations present in the movement. The ProMPs provides a variable stiffness controller that reproduces the movement but it requires a dynamics model of the system. Learning such a model is not a trivial task, and, therefore, we introduce the model-free ProMPs, that are learning jointly the movement and the necessary actions from a few demonstrations. We derive a variable stiffness controller analytically. We further extent the ProMPs to include force and torque signals, necessary for physical interaction. We evaluate our approach in simulated and real robot tasks.

@inproceedings{lirolem25752,
title = {Model-free Probabilistic Movement Primitives for physical interaction},
journal = {IEEE International Conference on Intelligent Robots and Systems},
volume = {2015-D},
year = {2015},
author = {A. Paraschos and E. Rueckert and J. Peters and G. Neumann},
booktitle = {IEEE/RSJ Conference on Intelligent Robots and Systems (IROS)},
pages = {2860--2866},
month = {September},
abstract = {Physical interaction in robotics is a complex problem
that requires not only accurate reproduction of the kinematic
trajectories but also of the forces and torques exhibited
during the movement. We base our approach on Movement
Primitives (MP), as MPs provide a framework for modelling
complex movements and introduce useful operations on the
movements, such as generalization to novel situations, time
scaling, and others. Usually, MPs are trained with imitation
learning, where an expert demonstrates the trajectories. However,
MPs used in physical interaction either require additional
learning approaches, e.g., reinforcement learning, or are based
on handcrafted solutions. Our goal is to learn and generate
movements for physical interaction that are learned with imitation
learning, from a small set of demonstrated trajectories.
The Probabilistic Movement Primitives (ProMPs) framework
is a recent MP approach that introduces beneficial properties,
such as combination and blending of MPs, and represents the
correlations present in the movement. The ProMPs provides
a variable stiffness controller that reproduces the movement
but it requires a dynamics model of the system. Learning such
a model is not a trivial task, and, therefore, we introduce the
model-free ProMPs, that are learning jointly the movement and
the necessary actions from a few demonstrations. We derive
a variable stiffness controller analytically. We further extent
the ProMPs to include force and torque signals, necessary for
physical interaction. We evaluate our approach in simulated
keywords = {ARRAY(0x56147f58c7d0)},
url = {http://eprints.lincoln.ac.uk/25752/}
}
• J. Peng, S. Yue, and H. Li, "NP/CMP equivalence: a phenomenon hidden among sparsity models l\_\0\ minimization and l\_\p\ minimization for information processing," IEEE Transactions on Information Theory, vol. 61, iss. 7, pp. 4028-4033, 2015.

In this paper, we have proved that in every underdetermined linear system Ax = b, there corresponds a constant p*(A, b) \ensuremath> 0 such that every solution to the l p-norm minimization problem also solves the l0-norm minimization problem whenever 0 \ensuremath

@article{lirolem17877,
pages = {4028--4033},
month = {June},
author = {Jigen Peng and Shigang Yue and Haiyang Li},
publisher = {IEEE},
number = {7},
year = {2015},
journal = {IEEE Transactions on Information Theory},
title = {NP/CMP equivalence: a phenomenon hidden among sparsity models l\_\{0\} minimization and l\_\{p\} minimization for information processing},
volume = {61},
abstract = {In this paper, we have proved that in every underdetermined linear system Ax = b, there corresponds a constant p*(A, b) {\ensuremath{>}} 0 such that every solution to the l p-norm minimization problem also solves the l0-norm minimization problem whenever 0 {\ensuremath{
• E. Rueckert, J. Mundo, A. Paraschos, J. Peters, and G. Neumann, "Extracting low-dimensional control variables for movement primitives," in IEEE International Conference on Robotics and Automation 2015, 2015, pp. 1511-1518.

Movement primitives (MPs) provide a powerful framework for data driven movement generation that has been successfully applied for learning from demonstrations and robot reinforcement learning. In robotics we often want to solve a multitude of different, but related tasks. As the parameters of the primitives are typically high dimensional, a common practice for the generalization of movement primitives to new tasks is to adapt only a small set of control variables, also called meta parameters, of the primitive. Yet, for most MP representations, the encoding of these control variables is pre-coded in the representation and can not be adapted to the considered tasks. In this paper, we want to learn the encoding of task-specific control variables also from data instead of relying on fixed meta-parameter representations. We use hierarchical Bayesian models (HBMs) to estimate a low dimensional latent variable model for probabilistic movement primitives (ProMPs), which is a recent movement primitive representation. We show on two real robot datasets that ProMPs based on HBMs outperform standard ProMPs in terms of generalization and learning from a small amount of data and also allows for an intuitive analysis of the movement. We also extend our HBM by a mixture model, such that we can model different movement types in the same dataset.

@inproceedings{lirolem25760,
pages = {1511--1518},
booktitle = {IEEE International Conference on Robotics and Automation 2015},
month = {May},
author = {E. Rueckert and J. Mundo and A. Paraschos and J. Peters and Gerhard Neumann},
number = {June},
year = {2015},
journal = {Proceedings - IEEE International Conference on Robotics and Automation},
volume = {2015-J},
title = {Extracting low-dimensional control variables for movement primitives},
abstract = {Movement primitives (MPs) provide a powerful framework for data driven movement generation that has been successfully applied for learning from demonstrations and robot reinforcement learning. In robotics we often want to solve a multitude of different, but related tasks. As the parameters of the primitives are typically high dimensional, a common practice for the generalization of movement primitives to new tasks is to adapt only a small set of control variables, also called meta parameters, of the primitive. Yet, for most MP representations, the encoding of these control variables is pre-coded in the representation and can not be adapted to the considered tasks. In this paper, we want to learn the encoding of task-specific control variables also from data instead of relying on fixed meta-parameter representations. We use hierarchical Bayesian models (HBMs) to estimate a low dimensional latent variable model for probabilistic movement primitives (ProMPs), which is a recent movement primitive representation. We show on two real robot datasets that ProMPs based on HBMs outperform standard ProMPs in terms of generalization and learning from a small amount of data and also allows for an intuitive analysis of the movement. We also extend our HBM by a mixture model, such that we can model different movement types in the same dataset.},
url = {http://eprints.lincoln.ac.uk/25760/},
keywords = {ARRAY(0x56147f79cb28)}
}
• V. Sandulescu, S. Andrews, D. Ellis, N. Bellotto, and O. M. Mozos, "Stress detection using wearable physiological sensors," Lecture Notes in Computer Science, vol. 9107, pp. 526-532, 2015.

As the population increases in the world, the ratio of health carers is rapidly decreasing. Therefore, there is an urgent need to create new technologies to monitor the physical and mental health of people during their daily life. In particular, negative mental states like depression and anxiety are big problems in modern societies, usually due to stressful situations during everyday activities including work. This paper presents a machine learning approach for stress detection on people using wearable physiological sensors with the ?final aim of improving their quality of life. The presented technique can monitor the state of the subject continuously and classify it into "stressful" or "non-stressful" situations. Our classification results show that this method is a good starting point towards real-time stress detection.

@article{lirolem17143,
title = {Stress detection using wearable physiological sensors},
year = {2015},
journal = {Lecture Notes in Computer Science},
volume = {9107},
month = {June},
pages = {526--532},
author = {Virginia Sandulescu and Sally Andrews and David Ellis and Nicola Bellotto and Oscar Martinez Mozos},
publisher = {Springer verlag},
note = {Series: Lecture Notes in Computer Science
Artificial Computation in Biology and Medicine: International Work-Conference on the Interplay Between Natural and Artificial Computation, IWINAC 2015, Elche, Spain, June 1-5, 2015, Proceedings, Part I},
abstract = {As the population increases in the world, the ratio of health carers is rapidly decreasing. Therefore, there is an urgent need to create new technologies to monitor the physical and mental health of people during their  daily life. In particular, negative mental states like depression and anxiety are big problems in modern societies, usually due to stressful situations during everyday activities including work. This paper presents a machine learning approach for stress detection on people using wearable physiological sensors with the ?final aim of improving their quality of life. The presented technique can monitor the state of the subject continuously and classify it into "stressful" or "non-stressful" situations. Our classification results show that this method is a good starting point towards real-time stress detection.},
keywords = {ARRAY(0x56147f51e760)},
url = {http://eprints.lincoln.ac.uk/17143/}
}
• J. Santos, T. Krajnik, J. P. Fentanes, and T. Duckett, "Lifelong exploration of dynamic environments," in IEEE International Conference on Robotics and Automation (ICRA), 2015.

We propose a novel spatio-temporal mobile-robot exploration method for dynamic, human-populated environments. In contrast to other exploration methods that model the environment as being static, our spatio-temporal exploration method creates and maintains a world model that not only represents the environment's structure, but also its dynamics over time. Consideration of the world dynamics adds an extra, temporal dimension to the explored space and makes the exploration task a never-ending data-gathering process to keep the robot's environment model up-to-date. Thus, the crucial question is not only where, but also when to observe the explored environment. We address the problem by application of information-theoretic exploration to world representations that model the environment states' uncertainties as probabilistic functions of time. The predictive ability of the spatio-temporal model allows the exploration method to decide not only where, but also when to make environment observations. To verify the proposed approach, an evaluation of several exploration strategies and spatio-temporal models was carried out using real-world data gathered over several months. The evaluation indicates that through understanding of the environment dynamics, the proposed spatio-temporal exploration method could predict which locations were going to change at a specific time and use this knowledge to guide the robot. Such an ability is crucial for long-term deployment of mobile robots in human-populated spaces that change over time.

@inproceedings{lirolem17951,
year = {2015},
title = {Lifelong exploration of dynamic environments},
author = {Joao Santos and Tomas Krajnik and Jaime Pulido Fentanes and Tom Duckett},
publisher = {IEEE},
month = {May},
booktitle = {IEEE International Conference on Robotics and Automation (ICRA)},
url = {http://eprints.lincoln.ac.uk/17951/},
keywords = {ARRAY(0x56147f72cd88)},
abstract = {We propose a novel spatio-temporal mobile-robot exploration method for dynamic, human-populated environments.
In contrast to other exploration methods that model the environment as being static, our spatio-temporal exploration method creates and maintains a world model that not only represents the environment's structure, but also its dynamics over time. Consideration of the world dynamics adds an extra, temporal dimension to the explored space and makes the exploration task a never-ending data-gathering process to keep the robot's environment model up-to-date. Thus, the crucial question is not only where, but also when to observe the explored environment. We address the problem by application of information-theoretic exploration to world representations that model the environment states' uncertainties as probabilistic functions of time. The predictive ability of the spatio-temporal model allows the exploration method to decide not only where, but also when to make environment observations.

To verify the proposed approach, an evaluation of several exploration strategies and spatio-temporal models was carried out using real-world data gathered over several months. The evaluation indicates that through understanding of the environment dynamics, the proposed spatio-temporal exploration method could predict which locations were going to change at a specific time and use this knowledge to guide the robot. Such an ability is crucial for long-term deployment of mobile robots in human-populated spaces that change over time.}
}
• D. Wang, S. Yue, J. Xu, X. Hou, and C. Liu, "A saliency-based cascade method for fast traffic sign detection," in Intelligent Vehicles Symposium, IV 2015, 2015, pp. 180-185.

We propose a cascade method for fast and accurate traffic sign detection. The main feature of the method is that mid-level saliency test is used to efficiently and reliably eliminate background windows. Fast feature extraction is adopted in the subsequent stages for rejecting more negatives. Combining with neighbor scales awareness in window search, the proposed method runs at 3\texttt\char1265 fps for high resolution (1360x800) images, 2\texttt\char1267 times as fast as most state-of-the-art methods. Compared with them, the proposed method yields competitive performance on prohibitory signs while sacrifices performance moderately on danger and mandatory signs. \copyright 2015 IEEE.

@inproceedings{lirolem20151,
booktitle = {Intelligent Vehicles Symposium, IV 2015},
pages = {180--185},
month = {July},
note = {Conference Code:117127},
publisher = {Institute of Electrical and Electronics Engineers Inc.},
author = {Dongdong Wang and Shigang Yue and Jiawei Xu and Xinwen Hou and Cheng-Lin Liu},
year = {2015},
volume = {2015-A},
title = {A saliency-based cascade method for fast traffic sign detection},
journal = {IEEE Intelligent Vehicles Symposium, Proceedings},
url = {http://eprints.lincoln.ac.uk/20151/},
keywords = {ARRAY(0x56147f7715a0)},
abstract = {We propose a cascade method for fast and accurate traffic sign detection. The main feature of the method is that mid-level saliency test is used to efficiently and reliably eliminate background windows. Fast feature extraction is adopted in the subsequent stages for rejecting more negatives. Combining with neighbor scales awareness in window search, the proposed method runs at 3{\texttt{\char126}}5 fps for high resolution (1360x800) images, 2{\texttt{\char126}}7 times as fast as most state-of-the-art methods. Compared with them, the proposed method yields competitive performance on prohibitory signs while sacrifices performance moderately on danger and mandatory signs. {\copyright} 2015 IEEE.}
}
• J. Xu and S. Yue, "Building up a bio-inspired visual attention model by integrating top-down shape bias and improved mean shift adaptive segmentation," International Journal of Pattern Recognition and Artificial Intelligence, vol. 29, iss. 4, 2015.

The driver-assistance system (DAS) becomes quite necessary in-vehicle equipment nowadays due to the large number of road traffic accidents worldwide. An efficient DAS detecting hazardous situations robustly is key to reduce road accidents. The core of a DAS is to identify salient regions or regions of interest relevant to visual attended objects in real visual scenes for further process. In order to achieve this goal, we present a method to locate regions of interest automatically based on a novel adaptive mean shift segmentation algorithm to obtain saliency objects. In the proposed mean shift algorithm, we use adaptive Bayesian bandwidth to find the convergence of all data points by iterations and the k-nearest neighborhood queries. Experiments showed that the proposed algorithm is efficient, and yields better visual salient regions comparing with ground-truth benchmark. The proposed algorithm continuously outperformed other known visual saliency methods, generated higher precision and better recall rates, when challenged with natural scenes collected locally and one of the largest publicly available data sets. The proposed algorithm can also be extended naturally to detect moving vehicles in dynamic scenes once integrated with top-down shape biased cues, as demonstrated in our experiments. Â\copyright 2015 World Scientific Publishing Company.

@article{lirolem20639,
volume = {29},
title = {Building up a bio-inspired visual attention model by integrating top-down shape bias and improved mean shift adaptive segmentation},
journal = {International Journal of Pattern Recognition and Artificial Intelligence},
year = {2015},
number = {4},
month = {June},
publisher = {World Scientific Publishing Co. Pte Ltd},
author = {Jiawei Xu and Shigang Yue},
keywords = {ARRAY(0x56147f8eb790)},
url = {http://eprints.lincoln.ac.uk/20639/},
abstract = {The driver-assistance system (DAS) becomes quite necessary in-vehicle equipment nowadays due to the large number of road traffic accidents worldwide. An efficient DAS detecting hazardous situations robustly is key to reduce road accidents. The core of a DAS is to identify salient regions or regions of interest relevant to visual attended objects in real visual scenes for further process. In order to achieve this goal, we present a method to locate regions of interest automatically based on a novel adaptive mean shift segmentation algorithm to obtain saliency objects. In the proposed mean shift algorithm, we use adaptive Bayesian bandwidth to find the convergence of all data points by iterations and the k-nearest neighborhood queries. Experiments showed that the proposed algorithm is efficient, and yields better visual salient regions comparing with ground-truth benchmark. The proposed algorithm continuously outperformed other known visual saliency methods, generated higher precision and better recall rates, when challenged with natural scenes collected locally and one of the largest publicly available data sets. The proposed algorithm can also be extended naturally to detect moving vehicles in dynamic scenes once integrated with top-down shape biased cues, as demonstrated in our experiments. {\^A}{\copyright} 2015 World Scientific Publishing Company.}
}
• Z. Zhang, S. Yue, and G. Zhang, "Fly visual system inspired artificial neural network for collision detection," Neurocomputing, vol. 153, iss. 4, pp. 221-234, 2015.

This work investigates one bio-inspired collision detection system based on fly visual neural structures, in which collision alarm is triggered if an approaching object in a direct collision course appears in the field of view of a camera or a robot, together with the relevant time region of collision. One such artificial system consists of one artificial fly visual neural network model and one collision detection mechanism. The former one is a computational model to capture membrane potentials produced by neurons. The latter one takes the outputs of the former one as its inputs, and executes three detection schemes: (i) identifying when a spike takes place through the membrane potentials and one threshold scheme; (ii) deciding the motion direction of a moving object by the Reichardt detector model; and (iii) sending collision alarms and collision regions. Experimentally, relying upon a series of video image sequences with different scenes, numerical results illustrated that the artificial system with some striking characteristics is a potentially alternative tool for collision detection.

@article{lirolem17881,
publisher = {Elsevier},
author = {Zhuhong Zhang and Shigang Yue and Guopeng Zhang},
pages = {221--234},
month = {April},
number = {4},
volume = {153},
year = {2015},
title = {Fly visual system inspired artificial neural network for collision detection},
journal = {Neurocomputing},
keywords = {ARRAY(0x56147f8c7a60)},
url = {http://eprints.lincoln.ac.uk/17881/},
abstract = {This work investigates one bio-inspired collision detection system based on fly visual neural structures, in which collision alarm is triggered if an approaching object in a direct collision course appears in the field of view of a camera or a robot, together with the relevant time region of collision. One such artificial system consists of one artificial fly visual neural network model and one collision detection mechanism. The former one is a computational model to capture membrane potentials produced by neurons. The latter one takes the outputs of the former one as its inputs, and executes three detection schemes: (i) identifying when a spike takes place through the membrane potentials and one threshold scheme; (ii) deciding the motion direction of a moving object by the Reichardt detector model; and (iii) sending collision alarms and collision regions. Experimentally, relying upon a series of video image sequences with different scenes, numerical results illustrated that the artificial system with some striking characteristics is a potentially alternative tool for collision detection.}
}

### 2014

• B. H. Amor, G. Neumann, S. Kamthe, O. Kroemer, and J. Peters, "Interaction primitives for human-robot cooperation tasks," in 2014 IEEE International Conference on Robotics and Automation (ICRA 2014), 2014, pp. 2831-2837.

To engage in cooperative activities with human partners, robots have to possess basic interactive abilities and skills. However, programming such interactive skills is a challenging task, as each interaction partner can have different timing or an alternative way of executing movements. In this paper, we propose to learn interaction skills by observing how two humans engage in a similar task. To this end, we introduce a new representation called Interaction Primitives. Interaction primitives build on the framework of dynamic motor primitives (DMPs) by maintaining a distribution over the parameters of the DMP. With this distribution, we can learn the inherent correlations of cooperative activities which allow us to infer the behavior of the partner and to participate in the cooperation. We will provide algorithms for synchronizing and adapting the behavior of humans and robots during joint physical activities.

@inproceedings{lirolem25773,
pages = {2831--2837},
booktitle = {2014 IEEE International Conference on Robotics and Automation (ICRA 2014)},
month = {June},
journal = {Proceedings - IEEE International Conference on Robotics and Automation},
year = {2014},
title = {Interaction primitives for human-robot cooperation tasks},
author = {H. Ben Amor and Gerhard Neumann and S. Kamthe and O. Kroemer and J. Peters},
keywords = {ARRAY(0x56147f964dd0)},
url = {http://eprints.lincoln.ac.uk/25773/},
abstract = {To engage in cooperative activities with human
partners, robots have to possess basic interactive abilities
and skills. However, programming such interactive skills is a
challenging task, as each interaction partner can have different
timing or an alternative way of executing movements. In this
paper, we propose to learn interaction skills by observing how
two humans engage in a similar task. To this end, we introduce
a new representation called Interaction Primitives. Interaction
primitives build on the framework of dynamic motor primitives
(DMPs) by maintaining a distribution over the parameters of
the DMP. With this distribution, we can learn the inherent
correlations of cooperative activities which allow us to infer the
behavior of the partner and to participate in the cooperation.
We will provide algorithms for synchronizing and adapting the
behavior of humans and robots during joint physical activities.}
}
• F. Arvin, J. Murray, L. Shi, C. Zhang, and S. Yue, "Development of an autonomous micro robot for swarm robotics," in IEEE International Conference on Mechatronics and Automation (ICMA), 2014, pp. 635-640.

Swarm robotic systems which are inspired from social behaviour of animals especially insects are becoming a fascinating topic for multi-robot researchers. Simulation software is mostly used for performing research in swarm robotics due the hardware complexities and cost of robot platforms. However, simulation of large numbers of these swarm robots is extremely complex and often inaccurate. In this paper we present the design of a low-cost, open-platform, autonomous micro robot (Colias) for swarm robotic applications. Colias uses a circular platform with a diameter of 4 cm. Long-range infrared modules with adjustable output power allow the robot to communicate with its direct neighbours. The robot has been tested in individual and swarm scenarios and the observed results demonstrate its feasibility to be used as a micro sized mobile robot as well as a low-cost platform for robot swarm applications.

@inproceedings{lirolem14837,
booktitle = {IEEE International Conference on Mechatronics and Automation (ICMA)},
pages = {635--640},
month = {August},
publisher = {IEEE},
author = {Farshad Arvin and John Murray and Licheng Shi and Chun Zhang and Shigang Yue},
title = {Development of an autonomous micro robot for swarm robotics},
year = {2014},
keywords = {ARRAY(0x56147f5bc278)},
url = {http://eprints.lincoln.ac.uk/14837/},
abstract = {Swarm robotic systems which are inspired from social behaviour of animals especially insects are becoming a fascinating topic for multi-robot researchers. Simulation software is mostly used for performing research in swarm robotics due the hardware complexities and cost of robot platforms. However, simulation of large numbers of these swarm robots is extremely complex and often inaccurate. In this paper we present the design of a low-cost, open-platform, autonomous micro robot (Colias) for swarm robotic applications. Colias uses a circular platform with a diameter of 4 cm. Long-range infrared modules with adjustable output power allow the robot to communicate with its direct neighbours. The robot has been tested in individual and swarm scenarios and the observed results demonstrate its feasibility to be used as a micro sized mobile robot as well as a low-cost platform for robot swarm applications.}
}
• F. Arvin, A. E. Turgut, F. Bazyari, K. B. Arikan, N. Bellotto, and S. Yue, "Cue-based aggregation with a mobile robot swarm: a novel fuzzy-based method," Adaptive Behavior, vol. 22, iss. 3, pp. 189-206, 2014.

Aggregation in swarm robotics is referred to as the gathering of spatially distributed robots into a single aggregate. Aggregation can be classified as cue-based or self-organized. In cue-based aggregation, there is a cue in the environment that points to the aggregation area, whereas in self-organized aggregation no cue is present. In this paper, we proposed a novel fuzzy-based method for cue-based aggregation based on the state-of-the-art BEECLUST algorithm. In particular, we proposed three different methods: naïve, that uses a deterministic decision-making mechanism; vector-averaging, using a vectorial summation of all perceived inputs; and fuzzy, that uses a fuzzy logic controller. We used different experiment settings: one-source and two-source environments with static and dynamic conditions to compare all the methods. We observed that the fuzzy method outperformed all the other methods and it is the most robust method against noise.

@article{lirolem13932,
publisher = {Sage for International Society for Adaptive Behavior (ISAB)},
author = {Farshad Arvin and Ali Emre Turgut and Farhad Bazyari and Kutluk Bilge Arikan and Nicola Bellotto and Shigang Yue},
pages = {189--206},
month = {June},
number = {3},
title = {Cue-based aggregation with a mobile robot swarm: a novel fuzzy-based method},
year = {2014},
volume = {22},
abstract = {Aggregation in swarm robotics is referred to as the gathering of spatially distributed robots into a single aggregate. Aggregation can be classified as cue-based or self-organized. In cue-based aggregation, there is a cue in the environment that points to the aggregation area, whereas in self-organized aggregation no cue is present. In this paper, we proposed a novel fuzzy-based method for cue-based aggregation based on the state-of-the-art BEECLUST algorithm. In particular, we proposed three different methods: na{\"i}ve, that uses a deterministic decision-making mechanism; vector-averaging, using a vectorial summation of all perceived inputs; and fuzzy, that uses a fuzzy logic controller. We used different experiment settings: one-source and two-source environments with static and dynamic conditions to compare all the methods. We observed that the fuzzy method outperformed all the other methods and it is the most robust method against noise.},
url = {http://eprints.lincoln.ac.uk/13932/},
keywords = {ARRAY(0x56147f9e1718)}
}
• F. Arvin, A. E. Turgut, N. Bellotto, and S. Yue, "Comparison of different cue-based swarm aggregation strategies," in International Conference in Swarm Intelligence, 2014, pp. 1-8.

In this paper, we compare different aggregation strategies for cue-based aggregation with a mobile robot swarm. We used a sound source as the cue in the environment and performed real robot and simulation based experiments. We compared the performance of two proposed aggregation algorithms we called as the vector averaging and naïve with the state-of-the-art cue-based aggregation strategy BEECLUST. We showed that the proposed strategies outperform BEECLUST method. We also illustrated the feasibility of the method in the presence of noise. The results showed that the vector averaging algorithm is more robust to noise when compared to the naïve method.

@inproceedings{lirolem14927,
title = {Comparison of different cue-based swarm aggregation strategies},
year = {2014},
month = {October},
pages = {1--8},
booktitle = {International Conference in Swarm Intelligence},
publisher = {Springer},
author = {Farshad Arvin and Ali Emre Turgut and Nicola Bellotto and Shigang Yue},
note = {Proceedings, Part I, series volume 8794},
abstract = {In this paper, we compare different aggregation strategies for cue-based aggregation with a mobile robot swarm. We used a sound source as the cue in the environment and performed real robot and simulation based experiments. We compared the performance of two proposed aggregation algorithms we called as the vector averaging and na{\"i}ve with the state-of-the-art cue-based aggregation strategy BEECLUST. We showed that the proposed strategies outperform BEECLUST method. We also illustrated the feasibility of the method in the presence of noise. The results showed that the vector averaging algorithm is more robust to noise when compared to the na{\"i}ve method.},
url = {http://eprints.lincoln.ac.uk/14927/},
keywords = {ARRAY(0x56147f934038)}
}
• F. Arvin, J. Murray, C. Zhang, and S. Yue, "Colias: an autonomous micro robot for swarm robotic applications," International Journal of Advanced Robotic Systems, vol. 11, iss. 113, pp. 1-10, 2014.

Robotic swarms that take inspiration from nature are becoming a fascinating topic for multi-robot researchers. The aim is to control a large number of simple robots enables them in order to solve common complex tasks. Due to the hardware complexities and cost of robot platforms, current research in swarm robotics is mostly performed by simulation software. Simulation of large numbers of these robots which are used in swarm robotic applications is extremely complex and often inaccurate due to poor modelling of external conditions. In this paper we present the design of a low-cost, open-platform, autonomous micro robot (Colias) for swarm robotic applications. Colias employs a circular platform with a diameter of 4 cm. It has a maximum speed of 35 cm/s that gives the ability to be used in swarm scenarios very quickly in large arenas. Long-range infrared modules with adjustable output power allow the robot to communicate with its direct neighbours from a range of 0.5 cm to 3 m. Colias has been designed as a complete platform with supporting software development tools for robotics education and research. It has been tested in individual and swarm scenarios and the observed results demonstrate its feasibility to be used as a micro sized mobile robot as well as a low-cost platform for robot swarm applications.

@article{lirolem14585,
number = {113},
title = {Colias: an autonomous micro robot for swarm robotic applications},
year = {2014},
journal = {International Journal of Advanced Robotic Systems},
volume = {11},
pages = {1--10},
month = {July},
publisher = {InTech},
author = {Farshad Arvin and John Murray and Chun Zhang and Shigang Yue},
abstract = {Robotic swarms that take inspiration from nature are becoming a fascinating topic for multi-robot researchers. The aim is to control a large number of simple robots enables them in order to solve common complex tasks. Due to the hardware complexities and cost of robot platforms, current research in swarm robotics is mostly performed by simulation software. Simulation of large numbers of these robots which are used in swarm robotic applications is extremely complex and often inaccurate due to poor modelling of external conditions. In this paper we present the design of a low-cost, open-platform, autonomous micro robot (Colias) for swarm robotic applications. Colias employs a circular platform with a diameter of 4 cm. It has a maximum speed of 35 cm/s that gives the ability to be used in swarm scenarios very quickly in large arenas. Long-range infrared modules with adjustable output power allow the robot to communicate with its direct neighbours from a range of 0.5 cm to 3 m. Colias has been designed as a complete platform with supporting software development tools for robotics education and research. It has been tested in individual and swarm scenarios and the observed results demonstrate its feasibility to be used as a micro sized mobile robot as well as a low-cost platform for robot swarm applications.},
keywords = {ARRAY(0x56147f9c2360)},
url = {http://eprints.lincoln.ac.uk/14585/}
}
• A. Attar, X. Xie, C. Zhang, Z. Wang, and S. Yue, "Wireless Micro-Ball endoscopic image enhancement using histogram information," in Conference proceedings of the 2014 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, Institute of Electrical and Electronics Engineers Inc., 2014, pp. 3337-3340.

Wireless endoscopy systems is a new innovative method widely used for gastrointestinal tract examination in recent decade. Wireless Micro-Ball endoscopy system with multiple image sensors is the newest proposed method which can make a full view image of the gastrointestinal tract. But still the quality of images from this new wireless endoscopy system is not satisfactory. It's hard for doctors and specialist to easily examine and interpret the captured images. The image features also are not distinct enough to be used for further processing. So as to enhance these low-contrast endoscopic images a new image enhancement method based on the endoscopic images features and color distribution is proposed in this work. The enhancement method is performed on three main steps namely color space transformation, edge preserving mask formation, and histogram information correction. The luminance component of CIE Lab, YCbCr, and HSV color space is enhanced in this method and then two other components added finally to form an enhanced color image. The experimental result clearly show the robustness of the method. \copyright 2014 IEEE.

@incollection{lirolem17582,
journal = {2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC 2014},
year = {2014},
title = {Wireless Micro-Ball endoscopic image enhancement using histogram information},
note = {Conference of 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC 2014 ; Conference Date: 26 - 30 August 2014; Chicago, USA  Conference Code:109045},
author = {Abdolrahman Attar and Xiang Xie and Chun Zhang and Zhihua Wang and Shigang Yue},
publisher = {Institute of Electrical and Electronics Engineers Inc.},
booktitle = {Conference proceedings of the 2014 Annual International Conference of the IEEE Engineering in Medicine and Biology Society},
pages = {3337--3340},
month = {August},
abstract = {Wireless endoscopy systems is a new innovative method widely used for gastrointestinal tract examination in recent decade. Wireless Micro-Ball endoscopy system with multiple image sensors is the newest proposed method which can make a full view image of the gastrointestinal tract. But still the quality of images from this new wireless endoscopy system is not satisfactory. It's hard for doctors and specialist to easily examine and interpret the captured images. The image features also are not distinct enough to be used for further processing. So as to enhance these low-contrast endoscopic images a new image enhancement method based on the endoscopic images features and color distribution is proposed in this work. The enhancement method is performed on three main steps namely color space transformation, edge preserving mask formation, and histogram information correction. The luminance component of CIE Lab, YCbCr, and HSV color space is enhanced in this method and then two other components added finally to form an enhanced color image. The experimental result clearly show the robustness of the method. {\copyright} 2014 IEEE.},
keywords = {ARRAY(0x56147f68b008)},
url = {http://eprints.lincoln.ac.uk/17582/}
}
• A. Colome, G. Neumann, J. Peters, and C. Torras, "Dimensionality reduction for probabilistic movement primitives," in Humanoid Robots (Humanoids), 2014 14th IEEE-RAS International Conference on, 2014, pp. 794-800.

Humans as well as humanoid robots can use a large number of degrees of freedom to solve very complex motor tasks. The high-dimensionality of these motor tasks adds difficulties to the control problem and machine learning algorithms. However, it is well known that the intrinsic dimensionality of many human movements is small in comparison to the number of employed DoFs, and hence, the movements can be represented by a small number of synergies encoding the couplings between DoFs. In this paper, we want to apply Dimensionality Reduction (DR) to a recent movement representation used in robotics, called Probabilistic Movement Primitives (ProMP). While ProMP have been shown to have many benefits, they suffer with the high-dimensionality of a robotic system as the number of parameters of a ProMP scales quadratically with the dimensionality. We use probablistic dimensionality reduction techniques based on expectation maximization to extract the unknown synergies from a given set of demonstrations. The ProMP representation is now estimated in the low-dimensional space of the synergies. We show that our dimensionality reduction is more efficient both for encoding a trajectory from data and for applying Reinforcement Learning with Relative Entropy Policy Search (REPS).

@inproceedings{lirolem25756,
journal = {IEEE-RAS International Conference on Humanoid Robots},
volume = {2015-F},
title = {Dimensionality reduction for probabilistic movement primitives},
year = {2014},
author = {A. Colome and G. Neumann and J. Peters and C. Torras},
month = {November},
pages = {794--800},
booktitle = {Humanoid Robots (Humanoids), 2014 14th IEEE-RAS International Conference on},
abstract = {Humans as well as humanoid robots can use a large number of degrees of freedom to solve very complex motor tasks. The high-dimensionality of these motor tasks adds difficulties to the control problem and machine learning algorithms. However, it is well known that the intrinsic dimensionality of many human movements is small in comparison to the number of employed DoFs, and hence, the movements can be represented by a small number of synergies encoding the couplings between DoFs. In this paper, we want to apply Dimensionality Reduction (DR) to a recent movement representation used in robotics, called Probabilistic Movement Primitives (ProMP). While ProMP have been shown to have many benefits, they suffer with the high-dimensionality of a robotic system as the number of parameters of a ProMP scales quadratically with the dimensionality. We use probablistic dimensionality reduction techniques based on expectation maximization to extract the unknown synergies from a given set of demonstrations. The ProMP representation is now estimated in the low-dimensional space of the synergies. We show that our dimensionality reduction is more efficient both for encoding a trajectory from data and for applying Reinforcement Learning with Relative Entropy Policy Search (REPS).},
keywords = {ARRAY(0x56147f6bf6d0)},
url = {http://eprints.lincoln.ac.uk/25756/}
}
• H. Cuayahuitl, L. Frommberger, N. Dethlefs, A. Raux, M. Marge, and H. Zender, "Introduction to the special issue on Machine learning for multiple modalities in interactive systems and robots," ACM Transactions on Interactive Intelligent Systems (TiiS), vol. 4, iss. 3, p. 12e, 2014.

This special issue highlights research articles that apply machine learning to robots and other systems that interact with users through more than one modality, such as speech, gestures, and vision. For example, a robot may coordinate its speech with its actions, taking into account (audio-)visual feedback during their execution. Machine learning provides interactive systems with opportunities to improve performance not only of individual components but also of the system as a whole. However, machine learning methods that encompass multiple modalities of an interactive system are still relatively hard to find. The articles in this special issue represent examples that contribute to filling this gap.

@article{lirolem22212,
number = {3},
journal = {ACM Transactions on Interactive Intelligent Systems (TiiS)},
year = {2014},
title = {Introduction to the special issue on Machine learning for multiple modalities in interactive systems and robots},
volume = {4},
author = {Heriberto Cuayahuitl and Lutz Frommberger and Nina Dethlefs and Antoine Raux and Mathew Marge and Hendrik Zender},
publisher = {Association for Computing Machinery (ACM)},
pages = {12e},
month = {October},
abstract = {This special issue highlights research articles that apply machine learning to robots and other systems that interact with users through more than one modality, such as speech, gestures, and vision. For example, a robot may coordinate its speech with its actions, taking into account (audio-)visual feedback during their execution. Machine learning provides interactive systems with opportunities to improve performance not only of individual components but also of the system as a whole. However, machine learning methods that encompass multiple modalities of an interactive system are still relatively hard to find. The articles in this special issue represent examples that contribute to filling this gap.},
keywords = {ARRAY(0x56147f82e738)},
url = {http://eprints.lincoln.ac.uk/22212/}
}
• H. Cuayahuitl, I. Kruijff-Korbayová, and N. Dethlefs, "Nonstrict hierarchical reinforcement learning for interactive systems and robots," ACM Transactions on Interactive Intelligent Systems (TiiS), vol. 4, iss. 3, p. 15, 2014.

Conversational systems and robots that use reinforcement learning for policy optimization in large domains often face the problem of limited scalability. This problem has been addressed either by using function approximation techniques that estimate the approximate true value function of a policy or by using a hierarchical decomposition of a learning task into subtasks. We present a novel approach for dialogue policy optimization that combines the benefits of both hierarchical control and function approximation and that allows flexible transitions between dialogue subtasks to give human users more control over the dialogue. To this end, each reinforcement learning agent in the hierarchy is extended with a subtask transition function and a dynamic state space to allow flexible switching between subdialogues. In addition, the subtask policies are represented with linear function approximation in order to generalize the decision making to situations unseen in training. Our proposed approach is evaluated in an interactive conversational robot that learns to play quiz games. Experimental results, using simulation and real users, provide evidence that our proposed approach can lead to more flexible (natural) interactions than strict hierarchical control and that it is preferred by human users.

@article{lirolem22211,
month = {October},
pages = {15},
publisher = {Association for Computing Machinery (ACM)},
author = {Heriberto Cuayahuitl and Ivana Kruijff-Korbayov{\'a} and Nina Dethlefs},
year = {2014},
journal = {ACM Transactions on Interactive Intelligent Systems (TiiS)},
title = {Nonstrict hierarchical reinforcement learning for interactive systems and robots},
volume = {4},
number = {3},
abstract = {Conversational systems and robots that use reinforcement learning for policy optimization in large domains often face the problem of limited scalability. This problem has been addressed either by using function approximation techniques that estimate the approximate true value function of a policy or by using a hierarchical decomposition of a learning task into subtasks. We present a novel approach for dialogue policy optimization that combines the benefits of both hierarchical control and function approximation and that allows flexible transitions between dialogue subtasks to give human users more control over the dialogue. To this end, each reinforcement learning agent in the hierarchy is extended with a subtask transition function and a dynamic state space to allow flexible switching between subdialogues. In addition, the subtask policies are represented with linear function approximation in order to generalize the decision making to situations unseen in training. Our proposed approach is evaluated in an interactive conversational robot that learns to play quiz games. Experimental results, using simulation and real users, provide evidence that our proposed approach can lead to more flexible (natural) interactions than strict hierarchical control and that it is preferred by human users.},
keywords = {ARRAY(0x56147f97d298)},
url = {http://eprints.lincoln.ac.uk/22211/}
}
• C. Dann, G. Neumann, and J. Peters, "Policy evaluation with temporal differences: a survey and comparison," Journal of Machine Learning Research, vol. 15, pp. 809-883, 2014.

Policy evaluation is an essential step in most reinforcement learning approaches. It yields a value function, the quality assessment of states for a given policy, which can be used in a policy improvement step. Since the late 1980s, this research area has been dominated by temporal-difference (TD) methods due to their data-efficiency. However, core issues such as stability guarantees in the off-policy scenario, improved sample efficiency and probabilistic treatment of the uncertainty in the estimates have only been tackled recently, which has led to a large number of new approaches. This paper aims at making these new developments accessible in a concise overview, with foci on underlying cost functions, the off-policy scenario as well as on regularization in high dimensional feature spaces. By presenting the first extensive, systematic comparative evaluations comparing TD, LSTD, LSPE, FPKF, the residual- gradient algorithm, Bellman residual minimization, GTD, GTD2 and TDC, we shed light on the strengths and weaknesses of the methods. Moreover, we present alternative versions of LSTD and LSPE with drastically improved off-policy performance.

@article{lirolem25768,
author = {C. Dann and G. Neumann and J. Peters},
publisher = {Massachusetts Institute of Technology Press (MIT Press) / Microtome Publishing},
month = {March},
pages = {809--883},
volume = {15},
journal = {Journal of Machine Learning Research},
title = {Policy evaluation with temporal differences: a survey and comparison},
year = {2014},
url = {http://eprints.lincoln.ac.uk/25768/},
keywords = {ARRAY(0x56147f7df718)},
abstract = {Policy evaluation is an essential step in most reinforcement learning approaches. It yields a value function, the quality assessment of states for a given policy, which can be used in a policy improvement step. Since the late 1980s, this research area has been dominated by temporal-difference (TD) methods due to their data-efficiency. However, core issues such as stability guarantees in the off-policy scenario, improved sample efficiency and probabilistic treatment of the uncertainty in the estimates have only been tackled recently, which has led to a large number of new approaches.

This paper aims at making these new developments accessible in a concise overview, with foci on underlying cost functions, the off-policy scenario as well as on regularization in high dimensional feature spaces. By presenting the first extensive, systematic comparative evaluations comparing TD, LSTD, LSPE, FPKF, the residual- gradient algorithm, Bellman residual minimization, GTD, GTD2 and TDC, we shed light on the strengths and weaknesses of the methods. Moreover, we present alternative versions of LSTD and LSPE with drastically improved off-policy performance.}
}
• N. Dethlefs and H. Cuayahuitl, "Hierarchical reinforcement learning for situated language generation," Natural Language Engineering, vol. 21, iss. 3, pp. 391-435, 2014.

Natural Language Generation systems in interactive settings often face a multitude of choices, given that the communicative effect of each utterance they generate depends crucially on the interplay between its physical circumstances, addressee and interaction history. This is particularly true in interactive and situated settings. In this paper we present a novel approach for situated Natural Language Generation in dialogue that is based on hierarchical reinforcement learning and learns the best utterance for a context by optimisation through trial and error. The model is trained from human?human corpus data and learns particularly to balance the trade-off between efficiency and detail in giving instructions: the user needs to be given sufficient information to execute their task, but without exceeding their cognitive load. We present results from simulation and a task-based human evaluation study comparing two different versions of hierarchical reinforcement learning: One operates using a hierarchy of policies with a large state space and local knowledge, and the other additionally shares knowledge across generation subtasks to enhance performance. Results show that sharing knowledge across subtasks achieves better performance than learning in isolation, leading to smoother and more successful interactions that are better perceived by human users.

@article{lirolem22213,
month = {May},
pages = {391--435},
publisher = {Cambridge University Press},
author = {Nina Dethlefs and Heriberto Cuayahuitl},
year = {2014},
journal = {Natural Language Engineering},
title = {Hierarchical reinforcement learning for situated language generation},
volume = {21},
number = {3},
abstract = {Natural Language Generation systems in interactive settings often face a multitude of choices, given that the communicative effect of each utterance they generate depends crucially on the interplay between its physical circumstances, addressee and interaction history. This is particularly true in interactive and situated settings. In this paper we present a novel approach for situated Natural Language Generation in dialogue that is based on hierarchical reinforcement learning and learns the best utterance for a context by optimisation through trial and error. The model is trained from human?human corpus data and learns particularly to balance the trade-off between efficiency and detail in giving instructions: the user needs to be given sufficient information to execute their task, but without exceeding their cognitive load. We present results from simulation and a task-based human evaluation study comparing two different versions of hierarchical reinforcement learning: One operates using a hierarchy of policies with a large state space and local knowledge, and the other additionally shares knowledge across generation subtasks to enhance performance. Results show that sharing knowledge across subtasks achieves better performance than learning in isolation, leading to smoother and more successful interactions that are better perceived by human users.},
url = {http://eprints.lincoln.ac.uk/22213/},
keywords = {ARRAY(0x56147f742458)}
}
• C. Dondrup, N. Bellotto, and M. Hanheide, "Social distance augmented qualitative trajectory calculus for human-robot spatial interaction," in Robot and Human Interactive Communication, 2014 RO-MAN, 2014, pp. 519-524.

In this paper we propose to augment a wellestablished Qualitative Trajectory Calculus (QTC) by incorporating social distances into the model to facilitate a richer and more powerful representation of Human-Robot Spatial Interaction (HRSI). By combining two variants of QTC that implement different resolutions and switching between them based on distance thresholds we show that we are able to both reduce the complexity of the representation and at the same time enrich QTC with one of the core HRSI concepts: proxemics. Building on this novel integrated QTC model, we propose to represent the joint spatial behaviour of a human and a robot employing a probabilistic representation based on Hidden Markov Models. We show the appropriateness of our approach by encoding different HRSI behaviours observed in a human-robot interaction study and show how the models can be used to represent and classify these behaviours using social distance-augmented QTC.

@inproceedings{lirolem15832,
pages = {519--524},
booktitle = {Robot and Human Interactive Communication, 2014 RO-MAN},
month = {October},
title = {Social distance augmented qualitative trajectory calculus for human-robot spatial interaction},
year = {2014},
author = {Christian Dondrup and Nicola Bellotto and Marc Hanheide},
publisher = {IEEE},
url = {http://eprints.lincoln.ac.uk/15832/},
keywords = {ARRAY(0x56147f997478)},
abstract = {In this paper we propose to augment a wellestablished Qualitative Trajectory Calculus (QTC) by incorporating social distances into the model to facilitate a richer and more powerful representation of Human-Robot Spatial Interaction (HRSI). By combining two variants of QTC that implement different resolutions and switching between them based on distance thresholds we show that we are able to both reduce the complexity of the representation and at the same time enrich QTC with one of the core HRSI concepts: proxemics. Building on this novel integrated QTC model, we propose to represent the joint spatial behaviour of a human and a robot employing a probabilistic representation based on Hidden Markov Models. We show the appropriateness of our approach by encoding different HRSI behaviours observed in a human-robot interaction study and show how the models can be used to represent and classify these behaviours using  social distance-augmented QTC.}
}
• C. Dondrup, C. Lichtenthaeler, and M. Hanheide, "Hesitation signals in human-robot head-on encounters: a pilot study," in 9th ACM/IEEE International Conference on Human Robot Interaction, 2014, pp. 154-155.

The motivation for this research stems from the future vision of being able to buy a mobile service robot for your own household, unpack it, switch it on, and have it behave in an intelligent way; but of course it also has to adapt to your personal preferences over time. My work is focusing on the spatial aspect of the robot?s behaviours, which means when it is moving in a confined, shared space with a human it will also take the communicative character of these movements into account. This adaptation to the users preferences should come from experience which the robot gathers throughout several days or months of interaction and not from a programmer hard-coding certain behaviours

@inproceedings{lirolem13570,
booktitle = {9th ACM/IEEE International Conference on Human Robot Interaction},
pages = {154--155},
month = {March},
author = {Christian Dondrup and Christina Lichtenthaeler and Marc Hanheide},
publisher = {IEEE},
year = {2014},
title = {Hesitation signals in human-robot head-on encounters: a pilot study},
keywords = {ARRAY(0x56147f7d1628)},
url = {http://eprints.lincoln.ac.uk/13570/},
abstract = {The motivation for this research stems from the future vision of being able to buy a mobile service robot for your own household, unpack it, switch it on, and have it behave in an intelligent way; but of course it also has to adapt to your personal preferences over time. My work is focusing on the spatial aspect of the robot?s behaviours, which means when it is moving in a confined, shared space with a human it will also take the communicative character of these movements into account. This adaptation to the users preferences should come from experience which the robot gathers throughout several days or months of interaction and not from a programmer hard-coding certain behaviours}
}
• C. Dondrup, M. Hanheide, and N. Bellotto, "A probabilistic model of human-robot spatial interaction using a qualitative trajectory calculus," in AAAI Spring Symposium: "Qualitative Representations for Robots", 2014.

In this paper we propose a probabilistic model for Human-Robot Spatial Interaction (HRSI) using a Qualitative Trajectory Calculus (QTC). In particular, we will build on previous work representing HRSI as a Markov chain of QTC states and evolve this to an approach using a Hidden Markov Model representation. Our model accounts for the invalidity of certain transitions within the QTC to reduce the complexity of the probabilistic model and to ensure state sequences in accordance to this representational framework. We show the appropriateness of our approach by using the probabilistic model to encode different HRSI behaviours observed in a human-robot interaction study and show how the models can be used to classify these behaviours reliably. Copyright Â\copyright 2014, Association for the Advancement of Artificial Intelligence. All rights reserved.

@inproceedings{lirolem13523,
year = {2014},
title = {A probabilistic model of human-robot spatial interaction using a qualitative trajectory calculus},
publisher = {AAAI / AI Access Foundation},
author = {Christian Dondrup and Marc Hanheide and Nicola Bellotto},
month = {March},
booktitle = {AAAI Spring Symposium: "Qualitative Representations for Robots"},
abstract = {In this paper we propose a probabilistic model for Human-Robot Spatial Interaction (HRSI) using a Qualitative Trajectory Calculus (QTC). In particular, we will build on previous work representing HRSI as a Markov chain of QTC states and evolve this to an approach using a Hidden Markov Model representation. Our model accounts for the invalidity of certain transitions within the QTC to reduce the complexity of the probabilistic model and to ensure state sequences in accordance to this representational framework. We show the appropriateness of our approach by using the probabilistic model to encode different HRSI behaviours observed in a human-robot interaction study and show how the models can be used to classify these behaviours reliably. Copyright {\^A}{\copyright} 2014, Association for the Advancement of Artificial Intelligence. All rights reserved.},
url = {http://eprints.lincoln.ac.uk/13523/},
keywords = {ARRAY(0x56147f9207f0)}
}
• T. Duckett and T. Krajnik, "A frequency-based approach to long-term robotic mapping," in ICRA 2014 Workshop on Long Term Autonomy, 2014.

While mapping of static environments has been widely studied, long-term mapping in non-stationary environments is still an open problem. In this talk, we present a novel approach for long-term representation of populated environments, where many of the observed changes are caused by humans performing their daily activities. We propose to model the environment's dynamics by its frequency spectrum, as a combination of harmonic functions that correspond to periodic processes influencing the environment. Such a representation not only allows representation of environment dynamics over arbitrary timescales with constant memory requirements, but also prediction of future environment states. The proposed approach can be applied to many of the state-of-the-art environment models. In particular, we show that occupancy grids, topological or landmark maps can be easily extended to represent dynamic environments. We present experiments using data collected by a mobile robot patrolling an indoor environment over a period of one month, where frequency-enhanced models were compared to their static counterparts in four scenarios: i) 3D map building, ii) environment state prediction, iii) topological localisation and iv) anomaly detection, in order to verify the model's ability to detect unusual events. In all these cases, the frequency-enhanced models outperformed their static counterparts.

@inproceedings{lirolem14422,
month = {June},
booktitle = {ICRA 2014 Workshop on Long Term Autonomy},
title = {A frequency-based approach to long-term robotic mapping},
year = {2014},
author = {Tom Duckett and Tomas Krajnik},
abstract = {While mapping of static environments has been widely studied, long-term mapping in non-stationary environments is still an open problem. In this talk, we present a novel approach for long-term representation of populated environments, where many of the observed changes are caused by humans performing their daily activities. We propose to model the environment's dynamics by its frequency spectrum, as a combination of harmonic functions that correspond to periodic processes influencing the environment. Such a representation not only allows representation of environment dynamics over arbitrary timescales with constant memory requirements, but also prediction of future environment states. The proposed approach can be applied to many of the state-of-the-art environment models. In particular, we show that occupancy grids, topological or landmark maps can be easily extended to represent dynamic environments. We present experiments using data collected by a mobile robot patrolling an indoor environment over a period of one month, where frequency-enhanced models were compared to their static counterparts in four scenarios: i) 3D map building, ii) environment state prediction, iii) topological localisation and iv) anomaly detection, in order to verify the model's ability to detect unusual events. In all these cases, the frequency-enhanced models outperformed their static counterparts.},
url = {http://eprints.lincoln.ac.uk/14422/},
keywords = {ARRAY(0x56147f5aab10)}
}
• V. Gomez, H. J. Kappen, J. Peters, and G. Neumann, "Policy search for path integral control," in Machine Learning and Knowledge Discovery in Databases - European Conference, ECML/PKDD 2014, 2014, pp. 482-497.

Path integral (PI) control defines a general class of control problems for which the optimal control computation is equivalent to an inference problem that can be solved by evaluation of a path integral over state trajectories. However, this potential is mostly unused in real-world problems because of two main limitations: first, current approaches can typically only be applied to learn open-loop controllers and second, current sampling procedures are inefficient and not scalable to high dimensional systems. We introduce the efficient Path Integral Relative-Entropy Policy Search (PI-REPS) algorithm for learning feedback policies with PI control. Our algorithm is inspired by information theoretic policy updates that are often used in policy search. We use these updates to approximate the state trajectory distribution that is known to be optimal from the PI control theory. Our approach allows for a principled treatment of different sampling distributions and can be used to estimate many types of parametric or non-parametric feedback controllers. We show that PI-REPS significantly outperforms current methods and is able to solve tasks that are out of reach for current methods.

@inproceedings{lirolem25770,
number = {PART 1},
journal = {Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)},
year = {2014},
title = {Policy search for path integral control},
volume = {8724 L},
booktitle = {Machine Learning and Knowledge Discovery in Databases - European Conference, ECML/PKDD 2014},
pages = {482--497},
author = {Vincenc Gomez and Hilbert J. Kappen and Jan Peters and Gerhard Neumann},
publisher = {Springer},
abstract = {Path integral (PI) control defines a general class of control problems for which the optimal control computation is equivalent to an inference problem that can be solved by evaluation of a path integral over state trajectories. However, this potential is mostly unused in real-world problems because of two main limitations: first, current approaches can typically only be applied to learn open-loop controllers and second, current sampling procedures are inefficient and not scalable to high dimensional systems. We introduce the efficient Path Integral Relative-Entropy Policy Search (PI-REPS) algorithm for learning feedback policies with PI control. Our algorithm is inspired by information theoretic policy updates that are often used in policy search. We use these updates to approximate the state trajectory distribution that is known to be optimal from the PI control theory. Our approach allows for a principled treatment of different sampling distributions and can be used to estimate many types of parametric or non-parametric feedback controllers. We show that PI-REPS significantly outperforms current methods and is able to solve tasks that are out of reach for current methods.},
keywords = {ARRAY(0x56147f89d1a8)},
url = {http://eprints.lincoln.ac.uk/25770/}
}
• C. Hu, F. Arvin, and S. Yue, "Development of a bio-inspired vision system for mobile micro-robots," in IEEE International Conferences on Development and Learning and Epigenetic Robotics (ICDL-Epirob), 2014, pp. 81-86.

In this paper, we present a new bio-inspired vision system for mobile micro-robots. The processing method takes inspiration from vision of locusts in detecting the fast approaching objects. Research suggested that locusts use wide field visual neuron called the lobula giant movement detector to respond to imminent collisions. We employed the locusts' vision mechanism to motion control of a mobile robot. The selected image processing method is implemented on a developed extension module using a low-cost and fast ARM processor. The vision module is placed on top of a micro-robot to control its trajectory and to avoid obstacles. The observed results from several performed experiments demonstrated that the developed extension module and the inspired vision system are feasible to employ as a vision module for obstacle avoidance and motion control.

@inproceedings{lirolem16334,
year = {2014},
title = {Development of a bio-inspired vision system for mobile micro-robots},
publisher = {IEEE},
author = {Cheng Hu and Farshad Arvin and Shigang Yue},
pages = {81--86},
booktitle = {IEEE International Conferences on Development and Learning and Epigenetic Robotics (ICDL-Epirob)},
month = {October},
abstract = {In this paper, we present a new bio-inspired vision system for mobile micro-robots. The processing method takes inspiration from vision of locusts in detecting the fast approaching objects. Research suggested that locusts use wide field visual neuron called the lobula giant movement detector to respond to imminent collisions. We employed the locusts' vision mechanism to motion control of a mobile robot. The selected image processing method is implemented on a developed extension module using a low-cost and fast ARM processor. The vision module is placed on top of a micro-robot to control its trajectory and to avoid obstacles. The observed results from several performed experiments demonstrated that the developed extension module and the inspired vision system are feasible to employ as a vision module for obstacle avoidance and motion control.},
url = {http://eprints.lincoln.ac.uk/16334/},
keywords = {ARRAY(0x56147f7aa338)}
}
• K. Iliopoulos, N. Bellotto, and N. Mavridis, "From sequence to trajectory and vice versa: solving the inverse QTC problem and coping with real-world trajectories," in AAAI Spring Symposium: "Qualitative Representations for Robots", 2014.

Spatial interactions between agents carry information of high value to human observers, as exemplified by the high-level interpretations that humans make when watching the Heider and Simmel movie, or other such videos which just contain motions of simple objects, such as points, lines and triangles. However, not all the information contained in a pair of continuous trajectories is important; and thus the need for qualitative descriptions of interaction trajectories arises. Towards that purpose, Qualitative Trajectory Calculus (QTC) has been proposed in (Van de Weghe, 2004). However, the original definition of QTC handles uncorrupted continuous-time trajectories, while real-world signals are noisy and sampled in discrete-time. Also, although QTC presents a method for transforming trajectories to qualitative descriptions, the inverse problem has not yet been studied. Thus, in this paper, after discussing several aspects of the transition from ideal QTC to discrete-time noisy QTC, we introduce a novel algorithm for solving the QTC inverse problem; i.e. transforming qualitative descriptions to archetypal trajectories that satisfy them. Both of these problems are particularly important for the successful application of qualitative trajectory calculus to Human-Robot Interaction.

@inproceedings{lirolem13519,
author = {Konstantinos Iliopoulos and Nicola Bellotto and Nikolaos Mavridis},
publisher = {AAAI},
year = {2014},
title = {From sequence to trajectory and vice versa: solving the inverse QTC problem and coping with real-world trajectories},
booktitle = {AAAI Spring Symposium: "Qualitative Representations for Robots"},
month = {March},
keywords = {ARRAY(0x56147f602328)},
url = {http://eprints.lincoln.ac.uk/13519/},
abstract = {Spatial interactions between agents carry information of high value to human observers, as exemplified by the high-level interpretations that humans make when watching the Heider and Simmel movie, or other such videos which just contain motions of simple objects, such as points, lines and triangles. However, not all the information contained in a pair of continuous trajectories is important; and thus the need for qualitative descriptions of interaction trajectories arises. Towards that purpose, Qualitative Trajectory Calculus (QTC) has been proposed in (Van de Weghe, 2004). However, the original definition of QTC handles uncorrupted continuous-time trajectories, while real-world signals are noisy and sampled in discrete-time. Also, although QTC presents a method for transforming trajectories to qualitative descriptions, the inverse problem has not yet been studied. Thus, in this paper, after discussing several aspects of the transition from ideal QTC to discrete-time noisy QTC, we introduce a novel algorithm for solving the QTC inverse problem; i.e. transforming qualitative descriptions to archetypal trajectories that satisfy them. Both of these problems are particularly important for the successful application of qualitative trajectory calculus to Human-Robot Interaction.}
}
• T. Krajnik, J. P. Fentanes, G. Cielniak, C. Dondrup, and T. Duckett, "Spectral analysis for long-term robotic mapping," in 2014 IEEE International Conference on Robotics and Automation (ICRA 2014), 2014.

This paper presents a new approach to mobile robot mapping in long-term scenarios. So far, the environment models used in mobile robotics have been tailored to capture static scenes and dealt with the environment changes by means of ?memory decay?. While these models keep up with slowly changing environments, their utilization in dynamic, real world environments is difficult. The representation proposed in this paper models the environment?s spatio-temporal dynamics by its frequency spectrum. The spectral representation of the time domain allows to identify, analyse and remember regularly occurring environment processes in a computationally efficient way. Knowledge of the periodicity of the different environment processes constitutes the model predictive capabilities, which are especially useful for long-term mobile robotics scenarios. In the experiments presented, the proposed approach is applied to data collected by a mobile robot patrolling an indoor environment over a period of one week. Three scenarios are investigated, including intruder detection and 4D mapping. The results indicate that the proposed method allows to represent arbitrary timescales with constant (and low) memory requirements, achieving compression rates up to 106 . Moreover, the representation allows for prediction of future environment?s state with $\sim$ 90\% precision.

@inproceedings{lirolem13273,
booktitle = {2014 IEEE International Conference on Robotics and Automation (ICRA 2014)},
month = {May},
title = {Spectral analysis for long-term robotic mapping},
year = {2014},
publisher = {IEEE},
author = {Tomas Krajnik and Jaime Pulido Fentanes and Grzegorz Cielniak and Christian Dondrup and Tom Duckett},
abstract = {This paper presents a new approach to mobile robot mapping in long-term scenarios. So far, the environment models used in mobile robotics have been tailored to capture static scenes and dealt with the environment changes by means of ?memory decay?. While these models keep up with slowly changing environments, their utilization in dynamic, real world
environments is difficult.

The representation proposed in this paper models the environment?s spatio-temporal dynamics by its frequency spectrum. The spectral representation of the time domain allows to identify, analyse and remember regularly occurring environment processes in a computationally efficient way. Knowledge of the periodicity of the different environment processes constitutes the model predictive capabilities, which are especially useful for long-term mobile robotics scenarios.

In the experiments presented, the proposed approach is applied to data collected by a mobile robot patrolling an indoor
environment over a period of one week. Three scenarios are investigated, including intruder detection and 4D mapping. The results indicate that the proposed method allows to represent arbitrary timescales with constant (and low) memory requirements, achieving compression rates up to 106 . Moreover, the representation allows for prediction of future environment?s state with {$\sim$} 90\% precision.},
url = {http://eprints.lincoln.ac.uk/13273/},
keywords = {ARRAY(0x56147f61ffd8)}
}
• T. Krajnik, N. Matias, J. Faigl, P. Vanek, M. Saska, L. Preucil, T. Duckett, and M. Marta, "A practical multirobot localization system," Journal of Intelligent and Robotic Systems, vol. 76, iss. 3-4, pp. 539-562, 2014.

We present a fast and precise vision-based software intended for multiple robot localization. The core component of the software is a novel and efficient algorithm for black and white pattern detection. The method is robust to variable lighting conditions, achieves sub-pixel precision and its computational complexity is independent of the processed image size. With off-the-shelf computational equipment and low-cost cameras, the core algorithm is able to process hundreds of images per second while tracking hundreds of objects with a millimeter precision. In addition, we present the method's mathematical model, which allows to estimate the expected localization precision, area of coverage, and processing speed from the camera's intrinsic parameters and hardware's processing capacity. The correctness of the presented model and performance of the algorithm in real-world conditions is verified in several experiments. Apart from the method description, we also make its source code public at $\backslash$emph\http://purl.org/robotics/whycon\; so, it can be used as an enabling technology for various mobile robotic problems.

@article{lirolem13653,
year = {2014},
title = {A practical multirobot localization system},
journal = {Journal of Intelligent and Robotic Systems},
volume = {76},
number = {3-4},
month = {December},
pages = {539--562},
publisher = {Springer Heidelberg},
author = {Tomas Krajnik and Nitsche Matias and Jan Faigl and Petr Vanek and Martin Saska and Libor Preucil and Tom Duckett and Mejail Marta},
url = {http://eprints.lincoln.ac.uk/13653/},
keywords = {ARRAY(0x56147f688b68)},
abstract = {We present a fast and precise vision-based software intended for multiple robot localization. The core component of the software is a novel and efficient algorithm for black and white pattern detection. The method is robust to variable lighting conditions, achieves sub-pixel precision and its computational complexity is independent of the processed image size. With off-the-shelf computational equipment and low-cost cameras, the core algorithm is able to process hundreds of images per second while tracking hundreds of objects with a millimeter precision. In addition, we present the method's mathematical model, which allows to estimate the expected localization precision, area of coverage, and processing speed from the camera's intrinsic parameters and hardware's processing capacity. The correctness of the presented model and performance of the algorithm in real-world conditions is verified in several experiments.  Apart from the method description, we also make its source code public at {$\backslash$}emph\{http://purl.org/robotics/whycon\}; so, it can be used as an enabling technology for various mobile robotic problems.}
}
• T. Krajnik, J. P. Fentanes, O. M. Mozos, T. Duckett, J. Ekekrantz, and M. Hanheide, "Long-term topological localisation for service robots in dynamic environments using spectral maps," in IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2014.

This paper presents a new approach for topological localisation of service robots in dynamic indoor environments. In contrast to typical localisation approaches that rely mainly on static parts of the environment, our approach makes explicit use of information about changes by learning and modelling the spatio-temporal dynamics of the environment where the robot is acting. The proposed spatio-temporal world model is able to predict environmental changes in time, allowing the robot to improve its localisation capabilities during long-term operations in populated environments. To investigate the proposed approach, we have enabled a mobile robot to autonomously patrol a populated environment over a period of one week while building the proposed model representation. We demonstrate that the experience learned during one week is applicable for topological localization even after a hiatus of three months by showing that the localization error rate is significantly lower compared to static environment representations.

@inproceedings{lirolem14423,
booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
month = {September},
author = {Tomas Krajnik and Jaime Pulido Fentanes and Oscar Martinez Mozos and Tom Duckett and Johan Ekekrantz and Marc Hanheide},
publisher = {IEEE},
title = {Long-term topological localisation for service robots in dynamic environments using spectral maps},
year = {2014},
keywords = {ARRAY(0x56147f681ab0)},
url = {http://eprints.lincoln.ac.uk/14423/},
abstract = {This paper presents a new approach for topological localisation of service robots in dynamic indoor environments. In contrast to typical localisation approaches that rely mainly on static parts of the environment, our approach makes explicit use of information about changes by learning and modelling the spatio-temporal dynamics of the environment where the robot is acting.  The proposed spatio-temporal world model is able to predict environmental changes in time, allowing the robot to improve its localisation capabilities during long-term operations in populated environments. To investigate the proposed approach, we have enabled a mobile robot to autonomously patrol a populated environment over a period of one week while building the proposed model representation. We demonstrate that the experience learned during one week is applicable for topological localization even after a hiatus of three months by showing that the localization error rate is significantly lower compared to static environment representations.}
}
• T. Krajnik, J. Santos, B. Seemann, and T. Duckett, "FROctomap: an efficient spatio-temporal environment representation," in Advances in Autonomous Robotics Systems, M. Mistry, A. Leonardis, and M. Witkowski, Eds., Springer International Publishing, 2014, vol. 8717, pp. 281-282.

We present a novel software tool intended for mobile robot mapping in long-term scenarios. The method allows for efficient volumetric representation of dynamic three-dimensional environments over long periods of time. It is based on a combination of a well-established 3D mapping framework called Octomaps and an idea to model environment dynamics by its frequency spectrum. The proposed method allows not only for efficient representation, but also reliable prediction of the future states of dynamic three-dimensional environments. Our spatio-temporal mapping framework is available as an open-source C++ library and a ROS module which allows its easy integration in robotics projects.

@incollection{lirolem14895,
pages = {281--282},
booktitle = {Advances in Autonomous Robotics Systems},
month = {September},
author = {Tomas Krajnik and Joao Santos and Bianca Seemann and Tom Duckett},
publisher = {Springer International Publishing},
editor = {Michael Mistry and Ale Leonardis and Mark Witkowski},
series = {Lecture Notes in Computer Science},
volume = {8717},
year = {2014},
title = {FROctomap: an efficient spatio-temporal environment representation},
keywords = {ARRAY(0x56147f84f840)},
url = {http://eprints.lincoln.ac.uk/14895/},
abstract = {We present a novel software tool intended for mobile robot mapping in long-term scenarios. The method allows for efficient volumetric representation of dynamic three-dimensional environments over long periods of time. It is based on a combination of a well-established 3D mapping framework called Octomaps and an idea to model environment dynamics by its frequency spectrum. The proposed method allows not only for efficient representation, but also reliable prediction of the future states of dynamic three-dimensional environments. Our spatio-temporal mapping framework is available as an open-source C++ library and a ROS module which allows its easy integration in robotics projects.}
}
• O. Kroemer, V. H. Hoof, G. Neumann, and J. Peters, "Learning to predict phases of manipulation tasks as hidden states," in 2014 IEEE International Conference on Robotics and Automation, 2014, pp. 4009-4014.

Phase transitions in manipulation tasks often occur when contacts between objects are made or broken. A switch of the phase can result in the robot?s actions suddenly influencing different aspects of its environment. Therefore, the boundaries between phases often correspond to constraints or subgoals of the manipulation task. In this paper, we investigate how the phases of manipulation tasks can be learned from data. The task is modeled as an autoregressive hidden Markov model, wherein the hidden phase transitions depend on the observed states. The model is learned from data using the expectation-maximization algorithm. We demonstrate the proposed method on both a pushing task and a pepper mill turning task. The proposed approach was compared to a standard autoregressive hidden Markov model. The experiments show that the learned models can accurately predict the transitions in phases during the manipulation tasks.

@inproceedings{lirolem25769,
pages = {4009--4014},
booktitle = {2014 IEEE International Conference on Robotics and Automation},
month = {September},
title = {Learning to predict phases of manipulation tasks as hidden states},
year = {2014},
journal = {Proceedings - IEEE International Conference on Robotics and Automation},
author = {O. Kroemer and H. Van Hoof and G. Neumann and J. Peters},
url = {http://eprints.lincoln.ac.uk/25769/},
keywords = {ARRAY(0x56147f858980)},
abstract = {Phase transitions in manipulation tasks often occur
when contacts between objects are made or broken. A
switch of the phase can result in the robot?s actions suddenly
influencing different aspects of its environment. Therefore, the
boundaries between phases often correspond to constraints or
In this paper, we investigate how the phases of manipulation
tasks can be learned from data. The task is modeled as an
autoregressive hidden Markov model, wherein the hidden phase
transitions depend on the observed states. The model is learned
from data using the expectation-maximization algorithm. We
demonstrate the proposed method on both a pushing task
and a pepper mill turning task. The proposed approach was
compared to a standard autoregressive hidden Markov model.
The experiments show that the learned models can accurately
predict the transitions in phases during the manipulation tasks.}
}
• S. Lemaignan, M. Hanheide, M. Karg, H. Khambhaita, L. Kunze, F. Lier, I. L~A?tkebohle, and G. Milliez, "Simulation and HRI recent perspectives with the MORSE simulator," Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 8810, pp. 13-24, 2014.

Simulation in robotics is often a love-hate relationship: while simulators do save us a lot of time and effort compared to regular deployment of complex software architectures on complex hardware, simulators are also known to evade many of the real issues that robots need to manage when they enter the real world. Because humans are the paragon of dynamic, unpredictable, complex, real world entities, simulation of human-robot interactions may look condemn to fail, or, in the best case, to be mostly useless. This collective article reports on five independent applications of the MORSE simulator in the field of human-robot interaction: It appears that simulation is already useful, if not essential, to successfully carry out research in the field of HRI, and sometimes in scenarios we do not anticipate. Â\copyright 2014 Springer International Publishing Switzerland.

@article{lirolem21430,
year = {2014},
title = {Simulation and HRI recent perspectives with the MORSE simulator},
journal = {Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)},
volume = {8810},
month = {October},
pages = {13--24},
author = {S. Lemaignan and M. Hanheide and M. Karg and H. Khambhaita and L. Kunze and F. Lier and I. L{\~A}?tkebohle and G. Milliez},
publisher = {Springer Verlag},
note = {Find out how to access preview-only content
Chapter
Simulation, Modeling, and Programming for Autonomous Robots
Volume 8810 of the series Lecture Notes in Computer Science pp 13-24
Simulation and HRI Recent Perspectives with the MORSE Simulator

S{\'e}verin Lemaignan, Marc Hanheide, Michael Karg, Harmish Khambhaita, Lars Kunze, Florian Lier, Ingo L{\"u}tkebohle, Gr{\'e}goire Milliez
\$29.95 / ?24.95 / {\pounds}19.95 * Buy eBook \$89.00 / ?67.82 / {\pounds}56.99*
* Final gross prices may vary according to local VAT.Get Access
Abstract
Simulation in robotics is often a love-hate relationship: while simulators do save us a lot of time and effort compared to regular deployment of complex software architectures on complex hardware, simulators are also known to evade many of the real issues that robots need to manage when they enter the real world. Because humans are the paragon of dynamic, unpredictable, complex, real world entities, simulation of human-robot interactions may look condemn to fail, or, in the best case, to be mostly useless. This collective article reports on five independent applications of the MORSE simulator in the field of human-robot interaction: It appears that simulation is already useful, if not essential, to successfully carry out research in the field of HRI, and sometimes in scenarios we do not anticipate.
Simulation, Modeling, and Programming for Autonomous RobotsSimulation, Modeling, and Programming for Autonomous Robots Look
Inside
Chapter Metrics
367
Provided by Bookmetrix
Reference tools
Export citation
Other actions
Reprints and Permissions
Share
Supplementary Material (0)
References (24)

Title
Simulation and HRI Recent Perspectives with the MORSE Simulator
Book Title
Simulation, Modeling, and Programming for Autonomous Robots
Book Subtitle
4th International Conference, SIMPAR 2014, Bergamo, Italy, October 20-23, 2014. Proceedings},
keywords = {ARRAY(0x56147f809f30)},
url = {http://eprints.lincoln.ac.uk/21430/},
abstract = {Simulation in robotics is often a love-hate relationship: while simulators do save us a lot of time and effort compared to regular deployment of complex software architectures on complex hardware, simulators are also known to evade many of the real issues that robots need to manage when they enter the real world. Because humans are the paragon of dynamic, unpredictable, complex, real world entities, simulation of human-robot interactions may look condemn to fail, or, in the best case, to be mostly useless. This collective article reports on five independent applications of the MORSE simulator in the field of human-robot interaction: It appears that simulation is already useful, if not essential, to successfully carry out research in the field of HRI, and sometimes in scenarios we do not anticipate. {\^A}{\copyright} 2014 Springer International Publishing Switzerland.}
}
• R. Lioutikov, A. Paraschos, J. Peters, and G. Neumann, "Sample-based information-theoretic stochastic optimal control," in Proceedings of 2014 IEEE International Conference on Robotics and Automation, 2014, pp. 3896-3902.

Many Stochastic Optimal Control (SOC) approaches rely on samples to either obtain an estimate of the value function or a linearisation of the underlying system model. However, these approaches typically neglect the fact that the accuracy of the policy update depends on the closeness of the resulting trajectory distribution to these samples. The greedy operator does not consider such closeness constraint to the samples. Hence, the greedy operator can lead to oscillations or even instabilities in the policy updates. Such undesired behaviour is likely to result in an inferior performance of the estimated policy. We reuse inspiration from the reinforcement learning community and relax the greedy operator used in SOC with an information theoretic bound that limits the ?distance? of two subsequent trajectory distributions in a policy update. The introduced bound ensures a smooth and stable policy update. Our method is also well suited for model-based reinforcement learning, where we estimate the system dynamics model from data. As this model is likely to be inaccurate, it might be dangerous to exploit the model greedily. Instead, our bound ensures that we generate new data in the vicinity of the current data, such that we can improve our estimate of the system dynamics model. We show that our approach outperforms several state of the art approaches on challenging simulated robot control tasks.

@inproceedings{lirolem25771,
author = {R. Lioutikov and A. Paraschos and J. Peters and G. Neumann},
title = {Sample-based information-theoretic stochastic optimal control},
year = {2014},
journal = {Proceedings - IEEE International Conference on Robotics and Automation},
booktitle = {Proceedings of 2014 IEEE International Conference on Robotics and Automation},
pages = {3896--3902},
month = {September},
url = {http://eprints.lincoln.ac.uk/25771/},
keywords = {ARRAY(0x56147f88b058)},
abstract = {Many Stochastic Optimal Control (SOC) approaches
rely on samples to either obtain an estimate of the
value function or a linearisation of the underlying system model.
However, these approaches typically neglect the fact that the
accuracy of the policy update depends on the closeness of the
resulting trajectory distribution to these samples. The greedy
operator does not consider such closeness constraint to the
samples. Hence, the greedy operator can lead to oscillations
or even instabilities in the policy updates. Such undesired
behaviour is likely to result in an inferior performance of the
estimated policy. We reuse inspiration from the reinforcement
learning community and relax the greedy operator used in SOC
with an information theoretic bound that limits the ?distance? of
two subsequent trajectory distributions in a policy update. The
introduced bound ensures a smooth and stable policy update.
Our method is also well suited for model-based reinforcement
learning, where we estimate the system dynamics model from
data. As this model is likely to be inaccurate, it might be
dangerous to exploit the model greedily. Instead, our bound
ensures that we generate new data in the vicinity of the current
data, such that we can improve our estimate of the system
dynamics model. We show that our approach outperforms
several state of the art approaches on challenging simulated
}
• R. Lioutikov, A. Paraschos, J. Peters, and G. Neumann, "Generalizing movements with information-theoretic stochastic optimal control," Journal of Aerospace Information Systems, vol. 11, iss. 9, pp. 579-595, 2014.

Stochastic optimal control is typically used to plan a movement for a specific situation. Although most stochastic optimal control methods fail to generalize this movement plan to a new situation without replanning, a stochastic optimal control method is presented that allows reuse of the obtained policy in a new situation, as the policy is more robust to slight deviations from the initial movement plan. To improve the robustness of the policy, we employ information-theoretic policy updates that explicitly operate on trajectory distributions instead of single trajectories. To ensure a stable and smooth policy update, the ?distance? is limited between the trajectory distributions of the old and the new control policies. The introduced bound offers a closed-form solution for the resulting policy and extends results from recent developments in stochastic optimal control. In contrast to many standard stochastic optimal control algorithms, the current approach can directly infer the system dynamics from data points, and hence can also be used for model-based reinforcement learning. This paper represents an extension of the paper by Lioutikov et al. (?Sample-Based Information-Theoretic Stochastic Optimal Control,? Proceedings of 2014 IEEE International Conference on Robotics and Automation (ICRA), IEEE, Piscataway, NJ, 2014, pp. 3896?3902). In addition to revisiting the content, an extensive theoretical comparison is presented of the approach with related work, additional aspects of the implementation are discussed, and further evaluations are introduced.

@article{lirolem25767,
number = {9},
year = {2014},
title = {Generalizing movements with information-theoretic stochastic optimal control},
journal = {Journal of Aerospace Information Systems},
volume = {11},
author = {R. Lioutikov and A. Paraschos and J. Peters and G. Neumann},
publisher = {American Institute of Aeronautics and Astronautics},
pages = {579--595},
month = {September},
keywords = {ARRAY(0x56147f8e9778)},
url = {http://eprints.lincoln.ac.uk/25767/},
abstract = {Stochastic optimal control is typically used to plan a movement for a specific situation. Although most stochastic optimal control methods fail to generalize this movement plan to a new situation without replanning, a stochastic optimal control method is presented that allows reuse of the obtained policy in a new situation, as the policy is more robust to slight deviations from the initial movement plan. To improve the robustness of the policy, we employ information-theoretic policy updates that explicitly operate on trajectory distributions instead of single trajectories. To ensure a stable and smooth policy update, the ?distance? is limited between the trajectory distributions of the old and the new control policies. The introduced bound offers a closed-form solution for the resulting policy and extends results from recent developments in stochastic optimal control. In contrast to many standard stochastic optimal control algorithms, the current approach can directly infer the system dynamics from data points, and hence can also be used for model-based reinforcement learning. This paper represents an extension of the paper by Lioutikov et al. (?Sample-Based Information-Theoretic Stochastic Optimal Control,? Proceedings of 2014 IEEE International Conference on Robotics and Automation (ICRA), IEEE, Piscataway, NJ, 2014, pp. 3896?3902). In addition to revisiting the content, an extensive theoretical comparison is presented of the approach with related work, additional aspects of the implementation are discussed, and further evaluations are introduced.}
}
• H. Liu and S. Yue, "An efficient method to structural static reanalysis with deleting support constraints," Structural Engineering and Mechanics, vol. 52, iss. 6, pp. 1121-1134, 2014.

Structural design is usually an optimization process. Numerous parameters such as the member shapes and sizes, the elasticity modulus of material, the locations of nodes and the support constraints can be selected as design variables. These variables are progressively revised in order to obtain a satisfactory structure. Each modification requires a fresh analysis for the displacements and stresses, and reanalysis can be employed to reduce the computational cost. This paper is focused on static reanalysis problem with modification of deleting some supports. An efficient reanalysis method is proposed. The method makes full use of the initial information and preserves the ease of implementation. Numerical examples show that the calculated results of the proposed method are the identical as those of the direct analysis, while the computational time is remarkably reduced.

@article{lirolem16505,
year = {2014},
title = {An efficient method to structural static reanalysis with deleting support constraints},
journal = {Structural Engineering and Mechanics},
volume = {52},
number = {6},
publisher = {Techno Press},
author = {H. Liu and Shigang Yue},
month = {December},
pages = {1121--1134},
keywords = {ARRAY(0x56147f816b58)},
url = {http://eprints.lincoln.ac.uk/16505/},
abstract = {Structural design is usually an optimization process. Numerous parameters such as the member shapes and sizes, the elasticity modulus of material, the locations of nodes and the support constraints can be selected as design variables. These variables are progressively revised in order to obtain a satisfactory structure. Each modification requires a fresh analysis for the displacements and stresses, and reanalysis can be employed to reduce the computational cost. This paper is focused on static reanalysis problem with modification of deleting some supports. An efficient reanalysis method is proposed. The method makes full use of the initial information and preserves the ease of implementation. Numerical examples show that the calculated results of the proposed method are the identical as those of the direct analysis, while the computational time is remarkably reduced.}
}
• D. Liu and S. Yue, "Spiking neural network for visual pattern recognition," in International Conference on Multisensor Fusion and Information Integration for Intelligent Systems, MFI 2014, 2014, pp. 1-5.

Most of visual pattern recognition algorithms try to emulate the mechanism of visual pathway within the human brain. Regarding of classic face recognition task, by using the spatiotemporal information extracted from Spiking neural network (SNN), batch learning rule and on-line learning rule stand out from their competitors. However, the former one simply considers the average pattern within the class, and the latter one just relies on the nearest relevant single pattern. In this paper, a novel learning rule and its SNN framework has been proposed. It considers all relevant patterns in the local domain around the undetermined sample rather than just nearest relevant single pattern. Experimental results show the proposed learning rule and its SNN framework obtains satisfactory testing results under the ORL face database.

@inproceedings{lirolem16638,
pages = {1--5},
booktitle = {International Conference on Multisensor Fusion and Information Integration for Intelligent Systems, MFI 2014},
title = {Spiking neural network for visual pattern recognition},
year = {2014},
journal = {Processing of 2014 International Conference on Multisensor Fusion and Information Integration for Intelligent Systems, MFI 2014},
author = {Daqi Liu and Shigang Yue},
publisher = {Institute of Electrical and Electronics Engineers Inc.},
abstract = {Most of visual pattern recognition algorithms try to emulate the mechanism of visual pathway within the human brain. Regarding of classic face recognition task, by using the spatiotemporal information extracted from Spiking neural network (SNN), batch learning rule and on-line learning rule stand out from their competitors. However, the former one simply considers the average pattern within the class, and the latter one just relies on the nearest relevant single pattern. In this paper, a novel learning rule and its SNN framework has been proposed. It considers all relevant patterns in the local domain around the undetermined sample rather than just nearest relevant single pattern. Experimental results show the proposed learning rule and its SNN framework obtains satisfactory testing results under the ORL face database.},
keywords = {ARRAY(0x56147f384da0)},
url = {http://eprints.lincoln.ac.uk/16638/}
}
• K. S. Luck, G. Neumann, E. Berger, J. Peters, and H. B. Amor, "Latent space policy search for robotics," in IEEE/RSJ Conference on Intelligent Robots and Systems (IROS), 2014, pp. 1434-1440.

Learning motor skills for robots is a hard task. In particular, a high number of degrees-of-freedom in the robot can pose serious challenges to existing reinforcement learning methods, since it leads to a highdimensional search space. However, complex robots are often intrinsically redundant systems and, therefore, can be controlled using a latent manifold of much smaller dimensionality. In this paper, we present a novel policy search method that performs efficient reinforcement learning by uncovering the low-dimensional latent space of actuator redundancies. In contrast to previous attempts at combining reinforcement learning and dimensionality reduction, our approach does not perform dimensionality reduction as a preprocessing step but naturally combines it with policy search. Our evaluations show that the new approach outperforms existing algorithms for learning motor skills with high-dimensional robots.

@inproceedings{lirolem25772,
month = {September},
pages = {1434--1440},
booktitle = {IEEE/RSJ Conference on Intelligent Robots and Systems (IROS)},
journal = {IEEE International Conference on Intelligent Robots and Systems},
year = {2014},
title = {Latent space policy search for robotics},
author = {K. S. Luck and G. Neumann and E. Berger and J. Peters and H. B. Amor},
abstract = {Learning motor skills for robots is a hard
task. In particular, a high number of degrees-of-freedom
in the robot can pose serious challenges to existing reinforcement
learning methods, since it leads to a highdimensional
search space. However, complex robots are
often intrinsically redundant systems and, therefore, can
be controlled using a latent manifold of much smaller
dimensionality. In this paper, we present a novel policy
search method that performs efficient reinforcement learning
by uncovering the low-dimensional latent space of
actuator redundancies. In contrast to previous attempts
at combining reinforcement learning and dimensionality
reduction, our approach does not perform dimensionality
reduction as a preprocessing step but naturally combines
it with policy search. Our evaluations show that the new
approach outperforms existing algorithms for learning
motor skills with high-dimensional robots.},
url = {http://eprints.lincoln.ac.uk/25772/},
keywords = {ARRAY(0x56147f5c61e0)}
}
• G. Maeda, M. Ewerton, R. Lioutikov, B. H. Amor, J. Peters, and G. Neumann, "Learning interaction for collaborative tasks with probabilistic movement primitives," in 14th IEEE-RAS International Conference on Humanoid Robots (Humanoids), 2014, pp. 527-534.

This paper proposes a probabilistic framework based on movement primitives for robots that work in collaboration with a human coworker. Since the human coworker can execute a variety of unforeseen tasks a requirement of our system is that the robot assistant must be able to adapt and learn new skills on-demand, without the need of an expert programmer. Thus, this paper leverages on the framework of imitation learning and its application to human-robot interaction using the concept of Interaction Primitives (IPs). We introduce the use of Probabilistic Movement Primitives (ProMPs) to devise an interaction method that both recognizes the action of a human and generates the appropriate movement primitive of the robot assistant. We evaluate our method on experiments using a lightweight arm interacting with a human partner and also using motion capture trajectories of two humans assembling a box. The advantages of ProMPs in relation to the original formulation for interaction are exposed and compared.

@inproceedings{lirolem25764,
author = {G. Maeda and M. Ewerton and R. Lioutikov and H. Ben Amor and J. Peters and G. Neumann},
pages = {527--534},
booktitle = {14th IEEE-RAS International Conference on Humanoid Robots (Humanoids)},
month = {November},
title = {Learning interaction for collaborative tasks with probabilistic movement primitives},
journal = {IEEE-RAS International Conference on Humanoid Robots},
volume = {2015-F},
year = {2014},
abstract = {This paper proposes a probabilistic framework
based on movement primitives for robots that work in collaboration
with a human coworker. Since the human coworker
can execute a variety of unforeseen tasks a requirement of our
system is that the robot assistant must be able to adapt and
learn new skills on-demand, without the need of an expert
programmer. Thus, this paper leverages on the framework
of imitation learning and its application to human-robot interaction
using the concept of Interaction Primitives (IPs).
We introduce the use of Probabilistic Movement Primitives
(ProMPs) to devise an interaction method that both recognizes
the action of a human and generates the appropriate movement
primitive of the robot assistant. We evaluate our method
on experiments using a lightweight arm interacting with a
human partner and also using motion capture trajectories of
two humans assembling a box. The advantages of ProMPs in
relation to the original formulation for interaction are exposed
and compared.},
keywords = {ARRAY(0x56147f66d5a0)},
url = {http://eprints.lincoln.ac.uk/25764/}
}
• F. Moreno, G. Cielniak, and T. Duckett, "Evaluation of laser range-finder mapping for agricultural spraying vehicles," in Towards autonomous robotic systems, A. Natraj, S. Cameron, C. Melhuish, and M. Witkowski, Eds., Springer Berlin Heidelberg, 2014, vol. 8069, pp. 210-221.

In this paper, we present a new application of laser range-finder sensing to agricultural spraying vehicles. The current generation of spraying vehicles use automatic controllers to maintain the height of the sprayer booms above the crop. However, these control systems are typically based on ultrasonic sensors mounted on the booms, which limits the accuracy of the measurements and the response of the controller to changes in the terrain, resulting in a sub-optimal spraying process. To overcome these limitations, we propose to use a laser scanner, attached to the front of the sprayer?s cabin, to scan the ground surface in front of the vehicle and to build a scrolling 3d map of the terrain. We evaluate the proposed solution in a series of field tests, demonstrating that the approach provides a more detailed and accurate representation of the environment than the current sonar-based solution, and which can lead to the development of more efficient boom control systems.

@incollection{lirolem19647,
year = {2014},
title = {Evaluation of laser range-finder mapping for agricultural spraying vehicles},
volume = {8069},
series = {Lecture Notes in Computer Science},
editor = {Ashutosh Natraj and Stephen Cameron and Chris Melhuish and Mark Witkowski},
note = {14th Annual Conference, TAROS 2013, Oxford, UK, August 28--30, 2013, Revised Selected Papers},
author = {Francisco-Angel Moreno and Grzegorz Cielniak and Tom Duckett},
publisher = {Springer Berlin Heidelberg},
pages = {210--221},
booktitle = {Towards autonomous robotic systems},
month = {June},
keywords = {ARRAY(0x56147f723ed8)},
url = {http://eprints.lincoln.ac.uk/19647/},
abstract = {In this paper, we present a new application of laser range-finder sensing to agricultural spraying vehicles. The current generation of spraying vehicles use automatic controllers to maintain the height of the sprayer booms above the crop. However, these control systems are typically based on ultrasonic sensors mounted on the booms, which limits the accuracy of the measurements and the response of the controller to changes in the terrain, resulting in a sub-optimal spraying process. To overcome these limitations, we propose to use a laser scanner, attached to the front of the sprayer?s cabin, to scan the ground surface in front of the vehicle and to build a scrolling 3d map of the terrain. We evaluate the proposed solution in a series of field tests, demonstrating that the approach provides a more detailed and accurate representation of the environment than the current sonar-based solution, and which can lead to the development of more efficient boom control systems.}
}
• G. Neumann, C. Daniel, A. Paraschos, A. Kupcsik, and J. Peters, "Learning modular policies for robotics," Frontiers in Computational Neuroscience, vol. 8, iss. JUN, 2014.

A promising idea for scaling robot learning to more complex tasks is to use elemental behaviors as building blocks to compose more complex behavior. Ideally, such building blocks are used in combination with a learning algorithm that is able to learn to select, adapt, sequence and co-activate the building blocks. While there has been a lot of work on approaches that support one of these requirements, no learning algorithm exists that unifies all these properties in one framework. In this paper we present our work on a unified approach for learning such a modular control architecture. We introduce new policy search algorithms that are based on information-theoretic principles and are able to learn to select, adapt and sequence the building blocks. Furthermore, we developed a new representation for the individual building block that supports co-activation and principled ways for adapting the movement. Finally, we summarize our experiments for learning modular control architectures in simulation and with real robots.

@article{lirolem25765,
year = {2014},
title = {Learning modular policies for robotics},
volume = {8},
journal = {Frontiers in Computational Neuroscience},
number = {JUN},
author = {G. Neumann and C. Daniel and A. Paraschos and A. Kupcsik and J. Peters},
publisher = {Frontiers Media},
month = {June},
abstract = {A promising idea for scaling robot learning to more complex tasks is to use elemental behaviors as building blocks to compose more complex behavior. Ideally, such building blocks are used in combination with a learning algorithm that is able to learn to select, adapt, sequence and co-activate the building blocks. While there has been a lot of work on approaches that support one of these requirements, no learning algorithm exists that unifies all these properties in one framework. In this paper we present our work on a unified approach for learning such a modular control architecture. We introduce new policy search algorithms that are based on information-theoretic principles and are able to learn to select, adapt and sequence the building blocks. Furthermore, we developed a new representation for the individual building block that supports co-activation and principled ways for adapting the movement. Finally, we summarize our experiments for learning modular control architectures in simulation and with real robots.},
url = {http://eprints.lincoln.ac.uk/25765/},
keywords = {ARRAY(0x56147f6ab148)}
}
• E. Rueckert, M. Mindt, J. Peters, and G. Neumann, "Robust policy updates for stochastic optimal control," in Humanoid Robots (Humanoids), 2014 14th IEEE-RAS International Conference on, 2014, pp. 388-393.

For controlling high-dimensional robots, most stochastic optimal control algorithms use approximations of the system dynamics and of the cost function (e.g., using linearizations and Taylor expansions). These approximations are typically only locally correct, which might cause instabilities in the greedy policy updates, lead to oscillations or the algorithms diverge. To overcome these drawbacks, we add a regularization term to the cost function that punishes large policy update steps in the trajectory optimization procedure. We applied this concept to the Approximate Inference Control method (AICO), where the resulting algorithm guarantees convergence for uninformative initial solutions without complex hand-tuning of learning rates. We evaluated our new algorithm on two simulated robotic platforms. A robot arm with five joints was used for reaching multiple targets while keeping the roll angle constant. On the humanoid robot Nao, we show how complex skills like reaching and balancing can be inferred from desired center of gravity or end effector coordinates.

@inproceedings{lirolem25754,
year = {2014},
title = {Robust policy updates for stochastic optimal control},
volume = {2015-F},
journal = {IEEE-RAS International Conference on Humanoid Robots},
author = {E. Rueckert and M. Mindt and J. Peters and G. Neumann},
pages = {388--393},
booktitle = {Humanoid Robots (Humanoids), 2014 14th IEEE-RAS International Conference on},
month = {November},
url = {http://eprints.lincoln.ac.uk/25754/},
keywords = {ARRAY(0x56147f852770)},
abstract = {For controlling high-dimensional robots, most stochastic optimal control algorithms use approximations of the system dynamics and of the cost function (e.g., using linearizations and Taylor expansions). These approximations are typically only locally correct, which might cause instabilities in the greedy policy updates, lead to oscillations or the algorithms diverge. To overcome these drawbacks, we add a regularization term to the cost function that punishes large policy update steps in the trajectory optimization procedure. We applied this concept to the Approximate Inference Control method (AICO), where the resulting algorithm guarantees convergence for uninformative initial solutions without complex hand-tuning of learning rates. We evaluated our new algorithm on two simulated robotic platforms. A robot arm with five joints was used for reaching multiple targets while keeping the roll angle constant. On the humanoid robot Nao, we show how complex skills like reaching and balancing can be inferred from desired center of gravity or end effector coordinates.}
}
• L. Shi, C. Zhang, and S. Yue, "Vector control IC for permanent magnet synchronous motor," in 2014 IEEE International Conference on Electron Devices and Solid-State Circuits (EDSSC), 2014.

This paper presents a full-digital vector control integrated circuit(IC) for permanent magnet synchronous motor (PMSM) with considering hardware structure. We adopted top-down and modular partitioning logic optimization design. Design specification of space vector pulse width modulation (SVPWM) unit, vector coordinate transformation are illustrated. All of the modules were implemented with pure hardware and designed with Verilog hardware description language (HDL). Moreover, the proposed design was verified by Simulink-Matlab and field programmable gate array (FPGA). \copyright 2014 IEEE.

@inproceedings{lirolem17531,
note = {Conference Code:111593},
author = {Licheng Shi and Chun Zhang and Shigang Yue},
publisher = {Institute of Electrical and Electronics Engineers Inc.},
booktitle = {2014 IEEE International Conference on Electron Devices and Solid-State Circuits (EDSSC)},
month = {June},
title = {Vector control IC for permanent magnet synchronous motor},
journal = {2014 IEEE International Conference on Electron Devices and Solid-State Circuits, EDSSC 2014},
year = {2014},
keywords = {ARRAY(0x56147f75f828)},
url = {http://eprints.lincoln.ac.uk/17531/},
abstract = {This paper presents a full-digital vector control integrated circuit(IC) for permanent magnet synchronous motor (PMSM) with considering hardware structure. We adopted top-down and modular partitioning logic optimization design. Design specification of space vector pulse width modulation (SVPWM) unit, vector coordinate transformation are illustrated. All of the modules were implemented with pure hardware and designed with Verilog hardware description language (HDL). Moreover, the proposed design was verified by Simulink-Matlab and field programmable gate array (FPGA). {\copyright} 2014 IEEE.}
}
• Y. Tang, J. Peng, and S. Yue, "Cyclic and simultaneous iterative methods to matrix equations of the form AiX Bi = Fi," Numerical Algorithms, vol. 66, iss. 2, pp. 379-397, 2014.

This paper deals with a general type of linear matrix equation problem. It presents new iterative algorithms to solve the matrix equations of the form AiX Bi = Fi. These algorithms are based on the incremental subgradient and the parallel subgradient methods. The convergence region of these algorithms are larger than other existing iterative algorithms. Finally, some experimental results are presented to show the efficiency of the proposed algorithms. Â\copyright 2013 Springer Science+Business Media New York.

@article{lirolem11574,
journal = {Numerical Algorithms},
year = {2014},
title = {Cyclic and simultaneous iterative methods to matrix equations of the form AiX Bi = Fi},
volume = {66},
number = {2},
author = {Yuchao Tang and Jigen Peng and Shigang Yue},
publisher = {Springer},
month = {June},
pages = {379--397},
abstract = {This paper deals with a general type of linear matrix equation problem. It presents new iterative algorithms to solve the matrix equations of the form AiX Bi = Fi. These algorithms are based on the incremental subgradient and the parallel subgradient methods. The convergence region of these algorithms are larger than other existing iterative algorithms. Finally, some experimental results are presented to show the efficiency of the proposed algorithms. {\^A}{\copyright} 2013 Springer Science+Business Media New York.},
url = {http://eprints.lincoln.ac.uk/11574/},
keywords = {ARRAY(0x56147f9a5410)}
}
• P. Urcola, T. Duckett, and G. Cielniak, "On-line trajectory planning for autonomous spraying vehicles," in International Workshop on Recent Advances in Agricultural Robotics, 2014.

In this paper, we present a new application of on-line trajectory planning for autonomous sprayers. The current generation of these vehicles use automatic controllers to maintain the height of the spraying booms above the crop. However, such systems are typically based on ultrasonic sensors mounted directly on the booms, which limits the response of the controller to changes in the terrain, resulting in a suboptimal spraying process. To overcome these limitations, we propose to use 3D maps of the terrain ahead of the spraying booms based on laser range-fi?nder measurements combined with GPS-based localisation. Four different boom trajectory planning solutions which utilise the 3D maps are considered and their accuracy and real-time suitability is evaluated based on data collected from ?field tests. The point optimisation and interpolation technique presents a practical solution demonstrating satisfactory performance under real-time constraints.

@inproceedings{lirolem14603,
year = {2014},
title = {On-line trajectory planning for autonomous spraying vehicles},
author = {Pablo Urcola and Tom Duckett and Grzegorz Cielniak},
booktitle = {International Workshop on Recent Advances in Agricultural Robotics},
month = {July},
abstract = {In this paper, we present a new application of on-line trajectory planning for autonomous sprayers. The current generation of these vehicles use automatic controllers to maintain the height of the spraying booms above the crop. However, such systems are typically based on ultrasonic sensors mounted directly on the booms, which limits the
response of the controller to changes in the terrain, resulting in a suboptimal spraying process. To overcome these limitations, we propose to use 3D maps of the terrain ahead of the spraying booms based on laser range-fi?nder measurements combined with GPS-based localisation. Four different boom trajectory planning solutions which utilise the 3D maps are considered and their accuracy and real-time suitability is evaluated based on data collected from ?field tests. The point optimisation and interpolation technique presents a practical solution demonstrating satisfactory performance under real-time constraints.},
url = {http://eprints.lincoln.ac.uk/14603/},
keywords = {ARRAY(0x56147f8527d0)}
}
• J. Xu, R. Wang, and S. Yue, "Bio-inspired classifier for road extraction from remote sensing imagery," Journal of Applied Remote Sensing, vol. 8, iss. 1, p. 83577, 2014.

An adaptive approach for road extraction inspired by the mechanism of primary visual cortex (V1) is proposed. The motivation is originated by the characteristics in the receptive field from V1. It has been proved that human or primate visual systems can distinguish useful cues from real scenes effortlessly while traditional computer vision techniques cannot accomplish this task easily. This idea motivates us to design a bio-inspired model for road extraction from remote sensing imagery. The proposed approach is an improved support vector machine (SVM) based on the pooling of feature vectors, using an improved Gaussian radial basis function (RBF) kernel with tuning on synaptic gains. The synaptic gains comprise the feature vectors through an iterative optimization process representing the strength and width of Gaussian RBF kernel. The synaptic gains integrate the excitation and inhibition stimuli based on internal connections from V1. The summation of synaptic gains contributes to pooling of feature vectors. The experimental results verify the correlation between the synaptic gain and classification rules, and then show better performance in comparison with hidden Markov model, SVM, and fuzzy classification approaches. Our contribution is an automatic approach to road extraction without prelabeling and postprocessing work. Another apparent advantage is that our method is robust for images taken even under complex weather conditions such as snowy and foggy weather. Â\copyright 2014 SPIE.

@article{lirolem14764,
pages = {083577},
month = {August},
author = {Jiawei Xu and Ruisheng Wang and Shigang Yue},
publisher = {Society of Photo-optical Instrumentation Engineers (SPIE)},
number = {1},
journal = {Journal of Applied Remote Sensing},
year = {2014},
volume = {8},
title = {Bio-inspired classifier for road extraction from remote sensing imagery},
abstract = {An adaptive approach for road extraction inspired by the mechanism of primary visual cortex (V1) is proposed. The motivation is originated by the characteristics in the receptive field from V1. It has been proved that human or primate visual systems can distinguish useful cues from real scenes effortlessly while traditional computer vision techniques cannot accomplish this task easily. This idea motivates us to design a bio-inspired model for road extraction from remote sensing imagery. The proposed approach is an improved support vector machine (SVM) based on the pooling of feature vectors, using an improved Gaussian radial basis function (RBF) kernel with tuning on synaptic gains. The synaptic gains comprise the feature vectors through an iterative optimization process representing the strength and width of Gaussian RBF kernel. The synaptic gains integrate the excitation and inhibition stimuli based on internal connections from V1. The summation of synaptic gains contributes to pooling of feature vectors. The experimental results verify the correlation between the synaptic gain and classification rules, and then show better performance in comparison with hidden Markov model, SVM, and fuzzy classification approaches. Our contribution is an automatic approach to road extraction without prelabeling and postprocessing work. Another apparent advantage is that our method is robust for images taken even under complex weather conditions such as snowy and foggy weather. {\^A}{\copyright} 2014 SPIE.},
keywords = {ARRAY(0x56147f5a7ed8)},
url = {http://eprints.lincoln.ac.uk/14764/}
}
• J. Xu and S. Yue, "Mimicking visual searching with integrated top down cues and low-level features," Neurocomputing, vol. 133, pp. 1-17, 2014.

Visual searching is a perception task involved with visual attention, attention shift and active scan of the visual environment for a particular object or feature. The key idea of our paper is to mimic the human visual searching under the static and dynamic scenes. To build up an artificial vision system that performs the visual searching could be helpful to medical and psychological application development to human machine interaction. Recent state-of-the-art researches focus on the bottom-up and top-down saliency maps. Saliency maps indicate that the saliency likelihood of each pixel, however, understanding the visual searching process can help an artificial vision system exam details in a way similar to human and they will be good for future robots or machine vision systems which is a deeper digest than the saliency map. This paper proposed a computational model trying to mimic human visual searching process and we emphasis the motion cues on the visual processing and searching. Our model analysis the attention shifts by fusing the top-down bias and bottom-up cues. This model also takes account the motion factor into the visual searching processing. The proposed model involves five modules: the pre-learning process; top-down biasing; bottom-up mechanism; multi-layer neural network and attention shifts. Experiment evaluation results via benchmark databases and real-time video showed the model demonstrated high robustness and real-time ability under complex dynamic scenes.

@article{lirolem13453,
pages = {1--17},
month = {June},
publisher = {Elsevier},
author = {Jiawei Xu and Shigang Yue},
title = {Mimicking visual searching with integrated top down cues and low-level features},
journal = {Neurocomputing},
volume = {133},
year = {2014},
abstract = {Visual searching is a perception task involved with visual attention, attention shift and active scan of the visual environment for a particular object or feature. The key idea of our paper is to mimic the human visual searching under the static and dynamic scenes. To build up an artificial vision system that performs the visual searching could be helpful to medical and psychological application development to human machine interaction. Recent state-of-the-art researches focus on the bottom-up and top-down saliency maps. Saliency maps indicate that the saliency likelihood of each pixel, however, understanding the visual searching process can help an artificial vision system exam details in a way similar to human and they will be good for future robots or machine vision systems which is a deeper digest than the saliency map. This paper proposed a computational model trying to mimic human visual searching process and we emphasis the motion cues on the visual processing and searching. Our model analysis the attention shifts by fusing the top-down bias and bottom-up cues. This model also takes account the motion factor into the visual searching processing. The proposed model involves five modules: the pre-learning process; top-down biasing; bottom-up mechanism; multi-layer neural network and attention shifts. Experiment evaluation results via benchmark databases and real-time video showed the model demonstrated high robustness and real-time ability under complex dynamic scenes.},
keywords = {ARRAY(0x56147f66b3a0)},
url = {http://eprints.lincoln.ac.uk/13453/}
}
• S. Yue, K. Harmer, K. Guo, K. Adams, and A. Hunter, "Automatic blush detection in ?concealed information? test using visual stimuli," International Journal of Data Mining, Modelling and Management, vol. 6, iss. 2, pp. 187-201, 2014.

Blushing has been identified as an indicator of deception, shame, anxiety and embarrassment. Although normally associated with the skin coloration of the face, a blush response also affects skin surface temperature. In this paper, an approach to detect a blush response automatically is presented using the Argus P7225 thermal camera from e2v. The algorithm was tested on a sample population of 51 subjects, while using visual stimuli to elicit a response, and achieved recognition rates of \texttt\char12677\% TPR and \texttt\char12660\% TNR, indicating a thermal image sensor is the prospective device to pick up subtle temperature change synchronised with stimuli.

@article{lirolem14660,
month = {June},
pages = {187--201},
author = {Shigang Yue and Karl Harmer and Kun Guo and Karen Adams and Andrew Hunter},
publisher = {Inderscience},
year = {2014},
title = {Automatic blush detection in ?concealed information? test using visual stimuli},
journal = {International Journal of Data Mining, Modelling and Management},
volume = {6},
number = {2},
url = {http://eprints.lincoln.ac.uk/14660/},
keywords = {ARRAY(0x56147f6c5c70)},
abstract = {Blushing has been identified as an indicator of deception, shame, anxiety and embarrassment. Although normally associated with the skin coloration of the face, a blush response also affects skin surface temperature. In this paper, an approach to detect a blush response automatically is presented using the Argus P7225 thermal camera from e2v. The algorithm was tested on a sample population of 51 subjects, while using visual stimuli to elicit a response,  and achieved recognition rates of {\texttt{\char126}}77\% TPR and {\texttt{\char126}}60\% TNR, indicating a thermal image sensor is the prospective device to pick up subtle temperature change synchronised with stimuli.}
}
• G. Zahi and S. Yue, "Reducing motion blurring associated with temporal summation in low light scenes for image quality enhancement," in International Conference on Multisensor Fusion and Information Integration for Intelligent Systems, MFI 2014, 2014, pp. 1-5.

In order to see under low light conditions nocturnal insects rely on neural strategies based on combinations of spatial and temporal summations. Though these summation techniques when modelled are effective in improving the quality of low light images, using the temporal summation in scenes where image velocity is high only come at a cost of motion blurring in the output scenes. Most recent research has been towards reducing motion blurring in scenes where motion is caused by moving objects rather than effectively reducing motion blurring in scenes where motion is caused by moving cameras. This makes it impossible to implement the night vision algorithm in moving robots or cars that operate under low light conditions. In this paper we present a generic new method that can replace the normal temporal summation in scenes where motion is detected. The proposed method is both suitable for motion caused by moving objects as well as moving cameras. The effectiveness of this new generic method is shown with relevant supporting experiments.

@inproceedings{lirolem16637,
author = {Gabriel Zahi and Shigang Yue},
publisher = {Institute of Electrical and Electronics Engineers Inc.},
year = {2014},
title = {Reducing motion blurring associated with temporal summation in low light scenes for image quality enhancement},
journal = {Processing of 2014 International Conference on Multisensor Fusion and Information Integration for Intelligent Systems, MFI 2014},
booktitle = {International Conference on Multisensor Fusion and Information Integration for Intelligent Systems, MFI 2014},
pages = {1--5},
url = {http://eprints.lincoln.ac.uk/16637/},
keywords = {ARRAY(0x56147f602928)},
abstract = {In order to see under low light conditions nocturnal insects rely on neural strategies based on combinations of spatial and temporal summations. Though these summation techniques when modelled are effective in improving the quality of low light images, using the temporal summation in scenes where image velocity is high only come at a cost of motion blurring in the output scenes. Most recent research has been towards reducing motion blurring in scenes where motion is caused by moving objects rather than effectively reducing motion blurring in scenes where motion is caused by moving cameras. This makes it impossible to implement the night vision algorithm in moving robots or cars that operate under low light conditions. In this paper we present a generic new method that can replace the normal temporal summation in scenes where motion is detected. The proposed method is both suitable for motion caused by moving objects as well as moving cameras. The effectiveness of this new generic method is shown with relevant supporting experiments.}
}
• Z. Zhang, S. Yue, M. Liao, and F. Long, "Danger theory based artificial immune system solving dynamic constrained single-objective optimization," Soft Computing, vol. 18, iss. 1, pp. 185-206, 2014.

In this paper, we propose an artificial immune system (AIS) based on the danger theory in immunology for solving dynamic nonlinear constrained single-objective optimization problems with time-dependent design spaces. Such proposed AIS executes orderly three modules-danger detection, immune evolution and memory update. The first module identifies whether there are changes in the optimization environment and decides the environmental level, which helps for creating the initial population in the environment and promoting the process of solution search. The second module runs a loop of optimization, in which three sub-populations each with a dynamic size seek simultaneously the location of the optimal solution along different directions through co-evolution. The last module stores and updates the memory cells which help the first module decide the environmental level. This optimization system is an on-line and adaptive one with the characteristics of simplicity, modularization and co-evolution. The numerical experiments and the results acquired by the nonparametric statistic procedures, based on 22 benchmark problems and an engineering problem, show that the proposed approach performs globally well over the compared algorithms and is of potential use for many kinds of dynamic optimization problems. Â\copyright 2013 Springer-Verlag Berlin Heidelberg.

@article{lirolem11410,
author = {Zhuhong Zhang and Shigang Yue and Min Liao and Fei Long},
publisher = {Springer Verlag (Germany)},
month = {January},
pages = {185--206},
title = {Danger theory based artificial immune system solving dynamic constrained single-objective optimization},
year = {2014},
journal = {Soft Computing},
volume = {18},
number = {1},
abstract = {In this paper, we propose an artificial immune system (AIS) based on the danger theory in immunology for solving dynamic nonlinear constrained single-objective optimization problems with time-dependent design spaces. Such proposed AIS executes orderly three modules-danger detection, immune evolution and memory update. The first module identifies whether there are changes in the optimization environment and decides the environmental level, which helps for creating the initial population in the environment and promoting the process of solution search. The second module runs a loop of optimization, in which three sub-populations each with a dynamic size seek simultaneously the location of the optimal solution along different directions through co-evolution. The last module stores and updates the memory cells which help the first module decide the environmental level. This optimization system is an on-line and adaptive one with the characteristics of simplicity, modularization and co-evolution. The numerical experiments and the results acquired by the nonparametric statistic procedures, based on 22 benchmark problems and an engineering problem, show that the proposed approach performs globally well over the compared algorithms and is of potential use for many kinds of dynamic optimization problems. {\^A}{\copyright} 2013 Springer-Verlag Berlin Heidelberg.},
url = {http://eprints.lincoln.ac.uk/11410/},
keywords = {ARRAY(0x56147f986e40)}
}

### 2013

• P. Baxter, J. D. Greeff, R. Wood, and T. Belpaeme, "Modelling concept prototype competencies using a developmental memory model," Paladyn, Journal of Behavioral Robotics, vol. 3, iss. 4, pp. 200-208, 2013.

The use of concepts is fundamental to human-level cognition, but there remain a number of open questions as to the structures supporting this competence. Specifically, it has been shown that humans use concept prototypes, a flexible means of representing concepts such that it can be used both for categorisation and for similarity judgements. In the context of autonomous robotic agents, the processes by which such concept functionality could be acquired would be particularly useful, enabling flexible knowledge representation and application. This paper seeks to explore this issue of autonomous concept acquisition. By applying a set of structural and operational principles, that support a wide range of cognitive competencies, within a developmental framework, the intention is to explicitly embed the development of concepts into a wider framework of cognitive processing. Comparison with a benchmark concept modelling system shows that the proposed approach can account for a number of features, namely concept-based classification, and its extension to prototype-like functionality.

@article{lirolem23077,
month = {April},
pages = {200--208},
author = {Paul Baxter and Joachim De Greeff and Rachel Wood and Tony Belpaeme},
publisher = {De Gruyter/Springer},
note = {Issue cover date: December 2012},
year = {2013},
volume = {3},
title = {Modelling concept prototype competencies using a developmental memory model},
journal = {Paladyn, Journal of Behavioral Robotics},
number = {4},
url = {http://eprints.lincoln.ac.uk/23077/},
keywords = {ARRAY(0x56147f71a320)},
abstract = {The use of concepts is fundamental to human-level cognition, but there remain a number of open questions as to the structures supporting this competence. Specifically, it has been shown that humans use concept prototypes, a flexible means of representing concepts such that it can be used both for categorisation and for similarity judgements. In the context of autonomous robotic agents, the processes by which such concept functionality could be acquired would be particularly useful, enabling flexible knowledge representation and application. This paper seeks to explore this issue of autonomous concept acquisition. By applying a set of structural and operational principles, that support a wide range of cognitive competencies, within a developmental framework, the intention is to explicitly embed the development of concepts into a wider framework of cognitive processing. Comparison with a benchmark concept modelling system shows that the proposed approach can account for a number of features, namely concept-based classification, and its extension to prototype-like functionality.}
}
• P. E. Baxter, J. de Greeff, and T. Belpaeme, "Cognitive architecture for human?robot interaction: towards behavioural alignment," Biologically Inspired Cognitive Architectures, vol. 6, pp. 30-39, 2013.

Abstract With increasingly competent robotic systems desired and required for social human?robot interaction comes the necessity for more complex means of control. Cognitive architectures (specifically the perspective where principles of structure and function are sought to account for multiple cognitive competencies) have only relatively recently been considered for applica- tion to this domain. In this paper, we describe one such set of architectural principles ? acti- vation dynamics over a developmental distributed associative substrate ? and show how this enables an account of a fundamental competence for social cognition: multi-modal behavioural alignment. Data from real human?robot interactions is modelled using a computational system based on this set of principles to demonstrate how this competence can therefore be consid- ered as embedded in wider cognitive processing. It is shown that the proposed system can model the behavioural characteristics of human subjects. While this study is a simulation using real interaction data, the results obtained validate the application of the proposed approach to this issue.

@article{lirolem23076,
author = {Paul E. Baxter and Joachim de Greeff and Tony Belpaeme},
publisher = {Elsevier B.V.},
month = {October},
pages = {30--39},
journal = {Biologically Inspired Cognitive Architectures},
volume = {6},
year = {2013},
title = {Cognitive architecture for human?robot interaction: towards behavioural alignment},
abstract = {Abstract With increasingly competent robotic systems desired and required for social human?robot interaction comes the necessity for more complex means of control. Cognitive architectures (specifically the perspective where principles of structure and function are sought to account for multiple cognitive competencies) have only relatively recently been considered for applica- tion to this domain. In this paper, we describe one such set of architectural principles ? acti- vation dynamics over a developmental distributed associative substrate ? and show how this enables an account of a fundamental competence for social cognition: multi-modal behavioural alignment. Data from real human?robot interactions is modelled using a computational system based on this set of principles to demonstrate how this competence can therefore be consid- ered as embedded in wider cognitive processing. It is shown that the proposed system can model the behavioural characteristics of human subjects. While this study is a simulation using real interaction data, the results obtained validate the application of the proposed approach to this issue.},
keywords = {ARRAY(0x56147f6fac38)},
url = {http://eprints.lincoln.ac.uk/23076/}
}
• N. Bellotto, M. Hanheide, and N. V. de Weghe, "Qualitative design and implementation of human-robot spatial interactions," in International Conference on Social Robotics (ICSR), 2013.

Despite the large number of navigation algorithms available for mobile robots, in many social contexts they often exhibit inopportune motion behaviours in proximity of people, often with very "unnatural" movements due to the execution of segmented trajectories or the sudden activation of safety mechanisms (e.g., for obstacle avoidance). We argue that the reason of the problem is not only the difficulty of modelling human behaviours and generating opportune robot control policies, but also the way human-robot spatial interactions are represented and implemented. In this paper we propose a new methodology based on a qualitative representation of spatial interactions, which is both flexible and compact, adopting the well-defined and coherent formalization of Qualitative Trajectory Calculus (QTC). We show the potential of a QTC-based approach to abstract and design complex robot behaviours, where the desired robot's behaviour is represented together with its actual performance in one coherent approach, focusing on spatial interactions rather than pure navigation problems.

@inproceedings{lirolem11637,
title = {Qualitative design and implementation of human-robot spatial interactions},
year = {2013},
author = {Nicola Bellotto and Marc Hanheide and Nico Van de Weghe},
publisher = {Springer},
month = {October},
booktitle = {International Conference on Social Robotics (ICSR)},
abstract = {Despite the large number of navigation algorithms available for mobile robots, in many social contexts they often exhibit inopportune motion behaviours in proximity of people, often with very "unnatural" movements due to the execution of segmented trajectories or the sudden activation of safety mechanisms (e.g., for obstacle avoidance). We argue that the reason of the problem is not only the difficulty of modelling human behaviours and generating opportune robot control policies, but also the way human-robot spatial interactions are represented and implemented.
In this paper we propose a new methodology based on a qualitative representation of spatial interactions, which is both flexible and compact, adopting the well-defined and coherent formalization of Qualitative Trajectory Calculus (QTC). We show the potential of a QTC-based approach to abstract and design complex robot behaviours, where the desired robot's behaviour is represented together with its actual performance in one coherent approach, focusing on spatial interactions rather than pure navigation problems.},
url = {http://eprints.lincoln.ac.uk/11637/},
keywords = {ARRAY(0x56147f709e68)}
}
• N. Bellotto, "A multimodal smartphone interface for active perception by visually impaired," in IEEE SMC Int. Workshop on Human-Machine Systems, Cyborgs and Enhancing Devices (HUMASCEND), 2013.

The diffuse availability of mobile devices, such as smartphones and tablets, has the potential to bring substantial benefits to the people with sensory impairments. The solution proposed in this paper is part of an ongoing effort to create an accurate obstacle and hazard detector for the visually impaired, which is embedded in a hand-held device. In particular, it presents a proof of concept for a multimodal interface to control the orientation of a smartphone's camera, while being held by a person, using a combination of vocal messages, 3D sounds and vibrations. The solution, which is to be evaluated experimentally by users, will enable further research in the area of active vision with human-in-the-loop, with potential application to mobile assistive devices for indoor navigation of visually impaired people.

@inproceedings{lirolem11636,
publisher = {IEEE},
author = {Nicola Bellotto},
title = {A multimodal smartphone interface for active perception by visually impaired},
year = {2013},
month = {October},
booktitle = {IEEE SMC Int. Workshop on Human-Machine Systems, Cyborgs and Enhancing Devices (HUMASCEND)},
keywords = {ARRAY(0x561479dc7440)},
url = {http://eprints.lincoln.ac.uk/11636/},
abstract = {The diffuse availability of mobile devices, such as smartphones and tablets, has the potential to bring substantial benefits to the people with sensory impairments. The solution proposed in this paper is part of an ongoing effort to create an accurate obstacle and hazard detector for the visually impaired, which is embedded in a hand-held device. In particular, it presents a proof of concept for a multimodal interface to control the orientation of a smartphone's camera, while being held by a person, using a combination of vocal messages, 3D sounds and vibrations. The solution, which is to be evaluated experimentally by users, will enable further research in the area of active vision with human-in-the-loop, with potential application to mobile assistive devices for indoor navigation of visually impaired people.}
}
• C. Cherino, G. Cielniak, P. Dickinson, and P. Geril, "FUBUTEC-ECEC'2013," , 2013.

This edition covers Risk Management, Management Techniques, Production Design Optimization and Video Applications

@manual{lirolem22903,
type = {Documentation},
year = {2013},
title = {FUBUTEC-ECEC'2013},
author = {Cristina Cherino and Grzegorz Cielniak and Patrick Dickinson and Philippe Geril},
publisher = {EUROSIS-ETI BVBA},
note = {FUBUTEC'2013, Future Business Technology Conference, June 10-12, 2013, University of Lincoln, Lincoln, UK},
month = {May},
keywords = {ARRAY(0x56147f820188)},
url = {http://eprints.lincoln.ac.uk/22903/},
abstract = {This edition covers Risk Management, Management Techniques, Production Design Optimization and Video Applications}
}
• G. Cielniak, N. Bellotto, and T. Duckett, "Integrating mobile robotics and vision with undergraduate computer science," IEEE Transactions on Education, vol. 56, iss. 1, pp. 48-53, 2013.

This paper describes the integration of robotics education into an undergraduate Computer Science curriculum. The proposed approach delivers mobile robotics as well as covering the closely related field of Computer Vision, and is directly linked to the research conducted at the authors? institution. The paper describes the most relevant details of the module content and assessment strategy, paying particular attention to the practical sessions using Rovio mobile robots. The specific choices are discussed that were made with regard to the mobile platform, software libraries and lab environment. The paper also presents a detailed qualitative and quantitative analysis of student results, including the correlation between student engagement and performance, and discusses the outcomes of this experience.

@article{lirolem6031,
number = {1},
journal = {IEEE Transactions on Education},
year = {2013},
title = {Integrating mobile robotics and vision with undergraduate computer science},
volume = {56},
publisher = {The IEEE Education Society},
author = {Grzegorz Cielniak and Nicola Bellotto and Tom Duckett},
pages = {48--53},
month = {February},
keywords = {ARRAY(0x56147f87e798)},
url = {http://eprints.lincoln.ac.uk/6031/},
abstract = {This paper describes the integration of robotics education into an undergraduate Computer Science curriculum. The proposed approach delivers mobile robotics as well as covering the closely related field of Computer Vision, and is directly linked to the research conducted at the authors? institution. The paper describes the most relevant details of the module content and assessment strategy, paying particular attention to the practical sessions using Rovio mobile robots. The specific choices are discussed that were made with regard to the mobile platform, software libraries and lab environment. The paper also presents a detailed qualitative and quantitative analysis of student results, including the correlation between student engagement and performance, and discusses the outcomes of this experience.}
}
• C. Daniel, G. Neumann, O. Kroemer, and J. Peters, "Learning sequential motor tasks," in IEEE International Conference on Robotics and Automation, 2013, pp. 2626-2632.

Many real robot applications require the sequential use of multiple distinct motor primitives. This requirement implies the need to learn the individual primitives as well as a strategy to select the primitives sequentially. Such hierarchical learning problems are commonly either treated as one complex monolithic problem which is hard to learn, or as separate tasks learned in isolation. However, there exists a strong link between the robots strategy and its motor primitives. Consequently, a consistent framework is needed that can learn jointly on the level of the individual primitives and the robots strategy. We present a hierarchical learning method which improves individual motor primitives and, simultaneously, learns how to combine these motor primitives sequentially to solve complex motor tasks. We evaluate our method on the game of robot hockey, which is both difficult to learn in terms of the required motor primitives as well as its strategic elements.

@inproceedings{lirolem25781,
journal = {Proceedings - IEEE International Conference on Robotics and Automation},
title = {Learning sequential motor tasks},
year = {2013},
month = {May},
booktitle = {IEEE International Conference on Robotics and Automation},
pages = {2626--2632},
author = {C. Daniel and G. Neumann and O. Kroemer and J. Peters},
note = {cited By 3},
url = {http://eprints.lincoln.ac.uk/25781/},
keywords = {ARRAY(0x56147f70cc58)},
abstract = {Many real robot applications require the sequential use of multiple distinct motor primitives. This requirement implies the need to learn the individual primitives as well as a strategy to select the primitives sequentially. Such hierarchical learning problems are commonly either treated as one complex monolithic problem which is hard to learn, or as separate tasks learned in isolation. However, there exists a strong link between the robots strategy and its motor primitives. Consequently, a consistent framework is needed that can learn jointly on the level of the individual primitives and the robots strategy. We present a hierarchical learning method which improves individual motor primitives and, simultaneously, learns how to combine these motor primitives sequentially to solve complex motor tasks. We evaluate our method on the game of robot hockey, which is both difficult to learn in terms of the required motor primitives as well as its strategic elements.}
}
• M. P. Deisenroth, G. Neumann, and J. Peters, "A survey on policy search for robotics," Foundations and Trends in Robotics, vol. 2, iss. 1-2, pp. 388-403, 2013.

Policy search is a subfield in reinforcement learning which focuses on finding good parameters for a given policy parametrization. It is well suited for robotics as it can cope with high-dimensional state and action spaces, one of the main challenges in robot learning. We review recent successes of both model-free and model-based policy search in robot learning. Model-free policy search is a general approach to learn policies based on sampled trajectories. We classify model-free methods based on their policy evaluation strategy, policy update strategy, and exploration strategy and present a unified view on existing algorithms. Learning a policy is often easier than learning an accurate forward model, and, hence, model-free methods are more frequently used in practice. However, for each sampled trajectory, it is necessary to interact with the * Both authors contributed equally. robot, which can be time consuming and challenging in practice. Modelbased policy search addresses this problem by first learning a simulator of the robot?s dynamics from data. Subsequently, the simulator generates trajectories that are used for policy learning. For both modelfree and model-based policy search methods, we review their respective properties and their applicability to robotic systems.

@article{lirolem28029,
year = {2013},
volume = {2},
journal = {Foundations and Trends in Robotics},
title = {A survey on policy search for robotics},
number = {1-2},
month = {August},
pages = {388--403},
publisher = {Now Publishers},
author = {M. P. Deisenroth and G. Neumann and J. Peters},
url = {http://eprints.lincoln.ac.uk/28029/},
keywords = {ARRAY(0x56147f871e88)},
abstract = {Policy search is a subfield in reinforcement learning which focuses on
finding good parameters for a given policy parametrization. It is well
suited for robotics as it can cope with high-dimensional state and action
spaces, one of the main challenges in robot learning. We review recent
successes of both model-free and model-based policy search in robot
learning.
Model-free policy search is a general approach to learn policies
based on sampled trajectories. We classify model-free methods based on
their policy evaluation strategy, policy update strategy, and exploration
strategy and present a unified view on existing algorithms. Learning a
policy is often easier than learning an accurate forward model, and,
hence, model-free methods are more frequently used in practice. However,
for each sampled trajectory, it is necessary to interact with the
* Both authors contributed equally.
robot, which can be time consuming and challenging in practice. Modelbased
policy search addresses this problem by first learning a simulator
of the robot?s dynamics from data. Subsequently, the simulator generates
trajectories that are used for policy learning. For both modelfree
and model-based policy search methods, we review their respective
properties and their applicability to robotic systems.}
}
• T. Duckett, M. Hanheide, T. Krajnik, J. P. Fentanes, and C. Dondrup, "Spatio-temporal representation for cognitive control in long-term scenarios," in International IEEE/EPSRC Workshop on Autonomous Cognitive Robotics, 2013.

The FP-7 Integrated Project STRANDS [1] is aimed at producing intelligent mobile robots that are able to operate robustly for months in dynamic human environments. To achieve long-term autonomy, the robots would need to understand the environment and how it changes over time. For that, we will have to develop novel approaches to extract 3D shapes, objects, people, and models of activity from sensor data gathered during months of autonomous operation. So far, the environment models used in mobile robotics have been tailored to capture static scenes and environment variations are largely treated as noise. Therefore, utilization of the static models in ever-changing, real world environments is difficult. We propose to represent the environment?s spatio-temporal dynamics by its frequency spectrum.

@inproceedings{lirolem14893,
booktitle = {International IEEE/EPSRC Workshop on Autonomous Cognitive Robotics},
month = {March},
title = {Spatio-temporal representation for cognitive control in long-term scenarios},
year = {2013},
author = {Tom Duckett and Marc Hanheide and Tomas Krajnik and Jaime Pulido Fentanes and Christian Dondrup},
abstract = {The FP-7 Integrated Project STRANDS [1] is aimed at producing intelligent mobile robots that are able to operate robustly for months in dynamic human environments. To achieve long-term autonomy, the robots would need to understand the environment and how it changes over time. For that, we will have to develop novel approaches to extract 3D shapes, objects, people, and models of activity from sensor data gathered during months of autonomous operation.
So far, the environment models used in mobile robotics have been tailored to capture static scenes and environment variations are largely treated as noise. Therefore, utilization of the static models in ever-changing, real world environments is difficult. We propose to represent the environment?s spatio-temporal dynamics by its frequency spectrum.},
keywords = {ARRAY(0x56147f8325f0)},
url = {http://eprints.lincoln.ac.uk/14893/}
}
• T. Duckett and A. Lilienthal, "Editorial," Robotics and Autonomous Systems, vol. 61, iss. 10, pp. 1049-1050, 2013.

.

@article{lirolem12768,
note = {Selected Papers from the 5th European Conference on Mobile Robots (ECMR 2011)},
publisher = {Elsevier for North-Holland / Intelligent Autonomous Systems (IAS) Society},
author = {Tom Duckett and Achim Lilienthal},
pages = {1049--1050},
month = {October},
number = {10},
year = {2013},
volume = {61},
title = {Editorial},
journal = {Robotics and Autonomous Systems},
abstract = {.},
keywords = {ARRAY(0x56147f84fc30)},
url = {http://eprints.lincoln.ac.uk/12768/}
}
• T. Krajnik, M. Nitsche, J. Faigl, M. Mejail, L. Preucil, and T. Duckett, "External localization system for mobile robotics," in 16th International Conference on Advanced Robotics (ICAR 2013), 2013.

We present a fast and precise vision-based software intended for multiple robot localization. The core component of the proposed localization system is an efficient method for black and white circular pattern detection. The method is robust to variable lighting conditions, achieves sub-pixel precision, and its computational complexity is independent of the processed image size. With off-the-shelf computational equipment and low-cost camera, its core algorithm is able to process hundreds of images per second while tracking hundreds of objects with millimeter precision. We propose a mathematical model of the method that allows to calculate its precision, area of coverage, and processing speed from the camera?s intrinsic parameters and hardware?s processing capacity. The correctness of the presented model and performance of the algorithm in real-world conditions are verified in several experiments. Apart from the method description, we also publish its source code; so, it can be used as an enabling technology for various mobile robotics problems.

@inproceedings{lirolem12670,
title = {External localization system for mobile robotics},
year = {2013},
journal = {International Conference on Advanced Robotics, ICAR 2013 (Proceedings)},
publisher = {IEEE},
author = {Tomas Krajnik and Matias Nitsche and Jan Faigl and Marta Mejail and Libor Preucil and Tom Duckett},
month = {November},
booktitle = {16th International Conference on Advanced Robotics (ICAR 2013)},
abstract = {We present a fast and precise vision-based software intended for multiple robot localization. The core component of
the proposed localization system is an efficient method for black and white circular pattern detection. The method is robust to variable lighting conditions, achieves sub-pixel precision, and its computational complexity is independent of the processed image size. With off-the-shelf computational equipment and low-cost camera, its core algorithm is able to process hundreds of images per second while tracking hundreds of objects with millimeter precision. We propose a mathematical model of the method that allows to calculate its precision, area of coverage, and processing speed from the camera?s intrinsic parameters and hardware?s processing capacity. The correctness of the presented model and
performance of the algorithm in real-world conditions are verified in several experiments. Apart from the method description, we also publish its source code; so, it can be used as an enabling technology for various mobile robotics problems.},
url = {http://eprints.lincoln.ac.uk/12670/},
keywords = {ARRAY(0x56147f649290)}
}
• A. Kucukyilmaz, T. M. Sezgin, and C. Basdogan, "Role allocation through haptics in physical human-robot interaction," in 21st Signal Processing and Communications Applications Conference 2013, (SIU'13), 2013, pp. 1-5.

This paper presents a summary of our efforts to enable dynamic role allocation between humans and robots in physical collaboration tasks. A major goal in physical human-robot interaction research is to develop tacit and natural communication between partners. In previous work, we suggested that the communication between a human and a robot would benefit from a decision making process in which the robot can dynamically adjust its control level during the task based on the intentions of the human. In order to do this, we define leader and follower roles for the partners, and using a role exchange mechanism, we enable the partners to negotiate solely through force information to exchange roles. We show that when compared to an ?equal control? condition, the role exchange mechanism improves task performance and the joint efficiency of the partners.

@inproceedings{lirolem29369,
month = {April},
booktitle = {21st Signal Processing and Communications Applications Conference 2013, (SIU'13)},
pages = {1--5},
author = {Ayse Kucukyilmaz and Tevfik Metin Sezgin and Cagatay Basdogan},
publisher = {IEEE},
year = {2013},
title = {Role allocation through haptics in physical human-robot interaction},
url = {http://eprints.lincoln.ac.uk/29369/},
keywords = {ARRAY(0x56147f842dc8)},
abstract = {This paper presents a summary of our efforts to enable dynamic role allocation between humans and robots in physical collaboration tasks. A major goal in physical human-robot interaction research is to develop tacit and natural communication between partners. In previous work, we suggested that the communication between a human and a robot would benefit from a decision making process in which the robot can dynamically adjust its control level during the task based on the intentions of the human. In order to do this, we define leader and follower roles for the partners, and using a role exchange mechanism, we enable the partners to negotiate solely through force information to exchange roles. We show that when compared to an ?equal control? condition, the role exchange mechanism improves task performance and the joint efficiency of the partners.}
}
• A. Kucukyilmaz, "Haptic role allocation and intention negotiation in human-robot collaboration," PhD Thesis, 2013.

This dissertation aims to present a perspective to build more natural shared control systems for physical human-robot cooperation. As the tasks become more complex and more dynamic, many shared control schemes fail to meet the expectation of an effortless interaction that resembles human-human sensory communication. Since such systems are mainly built to improve task performance, the richness of sensory communication is of secondary concern. We suggest that effective cooperation can be achieved when the human?s and the robot?s roles within the task are dynamically updated during the execution of the task. These roles define states for the system, in which the robot?s control leads or follows the human?s actions. In such a system, a state transition can occur at certain times if the robot can determine the user?s intention for gaining/relinquishing control. Specifically, with these state transitions we assign certain roles to the human and the robot. We believe that only by employing the robot with tools to change its behavior during collaboration, we can improve the collaboration experience. We explore how human-robot cooperation in virtual and physical worlds can be improved using a force-based role-exchange mechanism. Our findings indicate that the proposed role exchange framework is beneficial in a sense that it can improve task performance and the efficiency of the partners during the task, and decrease the energy requirement of the human. Moreover, the results imply that the subjective acceptability of the proposed model is attained only when role exchanges are performed in a smooth and transparent fashion. Finally, we illustrate that adding extra sensory cues on top of a role exchange scheme is useful for improving the sense of interaction during the task, as well as making the system more comfortable and easier to use, and the task more enjoyable.

@phdthesis{lirolem29374,
title = {Haptic role allocation and intention negotiation in human-robot collaboration},
year = {2013},
school = {Koc University},
author = {Ayse Kucukyilmaz},
month = {June},
url = {http://eprints.lincoln.ac.uk/29374/},
keywords = {ARRAY(0x56147f5d5658)},
abstract = {This dissertation aims to present a perspective to build more natural shared control systems for physical human-robot cooperation. As the tasks become more complex and more dynamic, many shared control schemes fail to meet the expectation of an effortless interaction that resembles human-human sensory communication. Since such systems are mainly built to improve task performance, the richness of sensory communication is of secondary concern. We suggest that effective cooperation can be achieved when the human?s and the robot?s roles within the task are dynamically updated during the execution of the task. These roles define states for the system, in which the robot?s control leads or follows the human?s actions. In such a system, a state transition can occur at certain times if the robot can determine the user?s intention for gaining/relinquishing control. Specifically, with these state transitions we assign certain roles to the human and the robot. We believe that only by employing the robot with tools to change its behavior during collaboration, we can improve the collaboration experience.

We explore how human-robot cooperation in virtual and physical worlds can be improved using a force-based role-exchange mechanism. Our findings indicate that the proposed role exchange framework is beneficial in a sense that it can improve task performance and the efficiency of the partners during the task, and decrease the energy requirement of the human. Moreover, the results imply that the subjective acceptability of the proposed model is attained only when role exchanges are performed in a smooth and transparent fashion. Finally, we illustrate that adding extra sensory cues on top of a role exchange scheme is useful for improving the sense of interaction during the task, as well as making the system more comfortable and easier to use, and the task more enjoyable.}
}
• A. Kucukyilmaz, T. M. Sezgin, and C. Basdogan, "Intention recognition for dynamic role exchange in haptic collaboration," IEEE Transactions on Haptics, vol. 6, iss. 1, pp. 58-68, 2013.

@article{lirolem29366,
publisher = {Institute of Electrical and Electronics Engineers (IEEE)},
author = {Ayse Kucukyilmaz and Tevfik Metin Sezgin and Cagatay Basdogan},
pages = {58--68},
month = {May},
number = {1},
year = {2013},
journal = {IEEE Transactions on Haptics},
volume = {6},
title = {Intention recognition for dynamic role exchange in haptic collaboration},
keywords = {ARRAY(0x56147f648a50)},
url = {http://eprints.lincoln.ac.uk/29366/}
}
• A. G. Kupcsik, M. P. Deisenroth, J. Peters, and G. Neumann, "Data-efficient generalization of robot skills with contextual policy search," Proceedings of the 27th AAAI Conference on Artificial Intelligence, AAAI 2013, pp. 1401-1407, 2013.

In robotics, controllers make the robot solve a task within a specific context. The context can describe the objectives of the robot or physical properties of the environment and is always specified before task execution. To generalize the controller to multiple contexts, we follow a hierarchical approach for policy learning: A lower-level policy controls the robot for a given context and an upper-level policy generalizes among contexts. Current approaches for learning such upper-level policies are based on model-free policy search, which require an excessive number of interactions of the robot with its environment. More data-efficient policy search approaches are model based but, thus far, without the capability of learning hierarchical policies. We propose a new model-based policy search approach that can also learn contextual upper-level policies. Our approach is based on learning probabilistic forward models for long-term predictions. Using these redictions, we use information-theoretic insights to improve the upper-level policy. Our method achieves a substantial improvement in learning speed compared to existing methods on simulated and real robotic tasks.

@article{lirolem25777,
pages = {1401--1407},
month = {July},
note = {27th AAAI Conference on Artificial Intelligence, AAAI 2013; Bellevue, WA; United States; 14 - 18 July 2013},
journal = {Proceedings of the 27th AAAI Conference on Artificial Intelligence, AAAI 2013},
year = {2013},
title = {Data-efficient generalization of robot skills with contextual policy search},
author = {A. G. Kupcsik and M. P. Deisenroth and J. Peters and Gerhard Neumann},
keywords = {ARRAY(0x56147f9b82f8)},
url = {http://eprints.lincoln.ac.uk/25777/},
abstract = {In robotics, controllers make the robot solve a task within a specific context. The context can describe the objectives of
the robot or physical properties of the environment and is always specified before task execution. To generalize the controller to multiple contexts, we follow a hierarchical approach for policy learning: A lower-level policy controls the robot for a given context and an upper-level policy generalizes among contexts. Current approaches for learning such upper-level policies are based on model-free policy search, which require an excessive number of interactions of the robot with its environment.
More data-efficient policy search approaches are model based but, thus far, without the capability of learning
hierarchical policies. We propose a new model-based policy search approach that can also learn contextual upper-level
policies. Our approach is based on learning probabilistic forward models for long-term predictions. Using these  redictions, we use information-theoretic insights to improve the upper-level policy. Our method achieves a substantial improvement in learning speed compared to existing methods on simulated and real robotic tasks.}
}
• C. Lang, S. Wachsmuth, M. Hanheide, and H. Wersing, "Facial communicative signal interpretation in human-robot interaction by discriminative video subsequence selection," in International Conference on Robotics and Automation (ICRA), 2013, pp. 170-177.

Facial communicative signals (FCSs) such as head gestures, eye gaze, and facial expressions can provide useful feedback in conversations between people and also in humanrobot interaction. This paper presents a pattern recognition approach for the interpretation of FCSs in terms of valence, based on the selection of discriminative subsequences in video data. These subsequences capture important temporal dynamics and are used as prototypical reference subsequences in a classi?cation procedure based on dynamic time warping and feature extraction with active appearance models. Using this valence classi?cation, the robot can discriminate positive from negative interaction situations and react accordingly. The approach is evaluated on a database containing videos of people interacting with a robot by teaching the names of several objects to it. The verbal answer of the robot is expected to elicit the display of spontaneous FCSs by the human tutor, which were classi?ed in this work. The achieved classi?cation accuracies are comparable to the average human recognition performance and outperformed our previous results on this task.

@inproceedings{lirolem7880,
year = {2013},
title = {Facial communicative signal interpretation in human-robot interaction by discriminative video subsequence selection},
pages = {170--177},
booktitle = {International Conference on Robotics and Automation (ICRA)},
month = {May},
note = {Facial communicative signals (FCSs) such as head gestures, eye gaze, and facial expressions can provide useful feedback in conversations between people and also in humanrobot interaction. This paper presents a pattern recognition approach for the interpretation of FCSs in terms of valence, based on the selection of discriminative subsequences in video data. These subsequences capture important temporal dynamics and are used as prototypical reference subsequences in a classi?cation procedure based on dynamic time warping and feature extraction with active appearance models. Using this valence classi?cation, the robot can discriminate positive from negative interaction situations and react accordingly. The approach is evaluated on a database containing videos of people interacting with a robot by teaching the names of several objects to it. The verbal answer of the robot is expected to elicit the display of spontaneous FCSs by the human tutor, which were classi?ed in this work. The achieved classi?cation accuracies are comparable to the average human recognition performance and outperformed our previous results on this task.},
author = {Christian Lang and Sven Wachsmuth and Marc Hanheide and Heiko Wersing},
publisher = {IEEE},
abstract = {Facial communicative signals (FCSs) such as head gestures, eye gaze, and facial expressions can provide useful feedback in conversations between people and also in humanrobot interaction. This paper presents a pattern recognition approach for the interpretation of FCSs in terms of valence, based on the selection of discriminative subsequences in video data. These subsequences capture important temporal dynamics and are used as prototypical reference subsequences in a classi?cation procedure based on dynamic time warping and feature extraction with active appearance models. Using this valence classi?cation, the robot can discriminate positive from negative interaction situations and react accordingly. The approach is evaluated on a database containing videos of people interacting with a robot by teaching the names of several objects to it. The verbal answer of the robot is expected to elicit the display of spontaneous FCSs by the human tutor, which were classi?ed in this work. The achieved classi?cation accuracies are comparable to the average human recognition performance and outperformed our previous results on this task.},
keywords = {ARRAY(0x56147f7a73a8)},
url = {http://eprints.lincoln.ac.uk/7880/}
}
• C. Lang, S. Wachsmuth, M. Hanheide, and H. Wersing, "Facial communicative signal interpretation in human-robot interaction by discriminative video subsequence selection," in IEEE International Conference on Robotics and Automation (ICRA), Karlsruhe, 2013, pp. 170-177.

Facial communicative signals (FCSs) such as head gestures, eye gaze, and facial expressions can provide useful feedback in conversations between people and also in human-robot interaction. This paper presents a pattern recognition approach for the interpretation of FCSs in terms of valence, based on the selection of discriminative subsequences in video data. These subsequences capture important temporal dynamics and are used as prototypical reference subsequences in a classification procedure based on dynamic time warping and feature extraction with active appearance models. Using this valence classification, the robot can discriminate positive from negative interaction situations and react accordingly. The approach is evaluated on a database containing videos of people interacting with a robot by teaching the names of several objects to it. The verbal answer of the robot is expected to elicit the display of spontaneous FCSs by the human tutor, which were classified in this work. The achieved classification accuracies are comparable to the average human recognition performance and outperformed our previous results on this task. Â\copyright 2013 IEEE.

@inproceedings{lirolem13775,
booktitle = {IEEE International Conference on Robotics and Automation (ICRA)},
pages = {170--177},
month = {May},
note = {Conference Code:100673},
publisher = {IEEE},
author = {C. Lang and S. Wachsmuth and M. Hanheide and H. Wersing},
title = {Facial communicative signal interpretation in human-robot interaction by discriminative video subsequence selection},
year = {2013},
keywords = {ARRAY(0x56147f66ece8)},
url = {http://eprints.lincoln.ac.uk/13775/},
abstract = {Facial communicative signals (FCSs) such as head gestures, eye gaze, and facial expressions can provide useful feedback in conversations between people and also in human-robot interaction. This paper presents a pattern recognition approach for the interpretation of FCSs in terms of valence, based on the selection of discriminative subsequences in video data. These subsequences capture important temporal dynamics and are used as prototypical reference subsequences in a classification procedure based on dynamic time warping and feature extraction with active appearance models. Using this valence classification, the robot can discriminate positive from negative interaction situations and react accordingly. The approach is evaluated on a database containing videos of people interacting with a robot by teaching the names of several objects to it. The verbal answer of the robot is expected to elicit the display of spontaneous FCSs by the human tutor, which were classified in this work. The achieved classification accuracies are comparable to the average human recognition performance and outperformed our previous results on this task. {\^A}{\copyright} 2013 IEEE.}
}
• C. Lang, S. Wachsmuth, M. Hanheide, and H. Wersing, "Facial communicative signal interpretation in human-robot interaction by discriminative video subsequence selection," in IEEE International Conference on Robotics and Automation, ICRA 2013, Karlsruhe, 2013, pp. 170-177.

Facial communicative signals (FCSs) such as head gestures, eye gaze, and facial expressions can provide useful feedback in conversations between people and also in human-robot interaction. This paper presents a pattern recognition approach for the interpretation of FCSs in terms of valence, based on the selection of discriminative subsequences in video data. These subsequences capture important temporal dynamics and are used as prototypical reference subsequences in a classification procedure based on dynamic time warping and feature extraction with active appearance models. Using this valence classification, the robot can discriminate positive from negative interaction situations and react accordingly. The approach is evaluated on a database containing videos of people interacting with a robot by teaching the names of several objects to it. The verbal answer of the robot is expected to elicit the display of spontaneous FCSs by the human tutor, which were classified in this work. The achieved classification accuracies are comparable to the average human recognition performance and outperformed our previous results on this task. Â\copyright 2013 IEEE.

@inproceedings{lirolem13462,
year = {2013},
title = {Facial communicative signal interpretation in human-robot interaction by discriminative video subsequence selection},
booktitle = {IEEE International Conference on Robotics and Automation, ICRA 2013},
pages = {170--177},
note = {Conference Code:100673},
publisher = {IEEE},
author = {C. Lang and S. Wachsmuth and M. Hanheide and H. Wersing},
keywords = {ARRAY(0x5614782d9e10)},
url = {http://eprints.lincoln.ac.uk/13462/},
abstract = {Facial communicative signals (FCSs) such as head gestures, eye gaze, and facial expressions can provide useful feedback in conversations between people and also in human-robot interaction. This paper presents a pattern recognition approach for the interpretation of FCSs in terms of valence, based on the selection of discriminative subsequences in video data. These subsequences capture important temporal dynamics and are used as prototypical reference subsequences in a classification procedure based on dynamic time warping and feature extraction with active appearance models. Using this valence classification, the robot can discriminate positive from negative interaction situations and react accordingly. The approach is evaluated on a database containing videos of people interacting with a robot by teaching the names of several objects to it. The verbal answer of the robot is expected to elicit the display of spontaneous FCSs by the human tutor, which were classified in this work. The achieved classification accuracies are comparable to the average human recognition performance and outperformed our previous results on this task. {\^A}{\copyright} 2013 IEEE.}
}
• F. Moreno, G. Cielniak, and T. Duckett, "Evaluation of laser range-finder mapping for agricultural spraying vehicles," in Towards Autonomous Robotic Systems, 2013, pp. 210-221.

In this paper, we present a new application of laser range-finder sensing to agricultural spraying vehicles. The current generation of spraying vehicles use automatic controllers to maintain the height of the sprayer booms above the crop. However, these control systems are typically based on ultrasonic sensors mounted on the booms, which limits the accuracy of the measurements and the response of the controller to changes in the terrain, resulting in a sub-optimal spraying process. To overcome these limitations, we propose to use a laser scanner, attached to the front of the sprayer's cabin, to scan the ground surface in front of the vehicle and to build a scrolling 3d map of the terrain. We evaluate the proposed solution in a series of field tests, demonstrating that the approach provides a more detailed and accurate representation of the environment than the current sonar-based solution, and which can lead to the development of more efficient boom control systems.

@inproceedings{lirolem11330,
year = {2013},
title = {Evaluation of laser range-finder mapping for agricultural spraying vehicles},
author = {Francisco-Angel Moreno and Grzegorz Cielniak and Tom Duckett},
pages = {210--221},
booktitle = {Towards Autonomous Robotic Systems},
month = {August},
keywords = {ARRAY(0x56147f968550)},
url = {http://eprints.lincoln.ac.uk/11330/},
abstract = {In this paper, we present a new application of laser range-finder sensing to agricultural spraying vehicles. The current generation of spraying vehicles use automatic controllers to maintain the height of the sprayer booms above the crop.
However, these control systems are typically based on ultrasonic sensors mounted on the booms, which limits the accuracy of the measurements and the response of the controller to changes in the terrain, resulting in a sub-optimal spraying process. To overcome these limitations, we propose to use a laser scanner, attached to the front of the sprayer's cabin, to scan the ground surface in front of the vehicle and to build a scrolling 3d map of the terrain. We evaluate the proposed solution in a series of field tests, demonstrating that the approach provides a more detailed and accurate representation of the environment than the current sonar-based solution, and which can lead to the development of more efficient boom control systems.}
}
• A. Paraschos, C. Daniel, J. Peters, and G. Neumann, "Probabilistic movement primitives," in Advances in Neural Information Processing Systems, (NIPS), 2013.

Movement Primitives (MP) are a well-established approach for representing modular and re-usable robot movement generators. Many state-of-the-art robot learning successes are based MPs, due to their compact representation of the inherently continuous and high dimensional robot movements. A major goal in robot learning is to combine multiple MPs as building blocks in a modular control architecture to solve complex tasks. To this effect, a MP representation has to allow for blending between motions, adapting to altered task variables, and co-activating multiple MPs in parallel. We present a probabilistic formulation of the MP concept that maintains a distribution over trajectories. Our probabilistic approach allows for the derivation of new operations which are essential for implementing all aforementioned properties in one framework. In order to use such a trajectory distribution for robot movement control, we analytically derive a stochastic feedback controller which reproduces the given trajectory distribution. We evaluate and compare our approach to existing methods on several simulated as well as real robot scenarios.

@inproceedings{lirolem25785,
author = {A. Paraschos and C. Daniel and J. Peters and G. Neumann},
year = {2013},
journal = {Advances in Neural Information Processing Systems},
title = {Probabilistic movement primitives},
booktitle = {Advances in Neural Information Processing Systems, (NIPS)},
month = {December},
keywords = {ARRAY(0x56147f89da78)},
url = {http://eprints.lincoln.ac.uk/25785/},
abstract = {Movement Primitives (MP) are a well-established approach for representing modular
and re-usable robot movement generators. Many state-of-the-art robot learning
successes are based MPs, due to their compact representation of the inherently
continuous and high dimensional robot movements. A major goal in robot learning
is to combine multiple MPs as building blocks in a modular control architecture
to solve complex tasks. To this effect, a MP representation has to allow for
multiple MPs in parallel. We present a probabilistic formulation of the MP concept
that maintains a distribution over trajectories. Our probabilistic approach
allows for the derivation of new operations which are essential for implementing
all aforementioned properties in one framework. In order to use such a trajectory
distribution for robot movement control, we analytically derive a stochastic feedback
controller which reproduces the given trajectory distribution. We evaluate
and compare our approach to existing methods on several simulated as well as
real robot scenarios.}
}
• A. Paraschos, G. Neumann, and J. Peters, "A probabilistic approach to robot trajectory generation," in 13th IEEE-RAS International Conference on Humanoid Robots (Humanoids), 2013, pp. 477-483.

Motor Primitives (MPs) are a promising approach for the data-driven acquisition as well as for the modular and re-usable generation of movements. However, a modular control architecture with MPs is only effective if the MPs support co-activation as well as continuously blending the activation from one MP to the next. In addition, we need efficient mechanisms to adapt a MP to the current situation. Common approaches to movement primitives lack such capabilities or their implementation is based on heuristics. We present a probabilistic movement primitive approach that overcomes the limitations of existing approaches. We encode a primitive as a probability distribution over trajectories. The representation as distribution has several beneficial properties. It allows encoding a time-varying variance profile. Most importantly, it allows performing new operations - a product of distributions for the co-activation of MPs conditioning for generalizing the MP to different desired targets. We derive a feedback controller that reproduces a given trajectory distribution in closed form. We compare our approach to the existing state-of-the art and present real robot results for learning from demonstration.

@inproceedings{lirolem25693,
publisher = {IEEE},
author = {A. Paraschos and G. Neumann and J. Peters},
booktitle = {13th IEEE-RAS International Conference on  Humanoid Robots (Humanoids)},
pages = {477--483},
month = {October},
number = {Februa},
year = {2013},
title = {A probabilistic approach to robot trajectory generation},
volume = {2015-F},
keywords = {ARRAY(0x56147f6e3d90)},
url = {http://eprints.lincoln.ac.uk/25693/},
abstract = {Motor Primitives (MPs) are a promising approach for the data-driven acquisition as well as for the modular and re-usable generation of movements. However, a modular control architecture with MPs is only effective if the MPs support co-activation as well as continuously blending the activation from one MP to the next. In addition, we need efficient mechanisms to adapt a MP to the current situation. Common approaches to movement primitives lack such capabilities or their implementation is based on heuristics. We present a probabilistic movement primitive approach that overcomes the limitations of existing approaches. We encode a primitive as a probability distribution over trajectories. The representation as distribution has several beneficial properties. It allows encoding a time-varying variance profile. Most importantly, it allows performing new operations - a product of distributions for the co-activation of MPs conditioning for generalizing the MP to different desired targets. We derive a feedback controller that reproduces a given trajectory distribution in closed form. We compare our approach to the existing state-of-the art and present real robot results for learning from demonstration.}
}
• E. A. Rueckert, G. Neumann, M. Toussaint, and W. Maass, "Learned graphical models for probabilistic planning provide a new class of movement primitives," Frontiers in Computational Neuroscience, vol. 6, 2013.

Biological movement generation combines three interesting aspects: its modular organization in movement primitives (MPs), its characteristics of stochastic optimality under perturbations, and its efficiency in terms of learning. A common approach to motor skill learning is to endow the primitives with dynamical systems. Here, the parameters of the primitive indirectly define the shape of a reference trajectory. We propose an alternative MP representation based on probabilistic inference in learned graphical models with new and interesting properties that complies with salient features of biological movement control. Instead of endowing the primitives with dynamical systems, we propose to endow MPs with an intrinsic probabilistic planning system, integrating the power of stochastic optimal control (SOC) methods within a MP. The parameterization of the primitive is a graphical model that represents the dynamics and intrinsic cost function such that inference in this graphical model yields the control policy. We parameterize the intrinsic cost function using task-relevant features, such as the importance of passing through certain via-points. The system dynamics as well as intrinsic cost function parameters are learned in a reinforcement learning (RL) setting. We evaluate our approach on a complex 4-link balancing task. Our experiments show that our movement representation facilitates learning significantly and leads to better generalization to new task settings without re-learning.

@article{lirolem25789,
publisher = {Frontiers Media},
author = {Elmar A. Rueckert and Gerhard Neumann and Marc Toussaint and Wolfgang Maass},
year = {2013},
volume = {6},
journal = {Frontiers in Computational Neuroscience},
title = {Learned graphical models for probabilistic planning provide a new class of movement primitives},
month = {January},
keywords = {ARRAY(0x56147f5d5d78)},
url = {http://eprints.lincoln.ac.uk/25789/},
abstract = {Biological movement generation combines three interesting aspects: its modular organization in movement primitives (MPs), its characteristics of stochastic optimality under perturbations, and its efficiency in terms of learning. A common approach to motor skill learning is to endow the primitives with dynamical systems. Here, the parameters of the primitive indirectly define the shape of a reference trajectory. We propose an alternative MP representation based on probabilistic inference in learned graphical models with new and interesting properties that complies with salient features of biological movement control. Instead of endowing the primitives with dynamical systems, we propose to endow MPs with an intrinsic probabilistic planning system, integrating the power of stochastic optimal control (SOC) methods within a MP. The parameterization of the primitive is a graphical model that represents the dynamics and intrinsic cost function such that inference in this graphical model yields the control policy. We parameterize the intrinsic cost function using task-relevant features, such as the importance of passing through certain via-points. The system dynamics as well as intrinsic cost function parameters are learned in a reinforcement learning (RL) setting. We evaluate our approach on a complex 4-link balancing task. Our experiments show that our movement representation facilitates learning significantly and leads to better generalization to new task settings without re-learning.}
}
• P. S. Teh, A. B. J. Teoh, and S. Yue, "A survey of keystroke dynamics biometrics," The Scientific World Journal, vol. 2013, p. 408280, 2013.

Research on keystroke dynamics biometrics has been increasing, especially in the last decade. The main motivation behind this effort is due to the fact that keystroke dynamics biometrics is economical and can be easily integrated into the existing computer security systems with minimal alteration and user intervention. Numerous studies have been conducted in terms of data acquisition devices, feature representations, classification methods, experimental protocols, and evaluations. However, an up-to-date extensive survey and evaluation is not yet available. The objective of this paper is to provide an insightful survey and comparison on keystroke dynamics biometrics research performed throughout the last three decades, as well as offering suggestions and possible future research directions.

@article{lirolem12817,
month = {December},
pages = {408280},
author = {Pin Shen Teh and Andrew Beng Jin Teoh and Shigang Yue},
publisher = {Hindawi Publishing Corporation / Scientific World},
title = {A survey of keystroke dynamics biometrics},
journal = {The Scientific World Journal},
year = {2013},
volume = {2013},
keywords = {ARRAY(0x56147f925430)},
url = {http://eprints.lincoln.ac.uk/12817/},
abstract = {Research on keystroke dynamics biometrics has been increasing, especially in the last decade. The main motivation behind this effort is due to the fact that keystroke dynamics biometrics is economical and can be easily integrated into the existing computer security systems with minimal alteration and user intervention. Numerous studies have been conducted in terms of data acquisition devices, feature representations, classification methods, experimental protocols, and evaluations. However, an up-to-date extensive survey and evaluation is not yet available. The objective of this paper is to provide an insightful survey and comparison on keystroke dynamics biometrics research performed throughout the last three decades, as well as offering suggestions and possible future research directions.}
}
• J. Xu, S. Yue, and Y. Tang, "A motion attention model based on rarity weighting and motion cues in dynamic scenes," International Journal of Pattern Recognition and Artificial Intelligence, vol. 27, iss. 06, p. 1355009, 2013.

Nowadays, motion attention model is a controversial topic in the biological computer vision area. The computational attention model can be decomposed into a set of features via predefined channels. Here we designed a bio-inspired vision attention model, and added the rarity measurement onto it. The priority of rarity is emphasized under the assumption of weighting effect upon the features logic fusion. At this stage, a final saliency map at each frame is adjusted by the spatiotemporal and rarity values. By doing this, the process of mimicking human vision attention becomes more realistic and logical to the real circumstance. The experiments are conducted on the benchmark dataset of static images and video sequences. We simulated the attention shift based on several dataset. Most importantly, our dynamic scenes are mostly selected from the objects moving on the highway and dynamic scenes. The former one can be developed on the detection of car collision and will be a useful tool for further application in robotics. We also conduct experiment on the other video clips to prove the rationality of rarity factor and feature cues fusion methods. Finally, the evaluation results indicate our visual attention model outperforms several state-of-the-art motion attention models. Read More: http://www.worldscientific.com/doi/abs/10.1142/S0218001413550094

@article{lirolem13793,
number = {06},
year = {2013},
title = {A motion attention model based on rarity weighting and motion cues in dynamic scenes},
journal = {International Journal of Pattern Recognition and Artificial Intelligence},
volume = {27},
publisher = {World Scientific Publishing},
author = {Jiawei Xu and Shigang Yue and Yuchao Tang},
pages = {1355009},
month = {September},
keywords = {ARRAY(0x56147f5b1508)},
url = {http://eprints.lincoln.ac.uk/13793/},
abstract = {Nowadays, motion attention model is a controversial topic in the biological computer vision area. The computational attention model can be decomposed into a set of features via predefined channels. Here we designed a bio-inspired vision attention model, and added the rarity measurement onto it. The priority of rarity is emphasized under the assumption of weighting effect upon the features logic fusion. At this stage, a final saliency map at each frame is adjusted by the spatiotemporal and rarity values. By doing this, the process of mimicking human vision attention becomes more realistic and logical to the real circumstance. The experiments are conducted on the benchmark dataset of static images and video sequences. We simulated the attention shift based on several dataset. Most importantly, our dynamic scenes are mostly selected from the objects moving on the highway and dynamic scenes. The former one can be developed on the detection of car collision and will be a useful tool for further application in robotics. We also conduct experiment on the other video clips to prove the rationality of rarity factor and feature cues fusion methods. Finally, the evaluation results indicate our visual attention model outperforms several state-of-the-art motion attention models.

}
• S. Yue and C. F. Rind, "Postsynaptic organizations of directional selective visual neural networks for collision detection," Neurocomputing, vol. 103, pp. 50-62, 2013.

In this paper, we studied the postsynaptic organizations of directional selective visual neurons for collision detection. Directional selective neurons can extract different directional visual motion cues fast and reliably by allowing inhibition spreads to further layers in specific directions with one or several time steps delay. Whether these directional selective neurons can be easily organised for other specific visual tasks is not known. Taking collision detection as the primary visual task, we investigated the postsynaptic organizations of these directional selective neurons through evolutionary processes. The evolved postsynaptic organizations demonstrated robust properties in detecting imminent collisions in complex visual environments with many of which achieved 94\% success rate after evolution suggesting active roles in collision detection directional selective neurons and its postsynaptic organizations can play.

@article{lirolem9308,
journal = {Neurocomputing},
volume = {103},
year = {2013},
title = {Postsynaptic organizations of directional selective visual neural networks for collision detection},
author = {Shigang Yue and F. Claire Rind},
publisher = {Elsevier Science Limited},
pages = {50--62},
month = {March},
keywords = {ARRAY(0x56147f8ffe88)},
url = {http://eprints.lincoln.ac.uk/9308/},
abstract = {In this paper, we studied the postsynaptic organizations of directional selective visual neurons for collision detection. Directional selective neurons can extract different directional visual motion cues fast and reliably by allowing inhibition spreads to further layers in specific directions with one or several time steps delay. Whether these directional selective neurons can be easily organised for other specific visual tasks is not known. Taking collision detection as the primary visual task, we investigated the postsynaptic organizations of these directional selective neurons through evolutionary processes. The evolved postsynaptic organizations demonstrated robust properties in detecting imminent collisions in complex visual environments with many of which achieved 94\% success rate after evolution suggesting active roles in collision detection directional selective neurons and its postsynaptic organizations can play.}
}
• S. Yue and C. F. Rind, "Redundant neural vision systems: competing for collision recognition roles," IEEE Transactions on Autonomous Mental Development, vol. 5, iss. 2, pp. 173-186, 2013.

Ability to detect collisions is vital for future robots that interact with humans in complex visual environments. Lobula giant movement detectors (LGMD) and directional selective neurons (DSNs) are two types of identified neurons found in the visual pathways of insects such as locusts. Recent modelling studies showed that the LGMD or grouped DSNs could each be tuned for collision recognition. In both biological and artificial vision systems, however, which one should play the collision recognition role and the way the two types of specialized visual neurons could be functioning together are not clear. In this modeling study, we compared the competence of the LGMD and the DSNs, and also investigate the cooperation of the two neural vision systems for collision recognition via artificial evolution. We implemented three types of collision recognition neural subsystems ? the LGMD, the DSNs and a hybrid system which combines the LGMD and the DSNs subsystems together, in each individual agent. A switch gene determines which of the three redundant neural subsystems plays the collision recognition role. We found that, in both robotics and driving environments, the LGMD was able to build up its ability for collision recognition quickly and robustly therefore reducing the chance of other types of neural networks to play the same role. The results suggest that the LGMD neural network could be the ideal model to be realized in hardware for collision recognition.

@article{lirolem9307,
number = {2},
volume = {5},
year = {2013},
journal = {IEEE Transactions on Autonomous Mental Development},
title = {Redundant neural vision systems: competing for collision recognition roles},
pages = {173--186},
month = {June},
author = {Shigang Yue and F. Claire Rind},
publisher = {IEEE / Institute of Electrical and Electronics Engineers Incorporated},
keywords = {ARRAY(0x56147f6143a8)},
url = {http://eprints.lincoln.ac.uk/9307/},
abstract = {Ability to detect collisions is vital for future robots that interact with humans in complex visual environments. Lobula giant movement detectors (LGMD) and directional selective neurons (DSNs) are two types of identified neurons found in the visual pathways of insects such as locusts. Recent modelling studies showed that the LGMD or grouped DSNs could each be tuned for collision recognition. In both biological and artificial vision systems, however, which one should play the collision recognition role and the way the two types of specialized visual neurons could be functioning together are not clear. In this modeling study, we compared the competence of the LGMD and the DSNs, and also investigate the cooperation of the two neural vision systems for collision recognition via artificial evolution. We implemented three types of collision recognition neural subsystems ? the LGMD, the DSNs and a hybrid system which combines the LGMD and the DSNs subsystems together, in each individual agent. A switch gene determines which of the three redundant neural subsystems plays the collision recognition role. We found that, in both robotics and driving environments, the LGMD was able to build up its ability for collision recognition quickly and robustly therefore reducing the chance of other types of neural networks to play the same role. The results suggest that the LGMD neural network could be the ideal model to be realized in hardware for collision recognition.}
}
• G. Zahi and S. Yue, "Automatic detection of low light images in a video sequence Shot under different light conditions," in Modelling Symposium (EMS), 2013 European, 2013, pp. 271-276.

Nocturnal insects have the ability to neurally sum visual signals in space and time to be able to see under very low light conditions. This ability shown by nocturnal insects has inspired many researchers to develop a night vision algorithm, that is capable of significantly improving the quality and reliability of digital images captured under very low light conditions. This algorithm however when applied to day time images rather degrades their quality. It is therefore not suitable to apply the night vision algorithms equally to an image stream with different light conditions. This paper introduces a quick method of automatically determining when to apply the nocturnal vision algorithm by analysing the cumulative intensity histogram of each image in the stream. The effectiveness of this method is demonstrated with relevant experiments in a good and acceptable way.

@inproceedings{lirolem13757,
author = {Gabriel Zahi and Shigang Yue},
publisher = {IEEE},
year = {2013},
title = {Automatic detection of low light images in a video sequence Shot under different light conditions},
booktitle = {Modelling Symposium (EMS), 2013 European},
pages = {271--276},
month = {November},
keywords = {ARRAY(0x56147f694018)},
url = {http://eprints.lincoln.ac.uk/13757/},
abstract = {Nocturnal insects have the ability to neurally sum visual signals in space and time to be able to see under very low light conditions. This ability shown by nocturnal insects has inspired many researchers to develop a night vision algorithm, that is capable of significantly improving the quality and reliability of digital images captured under very low light conditions. This algorithm however when applied to day time images rather degrades their quality. It is therefore not suitable to apply the night vision algorithms equally to an image stream with different light conditions. This paper introduces a quick method of automatically determining when to apply the nocturnal vision algorithm by analysing the cumulative intensity histogram of each image in the stream. The effectiveness of this method is demonstrated with relevant experiments in a good and acceptable way.}
}
• M. Zillich, K. Zhou, D. Skocaj, M. Kristan, A. Vrecko, M. Mahnic, M. Janicek, G. M. Kruijff, T. Keller, M. Hanheide, and N. Hawes, "Robot George: interactive continuous learning of visual concepts," in Proceedings of the 8th ACM/IEEE international conference on Human-robot interaction, 2013, p. 425.

The video presents the robot George learning visual concepts in dialogue with a tutor

@inproceedings{lirolem8365,
pages = {425},
booktitle = {Proceedings of the 8th ACM/IEEE international conference on Human-robot interaction},
month = {March},
title = {Robot George: interactive continuous learning of visual concepts},
year = {2013},
publisher = {IEEE Press},
author = {Michael Zillich and Kai Zhou and Danijel Skocaj and Matej Kristan and Alen Vrecko and Marko Mahnic and Miroslav Janicek and Geert-Jan M. Kruijff and Thomas Keller and Marc Hanheide and Nick Hawes},
keywords = {ARRAY(0x56147f51e9a0)},
url = {http://eprints.lincoln.ac.uk/8365/},
abstract = {The video presents the robot George learning visual concepts in dialogue with a tutor}
}

### 2012

• H. B. Amor, O. Kroemer, U. Hillenbrand, G. Neumann, and J. Peters, "Generalization of human grasping for multi-fingered robot hands," in International Conference on Robot Systems (IROS), 2012, pp. 2043-2050.

Multi-fingered robot grasping is a challenging problem that is difficult to tackle using hand-coded programs. In this paper we present an imitation learning approach for learning and generalizing grasping skills based on human demonstrations. To this end, we split the task of synthesizing a grasping motion into three parts: (1) learning efficient grasp representations from human demonstrations, (2) warping contact points onto new objects, and (3) optimizing and executing the reach-and-grasp movements. We learn low-dimensional latent grasp spaces for different grasp types, which form the basis for a novel extension to dynamic motor primitives. These latent-space dynamic motor primitives are used to synthesize entire reach-and-grasp movements. We evaluated our method on a real humanoid robot. The results of the experiment demonstrate the robustness and versatility of our approach.

@inproceedings{lirolem25788,
author = {Heni Ben Amor and Oliver Kroemer and Ulrich Hillenbrand and Gerhard Neumann and Jan Peters},
year = {2012},
title = {Generalization of human grasping for multi-fingered robot hands},
journal = {IEEE International Conference on Intelligent Robots and Systems},
booktitle = {International Conference on Robot Systems (IROS)},
pages = {2043--2050},
month = {December},
url = {http://eprints.lincoln.ac.uk/25788/},
keywords = {ARRAY(0x56147f7ac268)},
abstract = {Multi-fingered robot grasping is a challenging
problem that is difficult to tackle using hand-coded programs.
In this paper we present an imitation learning approach for
learning and generalizing grasping skills based on human
demonstrations. To this end, we split the task of synthesizing
a grasping motion into three parts: (1) learning efficient grasp
representations from human demonstrations, (2) warping contact
points onto new objects, and (3) optimizing and executing
the reach-and-grasp movements. We learn low-dimensional
latent grasp spaces for different grasp types, which form the
basis for a novel extension to dynamic motor primitives. These
latent-space dynamic motor primitives are used to synthesize
entire reach-and-grasp movements. We evaluated our method
on a real humanoid robot. The results of the experiment
demonstrate the robustness and versatility of our approach.}
}
• F. Arvin, A. E. Turgut, and S. Yue, "Fuzzy-based aggregation with a mobile robot swarm," Lecture Notes in Computer Science, vol. 7461, pp. 346-347, 2012.

Aggregation is a widely observed phenomenon in social insects and animals such as cockroaches, honeybees and birds. From swarm robotics perspective [3], aggregation can be defined as gathering randomly distributed robots to form an aggregate. Honeybee aggregation is an example of cue-based aggregation method that was studied in [4]. In that study, micro robots were deployed in a gradually lighted environment to mimic the behavior of honeybees which aggregate around a zone that has the optimal temperature (BEECLUST). In our previous study [2], two modifications on BEECLUST ? dynamic velocity and comparative waiting time ? were applied to increase the performance of aggregation.

@article{lirolem7328,
pages = {346--347},
month = {September},
publisher = {Springer},
author = {Farshad Arvin and Ali Emre Turgut and Shigang Yue},
year = {2012},
journal = {Lecture Notes in Computer Science},
volume = {7461},
title = {Fuzzy-based aggregation with a mobile robot swarm},
keywords = {ARRAY(0x56147f9bf338)},
url = {http://eprints.lincoln.ac.uk/7328/},
abstract = {Aggregation is a widely observed phenomenon in social insects and animals such as cockroaches, honeybees and birds. From swarm robotics perspective [3], aggregation can be defined as gathering randomly distributed robots to form an aggregate. Honeybee aggregation is an example of cue-based aggregation method that was studied in [4]. In that study, micro robots were deployed in a gradually lighted environment to mimic the behavior of honeybees which aggregate around a zone that has the optimal temperature (BEECLUST). In our previous study [2], two modifications on BEECLUST ? dynamic velocity and comparative waiting time ? were applied to increase the performance of aggregation.}
}
• M. Barnes, M. Dudbridge, and T. Duckett, "Polarised light stress analysis and laser scatter imaging for non-contact inspection of heat seals in food trays," Journal of Food Engineering, vol. 112, iss. 3, pp. 183-190, 2012.

This paper introduces novel non-contact methods for detecting faults in heat seals of food packages. Two alternative imaging technologies are investigated; laser scatter imaging and polarised light stress images. After segmenting the seal area from the rest of the respective image, a classifier is trained to detect faults in different regions of the seal area using features extracted from the pixels in the respective region. A very large set of candidate features, based on statistical information relating to the colour and texture of each region, is first extracted. Then an adaptive boosting algorithm (AdaBoost) is used to automatically select the best features for discriminating faults from non-faults. With this approach, different features can be selected and optimised for the different imaging methods. In experiments we compare the performance of classifiers trained using features extracted from laser scatter images only, polarised light stress images only, and a combination of both image types. The results show that the polarised light and laser scatter classifiers achieved accuracies of 96$\backslash$\% and 90$\backslash$\%, respectively, while the combination of both sensors achieved an accuracy of 95$\backslash$\%. These figures suggest that both systems have potential for commercial development.

@article{lirolem5513,
author = {Michael Barnes and Michael Dudbridge and Tom Duckett},
publisher = {Elsevier},
pages = {183--190},
month = {October},
number = {3},
title = {Polarised light stress analysis and laser scatter imaging for non-contact inspection of heat seals in food trays},
year = {2012},
journal = {Journal of Food Engineering},
volume = {112},
keywords = {ARRAY(0x56147f994a18)},
url = {http://eprints.lincoln.ac.uk/5513/},
abstract = {This paper introduces novel non-contact methods for detecting faults in heat seals of food packages. Two alternative imaging technologies are investigated; laser scatter imaging and polarised light stress images. After segmenting the seal area from the rest of the respective image, a classifier is trained to detect faults in different regions of the seal area using features extracted from the pixels in the respective region. A very large set of candidate features, based on statistical information relating to the colour and texture of each region, is first extracted. Then an adaptive boosting algorithm (AdaBoost) is used to automatically select the best features for discriminating faults from non-faults. With this approach, different features can be selected and optimised for the different imaging methods. In experiments we compare the performance of classifiers trained using features extracted from laser scatter images only, polarised light stress images only, and a combination of both image types. The results show that the polarised light and laser scatter classifiers achieved accuracies of 96{$\backslash$}\% and 90{$\backslash$}\%, respectively, while the combination of both sensors achieved an accuracy of 95{$\backslash$}\%. These figures suggest that both systems have potential for commercial development.}
}
• N. Bellotto, B. Benfold, H. Harland, H. Nagel, N. Pirlo, I. Reid, E. Sommerlade, and C. Zhao, "Cognitive visual tracking and camera control," Computer Vision and Image Understanding, vol. 116, iss. 3, pp. 457-471, 2012.

Cognitive visual tracking is the process of observing and understanding the behaviour of a moving person. This paper presents an efficient solution to extract, in real-time, high-level information from an observed scene, and generate the most appropriate commands for a set of pan-tilt-zoom (PTZ) cameras in a surveillance scenario. Such a high-level feedback control loop, which is the main novelty of our work, will serve to reduce uncertainties in the observed scene and to maximize the amount of information extracted from it. It is implemented with a distributed camera system using SQL tables as virtual communication channels, and Situation Graph Trees for knowledge representation, inference and high-level camera control. A set of experiments in a surveillance scenario show the effectiveness of our approach and its potential for real applications of cognitive vision.

@article{lirolem4823,
month = {March},
pages = {457--471},
author = {Nicola Bellotto and Ben Benfold and Hanno Harland and Hans-Hellmut Nagel and Nicola Pirlo and Ian Reid and Eric Sommerlade and Chuan Zhao},
publisher = {Elsevier},
year = {2012},
journal = {Computer Vision and Image Understanding},
title = {Cognitive visual tracking and camera control},
volume = {116},
number = {3},
keywords = {ARRAY(0x56147f863b68)},
url = {http://eprints.lincoln.ac.uk/4823/},
abstract = {Cognitive visual tracking is the process of observing and understanding the behaviour of a moving person. This paper presents an efficient solution to extract, in real-time, high-level information from an observed scene, and generate the most appropriate commands for a set of pan-tilt-zoom (PTZ) cameras in a surveillance scenario. Such a high-level feedback control loop, which is the main novelty of our work, will serve to reduce uncertainties in the observed scene and to maximize the amount of information extracted from it. It is implemented with a distributed camera system using SQL tables as virtual communication channels, and Situation Graph Trees for knowledge representation, inference and high-level camera control. A set of experiments in a surveillance scenario show the effectiveness of our approach and its potential for real applications of cognitive vision.}
}
• N. Bellotto, "Robot control based on qualitative representation of human trajectories," in AAAI Spring Symposium, "Designing Intelligent Robots: Reintegrating AI", 2012.

A major challenge for future social robots is the high-level interpretation of human motion, and the consequent generation of appropriate robot actions. This paper describes some fundamental steps towards the real-time implementation of a system that allows a mobile robot to transform quantitative information about human trajectories (i.e. coordinates and speed) into qualitative concepts, and from these to generate appropriate control commands. The problem is formulated using a simple version of qualitative trajectory calculus, then solved using an inference engine based on fuzzy temporal logic and situation graph trees. Preliminary results are discussed and future directions of the current research are drawn.

@inproceedings{lirolem4780,
month = {March},
booktitle = {AAAI Spring Symposium, "Designing Intelligent Robots: Reintegrating AI"},
year = {2012},
title = {Robot control based on qualitative representation of human trajectories},
publisher = {AAAI - Association for the Advancement of Artificial Intelligence},
author = {Nicola Bellotto},
note = {A major challenge for future social robots is the high-level interpretation of human motion, and the consequent generation of appropriate robot actions. This paper describes some fundamental steps towards the real-time implementation of a system that allows a mobile robot to transform quantitative information about human trajectories (i.e. coordinates and speed) into qualitative concepts, and from these to generate appropriate control commands. The problem is formulated using a simple version of qualitative trajectory calculus, then solved using an inference engine based on fuzzy temporal logic and situation graph trees. Preliminary results are discussed and future directions of the current research are drawn.},
abstract = {A major challenge for future social robots is the high-level interpretation of human motion, and the consequent generation of appropriate robot actions. This paper describes some fundamental steps towards the real-time implementation of a system that allows a mobile robot to transform quantitative information about human trajectories (i.e. coordinates and speed) into qualitative concepts, and from these to generate appropriate control commands. The problem is formulated using a simple version of qualitative trajectory calculus, then solved using an inference engine based on fuzzy temporal logic and situation graph trees. Preliminary results are discussed and future directions of the current research are drawn.},
url = {http://eprints.lincoln.ac.uk/4780/},
keywords = {ARRAY(0x56147f9e8ba8)}
}
• T. Belpaeme, P. Baxter, R. Read, R. Wood, H. Cuayáhuitl, B. Kiefer, S. Racioppa, I. Kruijff-Korbayová, G. Athanasopoulos, V. Enescu, R. Looije, M. Neerincx, Y. Demiris, R. Ros-Espinoza, A. Beck, L. Cañamero, A. Hiolle, M. Lewis, I. Baroni, M. Nalin, P. Cosi, G. Paci, F. Tesser, G. Sommavilla, and R. Humbert, "Multimodal child-robot interaction: building social bonds," Journal of Human-Robot Interaction, vol. 1, iss. 2, 2012.

For robots to interact effectively with human users they must be capable of coordinated, timely behavior in response to social context. The Adaptive Strategies for Sustainable Long-Term Social Interaction (ALIZ-E) project focuses on the design of long-term, adaptive social interaction between robots and child users in real-world settings. In this paper, we report on the iterative approach taken to scientific and technical developments toward this goal: advancing individual technical competencies and integrating them to form an autonomous robotic system for evaluation ?in the wild.? The first evaluation iterations have shown the potential of this methodology in terms of adaptation of the robot to the interactant and the resulting influences on engagement. This sets the foundation for an ongoing research program that seeks to develop technologies for social robot companions.

@article{lirolem22210,
title = {Multimodal child-robot interaction: building social bonds},
journal = {Journal of Human-Robot Interaction},
volume = {1},
year = {2012},
number = {2},
month = {December},
author = {Tony Belpaeme and Paul Baxter and Robin Read and Rachel Wood and Heriberto Cuay{\'a}huitl and Bernd Kiefer and Stefania Racioppa and Ivana Kruijff-Korbayov{\'a} and Georgios Athanasopoulos and Valentin Enescu and Rosemarijn Looije and Mark Neerincx and Yiannis Demiris and Raquel Ros-Espinoza and Aryel Beck and Lola Ca{\~n}amero and Antione Hiolle and Matthew Lewis and Ilaria Baroni and Marco Nalin and Piero Cosi and Giulio Paci and Fabio Tesser and Giacomo Sommavilla and Remi Humbert},
publisher = {Clear Facts Research},
abstract = {For robots to interact effectively with human users they must be capable of coordinated, timely behavior in response to social context. The Adaptive Strategies for Sustainable Long-Term Social Interaction (ALIZ-E) project focuses on the design of long-term, adaptive social interaction between robots and child users in real-world settings. In this paper, we report on the iterative approach taken to scientific and technical developments toward this goal: advancing individual technical competencies and integrating them to form an autonomous robotic system for evaluation ?in the wild.? The first evaluation iterations have shown the potential of this methodology in terms of adaptation of the robot to the interactant and the resulting influences on engagement. This sets the foundation for an ongoing research program that seeks to develop technologies for social robot companions.},
url = {http://eprints.lincoln.ac.uk/22210/},
keywords = {ARRAY(0x56147f88e478)}
}
• J. Bird, T. Feltwell, and G. Cielniak, "Real-time adaptive track generation in racing games," in GAMEON '2012, 2012, pp. 17-24.

Real-time Adaptive Track Generation in Racing Games

@inproceedings{lirolem6900,
year = {2012},
title = {Real-time adaptive track generation in racing games},
month = {November},
pages = {17--24},
booktitle = {GAMEON '2012},
author = {Jake Bird and Tom Feltwell and Grzegorz Cielniak},
publisher = {Eurosis},
note = {Real-time Adaptive Track Generation in Racing Games},
url = {http://eprints.lincoln.ac.uk/6900/},
keywords = {ARRAY(0x56147f852890)},
abstract = {Real-time Adaptive Track Generation in Racing Games}
}
• G. Cielniak, N. Bellotto, and T. Duckett, "Integrating vision and robotics into the computer science curriculum," in 3rd International Workshop Teaching Robotics Teaching with Robotics: Integrating Robotics in School Curriculum, 2012.

This paper describes our efforts in integrating Robotics education into the undergraduate Computer Science curriculum. Our approach delivers Mobile Robotics together with the closely related field of Computer Vision and is directly linked to the research conducted at our institution. The paper describes the most relevant details related to the module content and assessment strategy, paying particular attention to the practical sessions using Rovio mobile webcams. We discuss the specific choices made with regard to the mobile platform, software libraries and lab environment. We also present a detailed qualitative and quantitative analysis, including the correlation between student engagement and performance, and discuss the outcomes of this experience.

@inproceedings{lirolem5516,
note = {This paper describes our efforts in integrating Robotics education into the undergraduate Computer Science curriculum. Our approach delivers Mobile Robotics together with the closely related field of Computer Vision and is directly linked to the research conducted at our institution. The paper describes the most relevant details related to the module content and assessment strategy, paying particular attention to the practical sessions using Rovio mobile webcams. We discuss the specific choices made with regard to the mobile platform, software libraries and lab environment. We also present a detailed qualitative and quantitative analysis, including the correlation between student engagement and performance, and discuss the outcomes of this experience.},
author = {Grzegorz Cielniak and Nicola Bellotto and Tom Duckett},
title = {Integrating vision and robotics into the computer science curriculum},
year = {2012},
booktitle = {3rd International Workshop Teaching Robotics Teaching with Robotics: Integrating Robotics in School Curriculum},
month = {April},
abstract = {This paper describes our efforts in integrating Robotics education into the undergraduate Computer Science curriculum. Our approach delivers Mobile Robotics together with the closely related field of Computer Vision and is directly linked to the research conducted at our institution. The paper describes the most relevant details related to the module content and assessment strategy, paying particular attention to the practical sessions using Rovio mobile webcams. We discuss the specific choices made with regard to the mobile platform, software libraries and lab environment. We also present a detailed qualitative and quantitative analysis, including the correlation between student engagement and performance, and discuss the outcomes of this experience.},
url = {http://eprints.lincoln.ac.uk/5516/},
keywords = {ARRAY(0x56147f6dc9d0)}
}
• C. Daniel, G. Neumann, and J. Peters, "Learning concurrent motor skills in versatile solution spaces," in International Conference on Intelligent Robot Systems (IROS), 2012, pp. 3591-3597.

Future robots need to autonomously acquire motor skills in order to reduce their reliance on human programming. Many motor skill learning methods concentrate on learning a single solution for a given task. However, discarding information about additional solutions during learning unnecessarily limits autonomy. Such favoring of single solutions often requires re-learning of motor skills when the task, the environment or the robot?s body changes in a way that renders the learned solution infeasible. Future robots need to be able to adapt to such changes and, ideally, have a large repertoire of movements to cope with such problems. In contrast to current methods, our approach simultaneously learns multiple distinct solutions for the same task, such that a partial degeneration of this solution space does not prevent the successful completion of the task. In this paper, we present a complete framework that is capable of learning different solution strategies for a real robot Tetherball task.

@inproceedings{lirolem25787,
title = {Learning concurrent motor skills in versatile solution spaces},
year = {2012},
journal = {IEEE International Conference on Intelligent Robots and Systems},
author = {C. Daniel and G. Neumann and J. Peters},
month = {October},
pages = {3591--3597},
booktitle = {International Conference on Intelligent Robot Systems (IROS)},
abstract = {Future robots need to autonomously acquire motor
skills in order to reduce their reliance on human programming.
Many motor skill learning methods concentrate
unnecessarily limits autonomy. Such favoring of single solutions
often requires re-learning of motor skills when the task, the
environment or the robot?s body changes in a way that renders
the learned solution infeasible. Future robots need to be able to
adapt to such changes and, ideally, have a large repertoire of
movements to cope with such problems. In contrast to current
methods, our approach simultaneously learns multiple distinct
solutions for the same task, such that a partial degeneration of
this solution space does not prevent the successful completion
of the task. In this paper, we present a complete framework
that is capable of learning different solution strategies for a
keywords = {ARRAY(0x56147f917b50)},
url = {http://eprints.lincoln.ac.uk/25787/}
}
• C. Daniel, G. Neumann, and J. Peters, "Hierarchical relative entropy policy search," in Proceedings of the 15th International Conference on Artificial Intelligence and Statistics (AISTATS) 2012, 2012, pp. 273-281.

Many real-world problems are inherently hierarchically structured. The use of this structure in an agent?s policy may well be the key to improved scalability and higher performance. However, such hierarchical structures cannot be exploited by current policy search algorithms. We will concentrate on a basic, but highly relevant hierarchy -- the ?mixed option? policy. Here, a gating network first decides which of the options to execute and, subsequently, the option-policy determines the action. In this paper, we reformulate learning a hierarchical policy as a latent variable estimation problem and subsequently extend the Relative Entropy Policy Search (REPS) to the latent variable case. We show that our Hierarchical REPS can learn versatile solutions while also showing an increased performance in terms of learning speed and quality of the found policy in comparison to the nonhierarchical approach.

@inproceedings{lirolem25791,
volume = {22},
year = {2012},
title = {Hierarchical relative entropy policy search},
pages = {273--281},
booktitle = {Proceedings of the 15th International Conference on Artificial Intelligence and Statistics (AISTATS) 2012},
month = {April},
author = {Christian Daniel and Gerhard Neumann and Jan Peters},
publisher = {MIT Press},
abstract = {Many real-world problems are inherently hierarchically
structured. The use of this structure
in an agent?s policy may well be the
key to improved scalability and higher performance.
However, such hierarchical structures
cannot be exploited by current policy
search algorithms. We will concentrate on
a basic, but highly relevant hierarchy {--} the
?mixed option? policy. Here, a gating network
first decides which of the options to execute
and, subsequently, the option-policy determines
the action.
In this paper, we reformulate learning a hierarchical
policy as a latent variable estimation
problem and subsequently extend the
Relative Entropy Policy Search (REPS) to
the latent variable case. We show that our
Hierarchical REPS can learn versatile solutions
while also showing an increased performance
in terms of learning speed and quality
of the found policy in comparison to the nonhierarchical
approach.},
url = {http://eprints.lincoln.ac.uk/25791/},
keywords = {ARRAY(0x56147f8bdbf8)}
}
• T. Feltwell, P. Dickinson, and G. Cielniak, "A framework for quantitative analysis of user-generated spatial data," in GAMEON '2012, 2012, pp. 17-24.

This paper proposes a new framework for automated analysis of game-play metrics for aiding game designers in finding out the critical aspects of the game caused by factors like design modications, change in playing style, etc. The core of the algorithm measures similarity between spatial distribution of user generated in-game events and automatically ranks them in order of importance. The feasibility of the method is demonstrated on a data set collected from a modern, multiplayer First Person Shooter, together with application examples of its use. The proposed framework can be used to accompany traditional testing tools and make the game design process more efficient.

@inproceedings{lirolem6889,
year = {2012},
title = {A framework for quantitative analysis of user-generated spatial data},
note = {This paper proposes a new framework for automated
analysis of game-play metrics for aiding game designers
in finding out the critical aspects of the game caused
by factors like design modications, change in playing
style, etc. The core of the algorithm measures similarity
between spatial distribution of user generated in-game
events and automatically ranks them in order of importance. The feasibility of the method is demonstrated on
a data set collected from a modern, multiplayer First
Person Shooter, together with application examples of
its use. The proposed framework can be used to accompany traditional testing tools and make the game design
process more efficient.},
author = {Tom Feltwell and Patrick Dickinson and Grzegorz Cielniak},
publisher = {Eurosis},
booktitle = {GAMEON '2012},
pages = {17--24},
month = {November},
keywords = {ARRAY(0x56147f6e77a0)},
url = {http://eprints.lincoln.ac.uk/6889/},
abstract = {This paper proposes a new framework for automated
analysis of game-play metrics for aiding game designers
in finding out the critical aspects of the game caused
by factors like design modications, change in playing
style, etc. The core of the algorithm measures similarity
between spatial distribution of user generated in-game
events and automatically ranks them in order of importance. The feasibility of the method is demonstrated on
a data set collected from a modern, multiplayer First
Person Shooter, together with application examples of
its use. The proposed framework can be used to accompany traditional testing tools and make the game design
process more efficient.}
}
• S. Ghidoni, G. Cielniak, and E. Menegatti, "Texture-based crowd detection and localisation," in The 12th International Conference on Intelligent Autonomous Systems, 2012, pp. 725-736.

This paper presents a crowd detection system based on texture analysis. The state-of-the-art techniques based on co-occurrence matrix have been revisited and a novel set of features proposed. These features provide a richer description of the co-occurrence matrix, and can be exploited to obtain stronger classification results, especially when smaller portions of the image are considered. This is extremely useful for crowd localisation: acquired images are divided into smaller regions in order to perform a classification on each one. A thorough evaluation of the proposed system on a real world data set is also presented: this validates the improvements in reliability of the crowd detection and localisation.

@inproceedings{lirolem5935,
booktitle = {The 12th International Conference on Intelligent Autonomous Systems},
pages = {725--736},
month = {June},
publisher = {IEEE / Robotics and Automation Society},
author = {Stefano Ghidoni and Grzegorz Cielniak and Emanuele Menegatti},
year = {2012},
title = {Texture-based crowd detection and localisation},
keywords = {ARRAY(0x56147f650158)},
url = {http://eprints.lincoln.ac.uk/5935/},
abstract = {This paper presents a crowd detection system based on texture analysis. The state-of-the-art techniques based on co-occurrence matrix have been revisited and a novel set of features proposed. These features provide a richer description of the co-occurrence matrix, and can be exploited to obtain stronger classification results, especially when smaller portions of the image are considered. This is extremely useful for crowd localisation: acquired images are divided into smaller regions in order to perform a classification on each one. A thorough evaluation of the proposed system on a real world data set is also presented: this validates the improvements in reliability of the crowd detection and localisation.}
}
• M. Hanheide, M. Lohse, and H. Zender, "Expectations, intentions, and actions in human-robot interaction," Internation Journal of Social Robotics, vol. 4, iss. 2, pp. 107-108, 2012.

From the issue entitled "Expectations, Intentions & Actions" Human-robot interaction is becoming increasingly complex through the growing number of abilities, both cognitive and physical, available to today?s robots. At the same time, interaction is still often dif?cult because the users do not understand the robots? internal states, expectations, intentions, and actions. Vice versa, robots lack understanding of the users? expectations, intentions, actions, and social signals.

@article{lirolem6562,
author = {M. Hanheide and M. Lohse and H. Zender},
publisher = {Springer},
note = {From the issue entitled "Expectations, Intentions \& Actions"
Human-robot interaction is becoming increasingly complex
through the growing number of abilities, both cognitive and
physical, available to today?s robots. At the same time, interaction is still often dif?cult because the users do not understand the robots? internal states, expectations, intentions, and
actions. Vice versa, robots lack understanding of the users?
expectations, intentions, actions, and social signals.},
month = {April},
pages = {107--108},
volume = {4},
year = {2012},
journal = {Internation Journal of Social Robotics},
title = {Expectations, intentions, and actions in human-robot interaction},
number = {2},
abstract = {From the issue entitled "Expectations, Intentions \& Actions"
Human-robot interaction is becoming increasingly complex
through the growing number of abilities, both cognitive and
physical, available to today?s robots. At the same time, interaction is still often dif?cult because the users do not understand the robots? internal states, expectations, intentions, and
actions. Vice versa, robots lack understanding of the users?
expectations, intentions, actions, and social signals.},
url = {http://eprints.lincoln.ac.uk/6562/},
keywords = {ARRAY(0x56147f5d0470)}
}
• M. Hanheide, A. Peters, and N. Bellotto, "Analysis of human-robot spatial behaviour applying a qualitative trajectory calculus," in 21st IEEE International Symposium on Robot and Human Interactive Communication, 2012, pp. 689-694.

The analysis and understanding of human-robot joint spatial behaviour (JSB) such as guiding, approaching, departing, or coordinating movements in narrow spaces and its communicative and dynamic aspects are key requirements on the road towards more intuitive interaction, safe encounter, and appealing living with mobile robots. This endeavours demand for appropriate models and methodologies to represent JSB and facilitate its analysis. In this paper, we adopt a qualitative trajectory calculus (QTC) as a formal foundation for the analysis and representation of such spatial behaviour of a human and a robot based on a compact encoding of the relative trajectories of two interacting agents in a sequential model. We present this QTC together with a distance measure and a probabilistic behaviour model and outline its usage in an actual JSB study.We argue that the proposed QTC coding scheme and derived methodologies for analysis and modelling are flexible and extensible to be adapted for a variety of other scenarios and studies. I.

@inproceedings{lirolem6750,
editor = {B. Gottfried and H. Aghajan},
year = {2012},
title = {Analysis of human-robot spatial behaviour applying a qualitative trajectory calculus},
month = {September},
pages = {689--694},
booktitle = {21st IEEE International Symposium on Robot and Human Interactive Communication},
publisher = {IEEE},
author = {Marc Hanheide and Annika Peters and Nicola Bellotto},
abstract = {The analysis and understanding of human-robot joint spatial behaviour (JSB) such as guiding, approaching, departing, or coordinating movements in narrow spaces and its communicative and dynamic aspects are key requirements on the road towards more intuitive interaction, safe encounter, and appealing living with mobile robots. This endeavours demand for appropriate models and methodologies to represent JSB and facilitate its analysis. In this paper, we adopt a qualitative trajectory calculus (QTC) as a formal foundation for the analysis and representation of such spatial behaviour of a human and a robot based on a compact encoding of the relative trajectories of two interacting agents in a sequential model. We present this QTC together with a distance measure and a probabilistic behaviour model and outline its usage in an actual JSB study.We argue that the proposed QTC coding scheme and derived methodologies for analysis and modelling are flexible and extensible to be adapted for a variety of other scenarios and studies. I.},
url = {http://eprints.lincoln.ac.uk/6750/},
keywords = {ARRAY(0x56147f909630)}
}
• J. Hutton, G. Harper, and T. Duckett, "A prototype low-cost machine vision system for automatic identification and quantification of potato defects," in The Dundee Conference - Crop Protection in Northern Britain 2012, 2012, pp. 273-278.

This paper reports on a current project to develop a prototype system for the automatic identification and quantification of potato defects based on machine vision. The system developed uses off-the-shelf hardware, including a low-cost vision sensor and a standard desktop computer with a graphics processing unit (GPU), together with software algorithms to enable detection, identification and quantification of common defects affecting potatoes at near-real-time frame rates. The system uses state-of-the-art image processing and machine learning techniques to automatically learn the appearance of different defect types. It also incorporates an intuitive graphical user interface (GUI) to enable easy set-up of the system by quality control (QC) staff working in the industry.

@inproceedings{lirolem14511,
month = {February},
pages = {273--278},
booktitle = {The  Dundee Conference  - Crop  Protection  in  Northern  Britain  2012},
title = {A prototype low-cost machine vision system for automatic identification and quantification of potato defects},
year = {2012},
author = {Jamie Hutton and Glyn Harper and Tom Duckett},
publisher = {Proceedings Crop Protection in Northern Britain 2012},
abstract = {This paper reports on a current project to develop a prototype system
for the automatic identification and quantification of potato defects based on
machine vision. The system developed uses off-the-shelf hardware, including a
low-cost vision sensor and a standard desktop computer with a graphics processing
unit (GPU), together with software algorithms to enable detection, identification
and quantification of common defects affecting potatoes at near-real-time frame
rates. The system uses state-of-the-art image processing and machine learning
techniques to automatically learn the appearance of different defect types. It also
incorporates an intuitive graphical user interface (GUI) to enable easy set-up of the
system by quality control (QC) staff working in the industry.},
keywords = {ARRAY(0x56147f74ff78)},
url = {http://eprints.lincoln.ac.uk/14511/}
}
• C. Jayne, S. Yue, and L. Iliadis, "Engineering Applications of Neural Networks: 13th International Conference, EANN 2012 London, UK, September 20-23, 2012 Proceedings," in 13th International Conference, EANN 2012, Chengdu, 2012.

.

@inproceedings{lirolem11609,
booktitle = {13th International Conference, EANN 2012},
month = {September},
note = {Conference Code:98083},
publisher = {Springer},
author = {Chrisina Jayne and Shigang Yue and Lazaros Iliadis},
title = {Engineering Applications of Neural Networks: 13th International Conference, EANN 2012 London, UK, September 20-23, 2012 Proceedings},
year = {2012},
volume = {311},
abstract = {.},
url = {http://eprints.lincoln.ac.uk/11609/},
keywords = {ARRAY(0x56147fa283c0)}
}
• C. Jayne, S. Yue, and L. Iliadis, Engineering applications of neural networks, Heidelberg: Springer, 2012, vol. 311.

Proceeedings of the 13th International Conference, EANN 2012, London, UK, September 20-23, 2012

@book{lirolem7434,
publisher = {Springer},
author = {Crisina Jayne and Shigang Yue and Lazaros Iliadis},
note = {Proceeedings of the 13th International Conference, EANN 2012, London, UK, September 20-23, 2012},
series = {Communications in computer and information science},
title = {Engineering applications of neural networks},
volume = {311},
year = {2012},
keywords = {ARRAY(0x56147f85cd30)},
url = {http://eprints.lincoln.ac.uk/7434/},
abstract = {Proceeedings of the 13th International Conference, EANN 2012, London, UK, September 20-23, 2012}
}
• A. Kucukyilmaz, S. O. Oguz, T. M. Sezgin, and C. Basdogan, "Improving human-computer cooperation through haptic role exchange and negotiation," in Immersive multimodal interactive presence, A. Peer and C. Giachritsis, Eds., Springer London, 2012, pp. 229-254.

Even though in many systems, computers have been programmed to share control with human operators in order to increase task performance, the interaction in such systems is still artificial when compared to natural human-human cooperation. In complex tasks, cooperating human partners may have their own agendas and take initiatives during the task. Such initiatives contribute to a richer interaction between cooperating parties, yet little research exists on how this can be established between a human and a computer. In a cooperation involving haptics, the coupling between the human and the computer should be defined such that the computer can understand the intentions of the human operator and respond accordingly. We believe that this will make the haptic interactions between the human and the computer more natural and human-like. In this regard, we suggest (1) a role exchange mechanism that is activated based on the magnitude of the force applied by the cooperating parties and (2) a negotiation model that enables more human-like coupling between the cooperating parties. We argue that when presented through the haptic channel, the proposed role exchange mechanism and the negotiation model serve to communicate the cooperating parties dynamically, naturally, and seamlessly, in addition to improving the task efficiency of the user. In this chapter, we explore how human-computer cooperation can be improved using a role-exchange mechanism and a haptic negotiation framework. We also discuss the use of haptic negotiation in assigning different behaviors to the computer; and the effectiveness of visual and haptic cues in conveying negotiation-related complex affective states. Throughout this chapter, we will adopt a broad terminology and speak of cooperative systems, in which both parties take some part in control, as shared control schemes, but the term ?control? is merely used to address the partners? manipulation capacities on the task.

@incollection{lirolem29365,
publisher = {Springer London},
author = {Ayse Kucukyilmaz and Salih Ozgur Oguz and Tevfik Metin Sezgin and Cagatay Basdogan},
month = {January},
pages = {229--254},
booktitle = {Immersive multimodal interactive presence},
title = {Improving human-computer cooperation through haptic role exchange and negotiation},
year = {2012},
series = {Springer Series on Touch and Haptic Systems},
editor = {A. Peer and C. Giachritsis},
abstract = {Even though in many systems, computers have been programmed to share control with human operators in order to increase task performance, the interaction in such systems is still artificial when compared to natural human-human cooperation. In complex tasks, cooperating human partners may have their own agendas and take initiatives during the task. Such initiatives contribute to a richer interaction between cooperating parties, yet little research exists on how this can be established between a human and a computer. In a cooperation involving haptics, the coupling between the human and the computer should be defined such that the computer can understand the intentions of the human operator and respond accordingly. We believe that this will make the haptic interactions between the human and the computer more natural and human-like. In this regard, we suggest (1) a role exchange mechanism that is activated based on the magnitude of the force applied by the cooperating parties and (2) a negotiation model that enables more human-like coupling between the cooperating parties. We argue that when presented through the haptic channel, the proposed role exchange mechanism and the negotiation model serve to communicate the cooperating parties dynamically, naturally, and seamlessly, in addition to improving the task efficiency of the user. In this chapter, we explore how human-computer cooperation can be improved using a role-exchange mechanism and a haptic negotiation framework. We also discuss the use of haptic negotiation in assigning different behaviors to the computer; and the effectiveness of visual and haptic cues in conveying negotiation-related complex affective states. Throughout this chapter, we will adopt a broad terminology and speak of cooperative systems, in which both parties take some part in control, as shared control schemes, but the term ?control? is merely used to address the partners? manipulation capacities on the task.},
url = {http://eprints.lincoln.ac.uk/29365/},
keywords = {ARRAY(0x56147f8608d0)}
}
• C. Lang, S. Wachsmuth, M. Hanheide, and H. Wersing, "Facial communicative signals: valence recognition in task-oriented human-robot interaction," International Journal of Social Robotics, vol. 4, iss. 3, pp. 249-262, 2012.

From the issue entitled "Measuring Human-Robots Interactions" This paper investigates facial communicative signals (head gestures, eye gaze, and facial expressions) as nonverbal feedback in human-robot interaction. Motivated by a discussion of the literature, we suggest scenario-specific investigations due to the complex nature of these signals and present an object-teaching scenario where subjects teach the names of objects to a robot, which in turn shall term these objects correctly afterwards. The robot?s verbal answers are to elicit facial communicative signals of its interaction partners. We investigated the human ability to recognize this spontaneous facial feedback and also the performance of two automatic recognition approaches. The first one is a static approach yielding baseline results, whereas the second considers the temporal dynamics and achieved classification rates

@article{lirolem6561,
number = {3},
year = {2012},
volume = {4},
title = {Facial communicative signals: valence recognition in task-oriented human-robot interaction},
journal = {International Journal of Social Robotics},
note = {From the issue entitled "Measuring Human-Robots Interactions"
This paper investigates facial communicative signals (head gestures, eye gaze, and facial expressions) as nonverbal feedback in human-robot interaction. Motivated by a discussion of the literature, we suggest scenario-specific investigations due to the complex nature of these signals and present an object-teaching scenario where subjects teach the names of objects to a robot, which in turn shall term these objects correctly afterwards. The robot?s verbal answers are to elicit facial communicative signals of its interaction partners. We investigated the human ability to recognize this spontaneous facial feedback and also the performance of two automatic recognition approaches. The first one is a static approach yielding baseline results, whereas the second considers the temporal dynamics and achieved classification rates},
author = {Christian Lang and Sven Wachsmuth and Marc Hanheide and Heiko Wersing},
publisher = {Springer},
pages = {249--262},
month = {August},
keywords = {ARRAY(0x56147f842bd0)},
url = {http://eprints.lincoln.ac.uk/6561/},
abstract = {From the issue entitled "Measuring Human-Robots Interactions"
This paper investigates facial communicative signals (head gestures, eye gaze, and facial expressions) as nonverbal feedback in human-robot interaction. Motivated by a discussion of the literature, we suggest scenario-specific investigations due to the complex nature of these signals and present an object-teaching scenario where subjects teach the names of objects to a robot, which in turn shall term these objects correctly afterwards. The robot?s verbal answers are to elicit facial communicative signals of its interaction partners. We investigated the human ability to recognize this spontaneous facial feedback and also the performance of two automatic recognition approaches. The first one is a static approach yielding baseline results, whereas the second considers the temporal dynamics and achieved classification rates}
}
• G. R. Law, P. D. Baxter, and M. S. Gilthorpe, "Selection bias in epidemiologic studies," in Modern methods for epidemiology, Springer Netherlands, 2012, pp. 57-71.

Bias is inherent in epidemiology, and researchers go to great lengths to avoid introducing bias into their studies. However, some bias is inevitable, and bias due to selection is particularly common. We discuss ways to identify bias and how authors have approached removing or adjusting for bias using statistical methods. Â\copyright Springer Science+Business Media Dordrecht 2012.

@incollection{lirolem26517,
year = {2012},
journal = {Modern Methods for Epidemiology},
title = {Selection bias in epidemiologic studies},
month = {February},
pages = {57--71},
booktitle = {Modern methods for epidemiology},
publisher = {Springer Netherlands},
author = {Graham R. Law and Paul D. Baxter and Mark S. Gilthorpe},
abstract = {Bias is inherent in epidemiology, and researchers go to great lengths to avoid introducing bias into their studies. However, some bias is inevitable, and bias due to selection is particularly common. We discuss ways to identify bias and how authors have approached removing or adjusting for bias using statistical methods. {\^A}{\copyright} Springer Science+Business Media Dordrecht 2012.},
keywords = {ARRAY(0x56147f9b8b20)},
url = {http://eprints.lincoln.ac.uk/26517/}
}
• S. Liu, Y. Tang, C. Zhang, and S. Yue, "Self-map building in wireless sensor network based on TDOA measurements," in IASTED International Conference on Artificial Intelligence and Soft Computing, Hamburg, 2012, pp. 150-155.

Node localization has long been established as a key problem in the sensor networks. Self-mapping in wireless sensor network which enables beacon-based systems to build a node map on-the-fly extends the range of the sensor network's applications. A variety of self-mapping algorithms have been developed for the sensor networks. Some algorithms assume no information and estimate only the relative location of the sensor nodes. In this paper, we assume a very small percentage of the sensor nodes aware of their own locations, so the proposed algorithm estimates other node's absolute location using the distance differences. In particular, time difference of arrival (TDOA) technology is adopted to obtain the distance difference. The obtained time difference accuracy is 10ns which corresponds to a distance difference error of 3m. We evaluate self-mapping's accuracy with a small number of seed nodes. Overall, the accuracy and the coverage are shown to be comparable to those achieved results with other technologies and algorithms. Â\copyright 2012 IEEE.

@inproceedings{lirolem10769,
year = {2012},
title = {Self-map building in wireless sensor network based on TDOA measurements},
publisher = {IASTED},
author = {S. Liu and Y. Tang and C. Zhang and S. Yue},
note = {Conference Code:94291},
month = {September},
pages = {150--155},
booktitle = {IASTED International Conference on Artificial Intelligence and Soft Computing},
abstract = {Node localization has long been established as a key problem in the sensor networks. Self-mapping in wireless sensor network which enables beacon-based systems to build a node map on-the-fly extends the range of the sensor network's applications. A variety of self-mapping algorithms have been developed for the sensor networks. Some algorithms assume no information and estimate only the relative location of the sensor nodes. In this paper, we assume a very small percentage of the sensor nodes aware of their own locations, so the proposed algorithm estimates other node's absolute location using the distance differences. In particular, time difference of arrival (TDOA) technology is adopted to obtain the distance difference. The obtained time difference accuracy is 10ns which corresponds to a distance difference error of 3m. We evaluate self-mapping's accuracy with a small number of seed nodes. Overall, the accuracy and the coverage are shown to be comparable to those achieved results with other technologies and algorithms. {\^A}{\copyright} 2012 IEEE.},
url = {http://eprints.lincoln.ac.uk/10769/},
keywords = {ARRAY(0x56147f8f0378)}
}
• S. O. Oguz, A. Kucukyilmaz, T. M. Sezgin, and C. Basdogan, "Supporting negotiation behavior with haptics-enabled human-computer interfaces," IEEE Transactions on Haptics, vol. 5, iss. 3, pp. 274-284, 2012.

An active research goal for human-computer interaction is to allow humans to communicate with computers in an intuitive and natural fashion, especially in real-life interaction scenarios. One approach that has been advocated to achieve this has been to build computer systems with human-like qualities and capabilities. In this paper, we present insight on how human-computer interaction can be enriched by employing the computers with behavioral patterns that naturally appear in human-human negotiation scenarios. For this purpose, we introduce a two-party negotiation game specifically built for studying the effectiveness of haptic and audio-visual cues in conveying negotiation related behaviors. The game is centered around a real-time continuous two-party negotiation scenario based on the existing game-theory and negotiation literature. During the game, humans are confronted with a computer opponent, which can display different behaviors, such as concession, competition, and negotiation. Through a user study, we show that the behaviors that are associated with human negotiation can be incorporated into human-computer interaction, and the addition of haptic cues provides a statistically significant increase in the human-recognition accuracy of machine-displayed behaviors. In addition to aspects of conveying these negotiation-related behaviors, we also focus on and report game-theoretical aspects of the overall interaction experience. In particular, we show that, as reported in the game-theory literature, certain negotiation strategies such as tit-for-tat may generate maximum combined utility for the negotiating parties, providing an excellent balance between the energy spent by the user and the combined utility of the negotiating parties.

@article{lirolem29372,
month = {August},
pages = {274--284},
author = {Salih Ozgur Oguz and Ayse Kucukyilmaz and Tevfik Metin Sezgin and Cagatay Basdogan},
publisher = {Institute of Electrical and Electronics Engineers (IEEE)},
journal = {IEEE Transactions on Haptics},
year = {2012},
volume = {5},
title = {Supporting negotiation behavior with haptics-enabled human-computer interfaces},
number = {3},
abstract = {An active research goal for human-computer interaction is to allow humans to communicate with computers in an intuitive and natural fashion, especially in real-life interaction scenarios. One approach that has been advocated to achieve this has been to build computer systems with human-like qualities and capabilities. In this paper, we present insight on how human-computer interaction can be enriched by employing the computers with behavioral patterns that naturally appear in human-human negotiation scenarios. For this purpose, we introduce a two-party negotiation game specifically built for studying the effectiveness of haptic and audio-visual cues in conveying negotiation related behaviors. The game is centered around a real-time continuous two-party negotiation scenario based on the existing game-theory and negotiation literature. During the game, humans are confronted with a computer opponent, which can display different behaviors, such as concession, competition, and negotiation. Through a user study, we show that the behaviors that are associated with human negotiation can be incorporated into human-computer interaction, and the addition of haptic cues provides a statistically significant increase in the human-recognition accuracy of machine-displayed behaviors. In addition to aspects of conveying these negotiation-related behaviors, we also focus on and report game-theoretical aspects of the overall interaction experience. In particular, we show that, as reported in the game-theory literature, certain negotiation strategies such as tit-for-tat may generate maximum combined utility for the negotiating parties, providing an excellent balance between the energy spent by the user and the combined utility of the negotiating parties.},
url = {http://eprints.lincoln.ac.uk/29372/},
keywords = {ARRAY(0x56147f5d6838)}
}
• O. Szymanezyk, T. Duckett, and P. Dickinson, "Agent-based crowd simulation in airports using games technology," in Transactions on computational collective intelligence, SPRINGER, 2012.

We adapt popular video-games technology for an agent-based crowd simulation framework in an airport terminal. To achieve this, we investigate game technology, crowd simulation and the unique traits of airports. Our findings are implemented in a virtual airport environment that exploits a scalable layered intelligence technique in combination with physics middleware and a social force approach for crowd simulation. Our experiments show that the framework runs at interactive frame-rate and evaluate the scalability with increasing number of agents demonstrating event triggered airport behaviour.

@incollection{lirolem6574,
publisher = {SPRINGER},
author = {Oliver Szymanezyk and Tom Duckett and Patrick Dickinson},
note = {Volume VIII, Issue 7430},
month = {October},
booktitle = {Transactions on computational collective intelligence},
year = {2012},
title = {Agent-based crowd simulation in airports using games technology},
series = {Lecture Notes in Computer Science},
abstract = {We adapt popular video-games technology for an agent-based crowd simulation framework in an airport terminal. To achieve this, we investigate game technology, crowd simulation and the unique traits of airports. Our findings are implemented in a virtual airport environment that exploits a scalable layered intelligence technique in combination with physics middleware and a social force approach for crowd simulation. Our experiments show that
the framework runs at interactive frame-rate and evaluate the scalability with increasing number of agents demonstrating event triggered airport behaviour.},
url = {http://eprints.lincoln.ac.uk/6574/},
keywords = {ARRAY(0x561479eaccc0)}
}
• Y. Tang, J. Peng, S. Yue, and J. Xu, "A primal dual proximal point method of Chambolle-Pock algorithms for ?1-TV minimization problems in image reconstruction," in 5th International Conference on Biomedical Engineering and Informatics, BMEI 2012, Chongqing, 2012, pp. 12-16.

Computed tomography (CT) image reconstruction problems can be solved by finding the minimizer of a suitable objective function. The objective function usually consists of a data fidelity term and a regularization term. Total variation (TV) minimization problems are widely used for solving incomplete data problems in CT image reconstruction. In this paper, we focus on the CT image reconstruction model which combines the TV regularization and ?1 data error term. We introduce a primal dual proximal point method of Chambolle-Pock algorithm to solve the proposed optimization problem. We tested it on computer simulated data and the experiment results shown it exhibited good performance when used to few-view CT image reconstruction. \copyright 2012 IEEE.

@inproceedings{lirolem13409,
publisher = {IEEE},
author = {Y. Tang and Jigen Peng and Shigang Yue and Jiawei Xu},
pages = {12--16},
booktitle = {5th International Conference on Biomedical Engineering and Informatics, BMEI 2012},
journal = {2012 5th International Conference on Biomedical Engineering and Informatics, BMEI 2012},
title = {A primal dual proximal point method of Chambolle-Pock algorithms for ?1-TV minimization problems in image reconstruction},
year = {2012},
abstract = {Computed tomography (CT) image reconstruction problems can be solved by finding the minimizer of a suitable objective function. The objective function usually consists of a data fidelity term and a regularization term. Total variation (TV) minimization problems are widely used for solving incomplete data problems in CT image reconstruction. In this paper, we focus on the CT image reconstruction model which combines the TV regularization and ?1 data error term. We introduce a primal dual proximal point method of Chambolle-Pock algorithm to solve the proposed optimization problem. We tested it on computer simulated data and the experiment results shown it exhibited good performance when used to few-view CT image reconstruction. {\copyright} 2012 IEEE.},
url = {http://eprints.lincoln.ac.uk/13409/},
keywords = {ARRAY(0x56147f9fa2f0)}
}
• P. S. Teh, S. Yue, and A. B. J. Teoh, "Improving keystroke dynamics authentication system via multiple feature fusion scheme," in 2012 International Conference on Cyber Security, Cyber Warfare and Digital Forensic (CyberSec), Kuala Lumpur, 2012, pp. 277-282.

This paper reports the performance and effect of diverse keystroke features combination on keystroke dynamic authentication system by using fusion scheme. First of all, four types of keystroke features are acquired from our collected dataset, later then transformed into similarity scores by the use of Gaussian Probability Density Function (GPD) and Direction Similarity Measure (DSM). Next, three fusion schemes are introduced to merge the scores pairing with six fusion rules. Result shows that the finest performance is obtained by the combination of both dwell time and flight time collectively. Finally, this experiment also investigates the effect of using larger dataset on performance, which turns out to be rather consistent. Â\copyright 2012 IEEE.

@inproceedings{lirolem10860,
title = {Improving keystroke dynamics authentication system via multiple feature fusion scheme},
year = {2012},
booktitle = {2012 International Conference on Cyber Security, Cyber Warfare and Digital Forensic (CyberSec)},
pages = {277--282},
publisher = {IEEE},
author = {Pin Shen Teh and Shigang Yue and A. B. J. Teoh},
note = {Conference Code:92830},
abstract = {This paper reports the performance and effect of diverse keystroke features combination on keystroke dynamic authentication system by using fusion scheme. First of all, four types of keystroke features are acquired from our collected dataset, later then transformed into similarity scores by the use of Gaussian Probability Density Function (GPD) and Direction Similarity Measure (DSM). Next, three fusion schemes are introduced to merge the scores pairing with six fusion rules. Result shows that the finest performance is obtained by the combination of both dwell time and flight time collectively. Finally, this experiment also investigates the effect of using larger dataset on performance, which turns out to be rather consistent. {\^A}{\copyright} 2012 IEEE.},
url = {http://eprints.lincoln.ac.uk/10860/},
keywords = {ARRAY(0x56147f767f60)}
}
• Y. Utsumi, E. Sommerlade, N. Bellotto, and I. Reid, "Cognitive active vision for human identification," in IEEE International Conference on Robotics and Automation (ICRA 2012), 2012.

We describe an integrated, real-time multi-camera surveillance system that is able to find and track individuals, acquire and archive facial image sequences, and perform face recognition. The system is based around an inference engine that can extract high-level information from an observed scene, and generate appropriate commands for a set of pan-tilt-zoom (PTZ) cameras. The incorporation of a reliable facial recognition into the high-level feedback is a main novelty of our work, showing how high-level understanding of a scene can be used to deploy PTZ sensing resources effectively. The system comprises a distributed camera system using SQL tables as virtual communication channels, Situation Graph Trees for knowledge representation, inference and high-level camera control, and a variety of visual processing algorithms including an on-line acquisition of facial images, and on-line recognition of faces by comparing image sets using subspace distance. We provide an extensive evaluation of this method using our system for both acquisition of training data, and later recognition. A set of experiments in a surveillance scenario show the effectiveness of our approach and its potential for real applications of cognitive vision.

@inproceedings{lirolem4836,
note = {We describe an integrated, real-time multi-camera surveillance system that is able to find and track individuals, acquire and archive facial image sequences, and perform face recognition. The system is based around an inference engine that can extract high-level information from an observed scene, and generate appropriate commands for a set of pan-tilt-zoom (PTZ) cameras. The incorporation of a reliable facial recognition into the high-level feedback is a main novelty of our work, showing how high-level understanding of a scene can be used to deploy PTZ sensing resources effectively. The system comprises a distributed camera system using SQL tables as virtual communication channels, Situation Graph
Trees for knowledge representation, inference and high-level camera control, and a variety of visual processing algorithms including an on-line acquisition of facial images, and on-line recognition of faces by comparing image sets using subspace distance. We provide an extensive evaluation of this method using our system for both acquisition of training data, and later recognition. A set of experiments in a surveillance scenario show the effectiveness of our approach and its potential for real applications of cognitive vision.},
year = {2012},
title = {Cognitive active vision for human identification},
author = {Yuzuko Utsumi and Eric Sommerlade and Nicola Bellotto and Ian Reid},
booktitle = {IEEE International Conference on Robotics and Automation (ICRA 2012)},
month = {May},
abstract = {We describe an integrated, real-time multi-camera surveillance system that is able to find and track individuals, acquire and archive facial image sequences, and perform face recognition. The system is based around an inference engine that can extract high-level information from an observed scene, and generate appropriate commands for a set of pan-tilt-zoom (PTZ) cameras. The incorporation of a reliable facial recognition into the high-level feedback is a main novelty of our work, showing how high-level understanding of a scene can be used to deploy PTZ sensing resources effectively. The system comprises a distributed camera system using SQL tables as virtual communication channels, Situation Graph
Trees for knowledge representation, inference and high-level camera control, and a variety of visual processing algorithms including an on-line acquisition of facial images, and on-line recognition of faces by comparing image sets using subspace distance. We provide an extensive evaluation of this method using our system for both acquisition of training data, and later recognition. A set of experiments in a surveillance scenario show the effectiveness of our approach and its potential for real applications of cognitive vision.},
keywords = {ARRAY(0x56147f9a5200)},
url = {http://eprints.lincoln.ac.uk/4836/}
}
• R. Wood, P. Baxter, and T. Belpaeme, "A Review of long-term memory in natural and synthetic systems," Adaptive Behavior, vol. 20, iss. 2, pp. 81-103, 2012.

Memory may be broadly regarded as information gained from past experi- ence which is available in the service of ongoing and future adaptive behavior. The biological implementation ofmemory shares little with memory in synthetic cognitive systems where it is typically regarded as a passive storage structure. Neurophysiological evidence indicates that memory is neither passive nor cen- tralised. A review of the relevant literature in the biological and computer sciences is conducted and a novel methodology is applied that incorporates neuroethological approaches with general biological inspiration in the design of synthetic cognitive systems: a case study regarding episodic memory provides an illustration of the utility of this methodology. As a consequence of applying this approach to the reinterpretation of the implementation of memory in syn- thetic systems, four fundamental functional principles are derived that are in accordance with neuroscientific theory, and which may be applied to the design of more adaptive and robust synthetic cognitive systems: priming, cross-modal associations, cross-modal coordination without semantic information transfer, and global system behavior resulting from activation dynamics within the mem- ory system.

@article{lirolem23079,
author = {Rachel Wood and Paul Baxter and Tony Belpaeme},
publisher = {Sage for International Society for Adaptive Behavior (ISAB)},
pages = {81--103},
month = {April},
number = {2},
volume = {20},
year = {2012},
title = {A Review of long-term memory in natural and synthetic systems},
abstract = {Memory may be broadly regarded as information gained from past experi- ence which is available in the service of ongoing and future adaptive behavior. The biological implementation ofmemory shares little with memory in synthetic cognitive systems where it is typically regarded as a passive storage structure. Neurophysiological evidence indicates that memory is neither passive nor cen- tralised. A review of the relevant literature in the biological and computer sciences is conducted and a novel methodology is applied that incorporates neuroethological approaches with general biological inspiration in the design of synthetic cognitive systems: a case study regarding episodic memory provides an illustration of the utility of this methodology. As a consequence of applying this approach to the reinterpretation of the implementation of memory in syn- thetic systems, four fundamental functional principles are derived that are in accordance with neuroscientific theory, and which may be applied to the design of more adaptive and robust synthetic cognitive systems: priming, cross-modal associations, cross-modal coordination without semantic information transfer, and global system behavior resulting from activation dynamics within the mem- ory system.},
url = {http://eprints.lincoln.ac.uk/23079/},
keywords = {ARRAY(0x56147f51b910)}
}
• J. Xu and S. Yue, "A top-down attention model based on the semi-supervised learning," in 5th International Conference on Biomedical Engineering and Informatics, BMEI 2012, Chongqing, 2012, pp. 1011-1014.
@inproceedings{lirolem13408,
abstract = {In this paper, we proposed a top-down motion tracking model to detect the attention region. Many biological inspired systems have been studied and most of them are consisted by bottom-up mechanisms and top-down processes. Top-down attention is guided by task-driven information that is acquired through learning procedures. Our model improves the top-down mechanisms by using a probability map (PM). The PM follows to track if all the potential locations of targets based on the information contained in the frame sequences. By using this, PM can be regarded as a short term memory for attended saliency regions. This function is similar to the dorsal stream of V1 primary area. The se